Bloomverd — AI Integration & Innovation

Overview

Bloomverd integrates AI at three distinct levels:

1. Autonomous farm health analysis — a scheduled pipeline that collects sensor data, calls an LLM to analyze conditions, and can autonomously trigger physical IoT devices
2. Conversational AI assistant — a streaming chat interface where the LLM has tool access to live farm data and can control devices on the farmer's behalf
3. Computer vision predictions — an external Python/ML service that analyzes geotagged field photos to produce disease and yield risk assessments

These layers work together: health analysis reads from the same database that chat tools query, predictions feed into health context, and IoT actions taken by the LLM are recorded in the same audit log regardless of whether they were triggered by the scheduler or a farmer's chat request.

---

LLM Configuration

The LLM layer uses the OpenAI SDK pointed at an Ollama-compatible endpoint. This makes the system LLM-agnostic — any model that exposes an OpenAI-compatible API works without code changes:

OLLAMA_BASE_URL=http://localhost:11434   # Ollama local
OLLAMA_BASE_URL=https://api.openai.com   # OpenAI directly
OLLAMA_BASE_URL=https://your-gateway     # Any proxy/gateway

OLLAMA_MODEL=llama3.2       # Llama (Meta)
OLLAMA_MODEL=qwen2.5:7b     # Qwen (Alibaba)
OLLAMA_MODEL=gemma3:12b     # Gemma (Google)
OLLAMA_MODEL=gpt-4o-mini    # OpenAI (if using OpenAI endpoint)

The createLlmClient() factory reads OLLAMA_BASE_URL and OLLAMA_API_KEY at startup and injects the configured client into all consumers that need it. Swapping models requires only an environment variable change — no code deployment needed.

---

1. Health Monitoring Pipeline

This is the most architecturally significant AI integration in the system.

How it works

Scheduler — A cron job (configurable, default every 2 hours) runs across all active farms. For each farm it checks the computed_at timestamp of the most recent FarmHealth record. If the farm's health is stale (older than the farmer's healthReportIntervalSeconds setting), it enqueues a batch job.

Consumer — The consumer collects context from multiple sources before calling the LLM:

Data collection
├── IoT devices:     PostgreSQL (device type, label, active status)
├── Telemetry:       DynamoDB farm_telemetry table
│                    (moisture, humidity, temperature, pH — lookback configurable)
├── Yield history:   PostgreSQL (last 10 YieldComparison records)
├── Predictions:     PostgreSQL (current week's disease/yield predictions)
└── Sensor history:  PostgreSQL (last 10 SensorHistoryPoint records)

Autonomous IoT control

The LLM is given the trigger_iot_device tool during health analysis. It may call it up to 5 times per analysis cycle if conditions warrant action:

• Soil moisture critically low → trigger IRRIGATE command with duration
• Disease probability high → trigger CAPTURE_IMAGE to get fresh field photos
• Sensor offline → trigger ACTIVATE_SENSOR

Every tool call is logged to the IotToolCall entity with status tracking (PENDING → IN_PROGRESS → COMPLETED/FAILED). AWS IoT Jobs notifies the backend via webhook when execution completes.

Structured output

The LLM returns a structured JSON object that is validated before persisting:

{
  "overall_score": 85,
  "soil_health": 78,
  "crop_health": 90,
  "weather_stress": 20,
  "disease_risk": 15,
  "crop_field_health": [...],
  "disease_alerts": [...],
  "health_alerts": [...],
  "sensor_history": [...],
  "yield_comparisons": [...]
}

Results are saved in a cascade across 6 entity types, each with a foreign key back to the parent FarmHealth row.

---

2. Conversational AI Assistant

Architecture

The chat system is fully async — GraphQL mutations enqueue the message, and a BullMQ consumer runs the LLM call so there is no HTTP timeout risk. Tokens are delivered to the client over SSE as they are generated.

User sends message
  ↓ GraphQL mutation
ChatService saves user message → PostgreSQL
ChatProducer enqueues job → chat-queue
  ↓ BullMQ
ChatConsumer loads full message history
  ↓
ClaudeService.streamAndProcess()
  ↓ tokens + tool events
ChatPubSubService (Redis PubSub)
  ↓ SSE
Client receives token stream
  ↓ on done
ChatService saves assistant message → PostgreSQL

Tool loop

The LLM has access to 5 tools and can call them in any order, up to 5 rounds per session:

Message persistence

Every message is stored with raw_blocks — the full Anthropic ContentBlock[] JSON array. This enables:

• Exact API replay for debugging
• Full audit trail of which tools were called and what they returned
• Reconstruction of the conversation from any point

Streaming

ClaudeService uses streaming completion (stream: true). As chunks arrive, text tokens are published to the farm/chat-specific Redis channel. The SSE endpoint subscribes to that channel and forwards events to the connected client. Event types: token, tool_use, done, error.

---

3. Computer Vision Predictions

Bloomverd delegates to an external Python service that handles image analysis. This separation allows the ML team to iterate on models independently.

Request format

POST PREDICTION_BASE_URL/predict?verbose=true

{
  "crop": "Maize",
  "soil_type": "LOAM",
  "growth_stage": "vegetative",
  "subplots": [
    { "image_url": "...", "latitude": 5.6, "longitude": -0.2, "area_ha": 1.2 }
  ],
  "farm_metadata": {
    "farm_size_ha": 5,
    "latitude": 5.6,
    "longitude": -0.2,
    "planting_density": 44000,
    "elevation_m": 60,
    "days_to_maturity": 120
  }
}

Risk level derivation

Disease predictions:

• predicted_class == "healthy" → LOW
• severity >= 0.5 → HIGH
• otherwise → MODERATE

Yield predictions:

• water_stress_pct < 0.3 → LOW
• water_stress_pct 0.3–0.59 → MODERATE
• water_stress_pct >= 0.6 → HIGH

Weekly quota enforcement

Each farmer has a predictionWeeklyLimit (default 3, configurable per subscription plan). The PredictionRange entity tracks the current week's usage via a regeneration_count column. Attempts beyond the limit are rejected at the service layer before any job is enqueued.

---

4. Farm Data Dashboard Summarization

When the frontend requests a dashboard view, the system:

1. Checks Redis for a cached result (TTL: farmDataCacheTtlSeconds)
2. On cache miss, enqueues a generate-farm-data job
3. The consumer collects latest FarmHealth, IoT devices, and recent DynamoDB telemetry
4. Calls the LLM with a 1024-token output budget to produce a structured summary

The result is cached in Redis so repeated dashboard loads do not hit the LLM.

---

IoT Automation Audit Trail

Every device command issued by the AI is recorded in IotToolCall:

The requested_by field distinguishes AI-initiated actions from farmer-initiated ones, which matters for compliance and debugging.

---

Innovation Highlights

Agentic decision loop

The health pipeline does not just observe — it acts. When the LLM sees critically low soil moisture, it issues an irrigation command directly through the IoT layer. This closes the sensing-to-action loop without requiring a human to read an alert and manually respond.

LLM-agnostic architecture

The OpenAI-compatible client abstraction means the system can run on open-weight models (Llama, Qwen, Gemma) via Ollama, or on commercial APIs (OpenAI, Anthropic) by changing two environment variables. As open-weight models improve and costs drop, Bloomverd benefits automatically.

Multi-source context fusion

The health consumer fuses five distinct data sources (real-time telemetry, historical yields, current predictions, device inventory, sensor trends) into a single LLM call. The LLM performs the cross-source reasoning that would otherwise require custom rule engines for every combination of conditions.

Tool-use loops for both automated and interactive AI

Both the health pipeline and the chat assistant use multi-round tool-use loops (up to 5 rounds each). This allows the LLM to gather exactly the information it needs rather than receiving a fixed context dump.

Shared IoT control plane

The same trigger_iot_device tool definition and the same IotToolCall audit log are used by both the health pipeline and the chat assistant. Farmers can ask "did the AI irrigate my field last night?" and get a direct answer from the audit log, regardless of which AI subsystem issued the command.