Building a Harness with Jev: A New Era for AI Decision-Making

Introduction

In the evolving landscape of artificial intelligence, the integration of agents and large language models (LLMs) has posed significant challenges, particularly in software applications reliant on structured data. However, the introduction of Jev, a new model from TypeSafe AI, promises to streamline this process by offering rapid and cost-effective decision-making capabilities. This article delves into the workings of Jev, its role within the agent loop, and how it can be effectively utilised with LangChain.

What is Jev?

Jev is not a traditional LLM; rather, it is classified as a System One model. This innovative model is designed to make quick, structured decisions that can be directly employed by software applications. Unlike conventional LLMs that generate text, Jev evaluates a given state and returns typed answers along with probabilities, making it ideal for classification tasks.

Trained using reinforcement learning for calibrated decisions (RLCD), Jev significantly enhances the efficiency of the agent loop by reducing the need for full chat LLM calls for each decision. Instead, developers can send a state and specific questions to Jev, which processes them rapidly and returns structured results.

Example of Jev in Action

Consider a support ticket scenario where a user reports an urgent issue:

{ "model" : "jev-latest", "state" : "Hi, I've been trying to connect my Stripe account for 3 days and it keeps failing. I'm losing sales. Please help ASAP.", "questions" : { "is_urgent" : { "type" : "noul", "instructions" : "The message conveys urgency or time-sensitivity" } } }

The response from Jev might indicate a 99.9% probability that the message is urgent, allowing applications to prioritise the ticket effectively.

Key Features of Jev

Jev supports three types of questions:

• Choice: Select from a set of options, returning probabilities and a confidence score.
• Score: Rate an input against ordered levels (e.g., low, medium, high), providing a continuous score and confidence value.
• Noul: Answer yes-or-no questions, returning the probability that a statement is true.

A notable advantage of Jev is its ability to evaluate multiple questions about the same state in parallel, which enhances response time and reduces costs.

How to Use Jev with LangChain

LangChain's provider-agnostic framework is well-suited for integrating Jev alongside various other model providers. The integration is facilitated through the TypeSafeClassifier, which allows developers to invoke Jev and receive classification results instead of chat responses.

Installation and Example Code

To get started, install the langchain-typesafe package and set your TYPESAFE_API_KEY. Here’s a simple example:

from langchain_typesafe import Noul, TypeSafeClassifier
classifier = TypeSafeClassifier()
response = classifier.invoke({ "state" : ("The deploy failed twice and customers are seeing 500s. Can someone look now?"), "questions" : { "urgent" : Noul(instructions="Does this need attention right now?") } })
urgency = response.nouls["urgent"].noul

This code snippet demonstrates how to pass a state and questions to Jev, allowing for seamless integration into existing workflows.

Use Cases for Jev

While Jev is not a direct replacement for LLMs, it excels in handling classification tasks typically managed by LLMs, but with reduced latency and cost. This positions Jev as a valuable complement to models driving agents, enabling LLMs to focus on open-ended reasoning and generation while Jev handles fast, structured decisions.

Model Routing

Jev can also facilitate model routing, allowing for the selection of appropriate models based on task complexity. For example:

from langchain.agents import create_agent
from langchain_typesafe.experimental.middleware import (ModelChoice, ModelRouterMiddleware)
router = ModelRouterMiddleware(choices={ "fast" : ModelChoice(model="openai:luna", criteria="Direct lookups, extraction, and localized changes."), "powerful" : ModelChoice(model="openai:sol", criteria="Architecture and high-stakes decisions.") }, instructions="Choose the least costly model that can complete the task.")
agent = create_agent("openai:gpt-5.6-luna", middleware=[router])

This middleware assesses requests and selects the most suitable model, optimising performance and cost.

Auto Mode

To enhance trustworthiness in agents, Jev can be employed to classify potentially dangerous actions before execution. This is achieved through the AutoModeMiddleware, which utilises Jev to evaluate tool calls:

from langchain.agents import create_agent
from langchain_typesafe.experimental.middleware import (AutoModeMiddleware)
guardrail = AutoModeMiddleware(tools=["bash"])
agent = create_agent("openai:gpt-5.6-luna", middleware=[guardrail])

This middleware checks for risky decisions, providing an additional layer of security.

Conclusion

The introduction of Jev marks a significant advancement in AI decision-making, offering developers a powerful tool for enhancing the efficiency and cost-effectiveness of their applications. As the AI landscape continues to evolve, we are excited to see the innovative solutions that emerge from the integration of Jev with LangChain. We encourage developers to explore the potential of Jev and share their experiences within the community.