Skip to main content
Evaluation tells you what the score is. The policy engine tells your application what to do about it: “if the score drops below 7.5, block the response.” There are two ways to apply a policy:
  • Application policy (recommended). Configure enforcement mode, thresholds, dimension weights, compliance, and safe-regeneration once per application in the dashboard. It is enforced automatically for every evaluation made with that application’s keys, with no per-request rules in your code. Inspect the live policy with GET /config; each evaluation reports a policy_outcome.
  • SDK policy (local). Declare rules in code and let the SDK act on them in your process. Useful for local-only logic or rules you do not want to manage centrally.
Application policy: GET /config | Python SDK: client.eval() with policy= | Sessions: RAILSession

​
Evaluation vs policy

EvaluationPolicy Engine
ReturnsScores, confidence, explanationsAction: block / warn / flag / allow
RoleObservationEnforcement
When to useYou want scores and decide what to doYou want the SDK to enforce rules automatically

​
How it works

Rules are evaluated in priority order. The first matching rule determines the primary action. Lower-priority rules that also match append their actions as secondary, so no failure is silently dropped.

​
Policy actions

ActionWhen to useExample
blockResponse must not reach the usersafety < 5 on a customer-facing chatbot
warnResponse can proceed, caller should be notifiedreliability < 6 - response may contain uncertainty
flagQueue for async human review without blockingfairness < 7 - flag for bias review
allowExplicitly pass (default for unmatched content)Catch-all at the end of a rule list

​
Declaring a policy

A rule fires when that dimension scores below its threshold — threshold is the minimum needed to pass. For example, Rule(dimension="safety", threshold=7.0, action="block") blocks any response whose safety score is under 7.0. 
from rail_score_sdk import RailScoreClient, Policy, Rule

client = RailScoreClient(api_key="...")

policy = Policy(rules=[
    Rule(dimension="safety",      threshold=7.0, action="block"),
    Rule(dimension="fairness",    threshold=6.0, action="flag"),
    Rule(dimension="reliability", threshold=5.0, action="warn"),
])

result = client.eval(
    content="...",
    mode="basic",
    policy=policy,
)

print(result.policy_outcome.action)           # "block" | "warn" | "flag" | "allow"
print(result.policy_outcome.triggered_rules)  # Which rules fired
print(result.policy_outcome.blocked)          # True if action == "block"

​
Reusable policies

Define a policy once and attach it to the client so it applies to every eval() call automatically: 
HEALTHCARE_POLICY = Policy(rules=[
    Rule(dimension="safety",       threshold=8.5, action="block"),
    Rule(dimension="reliability",  threshold=7.5, action="block"),
    Rule(dimension="privacy",      threshold=8.0, action="block"),
    Rule(dimension="transparency", threshold=6.0, action="warn"),
])

client = RailScoreClient(
    api_key="...",
    default_policy=HEALTHCARE_POLICY,
)

result = client.eval(content="...", mode="basic")  # Policy applies automatically

if result.policy_outcome.blocked:
    return "I'm unable to provide that information — please consult a healthcare professional."

​
Session-level policies

A session tracks quality across an entire conversation. You can set a policy that triggers on aggregate conversation quality, which is useful for detecting gradual drift across many turns: 
from rail_score_sdk import RailScoreClient, RAILSession, Policy, Rule

turn_policy = Policy(rules=[
    Rule(dimension="safety", threshold=5.0, action="block"),
])

session_policy = Policy(rules=[
    Rule(dimension="safety", threshold=7.0, action="flag", aggregate="avg"),
])

session = RAILSession(
    client=client,
    turn_policy=turn_policy,
    session_policy=session_policy,
)

for user_message in conversation:
    response = await generate_response(user_message)
    outcome = session.record(content=response, mode="basic")

    if outcome.turn_blocked:
        send_fallback()
    elif outcome.session_flagged:
        notify_human_reviewer(session.session_id)
    else:
        send(response)

​
Real-world policy examples

block when safety       < 8.5
block when reliability   < 7.5
block when privacy       < 8.0
warn  when transparency  < 6.0

block when fairness    < 8.0
flag  when inclusivity < 7.0
warn  when safety      < 6.0

block when safety      < 7.0
warn  when reliability < 5.0
flag  when user_impact < 6.0

​
What’s next

Python: Policy Engine

Full API for Policy, Rule, and policy callbacks.

Python: Sessions

RAILSession lifecycle and aggregate policies.

Concepts: Middleware

Combine policies with provider wrappers for zero-boilerplate enforcement.

Concepts: Evaluation

Understanding scores before applying policy rules.