跳转到主要内容
Evaluation tells you what the score is. The policy engine tells your application what to do about it. You declare rules - “if safety drops below 7, block the response” - and the SDK enforces them automatically on every evaluated response.
Python SDK: client.eval() with policy= | Sessions: RAILSession

​
Evaluation vs policy

EvaluationPolicy Engine
ReturnsScores, confidence, explanationsAction: block / warn / flag / allow
RoleObservationEnforcement
When to useYou want scores and decide what to doYou want the SDK to enforce rules automatically

​
How it works

Rules are evaluated in priority order. The first matching rule determines the primary action. Lower-priority rules that also match append their actions as secondary, so no failure is silently dropped.

​
Policy actions

ActionWhen to useExample
blockResponse must not reach the usersafety < 5 on a customer-facing chatbot
warnResponse can proceed, caller should be notifiedreliability < 6 - response may contain uncertainty
flagQueue for async human review without blockingfairness < 7 - flag for bias review
allowExplicitly pass (default for unmatched content)Catch-all at the end of a rule list

​
Declaring a policy

from rail_score_sdk import RailScoreClient, Policy, Rule

client = RailScoreClient(api_key="...")

policy = Policy(rules=[
    Rule(dimension="safety",      threshold=7.0, action="block"),
    Rule(dimension="fairness",    threshold=6.0, action="flag"),
    Rule(dimension="reliability", threshold=5.0, action="warn"),
])

result = client.eval(
    content="...",
    mode="basic",
    policy=policy,
)

print(result.policy_outcome.action)           # "block" | "warn" | "flag" | "allow"
print(result.policy_outcome.triggered_rules)  # Which rules fired
print(result.policy_outcome.blocked)          # True if action == "block"

​
Reusable policies

Define a policy once and attach it to the client so it applies to every eval() call automatically: 
HEALTHCARE_POLICY = Policy(rules=[
    Rule(dimension="safety",       threshold=8.5, action="block"),
    Rule(dimension="reliability",  threshold=7.5, action="block"),
    Rule(dimension="privacy",      threshold=8.0, action="block"),
    Rule(dimension="transparency", threshold=6.0, action="warn"),
])

client = RailScoreClient(
    api_key="...",
    default_policy=HEALTHCARE_POLICY,
)

result = client.eval(content="...", mode="basic")  # Policy applies automatically

if result.policy_outcome.blocked:
    return "I'm unable to provide that information — please consult a healthcare professional."

​
Session-level policies

A session tracks quality across an entire conversation. You can set a policy that triggers on aggregate conversation quality, which is useful for detecting gradual drift across many turns: 
from rail_score_sdk import RailScoreClient, RAILSession, Policy, Rule

turn_policy = Policy(rules=[
    Rule(dimension="safety", threshold=5.0, action="block"),
])

session_policy = Policy(rules=[
    Rule(dimension="safety", threshold=7.0, action="flag", aggregate="avg"),
])

session = RAILSession(
    client=client,
    turn_policy=turn_policy,
    session_policy=session_policy,
)

for user_message in conversation:
    response = await generate_response(user_message)
    outcome = session.record(content=response, mode="basic")

    if outcome.turn_blocked:
        send_fallback()
    elif outcome.session_flagged:
        notify_human_reviewer(session.session_id)
    else:
        send(response)

​
Real-world policy examples

safety      >= 8.5 : block
reliability >= 7.5 : block
privacy     >= 8.0 : block
transparency>= 6.0 : warn

fairness    >= 8.0 : block
inclusivity >= 7.0 : flag
safety      >= 6.0 : warn

safety      >= 7.0 : block
reliability >= 5.0 : warn
user_impact >= 6.0 : flag

​
What’s next

Python: Policy Engine

Full API for Policy, Rule, and policy callbacks.

Python: Sessions

RAILSession lifecycle and aggregate policies.

Concepts: Middleware

Combine policies with provider wrappers for zero-boilerplate enforcement.

Concepts: Evaluation

Understanding scores before applying policy rules.