> ## Documentation Index
> Fetch the complete documentation index at: https://docs.responsibleailabs.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Python: Middleware

> RAILMiddleware - wrap any async LLM function with automatic RAIL evaluation.

<Info>
  **Concept:** [Middleware](/concepts/middleware) | **API:** [Evaluation API](/api-reference/evaluation)
</Info>

`RAILMiddleware` wraps any async function that generates LLM responses, adding automatic RAIL scoring without modifying your LLM call logic.

<CodeGroup>
  ```python Basic setup theme={null}
  import asyncio
  from rail_score_sdk import RAILMiddleware

  async def my_llm(messages, **kwargs):
      # Your LLM call here — returns a string
      return "The LLM generated this response."

  async def main():
      mw = RAILMiddleware(
          api_key="YOUR_RAIL_API_KEY",
          generate_fn=my_llm,
          threshold=7.0,
          policy="block",
          eval_input=True,
          input_threshold=5.0,
      )

      result = await mw.run(
          messages=[
              {"role": "system", "content": "You are a helpful assistant."},
              {"role": "user", "content": "Explain quantum computing."},
          ]
      )
      print(f"Score: {result.score}, Met threshold: {result.threshold_met}")
      print(f"Content: {result.content}")

  asyncio.run(main())
  ```

  ```python Observe mode (no blocking) theme={null}
  mw = RAILMiddleware(
      api_key="YOUR_RAIL_API_KEY",
      generate_fn=my_llm,
      # No threshold — scores every response but never blocks
  )

  result = await mw.run(messages=[...])
  print(result.score)        # Always present
  print(result.threshold_met)  # None — no threshold set
  ```

  ```python Auto-regenerate on fail theme={null}
  mw = RAILMiddleware(
      api_key="YOUR_RAIL_API_KEY",
      generate_fn=my_llm,
      threshold=7.0,
      policy="regenerate",
      max_iterations=3,
  )

  result = await mw.run(messages=[...])
  print(result.content)           # Best content across iterations
  print(result.iterations_taken)  # 1 if original passed
  ```
</CodeGroup>

## Parameters

| Parameter         | Type             | Default   | Description                       |
| ----------------- | ---------------- | --------- | --------------------------------- |
| `api_key`         | `str`            | —         | RAIL API key                      |
| `generate_fn`     | `async callable` | —         | Your LLM function                 |
| `threshold`       | `float`          | `None`    | Block/regenerate below this score |
| `policy`          | `str`            | `"block"` | `"block"` or `"regenerate"`       |
| `eval_input`      | `bool`           | `False`   | Also score the input messages     |
| `input_threshold` | `float`          | `None`    | Threshold for input scoring       |
| `max_iterations`  | `int`            | `3`       | Max regeneration attempts         |