> ## Documentation Index
> Fetch the complete documentation index at: https://ai-kb.automationanywhere.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Encrypted File Ingestion via an External Decryption Service

> Configure AAEKB to hand encrypted uploads to an external decryption service you host, then ingest the decrypted content.

**Automation Anywhere Enterprise Knowledge Base (AAEKB / EKB)** — Administrator and Integration Guide

## What this feature does

AAEKB can hand an uploaded file to an **external decryption service that you host and control**, receive the decrypted bytes back, and then run its normal Knowledge Base ingestion pipeline (text extraction, OCR, chunking, embedding, indexing) on the decrypted content.

This exists for organisations that apply enterprise DRM or envelope encryption to documents at rest and cannot ship decryption keys to a third-party application. The keys, the decryption logic, and the plaintext boundary all stay inside your network. AAEKB only knows how to call an HTTP endpoint and read a byte stream back.

The integration is deliberately **generic and configuration-driven**. There is no vendor-specific DRM SDK inside AAEKB. Any HTTP service that accepts a file and returns a file — synchronously or via a job/poll pattern — can be plugged in, provided it fits the contract in [The contract your decryption service must implement](#the-contract-your-decryption-service-must-implement).

Key characteristics:

* **Opt-in per upload.** Files are only sent to the decryption service when the user explicitly ticks a toggle. Normal uploads are completely unaffected.
* **Single global configuration.** One decryption service is configured per AAEKB deployment, by a Super Admin.
* **In-memory only.** The encrypted bytes, the request to your service, and the decrypted response are all handled in memory. AAEKB does not write the ciphertext to a temporary file on disk before calling your service.
* **No key material in AAEKB.** AAEKB stores an endpoint URL and, optionally, an API credential for calling your service. It never stores or handles document encryption keys.

## How it works end to end

```text theme={null}
 User (browser)                AAEKB backend                 Your decryption service
      |                              |                                  |
      |  1. Upload file +            |                                  |
      |     "Requires Decryption"    |                                  |
      |----------------------------->|                                  |
      |                              |                                  |
      |            2. File security validation                          |
      |               (runs on the ENCRYPTED bytes)                     |
      |                              |                                  |
      |                              |  3. POST encrypted file          |
      |                              |--------------------------------->|
      |                              |                                  |
      |                              |  4a. sync: 200 + decrypted bytes |
      |                              |<---------------------------------|
      |                              |                                  |
      |                              |  4b. async: 202 + { job_id }     |
      |                              |<---------------------------------|
      |                              |  5b. GET poll/{job_id}  (repeat) |
      |                              |--------------------------------->|
      |                              |  6b. 200 + decrypted bytes       |
      |                              |<---------------------------------|
      |                              |                                  |
      |            7. MIME type re-detected on decrypted bytes          |
      |            8. Decrypted file stored in KB raw storage           |
      |            9. Normal ingestion: extract, chunk, embed, index    |
      |                              |                                  |
      |  10. Resource appears in KB  |                                  |
      |<-----------------------------|                                  |
```

Step by step:

1. **Upload.** The user selects files in the Knowledge Base upload modal and enables the **Requires Decryption** toggle. The frontend sends `needs_decryption=true` as an extra form field alongside the file.
2. **File security validation.** AAEKB's standard upload security middleware runs *before* decryption, against the **encrypted** bytes and the original filename. This is the single most important compatibility constraint — see [Compatibility](#compatibility-what-will-and-will-not-work).
3. **Submit to your service.** AAEKB reads the file into memory and sends it to your configured decrypt endpoint.
4. **Receive the plaintext.** Either immediately (`sync` mode) or after polling a result endpoint (`async` mode).
5. **Re-detect the file type.** AAEKB re-derives the MIME type from the decrypted bytes: first from the filename extension, then, if that yields nothing, by sniffing the first 2 KB of the decrypted content.
6. **Store and ingest.** The **decrypted** file is written to the Knowledge Base raw file storage and then processed by the ordinary ingestion pipeline. Decryption metadata is merged into the document's metadata record.

Two upload paths exist and both support the feature identically:

| Path                                       | Endpoint                              | Where decryption runs              |
| ------------------------------------------ | ------------------------------------- | ---------------------------------- |
| Synchronous upload (legacy / Quick Upload) | `POST /v3/project/knowledge/add/file` | Inside the web request             |
| Background upload (default)                | `POST /v5/project/knowledge/add/file` | Inside a background worker process |

The practical implication: **both your web/API containers and your background worker containers must be able to reach the decryption service.**

## Configuring the service (Super Admin)

1. Sign in as a Super Admin.
2. Open **Super Admin → Platform & integrations → KB Decryption Service**.
3. Fill in the configuration described in [Full configuration reference](#full-configuration-reference).
4. Toggle **Enabled** on.
5. Click **Save Changes**.

Notes on the admin screen:

* The **Async Polling Configuration** card is only shown when **Response Mode** is set to *Asynchronous*.
* The auth credential field is masked when the configuration is read back. Only the first four characters are shown; the rest is replaced with bullet characters. Re-saving without touching the field leaves the stored credential unchanged. To rotate the credential, clear the field and type the new value in full.
* Saving is atomic across the whole form — there is no per-field save.
* Setting **Enabled** to off immediately hides the end-user toggle and causes any upload that still claims `needs_decryption=true` to be rejected with a clear error.

The same configuration is available over the REST API for automated provisioning:

| Method | Path                            | Access                 | Purpose                                              |
| ------ | ------------------------------- | ---------------------- | ---------------------------------------------------- |
| `GET`  | `/admin/kb-decryption-service`  | Super Admin            | Read configuration (credential masked)               |
| `PUT`  | `/admin/kb-decryption-service`  | Super Admin            | Partial update; only the fields you send are changed |
| `GET`  | `/kb-decryption-service/status` | Any authenticated user | Returns `{"available": true}`                        |

Example provisioning call:

```bash theme={null}
curl -X PUT "https://ekb.example.com/admin/kb-decryption-service" \
  -H "Authorization: Bearer <admin-token>" \
  -H "Content-Type: application/json" \
  -d '{
        "enabled": true,
        "display_name": "Corporate DRM Gateway",
        "description": "Intranet decryption service for classified documents",
        "decrypt_endpoint": "https://drm.corp.internal/api/v1/async/decrypt/stream",
        "decrypt_http_method": "POST",
        "decrypt_content_type": "multipart/form-data",
        "decrypt_file_field_name": "file",
        "decrypt_extra_headers": {"X-Tenant": "aa-ekb"},
        "decrypt_extra_body_fields": {"purpose": "ingestion"},
        "response_mode": "async",
        "poll_endpoint_template": "https://drm.corp.internal/api/v1/async/download/stream/{job_id}",
        "poll_interval_seconds": 5,
        "poll_max_attempts": 120,
        "poll_job_id_json_path": "job_id",
        "poll_status_json_path": "status",
        "poll_completed_status": "COMPLETED",
        "poll_failed_statuses": "FAILED,ERROR",
        "auth_type": "api_key_header",
        "auth_token": "s3cr3t-api-key",
        "auth_header_name": "X-API-Key"
      }'
```

## Full configuration reference

### General

| Field          | Type    | Default                      | Meaning                                                                                     |
| -------------- | ------- | ---------------------------- | ------------------------------------------------------------------------------------------- |
| `enabled`      | boolean | `false`                      | Master switch. When off, the end-user toggle is hidden and decryption requests are refused. |
| `display_name` | string  | `Default Decryption Service` | Friendly name. Recorded in each decrypted document's metadata.                              |
| `description`  | string  | *(empty)*                    | Free-text note for administrators. Not used at runtime.                                     |

### Decryption request

| Field                       | Type                                                | Default               | Meaning                                                                                                |
| --------------------------- | --------------------------------------------------- | --------------------- | ------------------------------------------------------------------------------------------------------ |
| `decrypt_endpoint`          | URL                                                 | *(empty)*             | Absolute URL AAEKB posts the encrypted file to. Must be reachable from both API and worker containers. |
| `decrypt_http_method`       | `POST` \| `PUT`                                     | `POST`                | HTTP verb used for the submit request.                                                                 |
| `decrypt_content_type`      | `multipart/form-data` \| `application/octet-stream` | `multipart/form-data` | How the file is transmitted. See the note below.                                                       |
| `decrypt_file_field_name`   | string                                              | `file`                | Multipart form field name carrying the file. Only used in multipart mode.                              |
| `decrypt_extra_headers`     | JSON object                                         | `{}`                  | Extra headers merged into **both** the submit request and every poll request.                          |
| `decrypt_extra_body_fields` | JSON object                                         | `{}`                  | Extra form fields sent alongside the file. **Only used in multipart mode.**                            |
| `response_mode`             | `sync` \| `async`                                   | `async`               | Whether the decrypted bytes come back on the submit call or via polling.                               |

<Warning>
  **`application/octet-stream` mode.** When content type is `application/octet-stream`, AAEKB sends the raw file bytes as the entire request body. In this mode the **filename is not transmitted** and `decrypt_extra_body_fields` is **ignored**, because there is no multipart envelope to carry them. If your service needs the filename or additional context, either use `multipart/form-data` or pass the context through `decrypt_extra_headers`.
</Warning>

### Async polling (only used when `response_mode = async`)

| Field                    | Type            | Default        | Meaning                                                                                                                   |
| ------------------------ | --------------- | -------------- | ------------------------------------------------------------------------------------------------------------------------- |
| `poll_endpoint_template` | URL template    | *(empty)*      | Result URL. Must contain the literal placeholder `{job_id}`, which is substituted with the extracted job identifier.      |
| `poll_interval_seconds`  | integer         | `5`            | Seconds to wait between polls. AAEKB **sleeps before the first poll**, so the minimum decryption latency is one interval. |
| `poll_max_attempts`      | integer         | `120`          | Maximum number of polls. Total budget = `poll_interval_seconds × poll_max_attempts` (default 600 s / 10 minutes).         |
| `poll_job_id_json_path`  | dot path        | `job_id`       | Where to find the job identifier in the submit response, e.g. `data.jobId`.                                               |
| `poll_status_json_path`  | dot path        | `status`       | Where to find the job status in a JSON poll response, e.g. `result.state`.                                                |
| `poll_completed_status`  | string          | `COMPLETED`    | Status value meaning "done". Compared case-insensitively.                                                                 |
| `poll_failed_statuses`   | comma-separated | `FAILED,ERROR` | Status values that abort the job immediately. Compared case-insensitively.                                                |

Dot paths are simple nested-dictionary lookups. `data.job.id` resolves `response["data"]["job"]["id"]`. **Array indexing is not supported** — a path segment cannot be a list index.

### Authentication

| Field              | Type                                                       | Default         | Meaning                                                                       |
| ------------------ | ---------------------------------------------------------- | --------------- | ----------------------------------------------------------------------------- |
| `auth_type`        | `none` \| `bearer` \| `api_key_header` \| `custom_headers` | `none`          | Authentication scheme. See [Authentication options](#authentication-options). |
| `auth_token`       | secret string                                              | *(empty)*       | The bearer token or API key value. Masked on read.                            |
| `auth_header_name` | string                                                     | `Authorization` | Header name used when `auth_type = api_key_header`.                           |

## The contract your decryption service must implement

This is the specification to hand to whoever builds or operates the decryption service.

### Non-negotiable requirements

1. **Speak HTTP(S).** One endpoint that accepts a file; in async mode, a second endpoint that returns the result for a job.
2. **Return the decrypted file as raw bytes**, not as base64, not wrapped in JSON, not as a redirect to a download URL. AAEKB streams the response body straight into the ingestion pipeline.
3. **Preserve the file's original format.** If the ciphertext decrypts to a PDF, return a well-formed PDF. AAEKB does no format conversion.
4. **Return a non-empty body on success.** An empty response is treated as a failure.
5. **Be reachable from AAEKB's API and worker containers**, with a TLS certificate those containers trust.
6. **Be idempotent enough to survive a repeated GET** on the poll endpoint. See [The double-GET behaviour](#the-double-get-behaviour-async-mode).

### Submit endpoint

**Multipart mode (recommended).** AAEKB sends:

* Method: as configured (`POST` or `PUT`)
* `Content-Type`: `multipart/form-data` with a generated boundary
* One file part, named by `decrypt_file_field_name`, with `filename` set to the original upload filename
* Zero or more additional form fields from `decrypt_extra_body_fields`
* Auth and extra headers as configured

**Octet-stream mode.** AAEKB sends the raw bytes as the request body with `Content-Type: application/octet-stream`. No filename, no extra body fields.

**Expected responses:**

| Mode    | Acceptable status        | Body                                                                                                          |
| ------- | ------------------------ | ------------------------------------------------------------------------------------------------------------- |
| `sync`  | `200` only               | Decrypted file bytes                                                                                          |
| `async` | `202` (typical) or `200` | JSON containing a job identifier, **or** — if `200` with a binary content type — the decrypted bytes directly |

In `async` mode AAEKB inspects the submit response before deciding whether to poll:

* `200` **and** a binary-looking `Content-Type` → the bytes are used directly, no polling. (This lets a service short-circuit for small files.)
* Anything else → the body is parsed as JSON and the job identifier is extracted using `poll_job_id_json_path`.

A "binary-looking" content type is one that contains `octet-stream`, or that starts with `application/` and does not contain `json`. So `application/octet-stream` and `application/pdf` are treated as binary; `application/json` is not. `text/plain` **is not treated as binary** — see [Content-Type guidance](#content-type-guidance).

### Poll endpoint (async mode)

AAEKB issues a `GET` to `poll_endpoint_template` with `{job_id}` substituted, carrying the same auth and extra headers as the submit request. It repeats this every `poll_interval_seconds`, up to `poll_max_attempts` times.

Your service should respond with one of:

| Situation     | Status   | Body                                                                | AAEKB behaviour                                                                                 |
| ------------- | -------- | ------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------- |
| Still working | `202`    | Anything (ignored)                                                  | Waits and polls again                                                                           |
| Still working | `200`    | JSON whose status field is neither the completed nor a failed value | Waits and polls again                                                                           |
| Done          | `200`    | Raw decrypted bytes with a binary `Content-Type`                    | **Preferred.** Bytes are used; polling stops                                                    |
| Done          | `200`    | JSON whose status field equals `poll_completed_status`              | Issues one more `GET` to the same URL expecting binary; if that is not binary, the upload fails |
| Failed        | `200`    | JSON whose status field is in `poll_failed_statuses`                | Aborts immediately with an error naming the status                                              |
| Error         | `>= 400` | Anything                                                            | **Aborts immediately.** Does not retry                                                          |

<Warning>
  Do not return `404` for a job that is merely not ready yet. Any status code of 400 or above ends the job with a hard failure. Use `202` for "not ready".
</Warning>

### The double-GET behaviour (async mode)

If your poll endpoint answers `200` with a JSON body saying the job is complete, AAEKB assumes the file must be fetched separately and immediately issues a **second** `GET` **to the very same URL**, this time expecting a binary stream. If that second call also returns JSON, the upload fails with *"Decryption job completed but no file stream was returned."*

There are two ways to satisfy this:

* **Preferred:** have the poll endpoint return the raw bytes directly with `Content-Type: application/octet-stream` as soon as the job is done. AAEKB detects this on the first poll and never issues the second request. This is the simplest and fastest design.
* **Acceptable:** have the endpoint return a JSON status while pending/complete, and switch to returning the byte stream on a subsequent call. The job must remain retrievable for at least one extra GET after it first reports completion.

What is **not** supported is a design where the completed JSON contains a *different* download URL. AAEKB will not follow it.

### Content-Type guidance

| Response                  | Recommended `Content-Type` |
| ------------------------- | -------------------------- |
| Decrypted file bytes      | `application/octet-stream` |
| Job accepted / job status | `application/json`         |

Returning the decrypted payload as `text/plain` or `text/csv` in **async** mode will cause AAEKB to try to parse it as a job-status JSON document and fail. In **sync** mode the content type of the response is not inspected at all — any `200` is taken as the decrypted bytes — but `application/octet-stream` is still recommended for consistency.

### What AAEKB never does

* It does not send the decryption key, user credentials, or the end user's identity to your service. If your service needs to authorise per-document or per-user, that context must come from `decrypt_extra_headers` / `decrypt_extra_body_fields`, which are static and deployment-wide.
* It does not retry a failed submit or a failed poll. One failure ends the upload.
* It does not follow redirects to alternative download hosts as part of the completion handshake.
* It does not chunk or range-request large files. The whole file goes in one request and comes back in one response.

## Worked examples: synchronous mode

Configuration:

```json theme={null}
{
  "enabled": true,
  "decrypt_endpoint": "https://drm.corp.internal/api/v1/decrypt",
  "decrypt_http_method": "POST",
  "decrypt_content_type": "multipart/form-data",
  "decrypt_file_field_name": "file",
  "decrypt_extra_body_fields": { "purpose": "ingestion" },
  "decrypt_extra_headers": { "X-Tenant": "aa-ekb" },
  "response_mode": "sync",
  "auth_type": "bearer",
  "auth_token": "eyJhbGciOi..."
}
```

Request AAEKB sends:

```http theme={null}
POST /api/v1/decrypt HTTP/1.1
Host: drm.corp.internal
Authorization: Bearer eyJhbGciOi...
X-Tenant: aa-ekb
Content-Type: multipart/form-data; boundary=----AAEKBBoundary7MA4YWxk
Content-Length: 2481037

------AAEKBBoundary7MA4YWxk
Content-Disposition: form-data; name="purpose"

ingestion
------AAEKBBoundary7MA4YWxk
Content-Disposition: form-data; name="file"; filename="Q3-Financials.pdf"
Content-Type: application/octet-stream

<...encrypted bytes...>
------AAEKBBoundary7MA4YWxk--
```

Successful response your service must return:

```http theme={null}
HTTP/1.1 200 OK
Content-Type: application/octet-stream
Content-Length: 2394112
Content-Disposition: attachment; filename="Q3-Financials.pdf"

%PDF-1.7
<...decrypted bytes...>
```

`Content-Disposition` is optional and ignored; AAEKB keeps the original upload filename.

Failure response — any non-`200` ends the upload. Include a short, human-readable body; AAEKB surfaces the first 500 characters of it to the end user.

```http theme={null}
HTTP/1.1 403 Forbidden
Content-Type: application/json

{"error": "Document policy forbids export for this identity"}
```

The user sees: `Decryption service returned HTTP 403: {"error": "Document policy forbids export for this identity"}`

## Worked examples: asynchronous (polling) mode

Configuration:

```json theme={null}
{
  "enabled": true,
  "decrypt_endpoint": "https://drm.corp.internal/api/v1/async/decrypt/stream",
  "decrypt_content_type": "multipart/form-data",
  "decrypt_file_field_name": "file",
  "response_mode": "async",
  "poll_endpoint_template": "https://drm.corp.internal/api/v1/async/download/stream/{job_id}",
  "poll_interval_seconds": 5,
  "poll_max_attempts": 120,
  "poll_job_id_json_path": "job_id",
  "poll_status_json_path": "status",
  "poll_completed_status": "COMPLETED",
  "poll_failed_statuses": "FAILED,ERROR",
  "auth_type": "api_key_header",
  "auth_token": "s3cr3t-api-key",
  "auth_header_name": "X-API-Key"
}
```

**Step 1 — Submit**

```http theme={null}
POST /api/v1/async/decrypt/stream HTTP/1.1
Host: drm.corp.internal
X-API-Key: s3cr3t-api-key
Content-Type: multipart/form-data; boundary=----AAEKBBoundary7MA4YWxk

------AAEKBBoundary7MA4YWxk
Content-Disposition: form-data; name="file"; filename="Q3-Financials.pdf"
Content-Type: application/octet-stream

<...encrypted bytes...>
------AAEKBBoundary7MA4YWxk--
```

Response:

```http theme={null}
HTTP/1.1 202 Accepted
Content-Type: application/json

{
  "job_id": "8f14e45f-ceea-467a-9f7a-2a1d4b0c9e33",
  "status": "PENDING"
}
```

With a nested identifier you would instead set `poll_job_id_json_path` to `data.job.id` and return:

```json theme={null}
{ "data": { "job": { "id": "8f14e45f-ceea-467a-9f7a-2a1d4b0c9e33" } } }
```

**Step 2 — Poll while pending**

AAEKB waits 5 seconds, then:

```http theme={null}
GET /api/v1/async/download/stream/8f14e45f-ceea-467a-9f7a-2a1d4b0c9e33 HTTP/1.1
Host: drm.corp.internal
X-API-Key: s3cr3t-api-key
```

Either of these keeps AAEKB waiting:

```http theme={null}
HTTP/1.1 202 Accepted
Content-Type: application/json

{"status": "PROCESSING", "message": "File is still being processed."}
```

```http theme={null}
HTTP/1.1 200 OK
Content-Type: application/json

{"status": "PROCESSING", "progress": 42}
```

**Step 3 — Poll when complete (recommended pattern)**

```http theme={null}
HTTP/1.1 200 OK
Content-Type: application/octet-stream
Content-Length: 2394112
Content-Disposition: attachment; filename="Q3-Financials.pdf"

%PDF-1.7
<...decrypted bytes...>
```

AAEKB stops polling and proceeds to ingestion.

**Step 3 (alternative) — Complete via JSON, then stream**

```http theme={null}
HTTP/1.1 200 OK
Content-Type: application/json

{"status": "COMPLETED"}
```

AAEKB immediately re-issues the same `GET`, and your service must then answer with the binary stream shown above. If it answers with JSON again, the upload fails.

**Step 3 (failure)**

```http theme={null}
HTTP/1.1 200 OK
Content-Type: application/json

{"status": "FAILED", "reason": "Key not available for principal"}
```

The user sees: `External decryption job failed with status: FAILED`

Note that only the *status value* is surfaced, not your `reason` field. If you want detail in the user-facing message, encode it into the status string or return a `4xx`/`5xx` with a descriptive body instead.

**Reference mock service**

A minimal echo-back mock, useful for validating the plumbing before the real service is ready:

```python theme={null}
# Minimal AAEKB-compatible async decryption mock (FastAPI).
# Returns the uploaded bytes unchanged.
import io, uuid
from fastapi import FastAPI, UploadFile, File
from fastapi.responses import JSONResponse, StreamingResponse

app = FastAPI()
JOBS: dict[str, tuple[bytes, str]] = {}


@app.post("/api/v1/async/decrypt/stream")
async def submit(file: UploadFile = File(...)):
    job_id = str(uuid.uuid4())
    JOBS[job_id] = (await file.read(), file.filename or "file")
    return JSONResponse(status_code=202, content={"job_id": job_id, "status": "PENDING"})


@app.get("/api/v1/async/download/stream/{job_id}")
async def download(job_id: str):
    entry = JOBS.pop(job_id, None)
    if entry is None:
        # Not ready (or unknown). Must NOT be a 4xx — that would abort the job.
        return JSONResponse(status_code=202, content={"status": "PROCESSING"})
    data, filename = entry
    return StreamingResponse(
        io.BytesIO(data),
        media_type="application/octet-stream",
        headers={
            "Content-Disposition": f'attachment; filename="{filename}"',
            "Content-Length": str(len(data)),
        },
    )
```

## Authentication options

| `auth_type`      | What AAEKB sends                     | Notes                                                                                                                             |
| ---------------- | ------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------- |
| `none`           | Only `decrypt_extra_headers`         | Use with network-level controls (mTLS terminated at a proxy, IP allowlist, service mesh).                                         |
| `bearer`         | `Authorization: Bearer <auth_token>` | Plus `decrypt_extra_headers`.                                                                                                     |
| `api_key_header` | `<auth_header_name>: <auth_token>`   | Plus `decrypt_extra_headers`. Typical: `X-API-Key`.                                                                               |
| `custom_headers` | Only `decrypt_extra_headers`         | Put the complete header set, including any credential, in the extra headers JSON. Use for signature schemes or multi-header auth. |

In every mode, `decrypt_extra_headers` is merged in, and the resulting header set is applied to **both** the submit request and every poll request.

**Not supported natively:** OAuth 2.0 client-credentials flows with token refresh, HMAC request signing, and mutual TLS with a client certificate presented by AAEKB. All three can be handled by placing a small authenticating reverse proxy or sidecar between AAEKB and the decryption service, and pointing `decrypt_endpoint` at the proxy. Static tokens configured here do not auto-refresh; a rotating credential must be re-saved in the admin screen or pushed via the `PUT` API.

## End-user experience

**Visibility.** The Knowledge Base upload modals call `GET /kb-decryption-service/status` when they open. The **Requires Decryption** toggle is rendered **only** when that returns `{"available": true}`, which requires the configuration row to exist and `enabled` to be `true`. Users of a deployment without a configured service never see it. The status result is cached briefly in the browser (about 30 seconds), so enabling or disabling the feature may take up to that long to appear or disappear in an already-open session.

**Where it appears.** Both KB upload entry points expose the toggle:

* The **Add Resources** modal (drag-and-drop / file picker within a Knowledge Base)
* The **Selected Files** modal (used when files are dropped onto the KB view)

**Scope.** The toggle is **per upload batch, not per file**. If it is on, *every* file in that batch — including every file inside a selected folder — is sent to the decryption service. Mixed batches of encrypted and unencrypted files are not supported; upload them separately.

**Default state.** Off. It resets to off each time the modal is closed and reopened.

**Failure behaviour.** If decryption fails, that file's upload fails and the error is shown in the upload progress list / toast. Other files in the batch are unaffected. Nothing partial is indexed.

**Progress.** The browser's progress bar tracks the upload of bytes to AAEKB only. Time spent in your decryption service happens server-side afterwards and appears as processing time, not upload progress. With the default polling settings, expect at least 5 seconds of additional latency per file even for an instant decryption.

## Compatibility: what will and will not work

<Warning>
  Read this section carefully. It is the source of nearly all integration surprises.
</Warning>

### The critical constraint: validation happens before decryption

AAEKB runs its standard upload security validation against the **encrypted bytes and the original filename**, *before* the file is sent to your decryption service. There is no bypass for `needs_decryption`. This means an encrypted file must still *look like* a legitimate, allowed file type at the moment of upload.

The validation performs, in order:

1. **Filename checks** — no path traversal, no null bytes, no Windows-illegal characters, max 255 characters, no suspicious double extensions.
2. **Extension allowlist** — the file **must** have an extension, and it must be one of the supported types.
3. **Size limit** — 4096 MB (4 GB) ceiling.
4. **Content sniffing (libmagic)** — the first 2 KB are inspected and compared against the claimed extension. Executable signatures (PE / ELF / Mach-O) are rejected outright.
5. **Type-specific content checks** — e.g. images are opened and verified, text files must decode cleanly as text.

### Supported file extensions

Only these extensions are accepted (script types only if your deployment explicitly enables script uploads):

* **Documents:** `pdf`, `doc`, `docx`, `xls`, `xlsx`, `pptx`
* **Text and data:** `txt`, `md`, `csv`, `tsv`, `json`, `xml`, `dita`, `yaml`, `yml`, `html`, `htm`
* **Mail:** `eml`, `msg`
* **Archives:** `zip`
* **Images:** `png`, `jpg`, `jpeg`, `gif`, `webp`
* **Audio:** `mp3`, `wav`, `m4a`, `ogg`
* **Video:** `mp4`, `mov`, `avi`, `wmv`, `webm`
* **Scripts (opt-in only):** `py`, `sh`, `bash`, `js`, `php`, `rb`, `pl`, `bat`, `cmd`, `ps1`

The live list for your deployment is available at `GET /upload-capabilities`.

### Filename and wrapper compatibility

| Uploaded filename                        | Result                                           |
| ---------------------------------------- | ------------------------------------------------ |
| `Report.pdf` (ciphertext inside)         | Depends on the ciphertext envelope — see below   |
| `Report.pdf.enc`                         | **Rejected.** `.enc` is not an allowed extension |
| `Report.enc`, `Report.drm`, `Report.aes` | **Rejected.** Not allowed extensions             |
| `Report` (no extension)                  | **Rejected.** "File must have an extension"      |
| `Report.pdf.gpg`                         | **Rejected.** `.gpg` is not allowed              |

**Your encryption scheme must preserve the original file extension.** A wrapper format that appends `.enc`, `.drm`, `.p7m` or similar will be rejected at the door. If your DRM system produces such filenames, rename to the original extension before upload, or configure the client-side export to retain it.

### Ciphertext envelope compatibility

This is the second gate. The encrypted bytes must survive content sniffing while still claiming the original extension.

| Encrypted format                                                                                                         | Compatible? | Why                                                                                                                                                                                                                            |
| ------------------------------------------------------------------------------------------------------------------------ | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Office documents (`.docx`, `.xlsx`, `.pptx`) encrypted with Microsoft Information Protection / OOXML password encryption | **Yes**     | These are wrapped in an OLE2 / CDF compound container. AAEKB explicitly accepts OLE2, CDFV2, `application/vnd.ms-office` and `application/octet-stream` when the extension is an Office type. This is the best-supported case. |
| Legacy Office (`.doc`, `.xls`) encrypted in place                                                                        | **Yes**     | Same OLE2 allowance.                                                                                                                                                                                                           |
| PDF with standard PDF encryption (`/Encrypt` dictionary, password-protected)                                             | **Yes**     | The file remains a structurally valid PDF; `%PDF-` is still the leading signature.                                                                                                                                             |
| PDF wrapped in an opaque envelope, but with `%PDF-` present within the first 1024 bytes                                  | **Yes**     | AAEKB searches the first 1 KB for the `%PDF-` marker and accepts on that basis. A DRM envelope that keeps a PDF preamble/header will pass.                                                                                     |
| PDF fully encrypted into opaque ciphertext with no `%PDF-` in the first 1 KB                                             | **No**      | Sniffing returns `application/octet-stream` (or something unrelated) for a `.pdf` extension → rejected with *"File content does not match extension"* or *"File detected as generic binary type."*                             |
| Plain text formats (`.txt`, `.csv`, `.json`, `.md`, `.xml`, `.html`) encrypted into binary ciphertext                    | **No**      | Sniffing does not return a `text/*` type, so the extension/content check fails.                                                                                                                                                |
| ZIP archives encrypted as a whole                                                                                        | **No**      | The outer archive must be a readable ZIP; AAEKB inspects its entries. Additionally, **ZIP files containing encrypted members are rejected** by the archive validator.                                                          |
| Images, audio, video encrypted into opaque ciphertext                                                                    | **No**      | Same sniffing mismatch.                                                                                                                                                                                                        |

**Rule of thumb:** the integration is designed for, and reliably works with, **encrypted Office documents and encrypted/DRM-protected PDFs that retain a recognisable header**. Formats where encryption destroys all structural markers will be blocked before your service is ever called.

If your DRM system produces fully opaque ciphertext, the supported workaround is to have the exporting system wrap the payload in an OOXML or PDF container that retains the appropriate header bytes, so the file remains recognisable in transit. Alternatively, contact your AAEKB representative — the content-validation layer is configurable at the deployment level and can be adjusted for on-premise installations.

### After decryption

Once the decrypted bytes come back, AAEKB determines the type as follows:

1. `mimetypes.guess_type()` on the **original filename** — so the extension you uploaded with determines the ingestion path in almost all cases.
2. If that yields nothing, libmagic sniffs the first 2 KB of the **decrypted** content.

The decrypted content is then routed to the appropriate handler: PDF, Word, Excel, PowerPoint, HTML, XML/DITA, plain text, JSON, CSV/TSV, EML, MSG, audio (transcription), video (transcription), or ZIP (extract and ingest members).

If the type cannot be determined, the file falls through to a generic handler that attempts best-effort text extraction. The upload will still succeed and the file will be stored and downloadable, but it may be indexed with empty text content — meaning it will not be retrievable by semantic search. This is the usual symptom of a decryption service returning something other than the expected format (for example, base64 text instead of raw bytes).

### Feature interactions

| Feature                                                                               | Interaction                                                                                                                                           |
| ------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Quick Upload** toggle                                                               | Compatible. Decryption runs inline in the request; the user waits for the whole decryption round-trip before the modal completes.                     |
| **Folder upload**                                                                     | Compatible; the toggle applies to every file in the folder.                                                                                           |
| **Encrypted ZIP as a container of encrypted files**                                   | Not supported. Decryption is applied to the uploaded file as a whole, before archive extraction. Members inside a ZIP are not individually decrypted. |
| **URL / connector ingestion** (SharePoint, Confluence, web crawl, Google Drive, etc.) | **Not supported.** Decryption applies only to direct file uploads through the Knowledge Base upload modals and the `add/file` API endpoints.          |
| **Chat-attachment uploads**                                                           | Not exposed in the UI.                                                                                                                                |
| **Re-sync / re-processing of an existing document**                                   | The stored file is already decrypted, so re-processing works normally and does not call your service again.                                           |

## Timeouts, limits and sizing

| Operation                                           | Limit                                                                       | Configurable     |
| --------------------------------------------------- | --------------------------------------------------------------------------- | ---------------- |
| Submit request to the decryption endpoint           | **300 seconds**                                                             | No               |
| Each individual poll request                        | **60 seconds**                                                              | No               |
| The follow-up binary fetch after a JSON "completed" | **120 seconds**                                                             | No               |
| Total polling window                                | `poll_interval_seconds × poll_max_attempts` — **600 s (10 min)** by default | Yes              |
| Maximum upload size                                 | 4096 MB (4 GB)                                                              | Deployment-level |
| Retries on failure                                  | **None**                                                                    | No               |

Sizing guidance:

* Decryption is entirely in-memory. Peak memory per concurrent decryption is roughly **2× the file size** (ciphertext plus plaintext), on top of normal request overhead. Size your API and worker containers accordingly, or cap practical upload sizes well below the 4 GB ceiling for encrypted uploads.
* If your service can take longer than 5 minutes to *accept* a submission, it must use async mode — the 300-second submit timeout is not adjustable.
* If decryption itself can take longer than 10 minutes, raise `poll_max_attempts` (or `poll_interval_seconds`) accordingly.
* Each in-flight decryption occupies a worker thread for its whole duration, including polling sleep time. High-volume bulk ingestion of encrypted files with long decryption times will consume worker capacity; plan concurrency with that in mind.

## Storage, security and audit behaviour

### What gets stored

**AAEKB stores the decrypted file** in the Knowledge Base raw file storage, not the ciphertext.

This is because the raw stored file is read back by several parts of the product after ingestion: in-app document preview, download, citation rendering, table and spreadsheet extraction, and any subsequent re-processing or re-indexing of the document. Storing the plaintext keeps all of those working directly, and means the decryption service is called exactly once per document — at upload — rather than on every read. It also removes your decryption service from the critical path for everyday user activity, so an outage there never affects access to documents that are already in the Knowledge Base.

Once ingested, the document is protected by AAEKB's own controls: encryption at rest provided by the underlying object storage, plus AAEKB's project-, team- and role-level access controls governing who can view, search or download it.

### Metadata written to each decrypted document

```json theme={null}
{
  "originally_encrypted": true,
  "decryption_service": "Corporate DRM Gateway",
  "decrypted_at": 1774000000.123
}
```

* `decryption_service` is the configured **Display Name**.
* `decrypted_at` is a Unix epoch timestamp (seconds, float).
* `originally_encrypted` is an **informational marker**, letting administrators and end users identify which documents arrived encrypted. It does not affect processing: because the stored file is already decrypted, re-processing a document never involves the decryption service again.

### Credential handling

* `auth_token` is stored in the AAEKB application database.
* It is **masked** whenever the configuration is read back through the admin API or UI: only the first four characters are returned, the remainder replaced with bullet characters. Saving a masked value is detected and ignored, so re-saving the form never corrupts the stored credential.
* Every change to the configuration is written to the **Super Admin audit log** as event `kb_decryption_service_updated`, recording before/after values for all fields **except** the credential, which is recorded only as `auth_token_rotated: true`. The token value never appears in audit records or logs.

### Access control

* Reading and writing the configuration requires **Super Admin** privileges.
* The availability status endpoint requires an authenticated session but no special role, and returns only a boolean.
* Non-admin users can never see the endpoint URL, headers, or credential.

### Logging

AAEKB logs the filename, byte counts, and job identifiers at INFO level during decryption. It does **not** log file content or credentials. Error messages returned to users include up to the first 500 characters of your service's error response body — avoid returning sensitive material in error bodies.

## Error reference and troubleshooting

| Message the user sees                                                                               | HTTP | Cause                                                                       | Fix                                                                                                         |
| --------------------------------------------------------------------------------------------------- | ---- | --------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------- |
| `File type '.enc' not allowed. Allowed types include: ...`                                          | 400  | Non-allowlisted extension                                                   | Preserve the original extension on export                                                                   |
| `File must have an extension`                                                                       | 400  | Filename has no extension                                                   | Add the correct extension                                                                                   |
| `File content does not match extension`                                                             | 400  | Ciphertext doesn't sniff as the claimed type                                | See [Ciphertext envelope compatibility](#ciphertext-envelope-compatibility)                                 |
| `File detected as generic binary type. Cannot verify file safety.`                                  | 400  | Opaque ciphertext with a non-Office extension                               | See [Ciphertext envelope compatibility](#ciphertext-envelope-compatibility)                                 |
| `Encrypted ZIP files are not allowed`                                                               | 400  | A ZIP with password-protected members                                       | Upload the documents individually, or use an unencrypted archive                                            |
| `File too large (max 4096MB)`                                                                       | 400  | Over the size ceiling                                                       | Split the document                                                                                          |
| `File was marked as needing decryption, but no external decryption service is currently enabled...` | 400  | `needs_decryption=true` sent while the service is disabled                  | Enable the service, or refresh the page so the toggle disappears                                            |
| `Failed to reach the external decryption service: ...`                                              | 502  | DNS, TLS, connection refused, or timeout                                    | Check network path, firewall, and TLS trust from *both* API and worker containers                           |
| `Decryption service returned HTTP 4xx/5xx: ...`                                                     | 502  | Your service rejected the submit                                            | Inspect the quoted body; check auth headers                                                                 |
| `Decryption service returned a non-JSON response without file content.`                             | 502  | Async submit returned neither a binary body nor parseable JSON              | Return `202` with a JSON job object, or `200` with `application/octet-stream`                               |
| `Could not extract job_id from decryption response using path 'job_id'.`                            | 502  | `poll_job_id_json_path` doesn't match your response shape                   | Correct the dot path, e.g. `data.jobId`                                                                     |
| `Decryption poll returned HTTP 404: ...`                                                            | 502  | Poll endpoint returned `4xx` for a not-yet-ready job                        | Return `202` while pending                                                                                  |
| `External decryption job failed with status: FAILED`                                                | 502  | Your service reported a failed status                                       | Investigate in your service's logs                                                                          |
| `Decryption job completed but no file stream was returned.`                                         | 502  | Poll reported completion in JSON but the follow-up GET didn't return binary | Return the byte stream on completion — see [The double-GET behaviour](#the-double-get-behaviour-async-mode) |
| `Decryption service returned empty content.`                                                        | 502  | Zero-length response body                                                   | Return the actual decrypted bytes                                                                           |
| `Decryption service timed out after 600s of polling.`                                               | 504  | Exceeded `interval × max_attempts`                                          | Raise `poll_max_attempts`, or speed up decryption                                                           |
| `Decryption service config not found. Run the migration first.`                                     | 404  | Database migration hasn't been applied                                      | Run AAEKB database migrations                                                                               |

**Diagnostic tips**

* The toggle doesn't appear → check `GET /kb-decryption-service/status`; if it returns `{"available": false}`, the config row is missing or `enabled` is false. Also allow \~30 s for the browser-side cache to expire.
* Works on Quick Upload but not on normal upload (or vice versa) → this is almost always a network reachability difference between the API container and the background worker container.
* Upload succeeds but the document has no searchable content → your service is probably returning something other than raw bytes of the original format (commonly base64 or a JSON wrapper). Verify with `curl` that the response body starts with the expected magic bytes.

## Deployment and networking requirements

1. **Reachability from two places.** Both the AAEKB API containers and the AAEKB background worker containers must be able to resolve and reach `decrypt_endpoint` and `poll_endpoint_template`. In Kubernetes deployments, verify that network policies permit egress from both workloads.
2. **TLS trust.** AAEKB validates TLS certificates using the standard trusted CA bundle in its container image. A decryption service presenting a certificate from a **private/internal CA will fail to connect** unless that CA is added to the AAEKB container trust store as part of your deployment. Plan for this — it is a common first-run blocker on intranet deployments. Certificate verification cannot be disabled from the admin UI.
3. **Plain HTTP.** `http://` endpoints work but are strongly discouraged, as plaintext documents would traverse the network unprotected on the response leg.
4. **Stable hostname.** The endpoint is stored as a literal URL. Use a stable internal DNS name or service address rather than a pod IP.
5. **Latency budget.** The submit call holds a connection for up to 300 seconds. Ensure any intermediate load balancers, ingress controllers or proxies have idle/read timeouts at least that long, otherwise they will sever the connection before AAEKB gives up.
6. **Database migration.** The feature requires the `kb_decryption_service_config` table, created by a standard AAEKB migration that also seeds a single disabled default row. The feature is inert until a Super Admin enables it.

## Rollout and validation checklist

**Before enabling in production**

* [ ] Decryption service deployed and reachable from a test pod in the AAEKB namespace (`curl` from inside the API pod *and* a worker pod)
* [ ] TLS certificate chain trusted by the AAEKB containers
* [ ] AAEKB database migrations applied; the **KB Decryption Service** tab loads without a "table not found" error
* [ ] Confirmed with a representative sample that your **encrypted files pass upload validation** — take three real encrypted documents of each format you intend to ingest and upload them. This validates [Ciphertext envelope compatibility](#ciphertext-envelope-compatibility), which is the most common failure point

**Configuration**

* [ ] `decrypt_endpoint` set, method and content type match your service
* [ ] `decrypt_file_field_name` matches the form field your service expects (multipart mode)
* [ ] `response_mode` matches your service's behaviour
* [ ] For async: `poll_endpoint_template` contains `{job_id}`; JSON paths verified against a real response captured from your service
* [ ] `poll_interval_seconds × poll_max_attempts` comfortably exceeds your worst-case decryption time
* [ ] Auth configured and verified with `curl` using exactly the headers AAEKB will send
* [ ] Display Name set to something meaningful — it is recorded in every decrypted document's metadata

**Functional testing**

* [ ] Upload an encrypted file with the toggle **on** → document ingests, content is searchable
* [ ] Upload an unencrypted file with the toggle **off** → unaffected
* [ ] Upload an unencrypted file with the toggle **on** → your service should return a clear error; verify the message surfaces sensibly to the user
* [ ] Force a service failure (stop the service) → user sees a `502`-class error, no partial document is indexed
* [ ] Test with your largest expected file and confirm timing sits inside the polling budget
* [ ] Test a multi-file batch and a folder upload
* [ ] Verify the decrypted document downloads correctly from the Knowledge Base and that the metadata shows `originally_encrypted: true`

**Operational**

* [ ] Credential rotation procedure documented (re-save via the admin UI or `PUT` API)
* [ ] Monitoring/alerting on the decryption service, including its own error rate and latency
* [ ] Super Admin audit log reviewed to confirm configuration changes are being captured

## Known limitations

* **One service per deployment.** There is no per-project, per-team or per-file-type routing to different decryption services.
* **Batch-level toggle only.** Encrypted and unencrypted files cannot be mixed in a single upload batch.
* **Direct uploads only.** Connector-based ingestion (SharePoint, Confluence, Google Drive, web crawl, and similar) does not invoke decryption.
* **No retries.** A single transient network blip fails the upload; the user must retry manually.
* **Static credentials.** No OAuth token refresh, HMAC signing, or client-certificate mTLS from AAEKB itself. Use an authenticating proxy for those schemes.
* **No per-user or per-document authorisation context** is forwarded to the decryption service. It cannot enforce per-principal DRM policy based on information AAEKB sends.
* **Files are stored decrypted after ingestion**, under AAEKB's own access controls and storage encryption. See [Storage, security and audit behaviour](#storage-security-and-audit-behaviour).
* **No "test connection" button.** Validation is done by performing a real upload.
* **Fully opaque ciphertext is blocked** by pre-decryption content validation for most formats. See [Ciphertext envelope compatibility](#ciphertext-envelope-compatibility).
* **Nested decryption is not supported.** Members inside a ZIP are not individually decrypted.

## FAQ

**Does AAEKB ever see our encryption keys?**
No. AAEKB sends ciphertext to an endpoint you control and receives plaintext back. It has no knowledge of key material, algorithms, or your DRM policy engine.

**Can we run the decryption service entirely on our intranet?**
Yes — that is the primary use case. The endpoint only needs to be reachable from the AAEKB API and worker containers. For an on-premise AAEKB deployment, nothing needs to leave your network.

**Which response mode should we use?**
Use `sync` if decryption reliably completes within a few seconds and the service can hold the connection. Use `async` for anything slower, batched, or queue-backed. `async` with the poll endpoint returning the byte stream directly on completion is the most robust configuration.

**What happens if the decryption service is down?**
Uploads with the toggle enabled fail with a clear error. Normal uploads are unaffected. Already-ingested documents are unaffected — they are stored decrypted and never need your service again.

**Can we disable the feature temporarily?**
Yes. Toggle **Enabled** off. The end-user toggle disappears within about 30 seconds, and any upload still claiming `needs_decryption=true` is rejected with an explanatory message.

**Can users tell which documents were originally encrypted?**
Yes — decrypted documents carry `originally_encrypted: true`, the decryption service's display name, and a decryption timestamp in their metadata.

**Our encrypted files come out of the DRM system named `document.pdf.encrypted`. Will that work?**
No. Rename to `document.pdf` before upload. AAEKB validates the extension before decryption and does not recognise wrapper extensions. See [Filename and wrapper compatibility](#filename-and-wrapper-compatibility).

**Our ciphertext is fully opaque — no recognisable header at all. What are our options?**
The upload will be rejected before reaching your service. Options: (a) have the exporting system preserve or prepend the original format header, (b) use OOXML/OLE2-based envelope encryption for Office documents, which is explicitly supported, or (c) discuss a deployment-level adjustment to content validation with your AAEKB representative for your on-premise installation.

**Can the decryption service return a download URL instead of bytes?**
No. AAEKB reads the decrypted content from the response body and does not follow a URL returned in JSON.

**Does this work with Quick Upload?**
Yes, though the user waits for the full decryption round-trip before the upload modal completes.

*Document version 1.0. Feature reference: EN-3043.*