# Set up human spot checks ## Spot Check Sampling Spot check sampling lets you route a configurable percentage of submissions for manual review before they are exported. This is useful during tenant onboarding when extraction quality needs to be validated, and can be gradually reduced as confidence grows. ### Configuration Spot checking is configured per tenant in the organization settings. The configuration is a JSON object added to the tenant's `integrations` array. #### Example ```json { "type": "spot-check", "enabled": true, "sampleSize": 50, "spotCheckerEmails": [ "reviewer@yourcompany.com", "triage-inbox@linear.app" ], "templateOverrides": [ { "templateId": 3, "sampleSize": 50 } ] } ``` ### How it works When a submission is fully processed and ready for export, the system rolls a random number against the configured sample rate. If selected: 1. The submission is **not exported** automatically 2. It stays in **"Ready for Spot Check"** status in the dashboard 3. All configured spot checkers receive a **notification email** 4. A reviewer opens the submission, assigns it to themselves, and reviews the extracted data 5. The reviewer clicks **Mark as completed** — this exports the submission, sets the status to **"Completed"**, and sends the end user a **confirmation email** 6. The end user receives a **confirmation email** that their submission was processed Submissions that are not selected continue through the normal automatic export flow with no change. #### Settings reference | Setting | Type | Required | Description | | -------------------------------- | ---------------- | --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `enabled` | boolean | Yes | Enables or disables spot checking for the tenant. Set to `false` to disable without removing the configuration. | | `sampleSize` | integer (0-100) | Yes | Percentage of submissions routed to spot check. `0` = no submissions selected (effectively disabled). `100` = every submission is selected. | | `spotCheckerEmails` | array of strings | Yes | Email addresses that receive a notification when a submission is selected. Supports multiple addresses. Can include team inboxes or Linear triage email addresses. | | `templateOverrides` | array of objects | No | Optional per-template overrides for the sample rate. | | `templateOverrides[].templateId` | integer | Yes (within override) | The ID of the template to override. | | `templateOverrides[].sampleSize` | integer (0-100) | Yes (within override) | The sample rate for this specific template. Takes precedence over the tenant-level `sampleSize`. | #### Template overrides You can set different sample rates for different templates. For example, a tenant might need 100% review on a new template while an established one only needs 10%: ```json { "sampleSize": 10, "templateOverrides": [ { "templateId": 7, "sampleSize": 100 } ] } ``` In this example, template 7 has every submission reviewed, while all other templates are sampled at 10%. ### Enabling spot checking for a tenant To enable spot checking for your tenant, contact the Nora team with: * The **email addresses** that should receive spot check notifications * The desired **sample size** * Any **per-template overrides** if needed > **Tip:** Start with `sampleSize: 100` during initial onboarding so every submission is reviewed. Reduce gradually as extraction quality is validated (e.g. 100 → 50 → 10 → 0). ### Disabling spot checking Either set `enabled` to `false` or set `sampleSize` to `0`. Both stop new submissions from being selected, but existing spot check candidates in the queue are not affected. ### Reviewer workflow #### 1. Receive notification When a submission is selected for spot check, each email address in `spotCheckerEmails` receives an email with the submission name and number. #### 2. Open the submission In the FormFlow dashboard, spot-checked submissions are displayed with an orange **"Ready for Spot Check"** status badge, making them easy to identify in the submissions list. #### 3. Assign and review Open the submission and assign it to yourself. The status changes to **"In Spot Check"**. Review the extracted data against the source documents to verify accuracy. #### 4. Complete the review When satisfied with the data, click **Mark as completed**. This exports the submission, delivers a confirmation email to the end user, and sets the status to **"Completed"**. ### End user experience The end user (the person who submitted the original documents) is not aware that their submission was selected for spot check. From their perspective: * They submit documents as usual * They receive the same confirmation email they would normally receive * The only difference is a potential delay while the reviewer checks the data ### Recommended ramp-down schedule | Phase | sampleSize | When | | ------------------- | ---------- | ------------------------------------ | | Initial onboarding | 100 | First 1-2 weeks — review everything | | Building confidence | 50 | Weeks 2-4 — spot check half | | Monitoring | 10-20 | Ongoing — periodic quality checks | | Fully automated | 0 | When confident in extraction quality | Adjust timelines based on submission volume and error rates observed during review. --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.ai.insly.com/nora/configuration-guide/set-up-human-spot-checks.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.