Skip to content

feat(exa): refresh Exa integration against current API, retire dead research endpoint - #6074

Merged
waleedlatif1 merged 5 commits into
stagingfrom
worktree-exa-api-refresh
Jul 30, 2026
Merged

feat(exa): refresh Exa integration against current API, retire dead research endpoint#6074
waleedlatif1 merged 5 commits into
stagingfrom
worktree-exa-api-refresh

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

Exa's dev-rel team sent over a list of places our integration had drifted from their current API. I validated every claim against the live API with a real key, and found several things their list didn't cover.

Broken in production today:

  • exa_research calls /research/v1, which returns HTTP 410 RESEARCH_RETIRED. Every workflow using the Research operation fails. Removed it and added an Agent operation on /agent/runs. Workflows saved on the old operation route to Agent, so they start working again instead of erroring.
  • The category dropdown sent values Exa no longer recognizes (research_paper, news_article, movie, song, pdf, github, tweet). Exa treats category as an unvalidated soft hint, so these never errored — they just silently stopped steering results. Replaced with the current taxonomy (company, publication, news, personal site, financial report, people) and remapped legacy values so existing workflows improve rather than break.
  • Live Crawl Mode defaulted to never, so every search was silently pinned to cache-only unless the user dug into advanced mode. Removed the default and exposed maxAgeHours instead.

API drift Exa flagged, all confirmed:

  • Search types are now instant/fast/auto/deep-lite/deep/deep-reasoning. Legacy neural/keyword still pass through for saved workflows.
  • numResults is 1–100, not the 25 we documented.
  • livecrawl is deprecated in favor of maxAgeHours (−1 to 720). Exa 400s if both are sent, so they're now mutually exclusive.
  • useAutoprompt dropped, crawl-date filters marked deprecated (still functional, so existing configs are untouched).
  • score isn't in the search result schema — it only comes back for the legacy neural type. Marked optional instead of claiming it's always present.
  • answer's text controls the cited sources' page text, not the answer. Fixed the description and removed a dead query field that was always empty.
  • /findSimilar is deprecated but still live, so it stays, labeled.

Capability gaps closed:

  • Search results now expose id, so results can be chained into Get Contents via ids (previously you had to round-trip through URLs).
  • Added outputSchema structured output with field-level grounding on Search, Answer, and Agent.
  • Added highlightScores, subpages, entities, extras, statuses, requestId.

Unrelated bug found along the way: copilot's search-online never requested page content from Exa, so every result snippet was an empty string. Now requests highlights.

Docs regenerate from the tool definitions, so the docs page Exa reviewed is fixed by these code changes.

Type of Change

  • Bug fix
  • New feature

Testing

Validated every change against the live Exa API with a real key — including the cases that 400 (livecrawl + maxAgeHours, ids + urls), the full Agent create/poll/complete cycle with structured output and cost metering, and search→contents id chaining. Added 33 unit tests covering the freshness mutual-exclusion, category remapping, request-body shape per endpoint, and the legacy research→agent routing. Full block/tool suite green (519 tests). Typecheck and check:api-validation clean.

Backwards compatibility

  • Legacy exa_research operation routes to Agent, with the old research models mapped onto effort levels.
  • Legacy category and search-type values still work.
  • livecrawl still accepted as a passthrough; deprecated crawl-date filters left in place.
  • The pi search handler passes text as an object rather than a boolean; that path is preserved and covered by a test.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

…esearch endpoint

Exa's dev-rel team flagged that our integration was written against a
retired version of their API. Validated every claim against the live API
with a real key.

- /research/v1 returns HTTP 410 RESEARCH_RETIRED, so the Research
  operation was hard-broken in production. Removed it and added an Agent
  operation on /agent/runs. Saved workflows on the old operation are
  routed to Agent so they start working again.
- Category dropdown sent values Exa no longer recognizes (research_paper,
  news_article, movie, song, ...). Exa accepts category as an unvalidated
  soft hint, so these silently stopped steering results rather than
  erroring. Replaced with the current taxonomy and remapped legacy values.
- Live crawl mode defaulted to 'never', silently forcing cache-only
  results on every search. Removed the default and exposed maxAgeHours,
  which replaces the deprecated livecrawl. Exa 400s when both are sent,
  so they are now mutually exclusive.
- numResults was capped at 25 in the UI; the API allows 1-100.
- Search types refreshed to instant/fast/auto/deep-lite/deep/
  deep-reasoning. Legacy neural/keyword still pass through.
- Exposed result id so search results can be chained into Get Contents
  via ids, plus highlightScores, subpages, entities, extras, statuses,
  requestId, and outputSchema structured output with grounding.
- answer text controls cited-source text, not the answer; fixed the
  description and the dead query field.
- Marked findSimilar and the crawl-date filters deprecated. Both still
  work, so existing workflows are unaffected.
- Copilot search-online never requested page content, so every snippet
  was empty. Now requests highlights.
@vercel

vercel Bot commented Jul 30, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Jul 30, 2026 1:08am

Request Review

@cursor

cursor Bot commented Jul 30, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Touches workflow execution, external API contracts, and legacy operation routing; behavior changes for saved Exa blocks (freshness defaults, removed fields) though migrations and compatibility shims are included.

Overview
Realigns the Exa integration with Exa’s current API and fixes workflows that were failing or silently misbehaving.

Research → Agent: Removes the dead exa_research tool (/research/v1 returns 410) and adds exa_agent on /agent/runs with polling, structured output, and cost metering. Saved exa_research blocks still route to Agent; legacy research models map to effort, and a research output shape is preserved for existing references.

Search / contents / answer: Updates search types, categories, and limits; adds maxAgeHours (and drops the default livecrawl: never that forced cache-only). Search nests content options under contents, supports outputSchema / structuredOutput, and exposes result **id**s for Get Contents via ids. Subblock migrations strip deprecated useAutoprompt and livecrawl. Shared utils handle freshness mutual-exclusion, category remapping, and JSON schema parsing.

Copilot: search-online now requests Exa highlights so snippets are not empty.

Docs, block UI, integrations.json, and a broad exa.test.ts suite accompany the tool changes.

Reviewed by Cursor Bugbot for commit 8e5a779. Configure here.

Comment thread apps/sim/tools/exa/agent.ts
Comment thread apps/sim/blocks/blocks/exa.ts
Comment thread apps/sim/tools/exa/agent.ts
Comment thread apps/sim/blocks/blocks/exa.ts
@greptile-apps

greptile-apps Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Refreshes the Exa integration against the current API and retires the dead Research endpoint.

  • Replaces exa_research with Agent (/agent/runs), routing legacy workflows to Agent with model→effort mapping and a compatibility research output shape.
  • Updates Search/Get Contents/Find Similar/Answer params (categories, search types, maxAgeHours, ids, structured output) and regenerates docs.
  • Enforces urls/ids XOR and freshness mutual exclusion at request time; adds unit coverage for routing, remapping, and agent settle behavior.

Confidence Score: 5/5

Prior Greptile threads are addressed or intentionally left as designed; the PR appears safe to merge.

Legacy research workflows get a one-element research output from agent settle; Get Contents ids-only no longer uses a broken conditional required; remaining selector XOR stays at execution time per repo pattern. No blocking failure remains.

Important Files Changed

Filename Overview
apps/sim/tools/exa/agent.ts Agent tool polls runs to completion, settles failures, and emits legacy research array for saved research workflows.
apps/sim/blocks/blocks/exa.ts Block UI/ops updated for Agent and current Exa params; Get Contents urls/ids left both optional with runtime XOR; conditional required reverted.
apps/sim/tools/exa/get_contents.ts Request body requires exactly one of urls or ids and applies shared freshness helpers.
apps/sim/tools/exa/exa.test.ts Tests cover research→agent routing, effort mapping, legacy research output, and terminal agent statuses.

Reviews (5): Last reviewed commit: "fix(exa): revert conditional required on..." | Re-trigger Greptile

- A run already terminal on creation went through a path that never set
  success=false, so a failed or cancelled run reported as successful.
  Both the create path and the poll loop now settle through one function.
- Routing exa_research to Agent dropped the research output shape, so
  saved workflows referencing research[0].text resolved to undefined. The
  agent tool now also emits that legacy shape.
- The Agent operation's inputs are conditioned on both exa_agent and
  exa_research so the serializer keeps carrying a stored research query;
  it drops any value whose sub-block condition no longer matches.
- Dropped the model to effort mapping and the unused ExaResearchParams:
  the serializer drops values for removed sub-blocks, so model never
  reached the params function.
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

Comment thread apps/sim/blocks/blocks/exa.ts
…n outputs

The subblock ID stability check caught the removed subblocks — that gate
exists precisely to stop removals from breaking deployed workflows.

- Restore the research model sub-block, scoped to the legacy exa_research
  operation so it never shows for new workflows but still serializes for
  saved ones, and restore the model to effort mapping. Removing it lost
  the configured research depth, silently falling back to effort auto.
- Register useAutoprompt and livecrawl in SUBBLOCK_ID_MIGRATIONS as
  intentional removals. Neither has a value-compatible replacement:
  livecrawl is a mode string and maxAgeHours a number, so mapping one to
  the other would send NaN.
- Replace vague json output descriptions with their inner field lists.
- Drop the separate compat test file; the coverage that guards real
  regressions now lives in exa.test.ts.
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

Comment thread apps/sim/blocks/blocks/exa.ts
…research depth

- A saved research workflow with no stored model fell through to the
  Agent default of auto rather than the standard depth the old Research
  operation used. Legacy research now always maps to an effort level,
  defaulting to medium, and the legacy model sub-block carries the same
  default the old dropdown had.
- Get Contents needed both selectors optional so the ids path is
  reachable, which left an empty config failing only at run time. URLs is
  now conditionally required, dropping the requirement when result IDs
  are supplied, so the editor flags the empty case. The exactly-one check
  in the request body stays as the backstop.
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

Follow-up on the Get Contents selector point from the last round: I'd initially left this to runtime enforcement, citing .claude/rules/sim-integrations.md (mutually exclusive sources marked required: false, exactly-one enforced at execution). That rule still holds for the shape — urls/ids are alternate identifiers, so a canonicalParamId pair would be wrong here.

But the editor-time gap was a fair criticism, and the block system does support conditional required. urls is now required on Get Contents, with the requirement inverted away when result IDs are supplied:

required: (values) => ({
  field: 'operation',
  value: 'exa_get_contents',
  not: hasValue(values?.ids),
})

So an empty config is flagged in the editor, the ids-only path stays reachable, and the exactly-one check in the request body remains as the backstop. I used the function form rather than a static and clause because evaluateSubBlockCondition compares with strict equality, and an untouched field can be undefined or '' — the function inspects the value instead of guessing which.

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

Comment thread apps/sim/blocks/blocks/exa.ts Outdated
Comment thread apps/sim/blocks/blocks/exa.ts Outdated
The conditional required callback did not work and introduced a
regression. `isFieldRequired` in webhook deploy calls `config.required()`
with no arguments, so the callback never saw `ids` and left URLs required
— an ids-only block would have been reported as missing a required field
on deploy. `collectBlockFieldIssues` skips sub-block required checks
whose id matches a tool param, so it never evaluated the callback either.

Both selectors go back to optional with the exactly-one check in the
request body, which is what the integration rules prescribe for mutually
exclusive alternate identifiers. Added a comment recording why a
conditional required cannot express this, so it is not reattempted.
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 8e5a779. Configure here.

@waleedlatif1
waleedlatif1 merged commit 911b958 into staging Jul 30, 2026
27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant