[Enhancement] Add Web Agents Evaluation Workflow

[NiraliPopat]

Summary

Added a comprehensive web agents evaluation framework to SyGra for evaluating browser automation agents with retry logic, inline evaluation, and advanced metrics tracking.

Explain the features implemented:

🎯 Core Evaluation Framework

• Task Executor (task_executor.py): Complete evaluation workflow with pre/post processors
  • RequestResponseLogger: Comprehensive request/response logging system with timestamped JSON logs
  • FetchNextActionPreProcessor: Pre-processes LLM requests with chat history management and retry hint injection
  • FetchNextActionPostProcessor: Post-processes LLM responses and extracts tool calls
  • InlineEvaluationLambda: Real-time evaluation of model predictions against golden responses
  • RetryFlow: Intelligent retry logic with failure detection
  • ShouldContinueCondition: Workflow continuation control
  • Flatten: Post-processor to flatten nested retry structures for analysis

🛠️ Browser Interaction Tools

• 10 browser automation tools (tools.py) using LangChain's @tool decorator:
  • screenshot_tool, click_tool, type_tool, typing_tool, scroll_tool
  • wait_tool, resume_tool, hil_tool, text_clear_tool, slider_tool

🔄 Intelligent Retry System

• Adaptive failure hints that guide the agent:
  • Tool Incorrect: Instructs agent to use different tool
  • Parameters Incorrect: Instructs agent to fix parameters for same tool
• Configurable retry limits (default: 3 retries)
• Server error detection to prevent unnecessary retries

📊 Evaluation Metrics

• Unit Metrics (Inline):
  • Tool Match: Exact tool name matching
  • Step Match: Comprehensive validation (tool + parameters + bounding box + direction + text)
• Aggregator Metrics (Post-processing):
  • Accuracy: Overall success rate
  • Pass@k: Probability of success in k attempts
  • Pass^k: Probability of success in all k attempts
  • Step Efficiency: Efficiency scoring based on retry attempts with configurable penalty

⚙️ Configuration & Constants

• Graph Configuration (graph_config.yaml): Complete workflow definition with system prompts, node configuration, and metrics setup
• Constants (constants.py): Centralized configuration for server errors, tool mappings, retry settings, state keys, and failure hints

📚 Documentation

• Comprehensive README: Added documentation covering architecture, components, workflow, usage, configuration, logging, error handling, best practices, and troubleshooting
• Docs integration: Added to docs/eval/agents/web_agents_eval.md

Performance impact (if any):

• N/A - New feature added

How to Test the feature

Steps for reviewers to verify functionality:

1. Run the task tasks/eval/agents/web_agents
2. Observe the Flatten_.json file for records level output
3. Observe the MetricCollatorPostProcessor_.json for overall metrics

Checklist

• Lint fixes and unit testing done
• End to end task testing
• Documentation updated