⚡️ Speed up method PrComment.to_json by 329% in PR #1318 (fix/js-jest30-loop-runner)

[codeflash-ai]

⚡️ This pull request contains optimizations for PR #1318

If you approve this dependent PR, these changes will be merged into the original PR branch fix/js-jest30-loop-runner.

This PR will be automatically closed if the original PR is merged.

---

📄 329% (3.29x) speedup for PrComment.to_json in codeflash/github/PrComment.py

⏱️ Runtime : 1.61 milliseconds → 374 microseconds (best of 31 runs)

📝 Explanation and details

This optimization achieves a 329% speedup (1.61ms → 374μs) by eliminating expensive third-party library calls and simplifying dictionary lookups:

Primary Optimization: humanize_runtime() - Eliminated External Library Overhead

The original code used humanize.precisedelta() and re.split() to format time values, which consumed 79.6% and 11.4% of the function's execution time respectively (totaling ~91% overhead). The optimized version replaces this with:

1. Direct unit determination via threshold comparisons: Instead of calling humanize.precisedelta() and then parsing its output with regex, the code now uses a simple cascading if-elif chain (time_micro < 1000, < 1000000, etc.) to directly determine the appropriate time unit.

2. Inline formatting: Time values are formatted with f-strings (f"{time_micro:.3g}") at the same point where units are determined, eliminating the need to parse formatted strings.

3. Removed regex dependency: The re.split(r",|\s", runtime_human)[1] call is completely eliminated since units are now determined algorithmically rather than extracted from formatted output.

Line profiler evidence: The original humanize.precisedelta() call took 3.73ms out of 4.69ms total (79.6%), while the optimized direct formatting approach reduced the entire function to 425μs - an 11x improvement in humanize_runtime() alone.

Secondary Optimization: TestType.to_name() - Simplified Dictionary Access

Changed from:

if self is TestType.INIT_STATE_TEST:
    return ""
return _TO_NAME_MAP[self]

To:

return _TO_NAME_MAP.get(self, "")

This eliminates a conditional branch and replaces a KeyError-raising dictionary access with a safe .get() call. Line profiler shows this reduced execution time from 210μs to 172μs (18% faster).

Performance Impact by Test Case

All test cases show 300-500% speedups, with the most significant gains occurring when:

• Multiple runtime conversions happen (seen in to_json() which calls humanize_runtime() twice)
• Test cases with larger time values (e.g., 1 hour in nanoseconds) that previously required more complex humanize processing

The optimization particularly benefits the PrComment.to_json() method, which calls humanize_runtime() twice per invocation. This is reflected in test results showing consistent 350-370% speedups across typical usage patterns.

Trade-offs

None - this is a pure performance improvement with identical output behavior and no regressions in any other metrics.

✅ Correctness verification report:

Test	Status
⚙️ Existing Unit Tests	🔘 None Found
🌀 Generated Regression Tests	✅ 77 Passed
⏪ Replay Tests	🔘 None Found
🔎 Concolic Coverage Tests	🔘 None Found
📊 Tests Coverage	100.0%

🌀 Click to see Generated Regression Tests

import pytest
from codeflash.github.PrComment import PrComment
from codeflash.models.models import (BenchmarkDetail, FunctionTestInvocation,
                                     TestResults)
from codeflash.models.test_type import TestType

def test_to_json_basic_with_minimal_inputs():
    """Test to_json with minimal required inputs and no optional fields."""
    # Create minimal TestResults with no test invocations
    behavior_results = TestResults()
    benchmark_results = TestResults()
    
    # Create PrComment with required fields only
    pr_comment = PrComment(
        optimization_explanation="Simple optimization",
        best_runtime=1000,
        original_runtime=2000,
        function_name="test_func",
        relative_file_path="path/to/file.py",
        speedup_x="2.0x",
        speedup_pct="50%",
        winning_behavior_test_results=behavior_results,
        winning_benchmarking_test_results=benchmark_results,
    )
    
    # Call to_json and verify basic structure
    codeflash_output = pr_comment.to_json(); result = codeflash_output # 96.7μs -> 22.7μs (326% faster)

def test_to_json_with_humanized_runtimes():
    """Test that runtimes are properly humanized in output."""
    behavior_results = TestResults()
    benchmark_results = TestResults()
    
    pr_comment = PrComment(
        optimization_explanation="Test",
        best_runtime=1000000,  # 1 millisecond in nanoseconds
        original_runtime=2000000,  # 2 milliseconds in nanoseconds
        function_name="func",
        relative_file_path="file.py",
        speedup_x="2.0x",
        speedup_pct="50%",
        winning_behavior_test_results=behavior_results,
        winning_benchmarking_test_results=benchmark_results,
    )
    
    codeflash_output = pr_comment.to_json(); result = codeflash_output # 68.4μs -> 15.5μs (342% faster)

def test_to_json_without_async_throughput():
    """Test to_json when async throughput fields are not provided."""
    behavior_results = TestResults()
    benchmark_results = TestResults()
    
    pr_comment = PrComment(
        optimization_explanation="No async",
        best_runtime=1000,
        original_runtime=2000,
        function_name="sync_func",
        relative_file_path="file.py",
        speedup_x="2.0x",
        speedup_pct="50%",
        winning_behavior_test_results=behavior_results,
        winning_benchmarking_test_results=benchmark_results,
    )
    
    codeflash_output = pr_comment.to_json(); result = codeflash_output # 67.4μs -> 14.3μs (372% faster)

def test_to_json_with_async_throughput():
    """Test to_json when both async throughput values are provided."""
    behavior_results = TestResults()
    benchmark_results = TestResults()
    
    pr_comment = PrComment(
        optimization_explanation="Async optimization",
        best_runtime=1000,
        original_runtime=2000,
        function_name="async_func",
        relative_file_path="file.py",
        speedup_x="2.0x",
        speedup_pct="50%",
        winning_behavior_test_results=behavior_results,
        winning_benchmarking_test_results=benchmark_results,
        original_async_throughput=1000,
        best_async_throughput=2000,
    )
    
    codeflash_output = pr_comment.to_json(); result = codeflash_output # 66.9μs -> 14.4μs (364% faster)

def test_to_json_with_benchmark_details():
    """Test to_json with benchmark_details provided."""
    benchmark_details = [
        BenchmarkDetail(
            benchmark_name="bench1",
            test_function="test_func1",
            original_timing="100ms",
            expected_new_timing="50ms",
            speedup_percent=50.0,
        ),
        BenchmarkDetail(
            benchmark_name="bench2",
            test_function="test_func2",
            original_timing="200ms",
            expected_new_timing="100ms",
            speedup_percent=50.0,
        ),
    ]
    
    behavior_results = TestResults()
    benchmark_results = TestResults()
    
    pr_comment = PrComment(
        optimization_explanation="With benchmarks",
        best_runtime=1000,
        original_runtime=2000,
        function_name="func",
        relative_file_path="file.py",
        speedup_x="2.0x",
        speedup_pct="50%",
        winning_behavior_test_results=behavior_results,
        winning_benchmarking_test_results=benchmark_results,
        benchmark_details=benchmark_details,
    )
    
    codeflash_output = pr_comment.to_json(); result = codeflash_output # 66.5μs -> 14.5μs (357% faster)

def test_to_json_loop_count_zero():
    """Test that loop_count is correctly extracted from benchmark results."""
    behavior_results = TestResults()
    benchmark_results = TestResults()
    
    pr_comment = PrComment(
        optimization_explanation="Test",
        best_runtime=1000,
        original_runtime=2000,
        function_name="func",
        relative_file_path="file.py",
        speedup_x="2.0x",
        speedup_pct="50%",
        winning_behavior_test_results=behavior_results,
        winning_benchmarking_test_results=benchmark_results,
    )
    
    codeflash_output = pr_comment.to_json(); result = codeflash_output # 65.5μs -> 13.8μs (374% faster)

def test_to_json_report_table_empty():
    """Test that report_table is empty dict when no test results are present."""
    behavior_results = TestResults()
    benchmark_results = TestResults()
    
    pr_comment = PrComment(
        optimization_explanation="Test",
        best_runtime=1000,
        original_runtime=2000,
        function_name="func",
        relative_file_path="file.py",
        speedup_x="2.0x",
        speedup_pct="50%",
        winning_behavior_test_results=behavior_results,
        winning_benchmarking_test_results=benchmark_results,
    )
    
    codeflash_output = pr_comment.to_json(); result = codeflash_output # 65.5μs -> 14.1μs (365% faster)

def test_to_json_with_empty_string_fields():
    """Test to_json with empty string values for text fields."""
    behavior_results = TestResults()
    benchmark_results = TestResults()
    
    pr_comment = PrComment(
        optimization_explanation="",
        best_runtime=1000,
        original_runtime=2000,
        function_name="",
        relative_file_path="",
        speedup_x="",
        speedup_pct="",
        winning_behavior_test_results=behavior_results,
        winning_benchmarking_test_results=benchmark_results,
    )
    
    codeflash_output = pr_comment.to_json(); result = codeflash_output # 64.7μs -> 14.1μs (359% faster)

def test_to_json_with_very_large_runtimes():
    """Test to_json with extremely large nanosecond values."""
    behavior_results = TestResults()
    benchmark_results = TestResults()
    
    # Very large runtime: 1 hour in nanoseconds
    large_runtime = 3600 * 1000 * 1000 * 1000
    
    pr_comment = PrComment(
        optimization_explanation="Large time",
        best_runtime=large_runtime,
        original_runtime=large_runtime * 2,
        function_name="slow_func",
        relative_file_path="file.py",
        speedup_x="2.0x",
        speedup_pct="50%",
        winning_behavior_test_results=behavior_results,
        winning_benchmarking_test_results=benchmark_results,
    )
    
    codeflash_output = pr_comment.to_json(); result = codeflash_output # 67.2μs -> 16.3μs (312% faster)

def test_to_json_with_very_small_runtimes():
    """Test to_json with very small nanosecond values."""
    behavior_results = TestResults()
    benchmark_results = TestResults()
    
    # Very small runtime: 1 nanosecond
    pr_comment = PrComment(
        optimization_explanation="Fast",
        best_runtime=1,
        original_runtime=2,
        function_name="fast_func",
        relative_file_path="file.py",
        speedup_x="2.0x",
        speedup_pct="50%",
        winning_behavior_test_results=behavior_results,
        winning_benchmarking_test_results=benchmark_results,
    )
    
    codeflash_output = pr_comment.to_json(); result = codeflash_output # 12.4μs -> 11.6μs (7.10% faster)

def test_to_json_with_special_characters_in_strings():
    """Test to_json with special characters in string fields."""
    behavior_results = TestResults()
    benchmark_results = TestResults()
    
    pr_comment = PrComment(
        optimization_explanation="Optimized: 50% faster! 🚀",
        best_runtime=1000,
        original_runtime=2000,
        function_name="func_with_unicode_αβγ",
        relative_file_path="path/to/file_with-dashes_and.dots.py",
        speedup_x="2.0x",
        speedup_pct="50%",
        winning_behavior_test_results=behavior_results,
        winning_benchmarking_test_results=benchmark_results,
    )
    
    codeflash_output = pr_comment.to_json(); result = codeflash_output # 66.1μs -> 14.4μs (358% faster)

def test_to_json_with_long_text_fields():
    """Test to_json with very long string values."""
    behavior_results = TestResults()
    benchmark_results = TestResults()
    
    long_explanation = "x" * 10000
    long_filename = "very/" * 100 + "long/path.py"
    
    pr_comment = PrComment(
        optimization_explanation=long_explanation,
        best_runtime=1000,
        original_runtime=2000,
        function_name="func",
        relative_file_path=long_filename,
        speedup_x="2.0x",
        speedup_pct="50%",
        winning_behavior_test_results=behavior_results,
        winning_benchmarking_test_results=benchmark_results,
    )
    
    codeflash_output = pr_comment.to_json(); result = codeflash_output # 65.0μs -> 13.9μs (369% faster)

def test_to_json_with_zero_runtimes():
    """Test to_json when runtimes are zero."""
    behavior_results = TestResults()
    benchmark_results = TestResults()
    
    pr_comment = PrComment(
        optimization_explanation="Zero time",
        best_runtime=0,
        original_runtime=0,
        function_name="func",
        relative_file_path="file.py",
        speedup_x="0x",
        speedup_pct="0%",
        winning_behavior_test_results=behavior_results,
        winning_benchmarking_test_results=benchmark_results,
    )
    
    codeflash_output = pr_comment.to_json(); result = codeflash_output # 12.2μs -> 11.5μs (6.09% faster)

def test_to_json_with_one_async_throughput_none():
    """Test that async throughput fields are excluded if only one is None."""
    behavior_results = TestResults()
    benchmark_results = TestResults()
    
    # Only best_async_throughput provided
    pr_comment = PrComment(
        optimization_explanation="Partial async",
        best_runtime=1000,
        original_runtime=2000,
        function_name="func",
        relative_file_path="file.py",
        speedup_x="2.0x",
        speedup_pct="50%",
        winning_behavior_test_results=behavior_results,
        winning_benchmarking_test_results=benchmark_results,
        original_async_throughput=None,
        best_async_throughput=2000,
    )
    
    codeflash_output = pr_comment.to_json(); result = codeflash_output # 66.2μs -> 14.1μs (369% faster)

def test_to_json_with_empty_benchmark_details_list():
    """Test to_json with an empty benchmark_details list."""
    behavior_results = TestResults()
    benchmark_results = TestResults()
    
    pr_comment = PrComment(
        optimization_explanation="Empty benchmarks",
        best_runtime=1000,
        original_runtime=2000,
        function_name="func",
        relative_file_path="file.py",
        speedup_x="2.0x",
        speedup_pct="50%",
        winning_behavior_test_results=behavior_results,
        winning_benchmarking_test_results=benchmark_results,
        benchmark_details=[],
    )
    
    codeflash_output = pr_comment.to_json(); result = codeflash_output # 65.1μs -> 14.0μs (365% faster)

def test_to_json_with_single_benchmark_detail():
    """Test to_json with exactly one benchmark detail."""
    benchmark_detail = BenchmarkDetail(
        benchmark_name="single",
        test_function="test_single",
        original_timing="100ms",
        expected_new_timing="50ms",
        speedup_percent=50.0,
    )
    
    behavior_results = TestResults()
    benchmark_results = TestResults()
    
    pr_comment = PrComment(
        optimization_explanation="Single benchmark",
        best_runtime=1000,
        original_runtime=2000,
        function_name="func",
        relative_file_path="file.py",
        speedup_x="2.0x",
        speedup_pct="50%",
        winning_behavior_test_results=behavior_results,
        winning_benchmarking_test_results=benchmark_results,
        benchmark_details=[benchmark_detail],
    )
    
    codeflash_output = pr_comment.to_json(); result = codeflash_output # 64.5μs -> 14.3μs (351% faster)

def test_to_json_return_type_is_dict():
    """Test that to_json returns a dictionary."""
    behavior_results = TestResults()
    benchmark_results = TestResults()
    
    pr_comment = PrComment(
        optimization_explanation="Test",
        best_runtime=1000,
        original_runtime=2000,
        function_name="func",
        relative_file_path="file.py",
        speedup_x="2.0x",
        speedup_pct="50%",
        winning_behavior_test_results=behavior_results,
        winning_benchmarking_test_results=benchmark_results,
    )
    
    codeflash_output = pr_comment.to_json(); result = codeflash_output # 64.5μs -> 14.0μs (360% faster)

def test_to_json_with_many_benchmark_details():
    """Test to_json with a large number of benchmark details."""
    # Create 100 benchmark details
    benchmark_details = [
        BenchmarkDetail(
            benchmark_name=f"bench_{i}",
            test_function=f"test_func_{i}",
            original_timing=f"{100 * i}ms",
            expected_new_timing=f"{50 * i}ms",
            speedup_percent=50.0,
        )
        for i in range(100)
    ]
    
    behavior_results = TestResults()
    benchmark_results = TestResults()
    
    pr_comment = PrComment(
        optimization_explanation="Many benchmarks",
        best_runtime=1000,
        original_runtime=2000,
        function_name="func",
        relative_file_path="file.py",
        speedup_x="2.0x",
        speedup_pct="50%",
        winning_behavior_test_results=behavior_results,
        winning_benchmarking_test_results=benchmark_results,
        benchmark_details=benchmark_details,
    )
    
    codeflash_output = pr_comment.to_json(); result = codeflash_output # 66.1μs -> 14.4μs (358% faster)
    for i in range(100):
        pass

def test_to_json_with_large_async_throughput_values():
    """Test to_json with very large async throughput numbers."""
    behavior_results = TestResults()
    benchmark_results = TestResults()
    
    # Large throughput values
    pr_comment = PrComment(
        optimization_explanation="High throughput",
        best_runtime=1000,
        original_runtime=2000,
        function_name="async_func",
        relative_file_path="file.py",
        speedup_x="2.0x",
        speedup_pct="50%",
        winning_behavior_test_results=behavior_results,
        winning_benchmarking_test_results=benchmark_results,
        original_async_throughput=1000000,
        best_async_throughput=2000000,
    )
    
    codeflash_output = pr_comment.to_json(); result = codeflash_output # 65.3μs -> 14.3μs (356% faster)

def test_to_json_preserves_all_input_fields():
    """Test that all input fields are correctly mapped to output keys."""
    benchmark_details = [
        BenchmarkDetail(
            benchmark_name="bench",
            test_function="test",
            original_timing="100ms",
            expected_new_timing="50ms",
            speedup_percent=50.0,
        )
    ]
    
    behavior_results = TestResults()
    benchmark_results = TestResults()
    
    explanation = "This is an optimization explanation"
    best_rt = 5000000000
    orig_rt = 10000000000
    func_name = "my_function"
    file_path = "src/module/file.py"
    speedup_x_val = "2.5x"
    speedup_pct_val = "60%"
    orig_async = 5000
    best_async = 12500
    
    pr_comment = PrComment(
        optimization_explanation=explanation,
        best_runtime=best_rt,
        original_runtime=orig_rt,
        function_name=func_name,
        relative_file_path=file_path,
        speedup_x=speedup_x_val,
        speedup_pct=speedup_pct_val,
        winning_behavior_test_results=behavior_results,
        winning_benchmarking_test_results=benchmark_results,
        benchmark_details=benchmark_details,
        original_async_throughput=orig_async,
        best_async_throughput=best_async,
    )
    
    codeflash_output = pr_comment.to_json(); result = codeflash_output # 67.2μs -> 15.9μs (323% faster)

def test_to_json_multiple_calls_same_instance():
    """Test that calling to_json multiple times on same instance produces consistent results."""
    behavior_results = TestResults()
    benchmark_results = TestResults()
    
    pr_comment = PrComment(
        optimization_explanation="Consistent test",
        best_runtime=1000,
        original_runtime=2000,
        function_name="func",
        relative_file_path="file.py",
        speedup_x="2.0x",
        speedup_pct="50%",
        winning_behavior_test_results=behavior_results,
        winning_benchmarking_test_results=benchmark_results,
    )
    
    # Call to_json multiple times
    codeflash_output = pr_comment.to_json(); result1 = codeflash_output # 65.3μs -> 13.8μs (372% faster)
    codeflash_output = pr_comment.to_json(); result2 = codeflash_output # 46.9μs -> 8.29μs (466% faster)
    codeflash_output = pr_comment.to_json(); result3 = codeflash_output # 43.9μs -> 6.80μs (545% faster)

def test_to_json_handles_numeric_edge_cases():
    """Test to_json with various numeric edge cases."""
    behavior_results = TestResults()
    benchmark_results = TestResults()
    
    # Test with minimum valid positive integers
    pr_comment = PrComment(
        optimization_explanation="Numeric edge case",
        best_runtime=1,
        original_runtime=2,
        function_name="func",
        relative_file_path="file.py",
        speedup_x="2.0x",
        speedup_pct="50%",
        winning_behavior_test_results=behavior_results,
        winning_benchmarking_test_results=benchmark_results,
        original_async_throughput=1,
        best_async_throughput=1,
    )
    
    codeflash_output = pr_comment.to_json(); result = codeflash_output # 12.5μs -> 12.0μs (4.35% faster)

def test_to_json_with_whitespace_in_strings():
    """Test to_json with various whitespace in string fields."""
    behavior_results = TestResults()
    benchmark_results = TestResults()
    
    pr_comment = PrComment(
        optimization_explanation="Multi\nline\nexplanation\nwith\ttabs",
        best_runtime=1000,
        original_runtime=2000,
        function_name="func name with spaces",
        relative_file_path="path / to / file.py",
        speedup_x="2.0x",
        speedup_pct="50%",
        winning_behavior_test_results=behavior_results,
        winning_benchmarking_test_results=benchmark_results,
    )
    
    codeflash_output = pr_comment.to_json(); result = codeflash_output # 66.3μs -> 13.7μs (382% faster)

def test_to_json_with_high_precision_floats_in_benchmarks():
    """Test to_json with high precision floating point values in benchmark details."""
    benchmark_details = [
        BenchmarkDetail(
            benchmark_name="precision_test",
            test_function="test_precision",
            original_timing="123.456789ms",
            expected_new_timing="61.7283945ms",
            speedup_percent=49.99999999,
        )
    ]
    
    behavior_results = TestResults()
    benchmark_results = TestResults()
    
    pr_comment = PrComment(
        optimization_explanation="Precision test",
        best_runtime=1000,
        original_runtime=2000,
        function_name="func",
        relative_file_path="file.py",
        speedup_x="2.0x",
        speedup_pct="50%",
        winning_behavior_test_results=behavior_results,
        winning_benchmarking_test_results=benchmark_results,
        benchmark_details=benchmark_details,
    )
    
    codeflash_output = pr_comment.to_json(); result = codeflash_output # 64.5μs -> 13.7μs (372% faster)

def test_to_json_output_dict_keys_immutable_after_creation():
    """Test that the output dictionary contains the expected keys."""
    behavior_results = TestResults()
    benchmark_results = TestResults()
    
    pr_comment = PrComment(
        optimization_explanation="Keys test",
        best_runtime=1000,
        original_runtime=2000,
        function_name="func",
        relative_file_path="file.py",
        speedup_x="2.0x",
        speedup_pct="50%",
        winning_behavior_test_results=behavior_results,
        winning_benchmarking_test_results=benchmark_results,
    )
    
    codeflash_output = pr_comment.to_json(); result = codeflash_output # 64.2μs -> 14.0μs (358% faster)
    
    # Verify exact set of keys present
    expected_keys = {
        "optimization_explanation",
        "best_runtime",
        "original_runtime",
        "function_name",
        "file_path",
        "speedup_x",
        "speedup_pct",
        "loop_count",
        "report_table",
        "benchmark_details",
    }
# codeflash_output is used to check that the output of the original code is the same as that of the optimized code.

To edit these changes git checkout codeflash/optimize-pr1318-2026-02-04T14.10.57 and push.