Skip to content

feat: use thread-local curl handle for multi-concurrency support #214

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
maxday wants to merge 14 commits into
base: master
Choose a base branch
from
Open

feat: use thread-local curl handle for multi-concurrency support #214

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
726641b
Use thread local curl handle for multi-concurrency support
maxday May 26, 2026
2c9df5c
Add unit tests for thread-local curl multi-concurrency
maxday May 26, 2026
e09e6bf
Add error response body to post failure log and comprehensive unit tests
maxday May 26, 2026
6a9536e
Revert logging change in do_post error path
maxday May 26, 2026
12eff27
fix: format
maxday May 26, 2026
3a73104
Fix unit test compilation errors and remove unsupported clang-tidy key
maxday May 26, 2026
f2ad905
Suppress clang-tidy checks unsupported by older versions
maxday May 26, 2026
9f31e9d
Fix clang-tidy warnings in runtime and response code
maxday May 26, 2026
14d6f70
Fix all clang-tidy warnings and revert .clang-tidy to master
maxday May 26, 2026
1415a02
Remove no-op placeholder test
maxday May 26, 2026
4416211
fix: conflicts
maxday Jul 2, 2026
8a4606d
fix: conflicts
maxday Jul 2, 2026
34a8b1c
fix: implementation
maxday Jul 2, 2026
c4ebacc
Merge branch 'master' into maxday/use-thread-local-curl-for-multiconc…
maxday Jul 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ci/integ/install-deps.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ install_system_deps() {
install_build_deps() {
case "$OS" in
al2023|al2023-arm)
dnf install -y cmake ninja-build gcc-c++ openssl-devel libcurl-devel zip
dnf install -y cmake ninja-build gcc-c++ openssl-devel libcurl-devel zip libasan libubsan
;;
ubuntu)
apt-get install -y clang zlib1g-dev libssl-dev libcurl4-openssl-dev cmake ninja-build zip
Expand Down
17 changes: 12 additions & 5 deletions ci/integ/unit-test.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,18 @@ set -euo pipefail

OS=$1

CMAKE_ARGS=""
if [ "$OS" = "arch" ]; then
export CC=/usr/bin/clang CXX=/usr/bin/clang++
CMAKE_ARGS="-DENABLE_SANITIZERS=ON"
fi
CMAKE_ARGS="-DENABLE_SANITIZERS=ON"
case "$OS" in
arch)
export CC=/usr/bin/clang CXX=/usr/bin/clang++
;;
ubuntu)
export CC=/usr/bin/clang CXX=/usr/bin/clang++
;;
alpine)
CMAKE_ARGS=""
;;
esac

mkdir -p build && cd build
cmake .. -GNinja \
Expand Down
2 changes: 1 addition & 1 deletion include/aws/http/response.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ class response {
inline void append_body(const char* p, size_t sz);
inline bool has_header(char const* header) const;
inline lambda_runtime::outcome<std::string, bool> get_header(char const* header) const;
inline response_code get_response_code() const { return m_response_code; }
response_code get_response_code() const { return m_response_code; }

@bmoffatt bmoffatt May 27, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why was this change made?

Figure it was from a clang-tidy during development, but didn't audit all the runs

ah


/home/runner/work/aws-lambda-cpp/aws-lambda-cpp/include/aws/http/response.h:37:5: error: function 'get_response_code' has inline specifier but is implicitly inlined [readability-redundant-inline-specifier,-warnings-as-errors]
   37 |     inline response_code get_response_code() const { return m_response_code; }
      |     ^~~~~~

inline void set_response_code(aws::http::response_code c);
inline void set_content_type(char const* ct);
inline std::string const& get_body() const;
Expand Down
3 changes: 1 addition & 2 deletions include/aws/lambda-runtime/runtime.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -191,14 +191,13 @@ class runtime {

private:
void set_curl_next_options();
void set_curl_post_result_options();
static void set_curl_post_result_options();
post_outcome do_post(
std::string const& url,
std::string const& request_id,
invocation_response const& handler_response);
std::string const m_user_agent_header;
std::array<std::string const, 3> const m_endpoints;
CURL* const m_curl_handle;
};

inline std::chrono::milliseconds invocation_request::get_time_remaining() const
Expand Down
3 changes: 2 additions & 1 deletion include/aws/logging/logging.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,12 @@
*/

#include <cstdarg>
#include <cstdint>

@bmoffatt bmoffatt May 27, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see in the commit history that there was some fighting with clang tidy, with options added at one point and later reverted. Are the changes in logging.h/logging.cpp still necessary to get the CI to pass? eg: Can the more recent clang-tidy recommendations land in a standalone PR to keep this PR diff a tiny bit cleaner


namespace aws {
namespace logging {

enum class verbosity {
enum class verbosity : std::uint8_t {
error,
info,
debug,
Expand Down
6 changes: 5 additions & 1 deletion src/logging.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,15 +14,17 @@
*/
#include "aws/logging/logging.h"
#include <array>
#include <cstdarg>
#include <cstdio>
#include <chrono>

#define LAMBDA_RUNTIME_API __attribute__((visibility("default")))

namespace aws {
namespace logging {
namespace {

static inline char const* get_prefix(verbosity v)
inline char const* get_prefix(verbosity v)
{
switch (v) {
case verbosity::error:
Expand All @@ -36,6 +38,8 @@ static inline char const* get_prefix(verbosity v)
}
}

} // namespace

LAMBDA_RUNTIME_API
void log(verbosity v, char const* tag, char const* msg, va_list args)
{
Expand Down
118 changes: 64 additions & 54 deletions src/runtime.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,21 @@ static constexpr auto COGNITO_IDENTITY_HEADER = "lambda-runtime-cognito-identity
static constexpr auto DEADLINE_MS_HEADER = "lambda-runtime-deadline-ms";
static constexpr auto FUNCTION_ARN_HEADER = "lambda-runtime-invoked-function-arn";
static constexpr auto TENANT_ID_HEADER = "lambda-runtime-aws-tenant-id";
struct curl_handle_wrapper {
CURL* handle;
curl_handle_wrapper() : handle(curl_easy_init()) {}
~curl_handle_wrapper()
{
if (handle) {
curl_easy_cleanup(handle);
}
}
curl_handle_wrapper(curl_handle_wrapper const&) = delete;
curl_handle_wrapper& operator=(curl_handle_wrapper const&) = delete;
};

thread_local static curl_handle_wrapper m_curl_handle_wrapper;
thread_local static CURL*& m_curl_handle = m_curl_handle_wrapper.handle;

enum Endpoints {
INIT,
Expand Down Expand Up @@ -155,7 +170,7 @@ static int rt_curl_debug_callback(CURL* handle, curl_infotype type, char* data,
(void)handle;
(void)type;
(void)userdata;
std::string s(data, size);
const std::string s(data, size);
logging::log_debug(LOG_TAG, "CURL DBG: %s", s.c_str());
return 0;
}
Expand All @@ -164,83 +179,78 @@ static int rt_curl_debug_callback(CURL* handle, curl_infotype type, char* data,
runtime::runtime(std::string const& endpoint) : runtime(endpoint, "AWS_Lambda_Cpp/" + std::string(get_version())) {}

runtime::runtime(std::string const& endpoint, std::string const& user_agent)
: m_user_agent_header("User-Agent: " + user_agent),
m_endpoints{
{endpoint + "/2018-06-01/runtime/init/error",
endpoint + "/2018-06-01/runtime/invocation/next",
endpoint + "/2018-06-01/runtime/invocation/"}},
m_curl_handle(curl_easy_init())
: m_user_agent_header("User-Agent: " + user_agent), m_endpoints{
{endpoint + "/2018-06-01/runtime/init/error",
endpoint + "/2018-06-01/runtime/invocation/next",
endpoint + "/2018-06-01/runtime/invocation/"}}
{
if (!m_curl_handle) {
if (!lambda_runtime::m_curl_handle) {
logging::log_error(LOG_TAG, "Failed to acquire curl easy handle for next.");
}
}

runtime::~runtime()
{
curl_easy_cleanup(m_curl_handle);
}
runtime::~runtime() = default;

void runtime::set_curl_next_options()
{
// lambda freezes the container when no further tasks are available. The freezing period could be longer than the
// request timeout, which causes the following get_next request to fail with a timeout error.
curl_easy_reset(m_curl_handle);
curl_easy_setopt(m_curl_handle, CURLOPT_TIMEOUT, 0L);
curl_easy_setopt(m_curl_handle, CURLOPT_CONNECTTIMEOUT, 1L);
curl_easy_setopt(m_curl_handle, CURLOPT_NOSIGNAL, 1L);
curl_easy_setopt(m_curl_handle, CURLOPT_TCP_NODELAY, 1L);
curl_easy_setopt(m_curl_handle, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1);
curl_easy_reset(lambda_runtime::m_curl_handle);
curl_easy_setopt(lambda_runtime::m_curl_handle, CURLOPT_TIMEOUT, 0L);
curl_easy_setopt(lambda_runtime::m_curl_handle, CURLOPT_CONNECTTIMEOUT, 1L);
curl_easy_setopt(lambda_runtime::m_curl_handle, CURLOPT_NOSIGNAL, 1L);
curl_easy_setopt(lambda_runtime::m_curl_handle, CURLOPT_TCP_NODELAY, 1L);
curl_easy_setopt(lambda_runtime::m_curl_handle, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1);

curl_easy_setopt(m_curl_handle, CURLOPT_HTTPGET, 1L);
curl_easy_setopt(m_curl_handle, CURLOPT_URL, m_endpoints[Endpoints::NEXT].c_str());
curl_easy_setopt(lambda_runtime::m_curl_handle, CURLOPT_HTTPGET, 1L);
curl_easy_setopt(lambda_runtime::m_curl_handle, CURLOPT_URL, m_endpoints[Endpoints::NEXT].c_str());

curl_easy_setopt(m_curl_handle, CURLOPT_WRITEFUNCTION, write_data);
curl_easy_setopt(m_curl_handle, CURLOPT_HEADERFUNCTION, write_header);
curl_easy_setopt(lambda_runtime::m_curl_handle, CURLOPT_WRITEFUNCTION, write_data);
curl_easy_setopt(lambda_runtime::m_curl_handle, CURLOPT_HEADERFUNCTION, write_header);

curl_easy_setopt(m_curl_handle, CURLOPT_PROXY, "");
curl_easy_setopt(lambda_runtime::m_curl_handle, CURLOPT_PROXY, "");

#ifndef NDEBUG
curl_easy_setopt(m_curl_handle, CURLOPT_VERBOSE, 1);
curl_easy_setopt(m_curl_handle, CURLOPT_DEBUGFUNCTION, rt_curl_debug_callback);
curl_easy_setopt(lambda_runtime::m_curl_handle, CURLOPT_VERBOSE, 1);
curl_easy_setopt(lambda_runtime::m_curl_handle, CURLOPT_DEBUGFUNCTION, rt_curl_debug_callback);
#endif
}

void runtime::set_curl_post_result_options()
{
curl_easy_reset(m_curl_handle);
curl_easy_setopt(m_curl_handle, CURLOPT_TIMEOUT, 0L);
curl_easy_setopt(m_curl_handle, CURLOPT_CONNECTTIMEOUT, 1L);
curl_easy_setopt(m_curl_handle, CURLOPT_NOSIGNAL, 1L);
curl_easy_setopt(m_curl_handle, CURLOPT_TCP_NODELAY, 1L);
curl_easy_setopt(m_curl_handle, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1);
curl_easy_reset(lambda_runtime::m_curl_handle);
curl_easy_setopt(lambda_runtime::m_curl_handle, CURLOPT_TIMEOUT, 0L);
curl_easy_setopt(lambda_runtime::m_curl_handle, CURLOPT_CONNECTTIMEOUT, 1L);
curl_easy_setopt(lambda_runtime::m_curl_handle, CURLOPT_NOSIGNAL, 1L);
curl_easy_setopt(lambda_runtime::m_curl_handle, CURLOPT_TCP_NODELAY, 1L);
curl_easy_setopt(lambda_runtime::m_curl_handle, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1);

curl_easy_setopt(m_curl_handle, CURLOPT_POST, 1L);
curl_easy_setopt(m_curl_handle, CURLOPT_READFUNCTION, read_data);
curl_easy_setopt(m_curl_handle, CURLOPT_WRITEFUNCTION, write_data);
curl_easy_setopt(m_curl_handle, CURLOPT_HEADERFUNCTION, write_header);
curl_easy_setopt(lambda_runtime::m_curl_handle, CURLOPT_POST, 1L);
curl_easy_setopt(lambda_runtime::m_curl_handle, CURLOPT_READFUNCTION, read_data);
curl_easy_setopt(lambda_runtime::m_curl_handle, CURLOPT_WRITEFUNCTION, write_data);
curl_easy_setopt(lambda_runtime::m_curl_handle, CURLOPT_HEADERFUNCTION, write_header);

curl_easy_setopt(m_curl_handle, CURLOPT_PROXY, "");
curl_easy_setopt(lambda_runtime::m_curl_handle, CURLOPT_PROXY, "");

#ifndef NDEBUG
curl_easy_setopt(m_curl_handle, CURLOPT_VERBOSE, 1);
curl_easy_setopt(m_curl_handle, CURLOPT_DEBUGFUNCTION, rt_curl_debug_callback);
curl_easy_setopt(lambda_runtime::m_curl_handle, CURLOPT_VERBOSE, 1);
curl_easy_setopt(lambda_runtime::m_curl_handle, CURLOPT_DEBUGFUNCTION, rt_curl_debug_callback);
#endif
}

runtime::next_outcome runtime::get_next()
{
http::response resp;
set_curl_next_options();
curl_easy_setopt(m_curl_handle, CURLOPT_WRITEDATA, &resp);
curl_easy_setopt(m_curl_handle, CURLOPT_HEADERDATA, &resp);
curl_easy_setopt(lambda_runtime::m_curl_handle, CURLOPT_WRITEDATA, &resp);
curl_easy_setopt(lambda_runtime::m_curl_handle, CURLOPT_HEADERDATA, &resp);

curl_slist* headers = nullptr;
headers = curl_slist_append(headers, m_user_agent_header.c_str());
curl_easy_setopt(m_curl_handle, CURLOPT_HTTPHEADER, headers);
curl_easy_setopt(lambda_runtime::m_curl_handle, CURLOPT_HTTPHEADER, headers);

logging::log_debug(LOG_TAG, "Making request to %s", m_endpoints[Endpoints::NEXT].c_str());
CURLcode curl_code = curl_easy_perform(m_curl_handle);
const CURLcode curl_code = curl_easy_perform(lambda_runtime::m_curl_handle);
logging::log_debug(LOG_TAG, "Completed request to %s", m_endpoints[Endpoints::NEXT].c_str());
curl_slist_free_all(headers);

Expand All @@ -255,13 +265,13 @@ runtime::next_outcome runtime::get_next()

{
long resp_code;
curl_easy_getinfo(m_curl_handle, CURLINFO_RESPONSE_CODE, &resp_code);
curl_easy_getinfo(lambda_runtime::m_curl_handle, CURLINFO_RESPONSE_CODE, &resp_code);
resp.set_response_code(static_cast<aws::http::response_code>(resp_code));
}

{
char* content_type = nullptr;
curl_easy_getinfo(m_curl_handle, CURLINFO_CONTENT_TYPE, &content_type);
curl_easy_getinfo(lambda_runtime::m_curl_handle, CURLINFO_CONTENT_TYPE, &content_type);
resp.set_content_type(content_type);
}

Expand Down Expand Up @@ -312,7 +322,7 @@ runtime::next_outcome runtime::get_next()
if (out.is_success()) {
auto const& deadline_string = std::move(out).get_result();
constexpr int base = 10;
unsigned long ms = strtoul(deadline_string.c_str(), nullptr, base);
const unsigned long ms = strtoul(deadline_string.c_str(), nullptr, base);
assert(ms > 0);
assert(ms < ULONG_MAX);
req.deadline += std::chrono::milliseconds(ms);
Expand Down Expand Up @@ -343,7 +353,7 @@ runtime::post_outcome runtime::do_post(
invocation_response const& handler_response)
{
set_curl_post_result_options();
curl_easy_setopt(m_curl_handle, CURLOPT_URL, url.c_str());
curl_easy_setopt(lambda_runtime::m_curl_handle, CURLOPT_URL, url.c_str());
logging::log_info(LOG_TAG, "Making request to %s", url.c_str());

curl_slist* headers = nullptr;
Expand All @@ -368,11 +378,11 @@ runtime::post_outcome runtime::do_post(

std::pair<std::string const&, size_t> ctx{payload, 0};
aws::http::response resp;
curl_easy_setopt(m_curl_handle, CURLOPT_WRITEDATA, &resp);
curl_easy_setopt(m_curl_handle, CURLOPT_HEADERDATA, &resp);
curl_easy_setopt(m_curl_handle, CURLOPT_READDATA, &ctx);
curl_easy_setopt(m_curl_handle, CURLOPT_HTTPHEADER, headers);
CURLcode curl_code = curl_easy_perform(m_curl_handle);
curl_easy_setopt(lambda_runtime::m_curl_handle, CURLOPT_WRITEDATA, &resp);
curl_easy_setopt(lambda_runtime::m_curl_handle, CURLOPT_HEADERDATA, &resp);
curl_easy_setopt(lambda_runtime::m_curl_handle, CURLOPT_READDATA, &ctx);
curl_easy_setopt(lambda_runtime::m_curl_handle, CURLOPT_HTTPHEADER, headers);
const CURLcode curl_code = curl_easy_perform(lambda_runtime::m_curl_handle);
curl_slist_free_all(headers);

if (curl_code != CURLE_OK) {
Expand All @@ -386,7 +396,7 @@ runtime::post_outcome runtime::do_post(
}

long http_response_code;
curl_easy_getinfo(m_curl_handle, CURLINFO_RESPONSE_CODE, &http_response_code);
curl_easy_getinfo(lambda_runtime::m_curl_handle, CURLINFO_RESPONSE_CODE, &http_response_code);

if (!is_success(aws::http::response_code(http_response_code))) {
logging::log_error(
Expand Down Expand Up @@ -452,7 +462,7 @@ void run_handler(std::function<invocation_response(invocation_request const&)> c

auto const req = std::move(next_outcome).get_result();
logging::log_info(LOG_TAG, "Invoking user handler");
invocation_response res = handler(req);
const invocation_response res = handler(req);
logging::log_info(LOG_TAG, "Invoking user handler completed.");

if (res.is_success()) {
Expand Down Expand Up @@ -480,7 +490,7 @@ static std::string json_escape(std::string const& in)
constexpr char last_non_printable_character = 31;
std::string out;
out.reserve(in.length()); // most strings will end up identical
for (char ch : in) {
for (const char ch : in) {
if (ch > last_non_printable_character && ch != '\"' && ch != '\\') {
out.append(1, ch);
}
Expand Down
7 changes: 2 additions & 5 deletions tests/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,6 @@ cmake_minimum_required(VERSION 3.11)
project(aws-lambda-runtime-tests LANGUAGES CXX)

include(FetchContent)
if(POLICY CMP0135)
cmake_policy(SET CMP0135 NEW)
endif()
FetchContent_Declare(gtest
URL https://github.com/google/googletest/archive/v1.12.0.tar.gz
)
Expand All @@ -14,7 +11,8 @@ set(INSTALL_GTEST OFF)
FetchContent_MakeAvailable(gtest)

add_executable(unit_tests
unit/no_op_test.cpp)
unit/thread_local_curl_test.cpp
unit/unit_tests.cpp)
target_link_libraries(unit_tests PRIVATE gtest_main aws-lambda-runtime)

include(GoogleTest)
Expand All @@ -23,6 +21,5 @@ gtest_discover_tests(unit_tests
LABELS "unit"
DISCOVERY_TIMEOUT 10)


add_subdirectory(resources)

7 changes: 7 additions & 0 deletions tests/unit/main.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#include "gtest/gtest.h"

int main(int argc, char** argv)
{
::testing::InitGoogleTest(&argc, argv);
return RUN_ALL_TESTS();
}
6 changes: 0 additions & 6 deletions tests/unit/no_op_test.cpp
View file
Open in desktop

This file was deleted.

Loading
Loading