FileSystemStorageClient writes snake_case data on disk instead of camelCase

[janbuchar]

• crawlee-python/src/crawlee/storage_clients/models.py

  Lines 58 to 74 in 2eda280

  	@docs_group('Storage data')
  	class RequestQueueMetadata(StorageMetadata):
  	"""Model for a request queue metadata."""
  	model_config = ConfigDict(validate_by_name=True, validate_by_alias=True, from_attributes=True)
  	had_multiple_clients: Annotated[bool, Field(alias='hadMultipleClients')]
  	"""Indicates whether the queue has been accessed by multiple clients (consumers)."""
  	handled_request_count: Annotated[int, Field(alias='handledRequestCount')]
  	"""The number of requests that have been handled from the queue."""
  	pending_request_count: Annotated[int, Field(alias='pendingRequestCount')]
  	"""The number of requests that are still pending in the queue."""
  	total_request_count: Annotated[int, Field(alias='totalRequestCount')]
  	"""The total number of requests that have been added to the queue."""

  - models such as RequestQueueMetadata specify camelCase aliases
  • this suggests that we originally intended to maintain the same on-disk format as crawlee-js
  • FileSystemStorageClient calls model_dump without by_alias=True - that means that the files will be snake_case
• I found this while working on the Rust file system storage client
• @vdusek perhaps you remember if this was intentional?

[vdusek]

@vdusek perhaps you remember if this was intentional?

I don't think so, we probably just forgot by_alias=True. Maybe a candidate for v2?

[janbuchar]

@vdusek perhaps you remember if this was intentional?

I don't think so, we probably just forgot by_alias=True. Maybe a candidate for v2?

Yeah, I see it the same way. I'll have crawlee_storage use snakeCase internally then - no need to have two different on-disk formats.