Skip to content

*** PermissionError: [Errno 13] Permission denied: '/LeanCLI/__pycache__' #515

Open

Description

@shawnngtq

Context

I'm testing lean-cli using starter code, during backtest encounter *** PermissionError: [Errno 13] Permission denied: '/LeanCLI/pycache'

Setup

# confirm login
x lean whoami
You are logged in as myUsername (myEmailAddress@gmail.com)

# create project
x lean project-create test_qc

# directory
x ls
data  lean.json  storage  test_qc

Error during backtest

x lean backtest test_qc --verbose
Label 'strict_python_version' found in image 'quantconnect/lean', value 3.11.7
Label 'python_version' found in image 'quantconnect/lean', value 3.11
Label 'target_framework' found in image 'quantconnect/lean', value net6.0
Label 'lean_version' found in image 'quantconnect/lean', value 16712
Context information:
  Python version: 3.12.3 | packaged by conda-forge | (main, Apr 15 2024, 18:38:13) [GCC 12.3.0]
  OS: Linux-5.15.0-124-generic-x86_64-with-glibc2.35
  Lean CLI version: 1.0.205
  .NET version: Not installed
  VS Code version: 1.94.2-384ff7382de624fb94dbaf6da11977bba1ecd427-x64
  VS Code installed versions: asciidoctor.asciidoctor-vscode@3.4.1
                              batisteo.vscode-django@1.15.0
                              davidanson.vscode-markdownlint@0.56.0
                              denoland.vscode-deno@3.42.0
                              dotjoshjohnson.xml@2.5.1
                              ducfilan.pug-formatter@0.6.0
                              esbenp.prettier-vscode@11.0.0
                              formulahendry.auto-close-tag@0.5.15
                              formulahendry.code-runner@0.12.2
                              foxundermoon.shell-format@7.2.5
                              golang.go@0.42.1
                              lextudio.restructuredtext@190.4.2
                              mechatroner.rainbow-csv@3.12.0
                              ms-azuretools.vscode-docker@1.29.3
                              ms-dotnettools.csharp@2.50.27
                              ms-dotnettools.vscode-dotnet-runtime@2.2.1
                              ms-python.autopep8@2024.0.0
                              ms-python.black-formatter@2024.4.0
                              ms-python.debugpy@2024.12.0
                              ms-python.isort@2023.10.1
                              ms-python.python@2024.16.1
                              ms-python.vscode-pylance@2024.10.1
                              ms-toolsai.jupyter@2024.9.1
                              ms-toolsai.jupyter-keymap@1.1.2
                              ms-toolsai.jupyter-renderers@1.0.19
                              ms-toolsai.vscode-jupyter-cell-tags@0.1.9
                              ms-toolsai.vscode-jupyter-slideshow@0.1.6
                              ms-vscode-remote.remote-containers@0.388.0
                              ms-vscode.powershell@2024.2.2
                              njpwerner.autodocstring@0.6.1
                              rust-lang.rust-analyzer@0.3.2154
                              streetsidesoftware.code-spell-checker@3.0.1
                              timonwong.shellcheck@0.37.1
                              trond-snekvik.simple-rst@1.5.4
                              vscode-icons-team.vscode-icons@12.9.0
                              yzane.markdown-pdf@1.5.0
                              yzhang.markdown-all-in-one@3.6.2
--> POST https://www.quantconnect.com/api/v2/organizations/list
Request response: 
{"organizations":[{"id":"myOrganizationId","name":"myUsername","type":"Free","ownerName":"You","members":1,"selected":false,"skylightSync":1,"preferred":true,"profileImage":"https:\/\/cdn.quantconnect.com\/i\/tu\/o
rganization-default-icon.png"}],"success":true}
--> POST https://www.quantconnect.com/api/v2/organizations/read with data:
{
    "organizationId": "myOrganizationId"
}
Request response: 
{"organization":{"id":"myOrganizationId","allowSkylightSync":1,"seats":1,"type":"free","billing":null,"credit":{"movements":[],"movementsCount":0,"balance":0,"automaticCreditUsage":0,"automaticCreditTopUp":0,"
creditTopUpMin":500,"creditTopUpAmount":2000},"subscriptions":[{"id":524828,"name":"B-MICRO","price":0,"start":"2024-10-14 10:02:49"},{"id":524829,"name":"R-MICRO","price":0,"start":"2024-10-14 
10:02:49"}],"products":[{"name":"Backtest Node","icon":"https:\/\/cdn.quantconnect.com\/i\/tu\/backtest-node-icon-v1.0.png","items":[{"productId":30,"name":"B-MICRO","quantity":1,"unitPrice":0,"total":0}]},{"name":"Research 
Node","icon":"https:\/\/cdn.quantconnect.com\/i\/tu\/research-node-icon-v1.0.png","items":[{"productId":29,"name":"R-MICRO","quantity":1,"unitPrice":0,"total":0}]}],"invoices":[],"invoicesCount":0,"about":"","name":"myUsername","s
upportSeats":{"bronze":{"seats":0,"unassigned":0},"silver":{"seats":0,"unassigned":0},"gold":{"seats":0,"unassigned":0}},"shortDescription":"Write a short 
description","statistics":[{"name":"ComputeHours","value":"-"},{"name":"Lines","value":"20"},{"name":"Projects","value":"1"},{"name":"Backtests","value":"1"},{"name":"Members","value":"1"}],"profileImage":"https:\/\/cdn.quant
connect.com\/i\/tu\/organization-default-icon.png","useResourcesListView":false,"useMembersListView":false,"isUsingPaidFeatures":false,"isAbleToDowngrade":true,"nodes":{"backtest":[{"speed":3,"price":{"monthly":0,"yearly":0},
"cpu":1,"ram":7.7999999999999998,"name":"Community B-MICRO","sku":"B-MICRO","description":"1 CPU @ 3.0GHz, 7.8GB 
Ram","usedBy":"-","userProfile":"","projectName":"","projectId":null,"busy":false,"id":"BN-67b249b967e2c078bd1a3d2b862e5c8e","assets":100,"host":null,"hasGpu":0}],"research":[{"speed":2.3999999999999999,"price":{"monthly":0,"
yearly":0},"cpu":1,"ram":2,"name":"Community R-MICRO","sku":"R-MICRO","description":"1 CPU @ 2.4GHz, 2GB 
Ram","usedBy":"-","userProfile":"","projectName":"","projectId":null,"busy":false,"id":"RN-aaa768b110fb79d5697138ba8a75d64f","assets":100,"host":null,"hasGpu":0}],"live":[]},"environment":{"files":0,"bytes":4096,"size":"4 
KB"},"data":{"signedTime":null,"current":false},"objectStore":[{"id":24,"name":"S-MICRO","price":0,"bytes":52428800,"files":1000,"storage":"50 
MB","active":true},{"id":25,"name":"S-10","price":12,"bytes":2147483648,"files":20000,"storage":"2 GB","active":false},{"id":26,"name":"S-20","price":24,"bytes":5368709120,"files":50000,"storage":"5 
GB","active":false},{"id":27,"name":"S-50","price":60,"bytes":10737418240,"files":100000,"storage":"10 GB","active":false},{"id":28,"name":"S-100","price":120,"bytes":53687091200,"files":500000,"storage":"50 
GB","active":false}],"objectStoreExportDerivative":false,"objectStoreHighestConsumers":[],"objectStorageUsed":4096,"objectStorageUsedHuman":"4 
KB","members":[{"id":313917,"name":"myUsername","isAdmin":true,"isRequester":true,"profile":"https:\/\/cdn.quantconnect.com\/web\/i\/users\/thumbnail\/100\/default.jpg","editable":false,"support":null,"linkedin":null,"lastLoginTim
estamp":"2024-10-27 00:47:00","email":"myEmailAddress@gmail.com","profileUrl":"\/u\/myUsername_13","publicId":"myUsername_13","online":true,"permissions":{"updateBilling":{"name":"Update 
Billing","value":true},"researchNodeStop":{"name":"Stop Research Nodes","value":true},"researchNodeCreate":{"name":"Create Research Nodes","value":true},"researchNodeDelete":{"name":"Delete Research 
Nodes","value":true},"liveNodeStop":{"name":"Stop Live Nodes","value":true},"liveNodeCreate":{"name":"Create Live Nodes","value":true},"liveNodeDelete":{"name":"Delete Live 
Nodes","value":true},"backtestNodeStop":{"name":"Stop Backtest Nodes","value":true},"backtestNodeCreate":{"name":"Create Backtest Nodes","value":true},"backtestNodeDelete":{"name":"Delete Backtest 
Nodes","value":true},"teamAdd":{"name":"Add Team Members to The Organization","value":true},"teamEdit":{"name":"Edit Team Members Permissions","value":false},"teamRemove":{"name":"Remove Team 
Members","value":true},"storageBilling":{"name":"Manage Storage Billing","value":true},"storageCreate":{"name":"Create Storage Files","value":true},"storageDelete":{"name":"Delete Storage 
Files","value":true}}}],"alpha":null,"courses":{"coursesCont":4,"enrolledMembers":0,"completedTasks":0,"courses":[{"id":1,"name":"Boot Camp 101 \/ US 
Equities","authorName":"QuantConnect","thumbnail":"https:\/\/cdn.quantconnect.com\/i\/tu\/bootcamp-101-thumbnail-image3.jpg","shortDescription":"Learn algorithmic trading with python for US Equities. Guided strategy 
development in easily digestible portions.","members":[]},{"id":2,"name":"Boot Camp 102 \/ 
FOREX","authorName":"QuantConnect","thumbnail":"https:\/\/cdn.quantconnect.com\/i\/tu\/bootcamp-102-thumbnail-image3.jpg","shortDescription":"Learn algorithmic trading with python for FX. Guided strategy development in easily
digestible portions.","members":[]},{"id":3,"name":"Boot Camp 103 \/ Futures","authorName":"QuantConnect","thumbnail":"https:\/\/cdn.quantconnect.com\/i\/tu\/bootcamp-103-thumbnail-image3.jpg","shortDescription":"Learn 
algorithmic trading with python for Futures. Guided strategy development in easily digestible portions.","members":[]},{"id":5,"name":"Algorithmic Trading A-Z | The Complete 
Course","authorName":"Louis","thumbnail":"https:\/\/cdn.quantconnect.com\/i\/tu\/python-for-algorithmic-trading-thumbnail-image.jpg","shortDescription":"In this algorithmic trading tutorial series you will learn everything 
you need to know to start writing your own trading bots using Python and the QuantConnect quantitative trading 
platform.","members":[]}]},"channel":"ORGac335894ff-myOrganizationId","minimumTransactionSizeLimit":0,"billingFrequency":"monthly","billingStatus":null,"billingPausedUntil":null,"hasCardOnFile":false,"joinOrga
nizationLink":"https:\/\/www.quantconnect.com\/invite?t=organization&o=myOrganizationId&k=a1b8343ada8ef23bbb2185de8d01cd191f81d9dd","outOfSampleDays":90,"trial":null,"contracts":{"institution":null}},"success"
:true}
logged in as myUsername (myEmailAddress@gmail.com)
Label 'lean_version' found in image 'quantconnect/lean', value 16712
Found module 'local' from given name
Configuring Local
_update_settings(Local): Settings: {'id': 'Local', 'data-provider': 'QuantConnect.Lean.Engine.DataFeeds.DefaultDataProvider'}
LeanRunner().run_lean: lean_config: {'data-folder': 'data', 'log-handler': 'QuantConnect.Logging.CompositeLogHandler', 'messaging-handler': 'QuantConnect.Messaging.Messaging', 'job-queue-handler': 
'QuantConnect.Queues.JobQueue', 'api-handler': 'QuantConnect.Api.Api', 'map-file-provider': 'QuantConnect.Data.Auxiliary.LocalDiskMapFileProvider', 'factor-file-provider': 
'QuantConnect.Data.Auxiliary.LocalDiskFactorFileProvider', 'data-provider': 'QuantConnect.Lean.Engine.DataFeeds.DefaultDataProvider', 'data-channel-provider': 'DataChannelProvider', 'object-store': 
'QuantConnect.Lean.Engine.Storage.LocalObjectStore', 'data-aggregator': 'QuantConnect.Lean.Engine.DataFeeds.AggregationManager', 'symbol-minute-limit': 10000, 'symbol-second-limit': 10000, 'symbol-tick-limit': 10000, 
'show-missing-data-logs': False, 'maximum-warmup-history-days-look-back': 5, 'maximum-data-points-per-chart-series': 1000000, 'maximum-chart-series': 30, 'force-exchange-always-open': False, 'transaction-log': '', 
'reserved-words-prefix': '@', 'live-data-url': 'ws://www.quantconnect.com/api/v2/live/data/', 'live-data-port': 8020, 'live-cash-balance': '', 'live-holdings': '[]', 'ib-account': '', 'ib-user-name': '', 'ib-password': '', 
'ib-agent-description': 'Individual', 'ib-trading-mode': 'paper', 'ib-enable-delayed-streaming-data': False, 'ib-weekly-restart-utc-time': '22:00:00', 'tradier-environment': 'paper', 'tradier-account-id': '', 
'tradier-access-token': '', 'oanda-environment': 'Practice', 'oanda-access-token': '', 'oanda-account-id': '', 'fxcm-server': 'http://www.fxcorporate.com/Hosts.jsp', 'fxcm-terminal': 'Demo', 'fxcm-user-name': '', 
'fxcm-password': '', 'fxcm-account-id': '', 'iqfeed-iqconnect': 'C:/Program Files (x86)/DTN/IQFeed/iqconnect.exe', 'iqfeed-username': '', 'iqfeed-password': '', 'iqfeed-productName': '', 'iqfeed-version': '1.0', 
'coinbase-rest-api': 'https://api.coinbase.com', 'coinbase-url': 'wss://advanced-trade-ws.coinbase.com', 'coinbase-api-key': '', 'coinbase-api-secret': '', 'bitfinex-api-secret': '', 'bitfinex-api-key': '', 
'binance-api-secret': '', 'binance-api-key': '', 'binance-api-url': 'https://api.binance.com', 'binance-websocket-url': 'wss://stream.binance.com:9443/ws', 'binance-fapi-url': 'https://fapi.binance.com', 
'binance-fwebsocket-url': 'wss://fstream.binance.com/ws', 'binance-dapi-url': 'https://dapi.binance.com', 'binance-dwebsocket-url': 'wss://dstream.binance.com/ws', 'binanceus-api-secret': '', 'binanceus-api-key': '', 
'binanceus-api-url': 'https://api.binance.us', 'binanceus-websocket-url': 'wss://stream.binance.us:9443/ws', 'bybit-api-secret': '', 'bybit-api-key': '', 'bybit-api-url': 'https://api.bybit.com', 'bybit-websocket-url': 
'wss://stream.bybit.com', 'eze-api-address-url': '', 'eze-port': '', 'eze-domain': '', 'eze-locale': '', 'eze-password': '', 'eze-user-name': '', 'kraken-api-secret': '', 'kraken-api-key': '', 'kraken-verification-tier': 
'Starter', 'wolverine-host': '', 'wolverine-port': '', 'wolverine-account': '', 'wolverine-sender-comp-id': '', 'wolverine-target-comp-id': '', 'wolverine-on-behalf-of-comp-id': '', 'wolverine-log-fix-messages': False, 
'tdameritrade-account-number': '', 'tdameritrade-api-key': '', 'tdameritrade-access-token': '', 'rbi-host': '', 'rbi-port': '', 'rbi-account': '', 'rbi-sender-comp-id': '', 'rbi-target-comp-id': '', 
'rbi-on-behalf-of-comp-id': '', 'rbi-log-fix-messages': False, 'tt-user-name': '', 'tt-session-password': '', 'tt-account-name': '', 'tt-rest-app-key': '', 'tt-rest-app-secret': '', 'tt-rest-environment': '', 
'tt-market-data-sender-comp-id': '', 'tt-market-data-target-comp-id': '', 'tt-market-data-host': '', 'tt-market-data-port': '', 'tt-order-routing-sender-comp-id': '', 'tt-order-routing-target-comp-id': '', 
'tt-order-routing-host': '', 'tt-order-routing-port': '', 'tt-log-fix-messages': False, 'trade-station-client-id': '', 'trade-station-client-secret': '', 'trade-station-redirect-url': 'http://localhost', 
'trade-station-refresh-token': '', 'trade-station-api-url': 'https://sim-api.tradestation.com', 'trade-station-account-type': 'Cash|Margin|Futures', 'trade-station-account-id': '', 'alpaca-api-key': '', 'alpaca-api-secret': 
'', 'alpaca-access-token': '', 'alpaca-paper-trading': True, 'exante-client-id': '', 'exante-application-id': '', 'exante-shared-key': '', 'exante-account-id': '', 'exante-platform-type': '', 'nasdaq-auth-token': '', 
'tiingo-auth-token': '', 'us-energy-information-auth-token': '', 'iex-cloud-api-key': '', 'coinapi-api-key': '', 'coinapi-product': 'free', 'polygon-api-key': '', 'zerodha-access-token': '', 'zerodha-api-key': '', 
'zerodha-product-type': 'MIS', 'zerodha-trading-segment': 'EQUITY', 'zerodha-history-subscription': 'false', 'samco-client-id': '', 'samco-client-password': '', 'samco-year-of-birth': '', 'samco-product-type': 'MIS', 
'samco-trading-segment': 'EQUITY', 'ftx-account-tier': 'Tier1', 'ftx-api-secret': '', 'ftx-api-key': '', 'ftxus-account-tier': 'Tier1', 'ftxus-api-secret': '', 'ftxus-api-key': '', 'regression-test-languages': ['CSharp', 
'Python'], 'python-additional-paths': [], 'environments': {'backtesting': {'live-mode': False, 'setup-handler': 'QuantConnect.Lean.Engine.Setup.BacktestingSetupHandler', 'result-handler': 
'QuantConnect.Lean.Engine.Results.BacktestingResultHandler', 'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.FileSystemDataFeed', 'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.BacktestingRealTimeHandler', 
'history-provider': ['QuantConnect.Lean.Engine.HistoricalData.SubscriptionDataReaderHistoryProvider'], 'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BacktestingTransactionHandler', 'id': 'Local', 
'data-provider': 'QuantConnect.Lean.Engine.DataFeeds.DefaultDataProvider'}, 'live-paper': {'live-mode': True, 'live-mode-brokerage': 'PaperBrokerage', 'setup-handler': 'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 
'result-handler': 'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 'data-queue-handler': 
['QuantConnect.Lean.Engine.DataFeeds.Queues.LiveDataQueue'], 'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 'transaction-handler': 
'QuantConnect.Lean.Engine.TransactionHandlers.BacktestingTransactionHandler'}, 'live-zerodha': {'live-mode': True, 'live-mode-brokerage': 'ZerodhaBrokerage', 'data-queue-handler': ['ZerodhaBrokerage'], 'setup-handler': 
'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 
'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': 
['BrokerageHistoryProvider', 'SubscriptionDataReaderHistoryProvider']}, 'live-samco': {'live-mode': True, 'live-mode-brokerage': 'SamcoBrokerage', 'data-queue-handler': ['SamcoBrokerage'], 'setup-handler': 
'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 
'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': 
['BrokerageHistoryProvider', 'SubscriptionDataReaderHistoryProvider']}, 'live-tradier': {'live-mode': True, 'tradier-save-tokens': True, 'live-mode-brokerage': 'TradierBrokerage', 'data-queue-handler': ['TradierBrokerage'], 
'setup-handler': 'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 
'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 'transaction-handler': 
'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': ['BrokerageHistoryProvider', 'SubscriptionDataReaderHistoryProvider']}, 'live-interactive': {'live-mode': True, 
'live-mode-brokerage': 'InteractiveBrokersBrokerage', 'setup-handler': 'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 
'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 'data-queue-handler': ['InteractiveBrokersBrokerage'], 'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 
'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': ['BrokerageHistoryProvider', 'SubscriptionDataReaderHistoryProvider']}, 'live-interactive-iqfeed': 
{'live-mode': True, 'live-mode-brokerage': 'InteractiveBrokersBrokerage', 'setup-handler': 'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler',
'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 'data-queue-handler': ['QuantConnect.IQFeed.IQFeedDataQueueHandler'], 'real-time-handler': 
'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': 
['QuantConnect.IQFeed.IQFeedDataQueueHandler', 'SubscriptionDataReaderHistoryProvider']}, 'live-fxcm': {'live-mode': True, 'live-mode-brokerage': 'FxcmBrokerage', 'data-queue-handler': ['FxcmBrokerage'], 'setup-handler': 
'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 
'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': 
['BrokerageHistoryProvider', 'SubscriptionDataReaderHistoryProvider']}, 'live-oanda': {'live-mode': True, 'live-mode-brokerage': 'OandaBrokerage', 'data-queue-handler': ['OandaBrokerage'], 'setup-handler': 
'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 
'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': 
['BrokerageHistoryProvider', 'SubscriptionDataReaderHistoryProvider']}, 'live-coinbase': {'live-mode': True, 'live-mode-brokerage': 'CoinbaseBrokerage', 'data-queue-handler': ['CoinbaseBrokerage'], 'setup-handler': 
'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 
'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': 
['BrokerageHistoryProvider', 'SubscriptionDataReaderHistoryProvider']}, 'live-bitfinex': {'live-mode': True, 'live-mode-brokerage': 'BitfinexBrokerage', 'data-queue-handler': ['BitfinexBrokerage'], 'setup-handler': 
'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 
'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': 
['BrokerageHistoryProvider', 'SubscriptionDataReaderHistoryProvider']}, 'live-binance': {'live-mode': True, 'live-mode-brokerage': 'BinanceBrokerage', 'data-queue-handler': ['BinanceBrokerage'], 'setup-handler': 
'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 
'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': 
['BrokerageHistoryProvider', 'SubscriptionDataReaderHistoryProvider']}, 'live-futures-binance': {'live-mode': True, 'live-mode-brokerage': 'BinanceFuturesBrokerage', 'data-queue-handler': ['BinanceFuturesBrokerage'], 
'setup-handler': 'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 
'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 'transaction-handler': 
'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': ['BrokerageHistoryProvider', 'SubscriptionDataReaderHistoryProvider']}, 'live-coin-futures-binance': {'live-mode': True, 
'live-mode-brokerage': 'BinanceCoinFuturesBrokerage', 'data-queue-handler': ['BinanceCoinFuturesBrokerage'], 'setup-handler': 'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 
'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 
'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': ['BrokerageHistoryProvider', 'SubscriptionDataReaderHistoryProvider']}, 'live-binanceus': {'live-mode': 
True, 'live-mode-brokerage': 'BinanceUSBrokerage', 'data-queue-handler': ['BinanceUSBrokerage'], 'setup-handler': 'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 
'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 
'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': ['BrokerageHistoryProvider', 'SubscriptionDataReaderHistoryProvider']}, 'live-bybit': {'live-mode': True, 
'live-mode-brokerage': 'BybitBrokerage', 'data-queue-handler': ['BybitBrokerage'], 'setup-handler': 'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 
'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 
'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': ['BrokerageHistoryProvider', 'SubscriptionDataReaderHistoryProvider']}, 'live-trade-station': 
{'live-mode': True, 'live-mode-brokerage': 'TradeStationBrokerage', 'data-queue-handler': ['TradeStationBrokerage'], 'setup-handler': 'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 
'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 
'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': ['BrokerageHistoryProvider', 'SubscriptionDataReaderHistoryProvider']}, 'live-alpaca': {'live-mode': True,
'live-mode-brokerage': 'AlpacaBrokerage', 'data-queue-handler': ['AlpacaBrokerage'], 'setup-handler': 'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 
'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 
'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': ['BrokerageHistoryProvider', 'SubscriptionDataReaderHistoryProvider']}, 'live-futures-bybit': 
{'live-mode': True, 'live-mode-brokerage': 'BybitFuturesBrokerage', 'data-queue-handler': ['BybitFuturesBrokerage'], 'setup-handler': 'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 
'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 
'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': ['BrokerageHistoryProvider', 'SubscriptionDataReaderHistoryProvider']}, 'live-trading-technologies': 
{'live-mode': True, 'live-mode-brokerage': 'TradingTechnologiesBrokerage', 'data-queue-handler': ['TradingTechnologiesBrokerage'], 'setup-handler': 'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 
'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 
'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler'}, 'live-kraken': {'live-mode': True, 'live-mode-brokerage': 'KrakenBrokerage', 'data-queue-handler': ['KrakenBrokerage'], 
'setup-handler': 'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 
'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 'transaction-handler': 
'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': ['BrokerageHistoryProvider', 'SubscriptionDataReaderHistoryProvider']}, 'live-ftx': {'live-mode': True, 'live-mode-brokerage': 
'QuantConnect.FTXBrokerage.FTXBrokerage', 'data-queue-handler': ['QuantConnect.FTXBrokerage.FTXBrokerage'], 'setup-handler': 'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 
'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 
'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': ['BrokerageHistoryProvider', 'SubscriptionDataReaderHistoryProvider']}, 'live-exante': {'live-mode': True,
'live-mode-brokerage': 'QuantConnect.ExanteBrokerage.ExanteBrokerage', 'data-queue-handler': ['QuantConnect.ExanteBrokerage.ExanteBrokerage'], 'setup-handler': 'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 
'result-handler': 'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 'real-time-handler': 
'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': 'BrokerageHistoryProvider'}, 'live-ftxus':
{'live-mode': True, 'live-mode-brokerage': 'QuantConnect.FTXBrokerage.FTXUSBrokerage', 'data-queue-handler': ['QuantConnect.FTXBrokerage.FTXUSBrokerage'], 'setup-handler': 
'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 
'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': 
['BrokerageHistoryProvider', 'SubscriptionDataReaderHistoryProvider']}, 'live-wolverine': {'live-mode': True, 'live-mode-brokerage': 'WolverineBrokerage', 'history-provider': ['SubscriptionDataReaderHistoryProvider'], 
'setup-handler': 'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 
'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 'transaction-handler': 
'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler'}, 'live-tdameritrade': {'live-mode': True, 'live-mode-brokerage': 'TDAmeritradeBrokerage', 'history-provider': ['BrokerageHistoryProvider', 
'SubscriptionDataReaderHistoryProvider'], 'setup-handler': 'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 
'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 'transaction-handler': 
'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'data-queue-handler': ['TDAmeritradeBrokerage']}, 'live-rbi': {'live-mode': True, 'live-mode-brokerage': 'RBIBrokerage', 'data-queue-handler': 
['QuantConnect.Lean.Engine.DataFeeds.Queues.LiveDataQueue'], 'history-provider': ['SubscriptionDataReaderHistoryProvider'], 'setup-handler': 'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 
'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 
'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler'}, 'live-eze': {'live-mode': True, 'live-mode-brokerage': 'QuantConnect.EzeBrokerage.EzeBrokerage', 'data-queue-handler': 
['QuantConnect.EzeBrokerage.EzeBrokerage'], 'setup-handler': 'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 
'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 'transaction-handler': 
'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': ['BrokerageHistoryProvider', 'SubscriptionDataReaderHistoryProvider']}}, 'job-organization-id': 
'myOrganizationId', 'organization-id': 'myOrganizationId', 'file-database-last-update': '10/27/2024', 'id': 'Local', 'environment': 'backtesting', 'close-automatically': True, 
'composer-dll-directory': '.', 'debugging': False, 'debugging-method': 'LocalCmdline', 'job-user-id': '317654', 'api-access-token': '2e868a1e2afb0d1c75751f066bb0a1641203fb17567303489de402062d271a73', 'ib-host': '127.0.0.1', 
'ib-port': '4002', 'ib-tws-dir': '/root/Jts', 'ib-version': '985', 'iqfeed-host': 'host.docker.internal', 'algorithm-type-name': 'main', 'algorithm-language': 'Python', 'algorithm-location': '/LeanCLI/main.py', 'parameters': 
{}, 'storage-limit-mb': '9999999', 'storage-file-count': '9999999', 'algorithm-id': '1239616393'}
Running 'quantconnect/lean:latest' with the following configuration:
{
    'environment': {},
    'stop_signal': 'SIGINT',
    'mounts': [
        {'Target': '/Lean/Launcher/bin/Debug/config.json', 'Source': '/tmp/lean-cli-00e1qj2o/config.json', 'Type': 'bind', 'ReadOnly': True},
        {'Target': '/lean-cli-start.sh', 'Source': '/tmp/lean-cli-9c4gg8_v/lean-cli-start.sh', 'Type': 'bind', 'ReadOnly': True}
    ],
    'volumes': {
        '/myFilePath/test_qc': {'bind': '/LeanCLI', 'mode': 'rw'},
        '/myFilePath/data': {'bind': '/Lean/Data', 'mode': 'rw'},
        '/myFilePath/test_qc/backtests/2024-10-27_08-47-16': {'bind': '/Results', 'mode': 'rw'},
        '/myFilePath/storage': {'bind': '/Storage', 'mode': 'rw'}
    },
    'ports': {},
    'name': 'lean_cli_dd26966306e6407f96bb77f5f74707be',
    'entrypoint': ['bash', '/lean-cli-start.sh'],
    'detach': True,
    'hostname': 'thanos-Standard-PC-Q35-ICH9-2009',
    'tty': True,
    'stdin_open': True,
    'extra_hosts': {'host.docker.internal': '172.17.0.1'},
    'network': 'lean_cli'
}
+ '[' -d /LeanCLI ']'
+ python -m compileall /LeanCLI/main.py /LeanCLI/research.ipynb
Compiling '/LeanCLI/main.py'...
*** PermissionError: [Errno 13] Permission denied: '/LeanCLI/__pycache__'
Traceback (most recent call last):
  File "/home/thanos/miniconda3/envs/globalfintel/lib/python3.12/site-packages/lean/main.py", line 94, in main
    lean.main(standalone_mode=False)
  File "/home/thanos/miniconda3/envs/globalfintel/lib/python3.12/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/thanos/miniconda3/envs/globalfintel/lib/python3.12/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/thanos/miniconda3/envs/globalfintel/lib/python3.12/site-packages/lean/click.py", line 231, in invoke
    result = super().invoke(ctx)
             ^^^^^^^^^^^^^^^^^^^
  File "/home/thanos/miniconda3/envs/globalfintel/lib/python3.12/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/thanos/miniconda3/envs/globalfintel/lib/python3.12/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/thanos/miniconda3/envs/globalfintel/lib/python3.12/site-packages/lean/commands/backtest.py", line 400, in backtest
    lean_runner.run_lean(lean_config,
  File "/home/thanos/miniconda3/envs/globalfintel/lib/python3.12/site-packages/lean/components/docker/lean_runner.py", line 174, in run_lean
    raise RuntimeError(
RuntimeError: Something went wrong while running 'test_qc' in the 'backtesting' environment, the output is stored in 'test_qc/backtests/2024-10-27_08-47-16'
Error: Something went wrong while running 'test_qc' in the 'backtesting' environment, the output is stored in 'test_qc/backtests/2024-10-27_08-47-16'

Activity

  1. Marinovsky commented on Oct 30, 2024

    @Marinovsky
    Contributor

    Hey, @shawnngtq!

    Thanks for sharing the error. It seems that the issue might be because the __pycache__ folder was created with root user permissions, which the Docker container can’t access.
    Does your project already have a __pycache__ folder? If so, did you use different users while creating and running the backtest? Could you also check the folder's permissions with ls -l ./__pycache__/ ?
    Besides, you could also try deleting the folder and running it again.

  2. shawnngtq commented on Nov 1, 2024

    @shawnngtq
    Author

    @Marinovsky

    The test_qc project is the starter code, I don't make any changes to that. Read 1st post on how I generate that.

    There is no *pycache* found in that directory.

    PermissionError: [Errno 13] Permission denied: '/LeanCLI/__pycache__'

    From my understanding /LeanCLI is the container's home directory, thus the issue is during the backtest. I believe the __pycache__ is generated via python -m compileall /LeanCLI/main.py /LeanCLI/research.ipynb?

    x find . -iname '*pycache*'
    x ls
    data  lean.json  storage  test_qc
    x tree test_qc 
    test_qc
    ├── backtests
    │   ├── 2024-10-26_15-36-38
    │   │   ├── code
    │   │   │   ├── main.py
    │   │   │   └── research.ipynb
    │   │   └── config
    │   ├── 2024-10-27_08-40-53
    │   │   ├── code
    │   │   │   ├── main.py
    │   │   │   └── research.ipynb
    │   │   └── config
    │   ├── 2024-10-27_08-47-16
    │   │   ├── code
    │   │   │   ├── main.py
    │   │   │   └── research.ipynb
    │   │   └── config
    │   ├── 2024-10-27_21-37-54
    │   │   ├── code
    │   │   │   ├── main.py
    │   │   │   └── research.ipynb
    │   │   └── config
    │   ├── 2024-10-27_21-39-30
    │   │   ├── code
    │   │   │   ├── main.py
    │   │   │   └── research.ipynb
    │   │   └── config
    │   ├── 2024-10-27_21-41-25
    │   │   ├── code
    │   │   │   ├── main.py
    │   │   │   └── research.ipynb
    │   │   └── config
    │   └── 2024-10-28_16-28-31
    │       ├── code
    │       │   ├── main.py
    │       │   └── research.ipynb
    │       └── config
    ├── config.json
    ├── main.py
    └── research.ipynb
    
    15 directories, 24 files
    
  3. Martin-Molinero commented on Nov 15, 2024

    @Martin-Molinero
    Member

    Hey @shawnngtq! Sorry we can't seem to pin this issue, is there anything special about the lean cli folder on the host? believe we've seen some issue with mac network folder for example. Have u tried using root user or not? To determine whether that would solve it. Is research working for you or is it only backtesting? Can you try C# default project and backtest?

  4. shawnngtq commented on Nov 16, 2024

    @shawnngtq
    Author

    @Martin-Molinero

    Test case 1 - ensure no unexpected files

    I deleted test_qc and created test_qc2, same error.

    Test case 2 - ensure not permission issue

    chmod 777 to the entire directory, don't work but different error

    Test case 3 - ensure not user issue

    execute as root, don't work but different error

    Test case 4 - literally all you can do for permission issue

    It works with chmod 777 to the entire directory + root user.

    Possible cause

    I guess the problem is with my setup (Linux OS with Linux network folder).

    1st, using my standard user to everything don't work -> I suspect maybe root is required to work with the dotnet 6 installed for LEAN engine (https://www.quantconnect.com/docs/v2/lean-engine/getting-started#04-Installation-Instructions)?

    2nd, 777 -> purely my problem caused by Linux network folder

  5. Martin-Molinero commented on Nov 18, 2024

    @Martin-Molinero
    Member

    I guess the problem is with my setup (Linux OS with Linux network folder)

    Yes I'd bet it's the linux network folder + how docker handles this.
    I'm glad to hear you found a work around for the issue, this will surely help others debug this. We will keep this open so we can document it

  6. added
    documentationImprovements or additions to documentation
    and removed
    bugSomething isn't working
    on Dec 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentation

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions