Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 44 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Welcome to the official Python library for Runpod API & SDK.
- [⚡ | Serverless Worker (SDK)](#--serverless-worker-sdk)
- [Quick Start](#quick-start)
- [Local Test Worker](#local-test-worker)
- [📚 | API Language Library (GraphQL Wrapper)](#--api-language-library-graphql-wrapper)
- [📚 | REST API v2 Wrapper](#--rest-api-v2-wrapper)
- [Endpoints](#endpoints)
- [GPU Cloud (Pods)](#gpu-cloud-pods)
- [📁 | Directory](#--directory)
Expand Down Expand Up @@ -162,9 +162,9 @@ with VolumeCache(dirs=["/root/.cache/huggingface"]):

See [Network-Volume Warm Cache](https://github.com/runpod/runpod-python/blob/main/docs/serverless/volume_cache.md) documentation for configuration and details.

## 📚 | API Language Library (GraphQL Wrapper)
## 📚 | REST API v2 Wrapper

When interacting with the Runpod API you can use this library to make requests to the API.
Use the API wrapper to manage Runpod resources through REST API v2.

```python
import runpod
Expand Down Expand Up @@ -281,36 +281,64 @@ import runpod

runpod.api_key = "your_runpod_api_key_found_under_settings"

# Get all my pods
# get all my pods
pods = runpod.get_pods()

# Get a specific pod
pod = runpod.get_pod(pod.id)
# get a specific pod
pod = runpod.get_pod(pods[0]["id"])

# Create a pod with GPU
pod = runpod.create_pod("test", "runpod/stack", "NVIDIA GeForce RTX 3070")
# create a pod with a gpu
pod = runpod.create_pod("test", "runpod/stack", "NVIDIA GeForce RTX 4090")

# Create a pod with CPU
# create a pod with a cpu
pod = runpod.create_pod("test", "runpod/stack", instance_id="cpu3c-2-4")

# Stop the pod
runpod.stop_pod(pod.id)
# stop the pod
runpod.stop_pod(pod["id"])

# Resume the pod
runpod.resume_pod(pod.id)
# resume the pod
runpod.resume_pod(pod["id"])

# Terminate the pod
runpod.terminate_pod(pod.id)
# terminate the pod
runpod.terminate_pod(pod["id"])
```

### Template and placement options

- With `create_pod(template_id=...)`, omitting `docker_args` inherits the template's
command. Pass `docker_args=""` to clear that inherited command.
- Omitting `volume_mount_path` preserves an inherited GPU template volume's path.
Setting it explicitly changes the path while retaining the inherited volume size.
New persistent volumes and network volume mounts default to `/runpod-volume`.
CPU pods do not inherit template persistent volumes; a CPU `volume_mount_path`
requires `network_volume_id`.
- For GPU pods, `min_memory_in_gb` and `min_vcpu_count` specify minimum host RAM
and vCPUs **per GPU**, not GPU VRAM or totals for the pod.
- CPU `instance_id` must use `<cpu-flavor>-<vcpu-count>-<memory>`, with positive
integer vCPU and memory values, for example `cpu3c-4-8`.
- REST v2 cannot require a public-IP-capable host. `support_public_ip=False` is
the default and does not disable public networking. `True` raises `ValueError`
rather than silently ignoring that requirement.
- `create_template(volume_in_gb=0)` creates a template without a persistent volume.
- `create_endpoint(locations="US-KS-2,EU-RO-1")` accepts comma-separated
datacenter IDs. Country codes such as `US` and `RO` are not supported.
Endpoint creation sends a single REST request without catalog lookups.
- Endpoint `gpu_ids` accepts pool IDs and excluded GPU types, for example
`gpu_ids="ADA_48_PRO,-NVIDIA L40"` selects that pool without NVIDIA L40 GPUs.
- `create_endpoint` accepts only `QUEUE_DELAY` and `REQUEST_COUNT` scaling.
`idle_timeout` applies to `QUEUE_DELAY` (default: 5 seconds); explicitly setting
it with `REQUEST_COUNT` raises `ValueError`.
- `resume_pod(pod_id)` keeps the existing GPU allocation. REST v2 does not support
resizing on resume, so providing `gpu_count` raises `ValueError`.

## 📁 | Directory

```BASH
.
├── docs # Documentation
├── examples # Examples
├── runpod # Package source code
│ ├── api_wrapper # Language library - API (GraphQL)
│ ├── api # rest api v2 wrapper
│ ├── cli # Command Line Interface Functions
│ ├── endpoint # Language library - Endpoints
│ └── serverless # SDK - Serverless Worker
Expand Down
26 changes: 22 additions & 4 deletions docs/api/handling_errors.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,28 @@
# Handling Errors
# Handling API errors

```Python
Authentication failures raise `AuthenticationError`:

```python
import runpod

try:
# Use runpod to make a request
runpod.get_pods()
except runpod.error.AuthenticationError as err:
# Authentication with the API failed
print(err)
```

REST API problem responses raise `QueryError`. The exception includes the HTTP
status code, request method and path, and request-validation errors when present:

```python
try:
runpod.create_pod(
"training",
"runpod/pytorch:1.0.2-cu1281-torch280-ubuntu2404",
"NVIDIA GeForce RTX 4090",
)
except runpod.error.QueryError as err:
print(err.status_code)
print(err.query)
print(err.errors)
```
38 changes: 28 additions & 10 deletions docs/api/queries.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,25 +11,43 @@ for gpu in gpus:
print(gpu)
```

### get_gpus Output
### get_gpus output

```json
{'id': 'NVIDIA A100 80GB PCIe', 'displayName': 'A100 80GB', 'memoryInGb': 80}
{'id': 'NVIDIA A100-SXM4-80GB', 'displayName': 'A100 SXM 80GB', 'memoryInGb': 80}
{'id': 'NVIDIA A30', 'displayName': 'A30', 'memoryInGb': 24}
```python
[{
"id": "NVIDIA GeForce RTX 4090",
"name": "RTX 4090",
"pool": "ADA_24",
"manufacturer": "NVIDIA",
"memory": 24,
"secure": True,
"community": True,
"price": {"secure": 0.44, "community": 0.31, "serverless": 1.1},
"maxCount": {"secure": 8, "community": 4},
}]
```

## get_gpu

```python
gpu_id = "NVIDIA A100 80GB PCIe"
gpu = runpod.get_gpu(gpu_id)
gpu_id = "NVIDIA GeForce RTX 4090"
gpu = runpod.get_gpu(gpu_id, gpu_quantity=2)

print(gpu)
```

### get_gpu Output
`get_gpu` requests pod availability for the requested GPU count.

### get_gpu output

```json
{'id': 'NVIDIA A100 80GB PCIe', 'displayName': 'A100 80GB', 'memoryInGb': 80, 'secureCloud': True, 'communityCloud': True, 'lowestPrice': {'minimumBidPrice': 1.158, 'uninterruptablePrice': 1.89}}
```python
{
"id": "NVIDIA GeForce RTX 4090",
"name": "RTX 4090",
"memory": 24,
"availability": "HIGH",
"dataCenters": [
{"id": "US-KS-2", "name": "US Kansas 2", "availability": "HIGH"}
],
}
```
44 changes: 0 additions & 44 deletions examples/graphql_wrapper.py

This file was deleted.

37 changes: 37 additions & 0 deletions examples/rest_wrapper.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
"""REST v2 wrapper for the Runpod API."""

import time

import runpod

runpod.api_key = "YOUR_RUNPOD_API_KEY"

gpus = runpod.get_gpus()
print(gpus)

gpu = runpod.get_gpu("NVIDIA GeForce RTX 4090")
print(gpu)

pod = runpod.create_pod(
"test",
"runpod/pytorch:1.0.2-cu1281-torch280-ubuntu2404",
"NVIDIA GeForce RTX 4090",
)
print(pod)

print("Waiting for pod to be created...")
time.sleep(10)

pod = runpod.stop_pod(pod["id"])
print(pod)

print("Waiting for pod to be stopped...")
time.sleep(10)

pod = runpod.resume_pod(pod["id"])
print(pod)

print("Waiting for pod to be resumed...")
time.sleep(10)

runpod.terminate_pod(pod["id"])
2 changes: 1 addition & 1 deletion runpod/api/__init__.py
Original file line number Diff line number Diff line change
@@ -1 +1 @@
""" Allows api_wrapper to be imported as a module."""
"""Runpod API wrapper."""
Loading