HELIX SDK Documentation

Semantic compression for AI training and storage optimization. 2-5x size reduction with identity preservation.

2-5x Compression

PyTorch Compatible

V1 & V2 Formats

Installation

From PyPI

bash

# Basic install
pip install helix-sdk

# With PyTorch/TensorFlow support
pip install helix-sdk[ml]

# Full install (all extras)
pip install helix-sdk[all]

From Source

bash

git clone https://github.com/DB0SZ1/PROJECT-HELIX.git
cd PROJECT-HELIX
pip install -e .

Environment Setup

bash

# Set your Gemini API key (optional, enables AI enhancement)
export GEMINI_API_KEY="your-key"

# Or in Python
import os
os.environ["GEMINI_API_KEY"] = "your-key"

🌐 Remote Mode (NEW in v1.0.2)

New! Use HELIX via API without installing heavy dependencies. Perfect for web apps, mobile backends, and data centers.

Step 1: Get Your API Key

In the dashboard, go to API Keys tab to create your key.

Step 2: Install Lightweight SDK

bash

# Only needs httpx, numpy, pillow - NO heavy AI models!
pip install helix-sdk

Step 3: Use with Your API Key

python

from helix_sdk import HelixSDK

# Initialize with your API URL and key
sdk = HelixSDK(
    base_url="https://your-helix-server.com",  # Your deployed HELIX API
    api_key="hlx_your_api_key_here"            # From dashboard API Keys
)

# Compress an image - sends to API, receives .hlx file
result = sdk.compress("photo.jpg", "photo.hlx")
print(f"Compressed! {result.compression_ratio:.1f}x smaller")

# Materialize at 4K - API does the heavy lifting
sdk.materialize("photo.hlx", "photo_4k.png", resolution="4K")

Using cURL (No SDK Required)

bash

# Encode an image
curl -X POST "https://your-api.com/api/encode/v2" \
  -H "Authorization: Bearer hlx_your_api_key" \
  -F "file=@photo.jpg" \
  -o photo.hlx

# Materialize to 4K
curl -X POST "https://your-api.com/api/materialize" \
  -H "Authorization: Bearer hlx_your_api_key" \
  -F "file=@photo.hlx" \
  -F "target_resolution=4K" \
  -o output.png

JavaScript/Fetch Example

javascript

// Encode an image
const formData = new FormData();
formData.append('file', imageFile);

const response = await fetch('https://your-api.com/api/encode/v2', {
    method: 'POST',
    headers: { 'Authorization': 'Bearer hlx_your_api_key' },
    body: formData
});

const { hlx_file_b64 } = await response.json();

// Save the HLX data or send to materialize endpoint

When to Use Remote Mode

✅ Web applications / Next.js / React apps
✅ Mobile app backends
✅ Serverless functions (AWS Lambda, Vercel)
✅ Data centers with centralized processing
✅ Any environment where you can't install heavy AI deps

Quick Start

Basic Compression

python

from helix_sdk import HelixSDK

# Initialize SDK
sdk = HelixSDK()

# Compress an image to HLX
result = sdk.compress("photo.jpg", "photo.hlx")
print(f"Original: {result.input_size / 1024:.1f} KB")
print(f"Compressed: {result.output_size / 1024:.1f} KB")
print(f"Ratio: {result.compression_ratio:.1f}x")

Materialize at Different Resolutions

python

from helix_sdk import HelixSDK, Resolution

sdk = HelixSDK()

# Materialize at 1080p
result = sdk.materialize("photo.hlx", "photo_1080p.png", resolution=Resolution.RES_1080P)

# Materialize at 4K
result = sdk.materialize("photo.hlx", "photo_4k.png", resolution="4K")

# Available resolutions: 256p, 512p, 720p, 1080p, 1440p, 4K, 8K
print(f"Materialized at {result.resolution}")

With API Key (Enhanced Quality)

python

from helix_sdk import HelixSDK

# With Gemini API key for AI-enhanced materialization
sdk = HelixSDK(api_key="your-gemini-api-key")

# Compress with semantic analysis
result = sdk.compress("photo.jpg", "photo.hlx")

# Materialize with AI enhancement
result = sdk.materialize("photo.hlx", "photo_enhanced.png", resolution="4K")

HLX Formats

V1 Format (Compact)

• Maximum compression (2-5x)
• Encrypted binary format
• HELIX-only readable
• Best for storage/archival

V2 Format (Cross-Platform)

• Valid PNG viewable by any app
• Embedded blueprint data
• Larger file size (includes preview)
• Best for sharing/compatibility

python

from helix_sdk import HelixSDK, HLXFormat

sdk = HelixSDK()

# V1: Maximum compression (HELIX only)
sdk.compress("photo.jpg", "photo.hlx", format_version=HLXFormat.V1)

# V2: Cross-platform (any app can view)
sdk.compress("photo.jpg", "photo.hlx", format_version=HLXFormat.V2)

# Set default format
sdk = HelixSDK(default_format=HLXFormat.V2)

ML Training Integration

Compress Dataset (One-time)

python

from helix_sdk import HelixSDK

sdk = HelixSDK()

# Compress entire dataset directory
stats = sdk.compress_directory(
    input_dir="/data/imagenet/train/",    # 150GB original
    output_dir="/data/imagenet_hlx/",     # ~50GB compressed
    workers=8                              # Parallel processing
)

print(f"Files processed: {stats.files_processed}")
print(f"Total input: {stats.total_input_bytes / 1e9:.1f} GB")
print(f"Total output: {stats.total_output_bytes / 1e9:.1f} GB")
print(f"Compression: {stats.compression_ratio:.1f}x")
print(f"Space saved: {stats.space_saved_percent:.1f}%")

PyTorch DataLoader

python

from helix_sdk import HelixDataset, HelixLoader
import torch

# Load compressed dataset
dataset = HelixDataset(
    path="/data/imagenet_hlx/",
    target_resolution="512p",     # Materialize at 512x512
    enable_variants=True,         # Enable variant generation
    cache_materializations=True   # Cache for faster epochs
)

# PyTorch-compatible loader
loader = HelixLoader(
    dataset,
    batch_size=64,
    shuffle=True,
    num_workers=4,
    variants_per_image=3  # 3x dataset size via variants!
)

# Training loop
for batch in loader:
    # batch is (B, H, W, C) numpy array
    images = torch.from_numpy(batch).permute(0, 3, 1, 2).float() / 255
    loss = model.train_step(images)
    
print(f"Dataset size: {len(dataset)}")
print(f"Batches per epoch: {len(loader)}")

Semantic Search

python

from helix_sdk import HelixDataset

dataset = HelixDataset("/data/training_hlx/")

# Search by semantic description
sunset_indices = dataset.search("sunset beach sunset")
portrait_indices = dataset.search("person face portrait")

# Get metadata without materializing
for idx in sunset_indices[:5]:
    meta = dataset.get_metadata(idx)
    print(f"File: {meta['identity']}")
    print(f"  Scene: {meta['scene_description']}")
    print(f"  Anchors: {meta['anchor_count']}")

Infinite Variant Generation

python

from helix_sdk import HelixDataset

dataset = HelixDataset("/data/hlx/", enable_variants=True)

# Get different variants of the same image
# Each variant is slightly different but identity-preserving
for variant_id in range(5):
    img = dataset.get_variant(0, variant_id)
    # All variants preserve identity but vary in background/details

CLI Reference

bash

# Compress single image
helix compress photo.jpg                    # → photo.hlx
helix compress photo.jpg -o output.hlx      # Custom output
helix compress photo.jpg --format v2        # V2 format

# Materialize from HLX
helix materialize photo.hlx                 # → photo.png
helix materialize photo.hlx -r 4K           # At 4K resolution
helix materialize photo.hlx -o output.png   # Custom output

# Batch operations
helix batch /input/folder/ /output/folder/  # Compress all
helix batch /input/ /output/ -w 8           # 8 workers
helix batch /input/ /output/ --format v2    # V2 format

# Inspect HLX file
helix info photo.hlx                        # Show metadata
helix info photo.hlx --json                 # JSON output

# Extract preview (V2 only)
helix preview photo.hlx -o preview.png

API Reference

HelixSDK

HelixSDK(api_key=None, default_format=HLXFormat.V2, verbose=True)

Initialize SDK. api_key enables Gemini AI enhancement.

compress(input_path, output_path, format_version=None)

Compress image to HLX. Returns CompressionResult.

materialize(input_path, output_path, resolution="1080p")

Reconstruct image from HLX at target resolution.

compress_directory(input_dir, output_dir, workers=4)

Batch compress directory with parallel workers. Returns BatchStats.

get_info(hlx_path)

Get HLX file metadata without materializing.

get_supported_formats()

List supported input/output formats.

HelixDataset

HelixDataset(path, target_resolution="1080p", enable_variants=True)

Load HLX files from directory for training.

__getitem__(idx)

Get materialized image as numpy array.

__len__()

Number of HLX files in dataset.

get_variant(idx, variant_id)

Get specific variant of image at index.

get_metadata(idx)

Get blueprint metadata without materializing.

search(query, limit=100)

Semantic search across dataset by description.

HelixLoader

HelixLoader(dataset, batch_size=32, shuffle=True, num_workers=4)

PyTorch-compatible DataLoader for HELIX datasets.

variants_per_image=1

Generate N variants per image for augmentation.

drop_last=False

Drop incomplete final batch.

transform=None

Optional transform function applied to each image.

Data Classes

python

from helix_sdk import CompressionResult, MaterializationResult, BatchStats

# CompressionResult
result = sdk.compress("photo.jpg", "photo.hlx")
result.input_path       # str: Original file path
result.output_path      # str: HLX file path
result.input_size       # int: Original size in bytes
result.output_size      # int: HLX size in bytes
result.compression_ratio # float: input/output
result.success          # bool: Success status
result.error            # str: Error message if failed
result.processing_time_ms # int: Processing time
result.anchors_detected # int: Number of anchors found

# BatchStats
stats = sdk.compress_directory(...)
stats.files_processed   # int
stats.total_input_bytes # int
stats.total_output_bytes # int
stats.compression_ratio # float: Overall ratio
stats.space_saved_percent # float: (1 - output/input) * 100

REST API Endpoints

Endpoint	Method	Description
/	GET	Health check with feature status
/api/encode	POST	Encode image to HLX v1 format
/api/encode/v2	POST	Encode to cross-platform HLX v2
/api/materialize	POST	Reconstruct image from HLX
/api/hlx/preview	POST	Extract PNG preview from HLX v2
/api/hlx/format-info	GET	Get format version details
/api/sdk/info	GET	SDK documentation

Example: Encode via API

python

import requests

# Encode image
with open("photo.jpg", "rb") as f:
    response = requests.post(
        "http://localhost:8001/api/encode/v2",
        files={"file": f}
    )

# Save HLX file
with open("photo.hlx", "wb") as f:
    f.write(response.content)

# Materialize
with open("photo.hlx", "rb") as f:
    response = requests.post(
        "http://localhost:8001/api/materialize",
        files={"file": f},
        params={"resolution": "4K"}
    )

with open("output.png", "wb") as f:
    f.write(response.content)

Architecture

text

Project-HELIX/
├── src/
│   ├── api/                    # FastAPI server
│   │   ├── server.py          # Main API endpoints
│   │   └── sdk_docs.py        # Backend docs
│   ├── core/
│   │   ├── pipeline.py        # Main encoding pipeline
│   │   ├── materializer.py    # Image reconstruction
│   │   ├── hlx_codec.py       # Encryption/encoding
│   │   ├── enhancement_layers.py  # Post-processing
│   │   └── resolution_engine.py   # Multi-resolution
│   ├── extractors/
│   │   ├── anchors.py         # OpenCV extraction
│   │   └── gemini_anchors.py  # AI-powered extraction
│   └── schema/
│       └── blueprint.py       # Data structures
├── helix_sdk/                  # Pip-installable SDK
│   ├── __init__.py            # Public API
│   ├── core.py                # HelixSDK class
│   ├── dataset.py             # HelixDataset
│   ├── loader.py              # HelixLoader
│   └── cli.py                 # CLI commands
└── ui/helix-app/              # Next.js frontend

Pipeline Flow

Encoding Pipeline

1. Normalize image
2. Detect saliency regions
3. Extract identity anchors
4. Build structural mesh
5. Set constraints
6. Encrypt and serialize

Materialization Pipeline

1. Decrypt and validate
2. Stitch anchor crops
3. Apply enhancement layers
4. Upscale to target resolution
5. Optional: AI refinement
6. Verify identity preservation