Fix: Handle Error for CUDA Configuration and API Request

[skywinder]

Issue

Running the following curl request:

curl -X POST http://localhost:7860/v1/completions \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "Hello, how are you?",
    "max_tokens": 0
  }'

results in an error:

raise AssertionError("Torch not compiled with CUDA enabled")

Steps to Reproduce
1. Start the API using the following command:

./start_macos.sh --api --api-port 7860 --verbose

2.	Test the endpoint with:

curl -X POST http://localhost:7860/v1/completions \
-H "Content-Type: application/json" \
-d '{
    "prompt": "Hello, how are you?",
    "max_tokens": 0
}'

Expected Behavior

The request should return a clear error or handle cases where CUDA is not enabled.

Proposed Solution
• Add a preflight check during the startup process to verify CUDA compatibility.
• Ensure the API gracefully handles requests even when CUDA is unavailable.

Impact
• Prevents crashes due to unhandled assertions.
• Improves debugging experience with clear error messages.

Checklist:

• I have read the Contributing guidelines.

[jfmherokiller]

I applied your fix locally because it allows the system to fallback to cpu when I am playing a game that is making heavy use of my gpu.

[skywinder]

I applied your fix locally because it allows the system to fallback to cpu when I am playing a game that is making heavy use of my gpu.

thanks, one more useful use case. hope it will be merged