new-fetcher

Add a new external data fetcher

Follow this checklist completely — every step, in order. The DI rule: domain code depends on an abstract interface; concrete providers are chosen only by a factory function based on settings flags. Views and services must never import a concrete provider.

0. Gather requirements

Before writing code, establish (ask the user if not stated):

• Domain name — what data is fetched (e.g. commodity_data, dividends). Determines interface and factory names.
• Provider/source name — the external service (e.g. AlphaVantage, yfinance).
• Interface methods — signatures and return shapes.
• Key-gated? — does the provider need an API key? This decides whether a NoOp variant and *_API_ENABLED flag are required.

1. Interface

Create backend/base/infrastructure/interfaces/<domain>.py, modeled on economic_calendar.py in the same directory: module docstring, class extending ABC, @abstractmethod methods with type hints and one-line docstrings.

Export it from backend/base/infrastructure/interfaces/__init__.py (import + __all__ entry).

2. Concrete provider

Create backend/base/infrastructure/providers/<source>.py (or extend an existing provider module like yfinance_fetchers.py if the source is already used). The class implements the interface. If key-gated, take api_key via the constructor — see AlphaVantageEconomicCalendarFetcher in providers/economic_calendar.py.

3. Mock provider

Add Mock<Domain>Fetcher to backend/base/infrastructure/providers/mock_fetchers.py, matching the style of the existing mocks: deterministic (seed randomness on the symbol/query so the same input always yields the same output), realistic shapes, no network access.

4. NoOp provider (key-gated sources only)

Add NoOp<Domain>Fetcher in the same module as the concrete provider, returning empty results (pattern: NoOpEconomicCalendarFetcher in providers/economic_calendar.py). A missing key or disabled flag must never raise — it degrades to no data.

5. Settings

In backend/backend/settings.py, next to the existing API config (~lines 41–56):

• Key: <SOURCE>_API_KEY = load_optional_from_file_or_env("<SOURCE>_API_KEY", "").strip()
• Toggle: USE_<SOURCE>_API = os.environ.get('USE_<SOURCE>_API', 'true').lower() == 'true'
• Computed flag: <SOURCE>_API_ENABLED = bool(<SOURCE>_API_KEY) and USE_<SOURCE>_API — key present AND flag true.

6. Factory

Add get_default_<domain>_fetcher() to backend/base/services/__init__.py and list it in __all__. Copy the structure of get_default_economic_calendar_fetcher in that file:

• All imports lazy, inside the function (django.conf.settings and providers).
• Branch order: USE_MOCK_DATA_FETCHER → return mock; <SOURCE>_API_ENABLED false → return NoOp; else → concrete provider with api_key=settings.<SOURCE>_API_KEY.

7. .env.example

Add #<SOURCE>_API_KEY=... and #USE_<SOURCE>_API=true to the commented "Optional" block in .env.example, with the same comment style as the existing entries.

8. Tests

Create backend/base/tests/test_<domain>_fetcher.py, modeled on backend/base/tests/test_mock_news_calendar.py:

• Mock determinism: same query → identical result; different queries → different results.
• Result shape: required keys/columns present, dates/numbers parse.
• Factory wiring via override_settings: mock flag on → Mock* instance; *_API_ENABLED false → NoOp* instance; enabled with a dummy key → concrete class (do not call the network in the enabled test — assert the type only).

9. Guardrails

• Never import a concrete provider from a view, service, selector, or management command — only the factory.
• Consuming services accept the fetcher as a parameter defaulting to the factory result, e.g. def get_x(..., fetcher=None): fetcher = fetcher or get_default_<domain>_fetcher(), so tests can inject a fake.

10. Verify

From backend/:

DJANGO_SETTINGS_MODULE=backend.settings_test python manage.py test base --no-input

All tests must pass. On Windows/PowerShell: $env:DJANGO_SETTINGS_MODULE = 'backend.settings_test'; python manage.py test base --no-input.