apple_fm

apple_fm

In-process Apple Foundation Models (AFM 3) engine.

Drives Apple's apple_fm_sdk against the on-device model behind Apple Intelligence, with no HTTP hop and no second process — so the energy measured around a request is the energy the request actually cost. (The apple_fm_shim + apple_fm engine pair remains available for pointing external OpenAI-compatible clients at the same model.)

Requires an Apple Silicon Mac on macOS 26+ with Apple Intelligence enabled, and a full Xcode — not just Command Line Tools — because the SDK compiles Swift bindings at install time::

DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer \
    uv pip install -e '.[afm]'

jarvis ask --engine afm --model afm-3-core "..."

Measurement caveats specific to this backend — read before comparing numbers against MLX or Ollama:

• session.stream_response yields cumulative text snapshots, each batching roughly 8-10 tokens. ttft is therefore time-to-first-chunk (an upper bound on true TTFT — around 450ms on an M1 Pro versus a few tens of ms of real first-token latency), and derived inter-token latencies are inter-chunk latencies. Neither is comparable to a backend that streams one token at a time. Token counts, throughput and per-token energy are unaffected.
• The SDK exposes no way to select AFM 3 Core (dense ~3B) versus AFM 3 Core Advanced (20B sparse MoE, 1-4B active per request); the framework's dynamic profile picks one from the host device's capabilities. model is a run label only — :meth:AppleFMEngine.describe records the SDK version, context size and host chip so a run can be attributed after the fact.
• There is no Private Cloud Compute path in the Python SDK, which suits OpenJarvis: off-device inference would make the on-device energy measurement meaningless.

Classes

AppleFMEngine

AppleFMEngine(instructions: str = '', use_case: str = 'general', guardrails: str = 'default', sampling: str = 'greedy')

Bases: InferenceEngine

Apple Foundation Models, driven in-process via apple_fm_sdk.

Source code in src/openjarvis/engine/apple_fm.py

def __init__(
    self,
    instructions: str = "",
    use_case: str = "general",
    guardrails: str = "default",
    sampling: str = "greedy",
) -> None:
    self._instructions = str(instructions or "").strip()
    self._use_case_name = str(use_case or "general").strip().lower()
    self._guardrails_name = str(guardrails or "default").strip().lower()
    self._sampling = str(sampling or "greedy").strip().lower()

    if self._use_case_name not in _USE_CASES:
        raise ValueError(
            f"Unknown AFM use_case {use_case!r}. Expected one of: "
            f"{', '.join(_USE_CASES)}."
        )
    if self._guardrails_name not in _GUARDRAILS:
        raise ValueError(
            f"Unknown AFM guardrails {guardrails!r}. Expected one of: "
            f"{', '.join(_GUARDRAILS)}."
        )
    # Validate eagerly so a typo fails at construction rather than midway
    # through a run.
    parse_sampling_spec({"sampling": self._sampling})

    self._model: Any = None
    self._context_size: Optional[int] = None
    self._prepared = False
    self._loop: Optional[AsyncLoopRunner] = None
    self._skipped: Dict[str, int] = {}

Functions

health

health() -> bool

Report on-device model availability.

Returns a bool for the ABC, but logs the SDK's reason first: a bare False would hide whether the cause is Apple Intelligence being switched off, an ineligible device, or assets still downloading.

Source code in src/openjarvis/engine/apple_fm.py

def health(self) -> bool:
    """Report on-device model availability.

    Returns a bool for the ABC, but logs the SDK's reason first: a bare
    False would hide whether the cause is Apple Intelligence being switched
    off, an ineligible device, or assets still downloading.
    """
    try:
        available, reason = self._ensure_model().is_available()
    except Exception as exc:  # pragma: no cover - depends on host state
        logger.warning("Could not initialize Apple Foundation Models: %s", exc)
        return False
    if not available:
        logger.warning(
            "Apple Foundation Models unavailable: %s. Check that Apple "
            "Intelligence is enabled in System Settings, that this device "
            "is eligible, and that model assets have finished downloading.",
            getattr(reason, "name", reason),
        )
    return bool(available)

can_serve

can_serve(model: str) -> bool

Only the AFM labels — this engine cannot serve arbitrary model ids.

Unlike the other local engines, "is the model installed" is not a separate concern here: there is exactly one on-device model, so accepting any id would let engine selection route e.g. a Llama request to AFM and silently answer with a different model.

Source code in src/openjarvis/engine/apple_fm.py

def can_serve(self, model: str) -> bool:
    """Only the AFM labels — this engine cannot serve arbitrary model ids.

    Unlike the other local engines, "is the model installed" is not a
    separate concern here: there is exactly one on-device model, so
    accepting any id would let engine selection route e.g. a Llama request
    to AFM and silently answer with a different model.
    """
    return (model or "").strip().lower() in MODEL_LABELS

prepare

prepare(model: str) -> None

Validate the label, then warm up outside the first energy window.

The first request after boot pays a large one-off cost for model asset load and XPC setup (~10s versus ~450ms warm on an M1 Pro). Burning it here keeps it out of the telemetry window of the first real request.

Source code in src/openjarvis/engine/apple_fm.py

def prepare(self, model: str) -> None:
    """Validate the label, then warm up outside the first energy window.

    The first request after boot pays a large one-off cost for model asset
    load and XPC setup (~10s versus ~450ms warm on an M1 Pro). Burning it
    here keeps it out of the telemetry window of the first real request.
    """
    validate_model_label(model)
    language_model = self._ensure_model()

    available, reason = language_model.is_available()
    if not available:
        raise EngineConnectionError(
            "Apple Foundation Models is unavailable "
            f"({getattr(reason, 'name', reason)}). Enable Apple "
            "Intelligence in System Settings and wait for model assets to "
            "download."
        )

    self._context_size = int(language_model.context_size)
    self._ensure_loop().run(self._warmup())
    self._prepared = True

describe

describe() -> Dict[str, Any]

Run metadata. The executing variant is not observable, so the host chip and SDK version are the only way to attribute a run later.

Source code in src/openjarvis/engine/apple_fm.py

def describe(self) -> Dict[str, Any]:
    """Run metadata. The executing variant is not observable, so the host
    chip and SDK version are the only way to attribute a run later."""
    try:
        sdk_version: Optional[str] = _dist_version("apple-fm-sdk")
    except PackageNotFoundError:  # pragma: no cover - installed by definition
        sdk_version = None
    return {
        "engine_id": self.engine_id,
        "afm_sdk_version": sdk_version,
        "afm_context_size": self._context_size,
        "afm_use_case": self._use_case_name,
        "afm_guardrails": self._guardrails_name,
        "afm_instructions": self._instructions or None,
        "afm_sampling": self._sampling,
        "host_chip": _host_chip(),
        "host_os_version": platform.mac_ver()[0] or None,
        "variant_selection": (
            "device-chosen by the Foundation Models dynamic profile; "
            "the model name is a label only"
        ),
        "declined_requests": dict(self._skipped) or None,
    }

Functions