cross-posted from: https://lemmy.world/post/16391311

Andrej Karpathy endorses Apple Intelligence

Actually, really liked the Apple Intelligence announcement. It must be a very exciting time at Apple as they layer AI on top of the entire OS. A few of the major themes.

Step 1 Multimodal I/O. Enable text/audio/image/video capability, both read and write. These are the native human APIs, so to speak.

Step 2 Agentic. Allow all parts of the OS and apps to inter-operate via “function calling”; kernel process LLM that can schedule and coordinate work across them given user queries.

Step 3 Frictionless. Fully integrate these features in a highly frictionless, fast, “always on”, and contextual way. No going around copy pasting information, prompt engineering, or etc. Adapt the UI accordingly.

Step 4 Initiative. Don’t perform a task given a prompt, anticipate the prompt, suggest, initiate.

Step 5 Delegation hierarchy. Move as much intelligence as you can on device (Apple Silicon very helpful and well-suited), but allow optional dispatch of work to cloud.

Step 6 Modularity. Allow the OS to access and support an entire and growing ecosystem of LLMs (e.g. ChatGPT announcement).

Step 7 Privacy. <3

We’re quickly heading into a world where you can open up your phone and just say stuff. It talks back and it knows you. And it just works. Super exciting and as a user, quite looking forward to it.

https://x.com/karpathy/status/1800242310116262150?s=46

  • Synapse
    1•6 months ago

    I may not be very well informed about it, I admit. Would you have documents to share ? All I seem to find about Intel security engine is about how closed it is and how it has been exploited by bad actors in the past to gain elevated privileges in targeted hacks. It sucks, that’s for sure. But it doesn’t have much to do with the mass data collection employed by Google, Apple, Open AI, etc.

    • @Ptsf@lemmy.world
      1•6 months ago

      I unfortunately don’t have much to share beyond a decent understanding of compute systems at an enterprise scale (where we utilize these low level subprocessors to do various things such as gather asset data or deploy operating system configurations, see: https://en.m.wikipedia.org/wiki/Intel_Active_Management_Technology). The point I’m trying to make though is that current operating models don’t allow for system trust. If you can’t trust apple with high level data like that needed for llm models on-device (which is how they’ve configured it, requiring a specific user approval and interaction before forwarding minimal data to private process servers) then you shouldn’t trust any device that lacks a complete open boot/firmware/ and OS stack because if these companies were going to exploit your data that egregiously, they already have the lowest level (best) access possible to a system that can transparently (without your knowledge) access encryption enclaves, networking, and storage. Truly open alternatives do exist by the way (see Coreboot, etc) but you’re going to be looking at devices 10-20 years old since almost the entire industry runs proprietary at that level and it takes time for the less heavily funded community players to get up to speed.