how to config AI for selfhost

[IamTaoChen]

Can we use AI in the self-host instance?

[wanghkkk]

I have the same question

[vuxng]

Turn on the copilot plugin at
.affine/self-host/config/affine.js

[lennon101]

Turn on the copilot plugin at
.affine/self-host/config/affine.js

nice find and thanks for this! I've gone ahead and uncommented the copilot plugin and added my OpenAI apikey (see below). And I in the Affine logs I can see the CoPilot is registered. But I'm still getting an error. Do I need to get a FAL and an unsplashKey as well?

affine.js

// /* Copilot Plugin */
AFFiNE.use('copilot', {
  openai: {
    apiKey: 'sk-proj-<withheld>',
  },
  // fal: {
  //   apiKey: 'your-key',
  // },
  // unsplashKey: 'your-key',
  // storage: {
  //   provider: 'cloudflare-r2',
  //   bucket: 'copilot',
  // }
})

Affine logs:

[Nest] 117  - 06/26/2024, 2:23:10 AM VERBOSE [CacheInterceptor] cache blobs:6ebf69e5-33b9-4e3b-9809-a99de7acab75 miss
[Nest] 117  - 06/26/2024, 2:23:18 AM   DEBUG [CopilotProviderService] Copilot provider openai registered, capabilities: text-to-text, text-to-embedding, text-to-image, image-to-text

UI error when trying to use AI:

[Bottleface]

@lennon101 were you ever able to get this working?

[AndreasWidmer]

For everybody interested: this works with the self hosted version. You have to use (a paid) Open AI account with a User API keys (Legacy) - not the new project API keys!

In the affine.js config set:

AFFiNE.use('copilot', {
   openai: {
           apiKey: 'your-key',
   },
//   fal: {
//     apiKey: 'your-key',
//   },
//   unsplashKey: 'your-key',
//   storage: {
//     provider: 'cloudflare-r2',
//     bucket: 'copilot',
//   }
 })

[madratzz]

You can modify it in the postgres database

[pyrox82]

Can I also choose which model is used? I think by default it's gpt-4o. But I rather want to use gpt-4o-mini.

[JoeBuildsStuff]

You might need to modify the model selection in the postgres table

[richardy2012]

At this stage, modifications from the database are no longer effective, and restrictions are made directly at the code level. Only a series of built-in large models such as gpt-4o and gpt-4o-mini are supported, so don't bother with it.

[Let1fer]

Is there a way to edit the models used? My provider provides access under slightly different names

[Nishan-Vivek]

Looks like openai don't allow new accounts to generate legacy API keys anymore. Anyway to get this to work with the project api keys?

[hnykda]

I made it work with the latest 0.20 version: #10549 . My openai API key starts with sk-svcacct--

[pierdevara]

i found another workaround, i use LiteLLM as a llm proxy, converts over 200 LLMs in openai compatible api calls. In the admin UI you have the option to give a fake name to the LLM, as gpt-4o, so it will be recognize by AFFine

AFFiNE.use('copilot', {
  openai: {
    baseURL: 'http://localhost:4000',
    apiKey: 'sk-xxxxxxxxxxxxxxx',
  }
});

[Steven-Low]

i found another workaround, i use LiteLLM as a llm proxy, converts over 200 LLMs in openai compatible api calls. In the admin UI you have the option to give a fake name to the LLM, as gpt-4o, so it will be recognize by AFFine AFFiNE.use('copilot', { openai: { baseURL: 'http://localhost:4000', apiKey: 'sk-xxxxxxxxxxxxxxx', } });

Really appreciate the help, that solved it! 💘
however the UI require setting up postgresql so instead I just declare the litellm_config.yaml:

model_list:
  - model_name: gpt-4o
    litellm_params:
      model: openrouter/anthropic/claude-3.5-haiku
      api_key: os.environ/OPENROUTER_API_KEY

[floli]

Does that approach still works with 0.21? I have set:

AFFiNE.use('copilot', {
  openai: {
    apiKey: process.env.COPILOT_OPENAI_API_KEY,
  },
})

and set the API key in the .env file. It used to work on 0.20, however, on 0.21 all AI functionality is gone.

Any idea?

[tareeko]

I'd love to know if they're planning to add it back.

[sudesh0sudesh]

I think this new config.json appeared after 21 is the answer for new configs
https://github.com/toeverything/AFFiNE/blob/canary/.docker/selfhost/config.example.json

[floli]

That's right, my config.json now looks like

{
  "$schema": "https://github.com/toeverything/affine/releases/latest/download/config.schema.json",
  "server": {
    "name": "domain.de",
    "host": "affine.domain.de",
    "https": true
  },
  "copilot": {
    "enabled": true,
    "providers.openai": {
      "apiKey": "sk-proj-xxx"
    }
  }
}

As already stated by someone else, while the AI in the side bar does not work, the AI on selected texts works just fine.

[sudesh0sudesh]

Admin section says they are working on AI

[anixon604]

what are they working on though? UI just needs a config entry...? This sniffs of trying to force a subscription fee..

[laelhalawani]

This should be a feature.
Model selection and settings to paste in the API key in the UI.