Skip to content

Blablador API access

Blablador (http://helmholtz-blablador.fz-juelich.de) is a service that allows researchers to add their own LLM models to an authenticated web interface.

It also offers a REST API to access the models. The API is compatible to the OpenAI Python Api. The GEOMAR center for Oceanographic Research developed python bindings for Blablador, too: https://git.geomar.de/everardo-gonzalez/blablador-python-bindings

In order to use the API, you need to obtain an API key, available at Helmholtz Codebase, Helmholtz's Gitlab server. This is how you do it:

Step 1: Register on GitLab

If you don't have a GitLab account yet, go to Helmholtz Codebase's website and register for a new account. You can log in with any EduGAIN account, such as your university account.

Step 2: Obtain an API key

Go to your profile link. Alternatively, you can reach it by clicking on your profile picture in the top left corner of the screen. Then, click on "Preferences" in the dropdown menu.

In the preferences page, you will see an "Access Tokens" option on the left. Click on it.

Here, you can create a new "Personal Access Token" - this is the API key you need to access Blablador's API. Click in "Add new token", give it a name, such as "Blablador API key", and select the "api" scope. Then, click on "Create personal access token".

You will see a long string of characters. This is your API key. Copy it and save it somewhere safe. You will need it to access Blablador's API.

Important: Keys are valid for at most a year on Helmholtz Codebase. After that, you will need to create a new one.

Done! You have all you need to access Blablador's API.

The API endpoints is at https://helmholtz-blablador.fz-juelich.de:8000/v1/ .

The API documentation is https://helmholtz-blablador.fz-juelich.de:8000/docs . Try your api key with the "Authorize" button.

One can test it with the command line tool curl - Just make sure to replace MY_API_KEY with your actual API key. This is an example output:

curl --header "Authorization: Bearer MY_API_KEY" https://helmholtz-blablador.fz-juelich.de:8000/v1/models

{
  "object": "list",
  "data": [
    {
      "id": "Mistral-7B-Instruct-v0.2",
      "object": "model",
      "created": 1705423388,
      "owned_by": "fastchat",
      "root": "Mistral-7B-Instruct-v0.2",
      "parent": null,
      "permission": [
        {
          "id": "modelperm-5vcVRz3BNpzXrK34vzLnKK",
          "object": "model_permission",
          "created": 1705423388,
          "allow_create_engine": false,
          "allow_sampling": true,
          "allow_logprobs": true,
          "allow_search_indices": true,
          "allow_view": true,
          "allow_fine_tuning": false,
          "organization": "*",
          "group": null,
          "is_blocking": false
        }
      ]
    },
    {
      "id": "Mixtral-8x7B-Instruct-v0.1",
      "object": "model",
      "created": 1705423388,
      "owned_by": "fastchat",
      "root": "Mixtral-8x7B-Instruct-v0.1",
      "parent": null,
      "permission": [
        {
          "id": "modelperm-3pcd6r7ujicX3MoviEmG6k",
          "object": "model_permission",
          "created": 1705423388,
          "allow_create_engine": false,
          "allow_sampling": true,
          "allow_logprobs": true,
          "allow_search_indices": true,
          "allow_view": true,
          "allow_fine_tuning": false,
          "organization": "*",
          "group": null,
          "is_blocking": false
        }
      ]
    },
  ]
}%

Other endpoints are documented at https://helmholtz-blablador.fz-juelich.de:8000/docs

Model aliases

In order to cope with the constant change of models, Blablador uses aliases for models. This way, we can refer aliases in our code and the actual model can be changed without breaking the code. Those aliases are not shown on Blablador's web interface.

Right now, the aliases are:

  • alias-code - A model that is specially trained for code. As of March 2024, it's Starcoder2-15B
  • alias-embeddings - Usually a model specially made for embeddings. As of March 2024, it's GritLM-7B.
  • alias-fast - This alias is for model with a high throughput. As of March 2024, it's Mistral-7B-Instruct-v0.2.
  • alias-large - The larges model we have. It's usually the most accurate, but also the slowest. As of March 2024, it's Mixtral-8x7B-Instruct-v0.1.

Alias for code with hardcoded model names

Some softwares like Langchain expect OpenAI model names. In this case, we can use the following aliases: gpt-3.5-turbo, text-davinci-003 and text-embedding-ada-002 - These are aliases for the alias-embeddings model (see above).

In the future, it's guaranteed that these models will change.