7 ms·

The trick is to access the "bard-jan-24-gemini-pro" model, available in direct chat mode here: https://chat.lmsys.org/ https://chat.lmsys.org/. Significantly be

by j-b 3y ago

The trick is to access the "bard-jan-24-gemini-pro" model, available in direct chat mode here: https://chat.lmsys.org/ https://chat.lmsys.org/. Significantly better than the prior model.

jxy 3y ago
how odd! What exactly is lmsys using? Some hidden API that google give them so they can have a better ranking there?
- declaredapple 3y ago
  > Some hidden API that google give them so they can have a better ranking there? I don't know about that second part - but it would make sense that google (and others) may want to use lmsys's arena to benchmark their models. After all, Human A/B tests are far better then the current automated benchmarks. I would like more info from lmsys as to how they're accessing these though.
- j-b 3y ago
  Most likely through this platform: https://console.cloud.google.com/vertex-ai https://console.cloud.google.com/vertex-ai
  - jxy 3y ago
    Thanks. I managed to google and get two different API endpoints. From the vertex ai: API_ENDPOINT="us-central1-aiplatform.googleapis.com" PROJECT_ID="test00" MODEL_ID="gemini-pro" LOCATION_ID="us-central1" curl \ -X POST \ -H "Authorization: Bearer $(gcloud auth print-access-token)" \ -H "Content-Type: application/json" \ "https://${API_ENDPOINT}/v1/projects/${PROJECT_ID}/locations/${LOCATION_ID}/publishers/google/models/${MODEL_ID}:streamGenerateContent" -d '@request.json' and from the makersuite: curl \ -X POST https://generativelanguage.googleapis.com/v1beta/models/gemini-pro:generateContent?key=${API_KEY} \ -H 'Content-Type: application/json' \ -d '@request.json'
    - j-b 3y ago
      Created a simple app to test Gemini here: https://github.com/dssjon/gemini/blob/main/app.py https://github.com/dssjon/gemini/blob/main/app.py
bazmattaz 3y ago
Thanks for sharing. Is this a free way to access GPT4-turbo then or are there some limitations?

by j-b 3y ago

The trick is to access the "bard-jan-24-gemini-pro" model, available in direct chat mode here: https://chat.lmsys.org/ https://chat.lmsys.org/. Significantly better than the prior model.

jxy 3y ago

how odd! What exactly is lmsys using? Some hidden API that google give them so they can have a better ranking there?

declaredapple 3y ago
> Some hidden API that google give them so they can have a better ranking there? I don't know about that second part - but it would make sense that google (and others) may want to use lmsys's arena to benchmark their models. After all, Human A/B tests are far better then the current automated benchmarks. I would like more info from lmsys as to how they're accessing these though.
j-b 3y ago
Most likely through this platform: https://console.cloud.google.com/vertex-ai https://console.cloud.google.com/vertex-ai
- jxy 3y ago
  Thanks. I managed to google and get two different API endpoints. From the vertex ai: API_ENDPOINT="us-central1-aiplatform.googleapis.com" PROJECT_ID="test00" MODEL_ID="gemini-pro" LOCATION_ID="us-central1" curl \ -X POST \ -H "Authorization: Bearer $(gcloud auth print-access-token)" \ -H "Content-Type: application/json" \ "https://${API_ENDPOINT}/v1/projects/${PROJECT_ID}/locations/${LOCATION_ID}/publishers/google/models/${MODEL_ID}:streamGenerateContent" -d '@request.json' and from the makersuite: curl \ -X POST https://generativelanguage.googleapis.com/v1beta/models/gemini-pro:generateContent?key=${API_KEY} \ -H 'Content-Type: application/json' \ -d '@request.json'
  - j-b 3y ago
    Created a simple app to test Gemini here: https://github.com/dssjon/gemini/blob/main/app.py https://github.com/dssjon/gemini/blob/main/app.py

bazmattaz 3y ago

Thanks for sharing. Is this a free way to access GPT4-turbo then or are there some limitations?