Google Colab에서 Ollama를 사용하여 Gemma 모델을 실행

Gemma - Run with Ollama (with Colab)

Google Colab에서 Ollama로 Gemma 실행하기

이 노트북은 Ollama를 사용하여 Gemma 모델에서 추론을 실행하는 방법을 보여줍니다 . Ollama는 로컬에서 LLM을 실행하기 위한 사용하기 쉬운 솔루션이며 Gemma가 기본으로 지원됩니다.

 

Gemma는 Google DeepMind에서 구축한 경량의 최첨단 개방형 모델 제품군입니다. 버전 1.1로 업데이트되었습니다.

Gemma는 2b 및 7b 매개변수 크기로 제공됩니다.

• ollama run gemma:2b
• ollama run gemma:7b (default)

gemma 모델정보 확인

gemma 7b

 

gemma 2b

 

Colab 링크

Ollama 설치

공식 설치 스크립트를 통해 Ollama를 설치합니다.

!curl -fsSL https://ollama.com/install.sh | sh

!curl -fsSL https://ollama.com/install.sh | sh
>>> Installing ollama to /usr/local
>>> Downloading Linux amd64 bundle
############################################################################################# 100.0%
>>> Creating ollama user...
>>> Adding ollama user to video group...
>>> Adding current user to ollama group...
>>> Creating ollama systemd service...
WARNING: Unable to detect NVIDIA/AMD GPU. Install lspci or lshw to automatically detect and install GPU dependencies.
>>> The Ollama API is now available at 127.0.0.1:11434.
>>> Install complete. Run "ollama" from the command line.

 

Ollama 시작 (실행)

nohup을 사용하여 백그라운드에서 Ollama를 시작합니다. API 서비스 포트는 11434 입니다.

!nohup ollama serve > ollama.log &

nohup: redirecting stderr to stdout

 

Ollama 모델 목록 확인

!ollama list

NAME    ID    SIZE    MODIFIED

 

추론

명령행(command line)을 이용해서 gemma 7B 모델을 받고 추론을 실행 

!ollama run gemma:7b "대한민국의 수도는 어디야?" 2> ollama.log

대한민국의 수도는 서울입니다.</end_of_turn>

 

영어로 질문해봅니다.

ollama run gemma:7b "What is the capital of Korea?" 2> ollama.log

The capital of Korea is **Seoul**. It is the largest and most populous city in the country.

 

실행한 모델 확인

!ollama list

NAME        ID              SIZE      MODIFIED       
gemma:7b    a72c7f4d0a15    5.0 GB    41 seconds ago

 

REST 엔드포인트를 통해 응답 생성

!curl http://localhost:11434/api/generate -d '{ \
  "model": "gemma:7b", \
  "prompt":"프랑스의 수도는 어디야?" \
}'

{"model":"gemma:7b","created_at":"2024-09-20T07:43:59.191815259Z","response":"프","done":false}
{"model":"gemma:7b","created_at":"2024-09-20T07:43:59.238195689Z","response":"랑","done":false}
{"model":"gemma:7b","created_at":"2024-09-20T07:43:59.284446079Z","response":"스","done":false}
{"model":"gemma:7b","created_at":"2024-09-20T07:43:59.330415772Z","response":"의","done":false}
{"model":"gemma:7b","created_at":"2024-09-20T07:43:59.376395312Z","response":" 수","done":false}
{"model":"gemma:7b","created_at":"2024-09-20T07:43:59.404399661Z","response":"도","done":false}
{"model":"gemma:7b","created_at":"2024-09-20T07:43:59.431299982Z","response":"는","done":false}
{"model":"gemma:7b","created_at":"2024-09-20T07:43:59.45782401Z","response":" 파","done":false}
{"model":"gemma:7b","created_at":"2024-09-20T07:43:59.487122476Z","response":"리","done":false}
{"model":"gemma:7b","created_at":"2024-09-20T07:43:59.514609513Z","response":"(","done":false}
{"model":"gemma:7b","created_at":"2024-09-20T07:43:59.540247914Z","response":"Paris","done":false}
{"model":"gemma:7b","created_at":"2024-09-20T07:43:59.565826889Z","response":")","done":false}
{"model":"gemma:7b","created_at":"2024-09-20T07:43:59.592181701Z","response":"입니다","done":false}
{"model":"gemma:7b","created_at":"2024-09-20T07:43:59.619404725Z","response":".","done":false}
{"model":"gemma:7b","created_at":"2024-09-20T07:43:59.646506512Z","response":"","done":true,"done_reason":"stop","context":[968,2997,235298,559,235298,15508,235313,1645,108,237964,239566,236354,236137,22618,236840,236214,53355,238173,238305,181537,615,235298,559,235298,15508,235313,108,235322,2997,235298,559,235298,15508,235313,2516,108,237964,239566,236354,236137,22618,236840,236214,59994,236432,235278,29437,235275,47555,235265],"total_duration":650239442,"load_duration":39196479,"prompt_eval_count":37,"prompt_eval_duration":114387000,"eval_count":15,"eval_duration":454674000}

 

REST 엔드포인트를 통해 Gemma와 채팅

!curl http://localhost:11434/api/chat -d '{ \
  "model": "gemma:7b", \
  "messages": [ \
    { "role": "user", "content": "스페인의 수도는 어디야?" } \
  ] \
}'

{"model":"gemma:7b","created_at":"2024-09-20T07:44:05.627626051Z","message":{"role":"assistant","content":"스"},"done":false}
{"model":"gemma:7b","created_at":"2024-09-20T07:44:05.673755341Z","message":{"role":"assistant","content":"페"},"done":false}
{"model":"gemma:7b","created_at":"2024-09-20T07:44:05.719617417Z","message":{"role":"assistant","content":"인"},"done":false}
{"model":"gemma:7b","created_at":"2024-09-20T07:44:05.765324012Z","message":{"role":"assistant","content":"의"},"done":false}
{"model":"gemma:7b","created_at":"2024-09-20T07:44:05.810472803Z","message":{"role":"assistant","content":" 수"},"done":false}
{"model":"gemma:7b","created_at":"2024-09-20T07:44:05.842618919Z","message":{"role":"assistant","content":"도"},"done":false}
{"model":"gemma:7b","created_at":"2024-09-20T07:44:05.865453528Z","message":{"role":"assistant","content":"는"},"done":false}
{"model":"gemma:7b","created_at":"2024-09-20T07:44:05.892675721Z","message":{"role":"assistant","content":" 마"},"done":false}
{"model":"gemma:7b","created_at":"2024-09-20T07:44:05.922734686Z","message":{"role":"assistant","content":"드"},"done":false}
{"model":"gemma:7b","created_at":"2024-09-20T07:44:05.94894934Z","message":{"role":"assistant","content":"리"},"done":false}
{"model":"gemma:7b","created_at":"2024-09-20T07:44:05.974847995Z","message":{"role":"assistant","content":"드"},"done":false}
{"model":"gemma:7b","created_at":"2024-09-20T07:44:06.000792996Z","message":{"role":"assistant","content":"입니다"},"done":false}
{"model":"gemma:7b","created_at":"2024-09-20T07:44:06.027347034Z","message":{"role":"assistant","content":"."},"done":false}
{"model":"gemma:7b","created_at":"2024-09-20T07:44:06.054820407Z","message":{"role":"assistant","content":""},"done_reason":"stop","done":true,"total_duration":612398322,"load_duration":38850549,"prompt_eval_count":37,"prompt_eval_duration":104264000,"eval_count":14,"eval_duration":427080000}