M1 Ollama+Gemma+LangChain 설치하고 간단히 돌려보기

해당 내용은 Mac (M1) 을 기준으로 작성되었습니다.

 

대규모 언어 모델을 로컬에서 실행하기 위해서 Ollama를 설치하고 구글의 최첨단 경량 오픈모델인 gemma를 다운받아서 간단히 돌려보고

LangChain으로 연결해봅니다.

 

간밤에 "LLM RAG Langchain 통합" 채팅방의 권진영님께서 친절하게 설치와 사용방법을 알려주셔서 다른분들도 간단히 설치해서 사용해보면 좋을거 같아서 정리해봅니다.

 

Ollama github에 가면 로컬환경에 설치가능한 설치파일들을 다운받을수있습니다.

https://github.com/ollama/ollama

GitHub - ollama/ollama: Get up and running with Llama 2, Mistral, Gemma, and other large language models.

 

 

이중 macOS에 해당하는 설치파일을 다운로드 받아서 설치합니다. 설치는 너무 간단해서 의외이기도 합니다. 다운로드 받으면 디렉토리에 설치파일이 생기게되고 더블클릭해서 설치합니다.

다운로드된 설치파일

 

실행하면 아래처럼 설치가 시작됩니다. Next 클릭

설치파일 실행하면 설치시작

설치는 금방 완료됩니다.

 

설치가 완료되면 Terminal을 실행해서 Ollama를 실행합니다.

 

Gemma는 두개 모델을 제공하는데 먼저 가장작은 모델로 시작해 보겠습니다.

% ollama run gemma:2b
pulling manifest
pulling c1864a5eb193...   5% ▕███                     ▏  87 MB/1.7 GB  8.0 MB/s   3m17s

 

설치가 완료되면 메시지를 호출 할수있는 창이 뜨면서 설치가 완료됩니다.

% ollama run gemma:2b
pulling manifest
pulling c1864a5eb193... 100% ▕██████████████████████████████████████████████▏ 1.7 GB
pulling 097a36493f71... 100% ▕██████████████████████████████████████████████▏ 8.4 KB
pulling 109037bec39c... 100% ▕██████████████████████████████████████████████▏  136 B
pulling 22a838ceb7fb... 100% ▕██████████████████████████████████████████████▏   84 B
pulling 887433b89a90... 100% ▕██████████████████████████████████████████████▏  483 B
verifying sha256 digest
writing manifest
removing any unused layers
success
>>> hi
Hi! 👋 How can I assist you today? 😊

Is there anything I can help you with?

>>> Send a message (/? for help)

 

간단하게 "hi"로 인사해 봤습니다.

 

이제 실행창에서 나오도록 합니다 "/bye"를 입력합니다.

>>> /bye
Hello! 👋 It's nice to hear from you. How can I help you today? 😊

Is there anything I can do for you?

>>> /bye
(base) dongsik@dongsikleeui-MacBookPro ~ %

 

바로 내보내주지는 않으면 한번더 "/bye" 합니다

 

설치되어있는 모델을 확인할수도 있습니다.

% ollama list
NAME    	ID          	SIZE  	MODIFIED
gemma:2b	b50d6c999e59	1.7 GB	6 minutes ago

 

또한 설치된 모델을 삭제도 할수있습니다. 설치와 삭제가 너무간단합니다.

% ollama rm gemma:2b
deleted 'gemma:2b'
% ollama list
NAME	ID	SIZE	MODIFIED
%

 

그럼 다시 설치하고 간단히 사용방법을 설명합니다.

 

1. LangChain 으로 실행하기

# LangChain 설치
pip install langchain

 

import langchain
# LangChain 버전 확인
print('LangChain version:', langchain.__version__)

결과
LangChain version: 0.1.12

 

로컬에 설치든 ollama gemma:2b 모델을 사용하도록 설정하고 실행합니다.

from langchain_community.llms import Ollama
import logging

# logging 설정
logging.basicConfig(level=logging.INFO)

try:
    llm = Ollama(model="gemma:2b")
    
    # 프롬프트가 잘 정의되어 있는지 확인하는 것이 필요합니다. (모델의 기능에 따라 조정가능)
    prompt = ("Why is the sky blue?")
    
    response = llm.invoke(prompt)
    print(response)
except ImportError:
    logging.error("Failed to import Ollama from langchain_community. Is the package installed?")
except Exception as e:
    logging.error(f"An unexpected error occurred: {e}")

 

결과 :
The sky appears blue due to Rayleigh scattering. This scattering process occurs when light interacts with molecules in the Earth's atmosphere. 

* **Blue light has a longer wavelength than other colors**. This means it can penetrate further into the atmosphere. 
* **Blue light waves have more energy** than other colors, so they are more likely to scatter. 
* **Water vapor molecules** in the atmosphere absorb blue light more efficiently than other colors. 
* **Scattered blue light** is scattered in all directions equally, giving the sky its blue color.

The amount and intensity of blue scattering depends on several factors, including:

* **Particle size and density of the particles**: Smaller particles scatter light more efficiently than larger particles. 
* **The wavelength of light**: Blue light is scattered more strongly than other colors. 
* **Atmospheric conditions**: Temperature, humidity, and air density can also affect scattering.

Overall, the scattering of sunlight in the atmosphere creates the blue color of the sky.

 

url call로 호출하고 결과를 streaming 방식으로 stand out으로 출력합니다.

from langchain.callbacks.manager import CallbackManager
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler
from langchain_community.llms.ollama import Ollama

llm = Ollama(
    base_url="http://localhost:11434",
    model="gemma:2b",
    callback_manager=CallbackManager(
        [StreamingStdOutCallbackHandler()],
    ),
)

prompt = ("Why is the sky blue?")
response = llm.invoke(prompt)
print(response)

 

결과 :
The sky appears blue due to Rayleigh scattering. Rayleigh scattering is the scattering of light by particles of a shorter wavelength, such as blue light. This scattering causes longer wavelengths, such as red and yellow light, to be scattered more than blue light. As a result, the sky appears blue to us.The sky appears blue due to Rayleigh scattering. Rayleigh scattering is the scattering of light by particles of a shorter wavelength, such as blue light. This scattering causes longer wavelengths, such as red and yellow light, to be scattered more than blue light. As a result, the sky appears blue to us.

 

 

https://github.com/ollama/ollama/blob/main/docs/api.md

2. Command 창에서 curl로 /api/generate

streaming 

Reqeust
% curl http://localhost:11434/api/generate -d '{
  "model": "gemma:2b",
  "prompt":"Why is the sky blue?"
}'

Response
{"model":"gemma:2b","created_at":"2024-04-10T01:46:35.254492Z","response":"The","done":false}
{"model":"gemma:2b","created_at":"2024-04-10T01:46:35.291573Z","response":" sky","done":false}
{"model":"gemma:2b","created_at":"2024-04-10T01:46:35.325664Z","response":" appears","done":false}
... <생략>
{"model":"gemma:2b","created_at":"2024-04-10T01:46:44.741546Z","response":" the","done":false}
{"model":"gemma:2b","created_at":"2024-04-10T01:46:44.775088Z","response":" atmosphere","done":false}
{"model":"gemma:2b","created_at":"2024-04-10T01:46:44.810226Z","response":".","done":false}
{"model":"gemma:2b","created_at":"2024-04-10T01:46:44.845784Z","response":"","done":true,"context":[106,1645,108,4385,603,573,8203,3868,235336,107,108,106,2516,108,651,8203,8149,3868,3402,577,153902,38497,235265,1417,38497,12702,1185,33365,113211,675,24582,575,573,10379,235303,235256,13795,235269,14076,24582,576,23584,578,16175,235265,109,235287,5231,5200,2611,919,476,5543,35571,688,1178,3868,2611,235265,1417,3454,674,1185,33365,30866,573,13795,235269,978,3868,2611,603,30390,3024,774,1167,2116,235265,108,235287,5231,10716,2611,919,476,25270,35571,688,578,603,30390,978,16347,1178,3118,2611,235265,108,235287,714,5231,10526,576,38497,688,12014,611,573,35571,576,573,2611,235265,11569,235269,3868,2611,603,30390,978,1178,1156,9276,235265,108,235287,714,13795,919,978,23584,24582,1178,16175,24582,235269,948,3454,674,978,3118,2611,603,30390,3024,235265,1417,603,3165,573,8203,8149,3868,235265,109,4858,708,1009,5942,4691,1105,573,3868,8203,235292,109,235287,714,3868,2881,603,5231,38131,576,5809,168428,1417,3454,674,573,8203,877,4824,3868,20853,576,1368,5342,689,7033,665,603,235265,108,235287,714,3868,2881,603,1170,5231,1665,10918,731,38636,168428,1417,3454,674,573,8203,877,4824,3868,793,4391,1368,1536,692,708,575,573,2134,235265,108,235287,714,3868,2881,603,476,5231,2667,576,2611,38497,168428,1417,3454,674,2611,603,30390,575,832,16759,731,24582,575,573,13795,235265,108,235287,714,3868,2881,576,573,8203,603,476,5231,28205,44299,168428,1417,603,1861,573,13795,603,780,13596,12876,235269,578,573,38497,2185,12014,611,573,6581,576,573,33365,8761,577,573,16071,575,573,13795,235265,107,108],"total_duration":13164771833,"load_duration":3454744833,"prompt_eval_count":15,"prompt_eval_duration":115430000,"eval_count":282,"eval_duration":9592944000}

 

No streaming 

Request
% curl http://localhost:11434/api/generate -d '{
  "model": "gemma:2b",
  "prompt":"Why is the sky blue?",
  "stream": false
}'

Reponse
{"model":"gemma:2b","created_at":"2024-04-10T01:49:38.296228Z","response":"The sky is blue due to Rayleigh scattering. Rayleigh scattering is the scattering of light by particles of a shorter wavelength. This means that blue light has a greater wavelength and is scattered more than other colors. This is why the sky appears blue.","done":true,"context":[106,1645,108,4385,603,573,8203,3868,235336,107,108,106,2516,108,651,8203,603,3868,3402,577,153902,38497,235265,153902,38497,603,573,38497,576,2611,731,16071,576,476,25270,35571,235265,1417,3454,674,3868,2611,919,476,6561,35571,578,603,30390,978,1178,1156,9276,235265,1417,603,3165,573,8203,8149,3868,235265,107,108],"total_duration":1936533375,"load_duration":3180292,"prompt_eval_duration":272404000,"eval_count":49,"eval_duration":1658092000}

 

3. Command 창에서 curl로 /api/chat 

 

Chat Request (Streaming)

Request
% curl http://localhost:11434/api/chat -d '{
  "model": "gemma:2b",
  "messages": [
    {
      "role": "user",
      "content": "why is the sky blue?"
    }
  ]
}'

Response
{"model":"gemma:2b","created_at":"2024-04-10T01:27:16.070998Z","message":{"role":"assistant","content":"The"},"done":false}
{"model":"gemma:2b","created_at":"2024-04-10T01:27:16.108371Z","message":{"role":"assistant","content":" sky"},"done":false}
{"model":"gemma:2b","created_at":"2024-04-10T01:27:16.142158Z","message":{"role":"assistant","content":" appears"},"done":false}
{"model":"gemma:2b","created_at":"2024-04-10T01:27:16.175229Z","message":{"role":"assistant","content":" blue"},"done":false}
{"model":"gemma:2b","created_at":"2024-04-10T01:27:16.207642Z","message":{"role":"assistant","content":" due"},"done":false}
... <생략>
{"model":"gemma:2b","created_at":"2024-04-10T01:27:25.624649Z","message":{"role":"assistant","content":" higher"},"done":false}
{"model":"gemma:2b","created_at":"2024-04-10T01:27:25.658043Z","message":{"role":"assistant","content":" temperatures"},"done":false}
{"model":"gemma:2b","created_at":"2024-04-10T01:27:25.692536Z","message":{"role":"assistant","content":"."},"done":false}
{"model":"gemma:2b","created_at":"2024-04-10T01:27:25.725932Z","message":{"role":"assistant","content":""},"done":true,"total_duration":12244362334,"load_duration":2484924542,"prompt_eval_count":15,"prompt_eval_duration":103298000,"eval_count":286,"eval_duration":9654690000}

 

Chat request (No streaming)

Request
% curl http://localhost:11434/api/chat -d '{
  "model": "gemma:2b",
  "messages": [
    {
      "role": "user",
      "content": "why is the sky blue?"
    }
  ],
  "stream": false
}'

Response
{"model":"gemma:2b","created_at":"2024-04-10T01:30:16.821415Z","message":{"role":"assistant","content":"The sky appears blue due to Rayleigh scattering. This phenomenon occurs when sunlight interacts with molecules in the Earth's atmosphere.\n\n**Rayleigh Scattering:**\n\n* Sunlight is composed of all colors of the spectrum, including blue, violet, yellow, orange, and red.\n* When sunlight enters the atmosphere, it interacts with molecules such as nitrogen and oxygen molecules.\n* These molecules have different sizes and structures, which cause different wavelengths of light to scatter in different directions.\n* Blue light, with its shorter wavelengths, is scattered more strongly than other colors due to its shorter path length through the atmosphere.\n\n**Blue Sky:**\n\n* As a result, blue light is scattered in all directions from the Sun.\n* This scattering effect spreads out the Sun's light throughout the atmosphere, making the sky appear blue.\n* The intensity of blue light can vary slightly depending on factors such as altitude, temperature, and atmospheric conditions.\n\n**Other Factors:**\n\n* The scattering process depends on the size and density of the molecules, which is why the sky appears blue even though the Sun is a star of much greater temperature.\n* The atmosphere is composed of different gases with varying densities, which influences the scattering process.\n* Cloud and pollution can also affect the sky's color, with clouds reflecting blue light more efficiently than other colors.\n\n**Conclusion:**\n\nThe blue color of the sky is primarily caused by Rayleigh scattering of sunlight by molecules in the Earth's atmosphere. This scattering process spreads out the Sun's light throughout the sky, making it appear blue to us on Earth."},"done":true,"total_duration":11417865583,"load_duration":4883667,"prompt_eval_duration":270041000,"eval_count":324,"eval_duration":11140497000}