!curl -fsSL https://ollama.com/install.sh | sh
>>> Installing ollama to /usr/local
>>> Downloading Linux amd64 bundle
############################################################################################# 100.0%
>>> Creating ollama user...
>>> Adding ollama user to video group...
>>> Adding current user to ollama group...
>>> Creating ollama systemd service...
WARNING: Unable to detect NVIDIA/AMD GPU. Install lspci or lshw to automatically detect and install GPU dependencies.
>>> The Ollama API is now available at 127.0.0.1:11434.
>>> Install complete. Run "ollama" from the command line.
Ollama 시작 (실행)
nohup을 사용하여 백그라운드에서 Ollama를 시작합니다. API 서비스 포트는 11434 입니다.
!nohup ollama serve > ollama.log &
nohup: redirecting stderr to stdout
Ollama 모델 목록 확인
!ollama list
NAME ID SIZE MODIFIED
추론
명령행(command line)을 이용해서 gemma 7B 모델을 받고 추론을 실행
!ollama run gemma:7b "대한민국의 수도는 어디야?" 2> ollama.log
대한민국의 수도는 서울입니다.</end_of_turn>
영어로 질문해봅니다.
ollama run gemma:7b "What is the capital of Korea?" 2> ollama.log
The capital of Korea is **Seoul**. It is the largest and most populous city in the country.
실행한 모델 확인
!ollama list
NAME ID SIZE MODIFIED
gemma:7b a72c7f4d0a15 5.0 GB 41 seconds ago
M1 교체 이전에 사용하던 MacBook Pro 2015에서 Ollama로 Gemma2를 구동해봤습니다.
MacBook Pro 2015
Processor:Multi-core CPU (Intel i5/i7/i9 or AMD equivalent) Memory:You should have at least 8 GB of RAM available to run the 7B models, 16 GB to run the 13B models, and 32 GB to run the 33B models. Storage:At least 20 GB of free space on an SSD Operating System:Recent versions of Windows, macOS, or Linux
Python 3.11.4로 생성된 Conda 가상환경으로 실행합니다.
(base) Gabrielui-MacBookPro:~ gabriel$ conda env list
# conda environments:
#
base * /usr/local/anaconda3
gabriel /usr/local/anaconda3/envs/gabriel
(base) Gabrielui-MacBookPro:~ gabriel$ conda activate gabriel
(gabriel) Gabrielui-MacBookPro:~ gabriel$
(gabriel) Gabrielui-MacBookPro:~ gabriel$ python -V
Python 3.11.4
(gabriel) Gabrielui-MacBookPro:~ gabriel$ ollama --version
ollama version is 0.3.9
(gabriel) Gabrielui-MacBookPro:~ gabriel$ ollama run gemma2
pulling manifest
pulling ff1d1fc78170... 100% ▕█████████████████████████████████████████████████████████████████▏ 5.4 GB
pulling 109037bec39c... 100% ▕█████████████████████████████████████████████████████████████████▏ 136 B
pulling 097a36493f71... 100% ▕█████████████████████████████████████████████████████████████████▏ 8.4 KB
pulling 2490e7468436... 100% ▕█████████████████████████████████████████████████████████████████▏ 65 B
pulling 10aa81da732e... 100% ▕█████████████████████████████████████████████████████████████████▏ 487 B
verifying sha256 digest
writing manifest
success
>>>
다운로드한 모델을 확인합니다.
(gabriel) Gabrielui-MacBookPro:~ gabriel$ ollama list
NAME ID SIZE MODIFIED
gemma2:latest ff02c3702f32 5.4 GB 14 minutes ago
(gabriel) Gabrielui-MacBookPro:~ gabriel$
Ollama + Gemma2 (9B)
ollama run gemma2 로 다운로드한 모델의 정보를 확인합니다.
>>> /show info
Model
arch gemma2
parameters 9.2B
quantization Q4_0
context length 8192
embedding length 3584
Parameters
stop "<start_of_turn>"
stop "<end_of_turn>"
License
Gemma Terms of Use
Last modified: February 21, 2024
>>>
프롬프트로 질문 "why is the sky blue?"
>>> why is the sky blue?
The sky appears blue due to a phenomenon called **Rayleigh scattering**.
Here's a breakdown:
* **Sunlight:** Sunlight is made up of all the colors of the rainbow.
* **Atmosphere:** When sunlight enters Earth's atmosphere, it collides with tiny air molecules (mainly nitrogen and
oxygen).
* **Scattering:** These molecules scatter the sunlight in all directions.
* **Wavelengths:** Blue light has a shorter wavelength than other colors in the visible spectrum. Shorter wavelengths are
scattered more effectively by the air molecules.
Therefore, blue light gets scattered much more than other colors, making the sky appear blue to our eyes.
**At sunrise and sunset:** The sunlight travels through more atmosphere to reach us. This means even more blue light is
scattered away, allowing longer wavelengths like orange and red to dominate, resulting in those beautiful hues.
>>> Send a message (/? for help)
Use Ctrl + d or /bye to exit.
(gabriel) Gabrielui-MacBookPro:~ gabriel$ ollama list
NAME ID SIZE MODIFIED
gemma:2b b50d6c999e59 1.7 GB 3 minutes ago
gemma2:latest ff02c3702f32 5.4 GB 40 minutes ago
(gabriel) Gabrielui-MacBookPro:~ gabriel$
Gemma2와 동일한 질문을 합니다. "why is the sky blue?"
>>> why is the sky blue?
The sky appears blue due to Rayleigh scattering.
**Rayleigh scattering** is a scattering phenomenon where light waves are scattered in different directions depending on their wavelength.
Blue light has a shorter wavelength than other colors of light, meaning it scatters more effectively. This is why the sky appears blue.
Here's a more detailed explanation:
* **Sunlight** contains all colors of the rainbow.
* **Blue light** has a shorter wavelength than other colors.
* **When blue light** interacts with molecules in the atmosphere, it **scatters more strongly** than other colors.
* This scattering causes blue light to be scattered **towards the observer**.
* The scattered blue light is what we see in the sky as **blue color**.
**Other factors that contribute to the blue color of the sky:**
* **Dust particles:** Air molecules can also scatter light, but their scattering effect is generally weaker than that of molecules in the
atmosphere.
* **Water droplets:** Water droplets in the atmosphere can also scatter light, but their effect is typically less significant than that of
molecules.
* **Sun's position:** The sky appears blue because the sun is lower in the sky, meaning it is directly overhead. This means that more blue
light reaches our eyes.
In summary, the blue color of the sky is due to Rayleigh scattering of light by molecules in the atmosphere, which primarily results from
the shorter wavelength of blue light.
>>> Send a message (/? for help)
실행시에 MacBook의 성능을 확인합니다.
실행한 모델 정보 확인
>>> /show info
Model
arch gemma
parameters 3B
quantization Q4_0
context length 8192
embedding length 2048
Parameters
repeat_penalty 1
stop "<start_of_turn>"
stop "<end_of_turn>"
License
Gemma Terms of Use
Last modified: February 21, 2024
>>> Send a message (/? for help)
(gabriel) Gabrielui-MacBookPro:~ gabriel$ ollama list
NAME ID SIZE MODIFIED
gemma:7b a72c7f4d0a15 5.0 GB 2 weeks ago
gemma:2b b50d6c999e59 1.7 GB 2 weeks ago
gemma2:latest ff02c3702f32 5.4 GB 2 weeks ago
Gemma2와 동일한 질문을 합니다. "why is the sky blue?"
>>> why is the sky blue?
**The sky is blue due to a phenomenon called Rayleigh scattering.**
* Sunlight is composed of all the colors of the rainbow, each with a different wavelength.
* Shorter wavelengths of light, such as blue light, have higher energy and more tightly spaced photons.
* When sunlight interacts with molecules in the atmosphere, such as nitrogen and oxygen, the shorter wavelengths are
scattered more effectively.
**Here's how it works:**
1. Sunlight enters the atmosphere and interacts with molecules of nitrogen and oxygen.
2. These molecules have electrons that can be excited by the energy of the incoming light.
3. The electrons absorb the energy and vibrate, then emit it in all directions.
4. However, **blue light is scattered more efficiently than other colors** because its shorter wavelength allows for more
frequent collisions between the photons and the molecules.
**This scattering process results in:**
* More blue light is scattered in all directions, reaching our eyes and making the sky appear blue.
* Longer wavelengths of light, such as red light, are scattered less efficiently, so we see more of them directly from
the sun.
**Additional factors influencing the sky's color:**
* **Time of day:** The sky is generally bluer at midday when the sunlight travels through less atmosphere.
* **Cloud cover:** Clouds can block the sunlight and scatter less blue light, making the sky appear less blue.
* **Pollution:** Pollution in the atmosphere can also scatter light and affect the sky's color.
>>> Send a message (/? for help)
실행한 모델 정보 확인
>>> /show info
Model
arch gemma
parameters 9B
quantization Q4_0
context length 8192
embedding length 3072
Parameters
penalize_newline false
repeat_penalty 1
stop "<start_of_turn>"
stop "<end_of_turn>"
License
Gemma Terms of Use
Last modified: February 21, 2024
>>> Send a message (/? for help)
질의 성능 확인
모델 실행 성능 확인
Intel CPU의 MacBook에서 Gemma2 9B 모델 그리고 Gemma 2B, 7B 모델까지 Ollama로 구동해 봤습니다.
% ollama run gemma:2b
pulling manifest
pulling c1864a5eb193... 100% ▕██████████████████████████████████████████████▏ 1.7 GB
pulling 097a36493f71... 100% ▕██████████████████████████████████████████████▏ 8.4 KB
pulling 109037bec39c... 100% ▕██████████████████████████████████████████████▏ 136 B
pulling 22a838ceb7fb... 100% ▕██████████████████████████████████████████████▏ 84 B
pulling 887433b89a90... 100% ▕██████████████████████████████████████████████▏ 483 B
verifying sha256 digest
writing manifest
removing any unused layers
success
>>> hi
Hi! 👋 How can I assist you today? 😊
Is there anything I can help you with?
>>> Send a message (/? for help)
간단하게 "hi"로 인사해 봤습니다.
이제 실행창에서 나오도록 합니다 "/bye"를 입력합니다.
>>> /bye
Hello! 👋 It's nice to hear from you. How can I help you today? 😊
Is there anything I can do for you?
>>> /bye
(base) dongsik@dongsikleeui-MacBookPro ~ %
바로 내보내주지는 않으면 한번더 "/bye" 합니다
설치되어있는 모델을 확인할수도 있습니다.
% ollama list
NAME ID SIZE MODIFIED
gemma:2b b50d6c999e59 1.7 GB 6 minutes ago
또한 설치된 모델을 삭제도 할수있습니다. 설치와 삭제가 너무간단합니다.
% ollama rm gemma:2b
deleted 'gemma:2b'
% ollama list
NAME ID SIZE MODIFIED
%
그럼 다시 설치하고 간단히 사용방법을 설명합니다.
1. LangChain 으로 실행하기
# LangChain 설치
pip install langchain
import langchain
# LangChain 버전 확인
print('LangChain version:', langchain.__version__)
결과
LangChain version: 0.1.12
로컬에 설치든 ollama gemma:2b 모델을 사용하도록 설정하고 실행합니다.
from langchain_community.llms import Ollama
import logging
# logging 설정
logging.basicConfig(level=logging.INFO)
try:
llm = Ollama(model="gemma:2b")
# 프롬프트가 잘 정의되어 있는지 확인하는 것이 필요합니다. (모델의 기능에 따라 조정가능)
prompt = ("Why is the sky blue?")
response = llm.invoke(prompt)
print(response)
except ImportError:
logging.error("Failed to import Ollama from langchain_community. Is the package installed?")
except Exception as e:
logging.error(f"An unexpected error occurred: {e}")
결과 :
The sky appears blue due to Rayleigh scattering. This scattering process occurs when light interacts with molecules in the Earth's atmosphere.
* **Blue light has a longer wavelength than other colors**. This means it can penetrate further into the atmosphere.
* **Blue light waves have more energy** than other colors, so they are more likely to scatter.
* **Water vapor molecules** in the atmosphere absorb blue light more efficiently than other colors.
* **Scattered blue light** is scattered in all directions equally, giving the sky its blue color.
The amount and intensity of blue scattering depends on several factors, including:
* **Particle size and density of the particles**: Smaller particles scatter light more efficiently than larger particles.
* **The wavelength of light**: Blue light is scattered more strongly than other colors.
* **Atmospheric conditions**: Temperature, humidity, and air density can also affect scattering.
Overall, the scattering of sunlight in the atmosphere creates the blue color of the sky.
url call로 호출하고 결과를 streaming 방식으로 stand out으로 출력합니다.
from langchain.callbacks.manager import CallbackManager
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler
from langchain_community.llms.ollama import Ollama
llm = Ollama(
base_url="http://localhost:11434",
model="gemma:2b",
callback_manager=CallbackManager(
[StreamingStdOutCallbackHandler()],
),
)
prompt = ("Why is the sky blue?")
response = llm.invoke(prompt)
print(response)
결과 :
The sky appears blue due to Rayleigh scattering. Rayleigh scattering is the scattering of light by particles of a shorter wavelength, such as blue light. This scattering causes longer wavelengths, such as red and yellow light, to be scattered more than blue light. As a result, the sky appears blue to us.The sky appears blue due to Rayleigh scattering. Rayleigh scattering is the scattering of light by particles of a shorter wavelength, such as blue light. This scattering causes longer wavelengths, such as red and yellow light, to be scattered more than blue light. As a result, the sky appears blue to us.
Request
% curl http://localhost:11434/api/generate -d '{
"model": "gemma:2b",
"prompt":"Why is the sky blue?",
"stream": false
}'
Reponse
{"model":"gemma:2b","created_at":"2024-04-10T01:49:38.296228Z","response":"The sky is blue due to Rayleigh scattering. Rayleigh scattering is the scattering of light by particles of a shorter wavelength. This means that blue light has a greater wavelength and is scattered more than other colors. This is why the sky appears blue.","done":true,"context":[106,1645,108,4385,603,573,8203,3868,235336,107,108,106,2516,108,651,8203,603,3868,3402,577,153902,38497,235265,153902,38497,603,573,38497,576,2611,731,16071,576,476,25270,35571,235265,1417,3454,674,3868,2611,919,476,6561,35571,578,603,30390,978,1178,1156,9276,235265,1417,603,3165,573,8203,8149,3868,235265,107,108],"total_duration":1936533375,"load_duration":3180292,"prompt_eval_duration":272404000,"eval_count":49,"eval_duration":1658092000}
Request
% curl http://localhost:11434/api/chat -d '{
"model": "gemma:2b",
"messages": [
{
"role": "user",
"content": "why is the sky blue?"
}
],
"stream": false
}'
Response
{"model":"gemma:2b","created_at":"2024-04-10T01:30:16.821415Z","message":{"role":"assistant","content":"The sky appears blue due to Rayleigh scattering. This phenomenon occurs when sunlight interacts with molecules in the Earth's atmosphere.\n\n**Rayleigh Scattering:**\n\n* Sunlight is composed of all colors of the spectrum, including blue, violet, yellow, orange, and red.\n* When sunlight enters the atmosphere, it interacts with molecules such as nitrogen and oxygen molecules.\n* These molecules have different sizes and structures, which cause different wavelengths of light to scatter in different directions.\n* Blue light, with its shorter wavelengths, is scattered more strongly than other colors due to its shorter path length through the atmosphere.\n\n**Blue Sky:**\n\n* As a result, blue light is scattered in all directions from the Sun.\n* This scattering effect spreads out the Sun's light throughout the atmosphere, making the sky appear blue.\n* The intensity of blue light can vary slightly depending on factors such as altitude, temperature, and atmospheric conditions.\n\n**Other Factors:**\n\n* The scattering process depends on the size and density of the molecules, which is why the sky appears blue even though the Sun is a star of much greater temperature.\n* The atmosphere is composed of different gases with varying densities, which influences the scattering process.\n* Cloud and pollution can also affect the sky's color, with clouds reflecting blue light more efficiently than other colors.\n\n**Conclusion:**\n\nThe blue color of the sky is primarily caused by Rayleigh scattering of sunlight by molecules in the Earth's atmosphere. This scattering process spreads out the Sun's light throughout the sky, making it appear blue to us on Earth."},"done":true,"total_duration":11417865583,"load_duration":4883667,"prompt_eval_duration":270041000,"eval_count":324,"eval_duration":11140497000}
