[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-9693":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":16,"stars7d":17,"stars30d":18,"stars90d":16,"forks30d":16,"starsTrendScore":16,"compositeScore":19,"rankGlobal":10,"rankLanguage":10,"license":20,"archived":21,"fork":21,"defaultBranch":22,"hasWiki":21,"hasPages":23,"topics":24,"createdAt":10,"pushedAt":10,"updatedAt":39,"readmeContent":40,"aiSummary":41,"trendingCount":16,"starSnapshotCount":16,"syncStatus":42,"lastSyncTime":43,"discoverSource":44},9693,"clip-as-service","jina-ai\u002Fclip-as-service","jina-ai","🏄 Scalable embedding, reasoning, ranking for images and sentences with CLIP","https:\u002F\u002Fclip-as-service.jina.ai",null,"Python",12826,2068,218,290,0,1,3,44.95,"Other",false,"main",true,[25,26,5,27,28,29,30,31,32,33,34,35,36,37,38],"bert","bert-as-service","clip-model","cross-modal-retrieval","cross-modality","deep-learning","image2vec","multi-modality","neural-search","onnx","openai","pytorch","sentence-encoding","sentence2vec","2026-06-12 02:02:11","\u003Cp align=\"center\">\n\u003Ca href=\"https:\u002F\u002Fclip-as-service.jina.ai\">\u003Cimg src=\"https:\u002F\u002Fgithub.com\u002Fjina-ai\u002Fclip-as-service\u002Fblob\u002Fmain\u002Fdocs\u002F_static\u002Flogo-light.svg?raw=true\" alt=\"CLIP-as-service logo: The data structure for unstructured data\" width=\"200px\">\u003C\u002Fa>\n\u003Cbr>\u003Cbr>\u003Cbr>\n\u003C\u002Fp>\n\n\n\u003Cp align=center>\n\u003Ca href=\"https:\u002F\u002Fpypi.org\u002Fproject\u002Fclip_server\u002F\">\u003Cimg alt=\"PyPI\" src=\"https:\u002F\u002Fimg.shields.io\u002Fpypi\u002Fv\u002Fclip_server?label=Release&style=flat-square\">\u003C\u002Fa>\n\u003Ca href=\"https:\u002F\u002Fdiscord.jina.ai\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fdiscord\u002F1106542220112302130?logo=discord&logoColor=white&style=flat-square\">\u003C\u002Fa>\n\u003Ca href=\"https:\u002F\u002Fcodecov.io\u002Fgh\u002Fjina-ai\u002Fclip-as-service\">\u003Cimg alt=\"Codecov branch\" src=\"https:\u002F\u002Fimg.shields.io\u002Fcodecov\u002Fc\u002Fgithub\u002Fjina-ai\u002Fclip-as-service\u002Fmain?logo=Codecov&logoColor=white&style=flat-square\">\u003C\u002Fa>\n\u003Ca href=\"https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002Fjina-ai\u002Fclip-as-service\u002Fblob\u002Fmain\u002Fdocs\u002Fhosting\u002Fcas-on-colab.ipynb\">\u003Cimg src=\"https:\u002F\u002Fimg.shields.io\u002Fbadge\u002FHost-on%20Google%20Colab%20(GPU\u002FTPU)-brightgreen?style=flat-square&logo=googlecolab&&logoColor=white\" alt=\"Host on Google Colab with GPU\u002FTPU support\">\u003C\u002Fa>\n\u003C\u002Fp>\n\n\u003C!-- start elevator-pitch -->\n\nCLIP-as-service is a low-latency high-scalability service for embedding images and text. It can be easily integrated as a microservice into neural search solutions.\n\n⚡ **Fast**: Serve CLIP models with TensorRT, ONNX runtime and PyTorch w\u002Fo JIT with 800QPS\u003Csup>[*]\u003C\u002Fsup>. Non-blocking duplex streaming on requests and responses, designed for large data and long-running tasks. \n\n🫐 **Elastic**: Horizontally scale up and down multiple CLIP models on single GPU, with automatic load balancing.\n\n🐥 **Easy-to-use**: No learning curve, minimalist design on client and server. Intuitive and consistent API for image and sentence embedding. \n\n👒 **Modern**: Async client support. Easily switch between gRPC, HTTP, WebSocket protocols with TLS and compression.\n\n🍱 **Integration**: Smooth integration with neural search ecosystem including [Jina](https:\u002F\u002Fgithub.com\u002Fjina-ai\u002Fjina) and [DocArray](https:\u002F\u002Fgithub.com\u002Fjina-ai\u002Fdocarray). Build cross-modal and multi-modal solutions in no time. \n\n\u003Csup>[*] with default config (single replica, PyTorch no JIT) on GeForce RTX 3090. \u003C\u002Fsup>\n\n\u003C!-- end elevator-pitch -->\n\n### Text & image embedding\n\n\u003Ctable>\n\u003Ctr>\n\u003Ctd> via HTTPS 🔐 \u003C\u002Ftd>\n\u003Ctd> via gRPC 🔐⚡⚡ \u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>\n\n```bash\ncurl \\\n-X POST https:\u002F\u002F\u003Cyour-inference-address>-http.wolf.jina.ai\u002Fpost \\\n-H 'Content-Type: application\u002Fjson' \\\n-H 'Authorization: \u003Cyour access token>' \\\n-d '{\"data\":[{\"text\": \"First do it\"}, \n    {\"text\": \"then do it right\"}, \n    {\"text\": \"then do it better\"}, \n    {\"uri\": \"https:\u002F\u002Fpicsum.photos\u002F200\"}], \n    \"execEndpoint\":\"\u002F\"}'\n```\n\n\u003C\u002Ftd>\n\u003Ctd>\n\n```python\n# pip install clip-client\nfrom clip_client import Client\n\nc = Client(\n    'grpcs:\u002F\u002F\u003Cyour-inference-address>-grpc.wolf.jina.ai',\n    credential={'Authorization': '\u003Cyour access token>'},\n)\n\nr = c.encode(\n    [\n        'First do it',\n        'then do it right',\n        'then do it better',\n        'https:\u002F\u002Fpicsum.photos\u002F200',\n    ]\n)\nprint(r)\n```\n\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003C\u002Ftable>\n\n### Visual reasoning\n\nThere are four basic visual reasoning skills: object recognition, object counting, color recognition, and spatial relation understanding. Let's try some:\n\n> You need to install [`jq` (a JSON processor)](https:\u002F\u002Fstedolan.github.io\u002Fjq\u002F) to prettify the results.\n\n\u003Ctable>\n\u003Ctr>\n\u003Ctd> Image \u003C\u002Ftd>\n\u003Ctd> via HTTPS 🔐 \u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>\n\u003Cimg src=\"https:\u002F\u002Fpicsum.photos\u002Fid\u002F1\u002F300\u002F300\">\n\u003C\u002Ftd>\n\u003Ctd>\n\n```bash\ncurl \\\n-X POST https:\u002F\u002F\u003Cyour-inference-address>-http.wolf.jina.ai\u002Fpost \\\n-H 'Content-Type: application\u002Fjson' \\\n-H 'Authorization: \u003Cyour access token>' \\\n-d '{\"data\":[{\"uri\": \"https:\u002F\u002Fpicsum.photos\u002Fid\u002F1\u002F300\u002F300\",\n\"matches\": [{\"text\": \"there is a woman in the photo\"},\n            {\"text\": \"there is a man in the photo\"}]}],\n            \"execEndpoint\":\"\u002Frank\"}' \\\n| jq \".data[].matches[] | (.text, .scores.clip_score.value)\"\n```\n\ngives:\n\n```\n\"there is a woman in the photo\"\n0.626907229423523\n\"there is a man in the photo\"\n0.37309277057647705\n```\n\n\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>\n\u003Cimg src=\"https:\u002F\u002Fpicsum.photos\u002Fid\u002F133\u002F300\u002F300\">\n\u003C\u002Ftd>\n\u003Ctd>\n\n```bash\ncurl \\\n-X POST https:\u002F\u002F\u003Cyour-inference-address>-http.wolf.jina.ai\u002Fpost \\\n-H 'Content-Type: application\u002Fjson' \\\n-H 'Authorization: \u003Cyour access token>' \\\n-d '{\"data\":[{\"uri\": \"https:\u002F\u002Fpicsum.photos\u002Fid\u002F133\u002F300\u002F300\",\n\"matches\": [\n{\"text\": \"the blue car is on the left, the red car is on the right\"},\n{\"text\": \"the blue car is on the right, the red car is on the left\"},\n{\"text\": \"the blue car is on top of the red car\"},\n{\"text\": \"the blue car is below the red car\"}]}],\n\"execEndpoint\":\"\u002Frank\"}' \\\n| jq \".data[].matches[] | (.text, .scores.clip_score.value)\"\n```\n\ngives:\n```\n\"the blue car is on the left, the red car is on the right\"\n0.5232442617416382\n\"the blue car is on the right, the red car is on the left\"\n0.32878655195236206\n\"the blue car is below the red car\"\n0.11064132302999496\n\"the blue car is on top of the red car\"\n0.03732786327600479\n```\n\n\u003C\u002Ftd>\n\u003C\u002Ftr>\n\n\n\u003Ctr>\n\u003Ctd>\n\u003Cimg src=\"https:\u002F\u002Fpicsum.photos\u002Fid\u002F102\u002F300\u002F300\">\n\u003C\u002Ftd>\n\u003Ctd>\n\n```bash\ncurl \\\n-X POST https:\u002F\u002F\u003Cyour-inference-address>-http.wolf.jina.ai\u002Fpost \\\n-H 'Content-Type: application\u002Fjson' \\\n-H 'Authorization: \u003Cyour access token>' \\\n-d '{\"data\":[{\"uri\": \"https:\u002F\u002Fpicsum.photos\u002Fid\u002F102\u002F300\u002F300\",\n\"matches\": [{\"text\": \"this is a photo of one berry\"},\n            {\"text\": \"this is a photo of two berries\"},\n            {\"text\": \"this is a photo of three berries\"},\n            {\"text\": \"this is a photo of four berries\"},\n            {\"text\": \"this is a photo of five berries\"},\n            {\"text\": \"this is a photo of six berries\"}]}],\n            \"execEndpoint\":\"\u002Frank\"}' \\\n| jq \".data[].matches[] | (.text, .scores.clip_score.value)\"\n```\n\ngives:\n```\n\"this is a photo of three berries\"\n0.48507222533226013\n\"this is a photo of four berries\"\n0.2377079576253891\n\"this is a photo of one berry\"\n0.11304923892021179\n\"this is a photo of five berries\"\n0.0731358453631401\n\"this is a photo of two berries\"\n0.05045759305357933\n\"this is a photo of six berries\"\n0.04057715833187103\n```\n\n\u003C\u002Ftd>\n\u003C\u002Ftr>\n\n\n\u003C\u002Ftable>\n\n\n## [Documentation](https:\u002F\u002Fclip-as-service.jina.ai)\n\n## Install\n\nCLIP-as-service consists of two Python packages `clip-server` and `clip-client` that can be installed _independently_. Both require Python 3.7+. \n\n### Install server\n\n\u003Ctable>\n\u003Ctr>\n\u003Ctd> Pytorch Runtime ⚡ \u003C\u002Ftd>\n\u003Ctd> ONNX Runtime ⚡⚡\u003C\u002Ftd>\n\u003Ctd> TensorRT Runtime ⚡⚡⚡ \u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>\n\n```bash\npip install clip-server\n```\n\n\u003C\u002Ftd>\n\u003Ctd>\n\n```bash\npip install \"clip-server[onnx]\"\n```\n\n\u003C\u002Ftd>\n\u003Ctd>\n\n```bash\npip install nvidia-pyindex \npip install \"clip-server[tensorrt]\"\n```\n\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003C\u002Ftable>\n\nYou can also [host the server on Google Colab](https:\u002F\u002Fclip-as-service.jina.ai\u002Fhosting\u002Fcolab\u002F), leveraging its free GPU\u002FTPU.\n\n### Install client\n\n```bash\npip install clip-client\n```\n\n### Quick check\n\nYou can run a simple connectivity check after install.\n\n\n\u003Ctable>\n\u003Ctr>\n\u003Cth> C\u002FS \u003C\u002Fth> \n\u003Cth> Command \u003C\u002Fth> \n\u003Cth> Expect output \u003C\u002Fth>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>\nServer\n\u003C\u002Ftd>\n\u003Ctd> \n\n```bash\npython -m clip_server\n```\n     \n\u003C\u002Ftd>\n\u003Ctd>\n\n\u003Cimg src=\"https:\u002F\u002Fgithub.com\u002Fjina-ai\u002Fclip-as-service\u002Fblob\u002Fmain\u002F.github\u002FREADME-img\u002Fserver-output.svg?raw=true\" alt=\"Expected server output\" width=\"300px\">\n\n\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>\nClient\n\u003C\u002Ftd>\n\u003Ctd> \n\n```python\nfrom clip_client import Client\n\nc = Client('grpc:\u002F\u002F0.0.0.0:23456')\nc.profile()\n```\n     \n\u003C\u002Ftd>\n\u003Ctd>\n\n\u003Cimg src=\"https:\u002F\u002Fgithub.com\u002Fjina-ai\u002Fclip-as-service\u002Fblob\u002Fmain\u002F.github\u002FREADME-img\u002Fpyclient-output.svg?raw=true\" alt=\"Expected clip-client output\" width=\"300px\">\n\n\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003C\u002Ftable>\n\n\nYou can change `0.0.0.0` to the intranet or public IP address to test the connectivity over private and public network. \n\n\n## Get Started\n\n### Basic usage\n\n1. Start the server: `python -m clip_server`. Remember its address and port.\n2. Create a client:\n   ```python\n    from clip_client import Client\n   \n    c = Client('grpc:\u002F\u002F0.0.0.0:51000')\n    ```\n3. To get sentence embedding:\n    ```python    \n    r = c.encode(['First do it', 'then do it right', 'then do it better'])\n    \n    print(r.shape)  # [3, 512] \n    ```\n4. To get image embedding:\n    ```python    \n    r = c.encode(['apple.png',  # local image \n                  'https:\u002F\u002Fclip-as-service.jina.ai\u002F_static\u002Ffavicon.png',  # remote image\n                  'data:image\u002Fgif;base64,R0lGODlhEAAQAMQAAORHHOVSKudfOulrSOp3WOyDZu6QdvCchPGolfO0o\u002FXBs\u002FfNwfjZ0frl3\u002Fzy7\u002F\u002F\u002F\u002FwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACH5BAkAABAALAAAAAAQABAAAAVVICSOZGlCQAosJ6mu7fiyZeKqNKToQGDsM8hBADgUXoGAiqhSvp5QAnQKGIgUhwFUYLCVDFCrKUE1lBavAViFIDlTImbKC5Gm2hB0SlBCBMQiB0UjIQA7'])  # in image URI\n    \n    print(r.shape)  # [3, 512]\n    ```\n\nMore comprehensive server and client user guides can be found in the [docs](https:\u002F\u002Fclip-as-service.jina.ai\u002F).\n\n### Text-to-image cross-modal search in 10 lines\n\nLet's build a text-to-image search using CLIP-as-service. Namely, a user can input a sentence and the program returns matching images. We'll use the [Totally Looks Like](https:\u002F\u002Fsites.google.com\u002Fview\u002Ftotally-looks-like-dataset) dataset and [DocArray](https:\u002F\u002Fgithub.com\u002Fjina-ai\u002Fdocarray) package. Note that DocArray is included within `clip-client` as an upstream dependency, so you don't need to install it separately.\n\n#### Load images\n\nFirst we load images. You can simply pull them from Jina Cloud:\n\n```python\nfrom docarray import DocumentArray\n\nda = DocumentArray.pull('ttl-original', show_progress=True, local_cache=True)\n```\n\n\u003Cdetails>\n\u003Csummary>or download TTL dataset, unzip, load manually\u003C\u002Fsummary>\n\nAlternatively, you can go to [Totally Looks Like](https:\u002F\u002Fsites.google.com\u002Fview\u002Ftotally-looks-like-dataset) official website, unzip and load images:\n\n```python\nfrom docarray import DocumentArray\n\nda = DocumentArray.from_files(['left\u002F*.jpg', 'right\u002F*.jpg'])\n```\n\n\u003C\u002Fdetails>\n\nThe dataset contains 12,032 images, so it may take a while to pull. Once done, you can visualize it and get the first taste of those images:\n\n```python\nda.plot_image_sprites()\n```\n\n\u003Cp align=\"center\">\n\u003Cimg src=\"https:\u002F\u002Fgithub.com\u002Fjina-ai\u002Fclip-as-service\u002Fblob\u002Fmain\u002F.github\u002FREADME-img\u002Fttl-image-sprites.png?raw=true\" alt=\"Visualization of the image sprite of Totally looks like dataset\" width=\"50%\">\n\u003C\u002Fp>\n\n#### Encode images\n\nStart the server with `python -m clip_server`. Let's say it's at `0.0.0.0:51000` with `GRPC` protocol (you will get this information after running the server).\n\nCreate a Python client script:\n\n```python\nfrom clip_client import Client\n\nc = Client(server='grpc:\u002F\u002F0.0.0.0:51000')\n\nda = c.encode(da, show_progress=True)\n```\n\nDepending on your GPU and client-server network, it may take a while to embed 12K images. In my case, it took about two minutes.\n\n\u003Cdetails>\n\u003Csummary>Download the pre-encoded dataset\u003C\u002Fsummary>\n\nIf you're impatient or don't have a GPU, waiting can be Hell. In this case, you can simply pull our pre-encoded image dataset:\n\n```python\nfrom docarray import DocumentArray\n\nda = DocumentArray.pull('ttl-embedding', show_progress=True, local_cache=True)\n```\n\n\u003C\u002Fdetails>\n\n#### Search via sentence \n\nLet's build a simple prompt to allow a user to type sentence:\n\n```python\nwhile True:\n    vec = c.encode([input('sentence> ')])\n    r = da.find(query=vec, limit=9)\n    r[0].plot_image_sprites()\n```\n\n#### Showcase\n\nNow you can input arbitrary English sentences and view the top-9 matching images. Search is fast and instinctive. Let's have some fun:\n\n\u003Ctable>\n\u003Ctr>\n\u003Cth> \"a happy potato\" \u003C\u002Fth> \n\u003Cth> \"a super evil AI\" \u003C\u002Fth> \n\u003Cth> \"a guy enjoying his burger\" \u003C\u002Fth>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>\n\n\u003Cp align=\"center\">\n\u003Cimg src=\"https:\u002F\u002Fgithub.com\u002Fjina-ai\u002Fclip-as-service\u002Fblob\u002Fmain\u002F.github\u002FREADME-img\u002Fa-happy-potato.png?raw=true\" alt=\"Visualization of the image sprite of Totally looks like dataset\" width=\"100%\">\n\u003C\u002Fp>\n\n\u003C\u002Ftd>\n\u003Ctd>\n\n\u003Cp align=\"center\">\n\u003Cimg src=\"https:\u002F\u002Fgithub.com\u002Fjina-ai\u002Fclip-as-service\u002Fblob\u002Fmain\u002F.github\u002FREADME-img\u002Fa-super-evil-AI.png?raw=true\" alt=\"Visualization of the image sprite of Totally looks like dataset\" width=\"100%\">\n\u003C\u002Fp>\n\n\u003C\u002Ftd>\n\u003Ctd>\n\n\u003Cp align=\"center\">\n\u003Cimg src=\"https:\u002F\u002Fgithub.com\u002Fjina-ai\u002Fclip-as-service\u002Fblob\u002Fmain\u002F.github\u002FREADME-img\u002Fa-guy-enjoying-his-burger.png?raw=true\" alt=\"Visualization of the image sprite of Totally looks like dataset\" width=\"100%\">\n\u003C\u002Fp>\n\n\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003C\u002Ftable>\n\n\n\u003Ctable>\n\u003Ctr>\n\u003Cth> \"professor cat is very serious\" \u003C\u002Fth> \n\u003Cth> \"an ego engineer lives with parent\" \u003C\u002Fth> \n\u003Cth> \"there will be no tomorrow so lets eat unhealthy\" \u003C\u002Fth>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>\n\n\u003Cp align=\"center\">\n\u003Cimg src=\"https:\u002F\u002Fgithub.com\u002Fjina-ai\u002Fclip-as-service\u002Fblob\u002Fmain\u002F.github\u002FREADME-img\u002Fprofessor-cat-is-very-serious.png?raw=true\" alt=\"Visualization of the image sprite of Totally looks like dataset\" width=\"100%\">\n\u003C\u002Fp>\n\n\u003C\u002Ftd>\n\u003Ctd>\n\n\u003Cp align=\"center\">\n\u003Cimg src=\"https:\u002F\u002Fgithub.com\u002Fjina-ai\u002Fclip-as-service\u002Fblob\u002Fmain\u002F.github\u002FREADME-img\u002Fan-ego-engineer-lives-with-parent.png?raw=true\" alt=\"Visualization of the image sprite of Totally looks like dataset\" width=\"100%\">\n\u003C\u002Fp>\n\n\u003C\u002Ftd>\n\u003Ctd>\n\n\u003Cp align=\"center\">\n\u003Cimg src=\"https:\u002F\u002Fgithub.com\u002Fjina-ai\u002Fclip-as-service\u002Fblob\u002Fmain\u002F.github\u002FREADME-img\u002Fthere-will-be-no-tomorrow-so-lets-eat-unhealthy.png?raw=true\" alt=\"Visualization of the image sprite of Totally looks like dataset\" width=\"100%\">\n\u003C\u002Fp>\n\n\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003C\u002Ftable>\n\nLet's save the embedding result for our next example: \n\n```python\nda.save_binary('ttl-image')\n```\n\n### Image-to-text cross-modal search in 10 Lines\n\nWe can also switch the input and output of the last program to achieve image-to-text search. Precisely, given a query image find the sentence that best describes the image.\n\nLet's use all sentences from the book \"Pride and Prejudice\". \n\n```python\nfrom docarray import Document, DocumentArray\n\nd = Document(uri='https:\u002F\u002Fwww.gutenberg.org\u002Ffiles\u002F1342\u002F1342-0.txt').load_uri_to_text()\nda = DocumentArray(\n    Document(text=s.strip()) for s in d.text.replace('\\r\\n', '').split('.') if s.strip()\n)\n```\n\nLet's look at what we got:\n\n```python\nda.summary()\n```\n\n```text\n            Documents Summary            \n                                         \n  Length                 6403            \n  Homogenous Documents   True            \n  Common Attributes      ('id', 'text')  \n                                         \n                     Attributes Summary                     \n                                                            \n  Attribute   Data type   #Unique values   Has empty value  \n ────────────────────────────────────────────────────────── \n  id          ('str',)    6403             False            \n  text        ('str',)    6030             False            \n```\n\n#### Encode sentences\n\nNow encode these 6,403 sentences, it may take 10 seconds or less depending on your GPU and network: \n\n```python\nfrom clip_client import Client\n\nc = Client('grpc:\u002F\u002F0.0.0.0:51000')\n\nr = c.encode(da, show_progress=True)\n```\n\n\u003Cdetails>\n\u003Csummary>Download the pre-encoded dataset\u003C\u002Fsummary>\n\nAgain, for people who are impatient or don't have a GPU, we have prepared a pre-encoded text dataset:\n\n```python\nfrom docarray import DocumentArray\n\nda = DocumentArray.pull('ttl-textual', show_progress=True, local_cache=True)\n```\n\n\u003C\u002Fdetails>\n\n#### Search via image\n\nLet's load our previously stored image embedding, randomly sample 10 image Documents, then find top-1 nearest neighbour of each.\n\n```python\nfrom docarray import DocumentArray\n\nimg_da = DocumentArray.load_binary('ttl-image')\n\nfor d in img_da.sample(10):\n    print(da.find(d.embedding, limit=1)[0].text)\n```\n\n#### Showcase\n\nFun time! Note, unlike the previous example, here the input is an image and the sentence is the output. All sentences come from the book \"Pride and Prejudice\". \n\n\u003Ctable>\n\u003Ctr>\n\u003Ctd>\n\u003Cp align=\"center\">\n\u003Cimg src=\"https:\u002F\u002Fgithub.com\u002Fjina-ai\u002Fclip-as-service\u002Fblob\u002Fmain\u002F.github\u002FREADME-img\u002FBesides,-there-was-truth-in-his-looks.png?raw=true\" alt=\"Visualization of the image sprite of Totally looks like dataset\" height=\"100px\">\n\u003C\u002Fp>\n\n\n\u003C\u002Ftd>\n\u003Ctd>\n\n\u003Cp align=\"center\">\n\u003Cimg src=\"https:\u002F\u002Fgithub.com\u002Fjina-ai\u002Fclip-as-service\u002Fblob\u002Fmain\u002F.github\u002FREADME-img\u002FGardiner-smiled.png?raw=true\" alt=\"Visualization of the image sprite of Totally looks like dataset\" height=\"100px\">\n\u003C\u002Fp>\n\n\u003C\u002Ftd>\n\u003Ctd>\n\n\u003Cp align=\"center\">\n\u003Cimg src=\"https:\u002F\u002Fgithub.com\u002Fjina-ai\u002Fclip-as-service\u002Fblob\u002Fmain\u002F.github\u002FREADME-img\u002Fwhat’s-his-name.png?raw=true\" alt=\"Visualization of the image sprite of Totally looks like dataset\" height=\"100px\">\n\u003C\u002Fp>\n\n\u003C\u002Ftd>\n\u003Ctd>\n\n\u003Cp align=\"center\">\n\u003Cimg src=\"https:\u002F\u002Fgithub.com\u002Fjina-ai\u002Fclip-as-service\u002Fblob\u002Fmain\u002F.github\u002FREADME-img\u002FBy-tea-time,-however,-the-dose-had-been-enough,-and-Mr.png?raw=true\" alt=\"Visualization of the image sprite of Totally looks like dataset\" height=\"100px\">\n\u003C\u002Fp>\n\n\u003C\u002Ftd>\n\n\u003Ctd>\n\n\u003Cp align=\"center\">\n\u003Cimg src=\"https:\u002F\u002Fgithub.com\u002Fjina-ai\u002Fclip-as-service\u002Fblob\u002Fmain\u002F.github\u002FREADME-img\u002FYou-do-not-look-well.png?raw=true\" alt=\"Visualization of the image sprite of Totally looks like dataset\" height=\"100px\">\n\u003C\u002Fp>\n\n\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>Besides, there was truth in his looks\u003C\u002Ftd>\n\u003Ctd>Gardiner smiled\u003C\u002Ftd>\n\u003Ctd>what’s his name\u003C\u002Ftd>\n\u003Ctd>By tea time, however, the dose had been enough, and Mr\u003C\u002Ftd>\n\u003Ctd>You do not look well\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003C\u002Ftable>\n\n\u003Ctable>\n\u003Ctr>\n\u003Ctd>\n\u003Cp align=\"center\">\n\u003Cimg src=\"https:\u002F\u002Fgithub.com\u002Fjina-ai\u002Fclip-as-service\u002Fblob\u002Fmain\u002F.github\u002FREADME-img\u002F“A-gamester!”-she-cried.png?raw=true\" alt=\"Visualization of the image sprite of Totally looks like dataset\" height=\"100px\">\n\u003C\u002Fp>\n\n\n\u003C\u002Ftd>\n\u003Ctd>\n\n\u003Cp align=\"center\">\n\u003Cimg src=\"https:\u002F\u002Fgithub.com\u002Fjina-ai\u002Fclip-as-service\u002Fblob\u002Fmain\u002F.github\u002FREADME-img\u002FIf-you-mention-my-name-at-the-Bell,-you-will-be-attended-to.png?raw=true\" alt=\"Visualization of the image sprite of Totally looks like dataset\" height=\"100px\">\n\u003C\u002Fp>\n\n\u003C\u002Ftd>\n\u003Ctd>\n\n\u003Cp align=\"center\">\n\u003Cimg src=\"https:\u002F\u002Fgithub.com\u002Fjina-ai\u002Fclip-as-service\u002Fblob\u002Fmain\u002F.github\u002FREADME-img\u002FNever-mind-Miss-Lizzy’s-hair.png?raw=true\" alt=\"Visualization of the image sprite of Totally looks like dataset\" height=\"100px\">\n\u003C\u002Fp>\n\n\u003C\u002Ftd>\n\u003Ctd>\n\n\u003Cp align=\"center\">\n\u003Cimg src=\"https:\u002F\u002Fgithub.com\u002Fjina-ai\u002Fclip-as-service\u002Fblob\u002Fmain\u002F.github\u002FREADME-img\u002FElizabeth-will-soon-be-the-wife-of-Mr.png?raw=true\" alt=\"Visualization of the image sprite of Totally looks like dataset\" height=\"100px\">\n\u003C\u002Fp>\n\n\u003C\u002Ftd>\n\n\u003Ctd>\n\n\u003Cp align=\"center\">\n\u003Cimg src=\"https:\u002F\u002Fgithub.com\u002Fjina-ai\u002Fclip-as-service\u002Fblob\u002Fmain\u002F.github\u002FREADME-img\u002FI-saw-them-the-night-before-last.png?raw=true\" alt=\"Visualization of the image sprite of Totally looks like dataset\" height=\"100px\">\n\u003C\u002Fp>\n\n\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\n\u003Ctd>“A gamester!” she cried\u003C\u002Ftd>\n\u003Ctd>If you mention my name at the Bell, you will be attended to\u003C\u002Ftd>\n\u003Ctd>Never mind Miss Lizzy’s hair\u003C\u002Ftd>\n\u003Ctd>Elizabeth will soon be the wife of Mr\u003C\u002Ftd>\n\u003Ctd>I saw them the night before last\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003C\u002Ftable>\n\n\n\n### Rank image-text matches via CLIP model\n\nFrom `0.3.0` CLIP-as-service adds a new `\u002Frank` endpoint that re-ranks cross-modal matches according to their joint likelihood in CLIP model. For example, given an image Document with some predefined sentence matches as below:\n\n```python\nfrom clip_client import Client\nfrom docarray import Document\n\nc = Client(server='grpc:\u002F\u002F0.0.0.0:51000')\nr = c.rank(\n    [\n        Document(\n            uri='.github\u002FREADME-img\u002Frerank.png',\n            matches=[\n                Document(text=f'a photo of a {p}')\n                for p in (\n                    'control room',\n                    'lecture room',\n                    'conference room',\n                    'podium indoor',\n                    'television studio',\n                )\n            ],\n        )\n    ]\n)\n\nprint(r['@m', ['text', 'scores__clip_score__value']])\n```\n\n```text\n[['a photo of a television studio', 'a photo of a conference room', 'a photo of a lecture room', 'a photo of a control room', 'a photo of a podium indoor'], \n[0.9920725226402283, 0.006038925610482693, 0.0009973491542041302, 0.00078492151806131, 0.00010626466246321797]]\n```\n\nOne can see now `a photo of a television studio` is ranked to the top with `clip_score` score at `0.992`. In practice, one can use this endpoint to re-rank the matching result from another search system, for improving the cross-modal search quality.\n\n\u003Ctable>\n\u003Ctr>\n\u003Ctd>\n\u003Cimg src=\"https:\u002F\u002Fgithub.com\u002Fjina-ai\u002Fclip-as-service\u002Fblob\u002Fmain\u002F.github\u002FREADME-img\u002Frerank.png?raw=true\" alt=\"Rerank endpoint image input\" height=\"150px\">\n\u003C\u002Ftd>\n\u003Ctd>\n\u003Cimg src=\"https:\u002F\u002Fgithub.com\u002Fjina-ai\u002Fclip-as-service\u002Fblob\u002Fmain\u002F.github\u002FREADME-img\u002Frerank-chart.svg?raw=true\" alt=\"Rerank endpoint output\">\n\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003C\u002Ftable>\n\n### Rank text-image matches via CLIP model\n\nIn the [DALL·E Flow](https:\u002F\u002Fgithub.com\u002Fjina-ai\u002Fdalle-flow) project, CLIP is called for ranking the generated results from DALL·E. [It has an Executor wrapped on top of `clip-client`](https:\u002F\u002Fgithub.com\u002Fjina-ai\u002Fdalle-flow\u002Fblob\u002Fmain\u002Fexecutors\u002Frerank\u002Fexecutor.py), which calls `.arank()` - the async version of `.rank()`:\n\n```python\nfrom clip_client import Client\nfrom jina import Executor, requests, DocumentArray\n\n\nclass ReRank(Executor):\n    def __init__(self, clip_server: str, **kwargs):\n        super().__init__(**kwargs)\n        self._client = Client(server=clip_server)\n\n    @requests(on='\u002F')\n    async def rerank(self, docs: DocumentArray, **kwargs):\n        return await self._client.arank(docs)\n```\n\n\u003Cp align=\"center\">\n\u003Cimg src=\"https:\u002F\u002Fgithub.com\u002Fjina-ai\u002Fclip-as-service\u002Fblob\u002Fmain\u002F.github\u002FREADME-img\u002Fclient-dalle.png?raw=true\" alt=\"CLIP-as-service used in DALLE Flow\" width=\"300px\">\n\u003C\u002Fp>\n\nIntrigued? That's only scratching the surface of what CLIP-as-service is capable of. [Read our docs to learn more](https:\u002F\u002Fclip-as-service.jina.ai).\n\n\u003C!-- start support-pitch -->\n## Support\n\n- Join our [Discord community](https:\u002F\u002Fdiscord.jina.ai) and chat with other community members about ideas.\n- Watch our [Engineering All Hands](https:\u002F\u002Fyoutube.com\u002Fplaylist?list=PL3UBBWOUVhFYRUa_gpYYKBqEAkO4sxmne) to learn Jina's new features and stay up-to-date with the latest AI techniques.\n- Subscribe to the latest video tutorials on our [YouTube channel](https:\u002F\u002Fyoutube.com\u002Fc\u002Fjina-ai)\n\n## Join Us\n\nCLIP-as-service is backed by [Jina AI](https:\u002F\u002Fjina.ai) and licensed under [Apache-2.0](.\u002FLICENSE). [We are actively hiring](https:\u002F\u002Fjobs.jina.ai) AI engineers, solution engineers to build the next neural search ecosystem in open-source.\n\n\u003C!-- end support-pitch -->\n","CLIP-as-service 是一个用于图像和文本嵌入的低延迟高可扩展性服务。其核心功能包括使用 TensorRT、ONNX 运行时和 PyTorch 无 JIT 编译来提供高达 800QPS 的 CLIP 模型服务，支持非阻塞双工流处理，适用于大数据量和长时间运行的任务。该服务具有弹性伸缩能力，能够在单个 GPU 上水平扩展多个 CLIP 模型，并自动负载均衡。此外，它提供了简洁易用的客户端和服务端设计，以及直观一致的 API 接口。CLIP-as-service 支持异步客户端，可以轻松切换 gRPC、HTTP 和 WebSocket 协议，并具备 TLS 和压缩功能。该项目非常适合需要快速集成到神经搜索解决方案中的场景，如构建跨模态或多模态应用。",2,"2026-06-11 03:24:14","top_topic"]