Skip to content

Commit ff56efd

Browse files
authored
Add files via upload
Upgraded from a command line tool to a library
1 parent 8eb9906 commit ff56efd

7 files changed

Lines changed: 1344 additions & 745 deletions

File tree

Dockerfile

Lines changed: 12 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,12 @@
1-
# syntax=docker/dockerfile:1
2-
3-
FROM python:latest
4-
5-
WORKDIR /app
6-
7-
COPY . .
8-
9-
RUN pip install --upgrade pip && pip install build && python -m build
10-
RUN pip install dist/*.whl
11-
12-
ENTRYPOINT ["searchcode"]
1+
# syntax=docker/dockerfile:1
2+
3+
FROM python:latest
4+
5+
WORKDIR /app
6+
7+
COPY . .
8+
9+
RUN pip install --upgrade pip && pip install build && python -m build
10+
RUN pip install dist/*.whl
11+
12+
ENTRYPOINT ["searchcode"]

LICENSE

Lines changed: 674 additions & 674 deletions
Large diffs are not rendered by default.

README.md

Lines changed: 30 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -1,23 +1,30 @@
1-
![searchcode-cli_banner](https://user-images.githubusercontent.com/74001397/203441377-ad53a2ab-16d6-42b3-bbec-542c9ed43534.png)
2-
3-
4-
A Python wrapper around the [Searchcode API](https://searchcode.com/api/)
5-
6-
[![Upload Python Package](https://github.com/rly0nheart/searchcode-cli/actions/workflows/python-publish.yml/badge.svg)](https://github.com/rly0nheart/searchcode-cli/actions/workflows/python-publish.yml) [![CodeQL](https://github.com/rly0nheart/searchcode-cli/actions/workflows/codeql.yml/badge.svg)](https://github.com/rly0nheart/searchcode-cli/actions/workflows/codeql.yml)
7-
8-
# Documentation
9-
[Refer to the Wiki](https://github.com/rly0nheart/searchcode-cli/wiki) for installation instructions, code examples, in addition to all other documentation.
10-
11-
# Notice
12-
> This project is developed with permission from [Benjamin Boyter](https://boyter.org/about/), the developer of [searchcode.com](https://searchcode.com).
13-
Therefore, I only take credit for this project. Not the API.
14-
15-
# About SearchCode
16-
Read more about searchcode [here](https://searchcode.com/about/)
17-
18-
# Support
19-
If you like this project and would like to show support, you can Buy Me A Coffee using the button below
20-
21-
<a href="https://www.buymeacoffee.com/189381184" target="_blank"><img src="https://cdn.buymeacoffee.com/buttons/default-orange.png" alt="Buy Me A Coffee" height="41" width="174"></a>
22-
23-
Your support will be much appreciated!😊
1+
# searchcode-cli
2+
3+
![searchcode-cli_banner](https://user-images.githubusercontent.com/74001397/203441377-ad53a2ab-16d6-42b3-bbec-542c9ed43534.png)
4+
5+
[![Upload Python Package](https://github.com/rly0nheart/searchcode-cli/actions/workflows/python-publish.yml/badge.svg)](https://github.com/rly0nheart/searchcode-cli/actions/workflows/python-publish.yml) [![CodeQL](https://github.com/rly0nheart/searchcode-cli/actions/workflows/codeql.yml/badge.svg)](https://github.com/rly0nheart/searchcode-cli/actions/workflows/codeql.yml)
6+
7+
Command line client for [Searchcode](https://searchcode.com)
8+
9+
# Features
10+
- [x] Queries the code index and returns at most 100 results.
11+
- [x] Returns the raw data from a code file given the code id which can be found as the id in a code search result.
12+
- [x] Returns an array of results given a searchcode unique code id which are considered to be duplicates.
13+
- [ ] Saves results to a JSON file (coming soon...)
14+
15+
# TODO
16+
- [ ] Create GUI (in progress...)
17+
18+
# Wiki
19+
[Refer to the Wiki](https://github.com/rly0nheart/searchcode-cli/wiki) for installation instructions, in addition to all other documentation.
20+
21+
22+
# About SearchCode
23+
Read more about searchcode [here](https://searchcode.com/about/)
24+
25+
# Donations
26+
If you like **searchcode-cli** and would like to donate, you could Buy Me A Coffee using the button below
27+
28+
<a href="https://www.buymeacoffee.com/189381184" target="_blank"><img src="https://cdn.buymeacoffee.com/buttons/default-orange.png" alt="Buy Me A Coffee" height="41" width="174"></a>
29+
30+
Your support will be much appreciated!😊

pyproject.toml

Lines changed: 31 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -1,35 +1,31 @@
1-
[build-system]
2-
requires = ["setuptools", "setuptools-scm"]
3-
build-backend = "setuptools.build_meta"
4-
5-
[project]
6-
name = "searchcode-cli"
7-
version = "1.3.1"
8-
description = "Command line client for Searchcode.com"
9-
readme = "README.md"
10-
requires-python = ">=3.8"
11-
license = {file = "LICENSE"}
12-
keywords = ["searchcode", "search-engine", "codesearch", "searchcode-server"]
13-
authors = [{name = "Richard Mwewa", email = "rly0nheart@duck.com"}]
14-
classifiers = [
15-
"Development Status :: 5 - Production/Stable",
16-
"Programming Language :: Python :: 3",
17-
"Intended Audience :: Information Technology",
18-
"License :: OSI Approved :: GNU General Public License v3 (GPLv3)",
19-
"Operating System :: OS Independent",
20-
"Natural Language :: English",
21-
""
22-
]
23-
24-
dependencies = [
25-
"rich",
26-
"requests"
27-
]
28-
29-
[project.urls]
30-
homepage = "https://github.com/rly0nheart/searchcode-cli"
31-
documentation = "https://github.com/rly0nheart/searchcode-cli/wiki"
32-
repository = "https://github.com/rly0nheart/searchcode-cli.git"
33-
34-
[project.scripts]
35-
searchcode = "searchcode.searchcode:searchcode"
1+
[build-system]
2+
requires = ["setuptools", "setuptools-scm"]
3+
build-backend = "setuptools.build_meta"
4+
5+
[project]
6+
name = "searchcode-cli"
7+
version = "2.0.0"
8+
description = "A Python wrapper around the searchcode API"
9+
readme = "README.md"
10+
requires-python = ">=3.9"
11+
license = {file = "LICENSE"}
12+
keywords = ["searchcode", "search-engine", "codesearch", "api-wrapper"]
13+
authors = [{name = "Richard Mwewa", email = "rly0nheart@duck.com"}]
14+
classifiers = [
15+
"Development Status :: 5 - Production/Stable",
16+
"Programming Language :: Python :: 3",
17+
"Intended Audience :: Information Technology",
18+
"License :: OSI Approved :: GNU General Public License v3 (GPLv3)",
19+
"Operating System :: OS Independent",
20+
"Natural Language :: English",
21+
""
22+
]
23+
24+
[tool.setuptools]
25+
packages = ["searchcode"]
26+
27+
[project.urls]
28+
homepage = "https://github.com/rly0nheart/searchcode"
29+
documentation = "https://github.com/rly0nheart/searchcode-cli/wiki"
30+
repository = "https://github.com/rly0nheart/searchcode-cli.git"
31+

searchcode/__init__.py

Lines changed: 141 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1,141 @@
1-
1+
# Search code api endpoints
2+
from searchcode.connection import __api_handler
3+
from searchcode.miscellaneous import __code_sources, __code_languages, __api_endpoints
4+
5+
6+
def __join_parameters(names, sources) -> str:
7+
"""
8+
Returns a single string of joined parameters for the given list of filter names and sources.
9+
10+
Parameters:
11+
-----------
12+
names (list):
13+
A list of names used as filters.
14+
sources (list):
15+
A list of tuples representing the sources to be filtered.
16+
17+
Returns:
18+
--------
19+
str: A single string of joined parameters for the given list of filter names.
20+
21+
About
22+
-----
23+
The __join_parameters function takes a list of names and sources as input and iterates over each name.
24+
For each name, it iterates over the sources list and compares the lowercase version of the
25+
source name to the lowercase version of each source name in the sources list.
26+
If a match is found, the corresponding parameter is added to the parameters string,
27+
and the loop breaks out of the inner loop to move on to the next source name.
28+
After iterating over all the source names, the function returns the parameters string.
29+
"""
30+
parameters = ""
31+
for name in names:
32+
for source_name, source_parameter in sources:
33+
if source_name.lower() == name.lower():
34+
parameters += source_parameter
35+
break
36+
37+
return parameters
38+
39+
40+
def code_search(query: str, page_number: int = 0, per_page: int = 10, max_retries: int = 3, backoff_time: int = 10,
41+
code_sources: list = None, code_languages: list = None) -> list:
42+
"""
43+
Queries the code index and returns at most 10 results.
44+
If the results list is empty, then this indicates that you have reached the end of the available results.
45+
To fetch all results for a given query, keep incrementing 'page_number'
46+
until you get a page with an empty results list.
47+
48+
Parameters:
49+
----------
50+
query (str):
51+
The search query string.
52+
page_number (int):
53+
Result page starting at 0 through to 49.
54+
per_page (int):
55+
Number of results wanted per page max 100.
56+
max_retries (int):
57+
The maximum number of retries in case of a failed request.
58+
backoff_time (int):
59+
The number of seconds to wait before retrying a failed request.
60+
code_sources : list
61+
A list of source names used as filters.
62+
code_languages : list
63+
A list of programming language names used as filters.
64+
65+
Returns:
66+
-------
67+
list: A list of code search results.
68+
69+
About
70+
-----
71+
The code_search function queries the code index by sending an HTTP request to the code_search API endpoint.
72+
It accepts several optional parameters, including query, page_number, per_page, max_retries, backoff_time,
73+
code_sources, and code_languages.
74+
The sources and languages lists are filtered by calling the __join_parameters function,
75+
which returns a single string of joined parameters based on the filter names and sources.
76+
The function then calls the __api_handler function to handle the API request, which returns a dictionary of results.
77+
The results list is then extracted and returned.
78+
If the results list is empty, this indicates that you have reached the end of the available results.
79+
To fetch all results for a given query, keep incrementing 'page_number' until you get a page with an empty results list.
80+
"""
81+
if code_languages is None:
82+
code_languages = []
83+
if code_sources is None:
84+
code_sources = []
85+
86+
code_search_endpoint = __api_endpoints()[0]
87+
sources = __join_parameters(code_sources, __code_sources())
88+
languages = __join_parameters(code_languages, __code_languages())
89+
90+
return __api_handler(code_search_endpoint.format(query, page_number, per_page, sources, languages),
91+
max_retries, backoff_time)['results']
92+
93+
94+
def code_result(code_id: int = None, max_retries: int = 3, backoff_time: int = 10) -> str:
95+
"""
96+
Retrieves the raw data for a code file based on the given code ID.
97+
98+
Parameters:
99+
-----------
100+
code_id (int):
101+
The ID of the code file to retrieve.
102+
max_retries (int):, optional
103+
The maximum number of times to retry the API call in case of a failure. Defaults to 3.
104+
backoff_time (int):, optional
105+
The number of seconds to wait before retrying the API call in case of a failure. Defaults to 5.
106+
107+
Returns:
108+
--------
109+
str: The raw data for the code file, if the API call is successful.
110+
111+
About
112+
-----
113+
This function takes a `code_id` parameter which is the ID of a code file as returned by a code_search
114+
result. It then sends a request to the corresponding API endpoint to retrieve the raw data for that
115+
code file. If the request is successful, the raw data is returned.
116+
"""
117+
code_result_endpoint = __api_endpoints()[1]
118+
return __api_handler(code_result_endpoint.format(code_id), max_retries, backoff_time)['code']
119+
120+
121+
def related_results(code_id: int = None, max_retries: int = 3, backoff_time: int = 10) -> list:
122+
"""
123+
Queries the related code results endpoint of the Searchcode API and returns an array of results
124+
that are considered to be duplicates of the code file identified by the given code_id.
125+
126+
Parameters:
127+
-----------
128+
code_id (int):
129+
The unique code id of the code file for which to retrieve related results.
130+
max_retries (int):
131+
The maximum number of times to retry the API call in case of failures. Default is 3.
132+
backoff_time (int):
133+
The time in seconds to wait between retries in case of failures. Default is 5.
134+
135+
Returns:
136+
--------
137+
list: A list of related results, where each result is a dictionary containing metadata
138+
about a code file, such as its filename, repository, and lines of code.
139+
"""
140+
related_results_endpoint = __api_endpoints()[2]
141+
return __api_handler(related_results_endpoint.format(code_id), max_retries, backoff_time)

searchcode/connection.py

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
import json
2+
import urllib.error
3+
import urllib.request
4+
from typing import Union
5+
6+
7+
def __api_handler(endpoint: str) -> Union[dict, list, str]:
8+
"""
9+
Sends a request to the designated API endpoint and returns the response.
10+
If the API returns a 429 error code, indicating that the user has sent too many requests in a short period of time,
11+
the handler will extract the retry-after header value and return it as an integer.
12+
13+
Parameters:
14+
-----------
15+
endpoint (str): A string representing the API endpoint to send a request to.
16+
17+
Returns:
18+
--------
19+
Union[dict, list, str]: If the response is successful, returns the response data as a dictionary, list or string,
20+
depending on the API endpoint.
21+
22+
Raises:
23+
-------
24+
Exception: If the function was unable to get a valid response after making the request, or if the retry-after
25+
header is missing in the API response.
26+
"""
27+
try:
28+
with urllib.request.urlopen(endpoint) as response:
29+
if response.status == 429:
30+
# If the API returns a 429 response, extract the retry-after header value
31+
retry_after = response.getheader("Retry-After")
32+
if retry_after:
33+
raise Exception(f"Too many requests. Please try again in {retry_after}.")
34+
else:
35+
# Retry-after header not found
36+
raise Exception("Too many requests. Retry-after header not found in the response.")
37+
else:
38+
api_data = response.read().decode("utf-8")
39+
return json.loads(api_data)
40+
except (urllib.error.HTTPError, urllib.error.URLError) as error:
41+
# Catch any network errors or timeouts
42+
raise Exception(error)
43+

0 commit comments

Comments
 (0)