CoP: Data Science: Create district types reusable tool (API, single dataset, etc.)

Overview

We need to create a tool so that each project at H4LA that renders points on a map can use District Files to help people analyze or view the data.

Action Items

Identify large groups/districts
Identify links for groups/districts
Locate and obtain shape files for these districts Obtain Shape Files for Different District Types as of Nov/Dec 2021 #124
Determine what files types we will make these available (shp, npm, and/or GeoJSON)
Put files in GitHub repository so they are available to use in the organization.
research how we will create a data set out of this info that will be self updating (meaning are there apis for these groups)
...

Resources

Example Neighborhood Council Shape File

Initial Identification of Large Groups/Districts

City Neighborhood Councils (NCs)
County Supervisory Districts
City Business Improvement Districts (BIDs)
National U.S. House of Representatives that serve LA
State California State Senators that serve LA
State California Assembly members that serve LA

Shapefiles

create a npm package for delivering the data. We need to get a backend person involved and we need to make one for each time they change, so la-shape-files-2021, la-shape-files-2022

next steps are talking to 311 team, tdm team, food oasis, luckparking

Feedback from Mike Morgan on 12/9: Since the shape files for the various districts are small enough (less than 50MB, see here), they can be stored in a repository. We should also consider making these available as npm and GeoJSON.

Notes from 3/11 meeting with Abe, Bonnie, John (Food Oasis) and Mike:

Food Oasis uses PostGRES DB's own geometry data type to run scripts, and then converts to geojson to send to client.

Can take lat/lon and returns NC
Can render a neighborhood on a map

PostGRES can also consume geojson to convert to its proprietary geometric data type.

The recording of the meeting

*Update:* The data collection script now saves *both JSON and GeoJSON* versions of the district boundary files into the shapefiles_output/ folder. An example Jupyter notebook has also been added to show how to load and plot the data, geohub district data collection <https://github.com/parcheesime/geohub-datacollection> For tomorrow’s meeting, maybe we can discuss: - Where we want to run the GitHub Actions workflow (weekly or monthly) - Where the output files should be stashed (repo vs external storage like Google Drive) website: portfolio <https://parcheesime.github.io/portfolio-site/> <https://sites.google.com/view/aletiatrepte/home> linkedin: aletia-trepte <https://www.linkedin.com/in/aletia-trepte/> github: parcheesime <https://github.com/parcheesime>

…

On Mon, Sep 8, 2025 at 5:55 PM Bonnie Wolfe ***@***.***> wrote: *ExperimentsInHonesty* left a comment (hackforla/data-science#118) <#118 (comment)> @parcheesime <https://github.com/parcheesime> @ryanfchase <https://github.com/ryanfchase> and @ExperimentsInHonesty <https://github.com/ExperimentsInHonesty> have a meeting at 5pm this Thursday — Reply to this email directly, view it on GitHub <#118 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AJDAJOFXTI4T6K4EBFTMWVD3RYQPDAVCNFSM6AAAAAB2K4FBH6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTENRYGQ3TQMRYGY> . You are receiving this because you were mentioned.Message ID: ***@***.***>

Update:

Progress: I forked and cloned the new h4la district files, ran the code, updated gitgnore so that json files are available in the repo.
Blockers: None
Availability: I can meet Thursday evenings, after 5.
ETA: 2-3 weeks
Next: create csv log file of most recent shape files collected.

My fork

H4LA district files repo

Update:

Progress: Added logger file to capture run data into csv, commeted out yml for now while testing, updated readme
Availability: I can meet Thursday evenings, after 5.
ETA: 2 weeks
Next: add a few text scripts, set up github action, run workflow, test runs

@parcheesime

Please provide update @parcheesime

Instructions

Progress: "What is the current status of your project? What have you completed and what is left to do?"
Blockers: "Difficulties or errors encountered."
Availability: "How much time will you have this week to work on this issue?"
ETA: "When do you expect this issue to be completed?"
Pictures (if necessary): "Add any pictures that will help illustrate what you are working on."

You can use this template

1. Progress: 
2. Blockers: 
3. Availability:
4. ETA:
5. Pictures (if necessary):

I will give a full update next week. There was a wayer leak in my place and have been displaced and going through repairs. website: portfolio <https://parcheesime.github.io/portfolio-site/> <https://sites.google.com/view/aletiatrepte/home> linkedin: aletia-trepte <https://www.linkedin.com/in/aletia-trepte/> github: parcheesime <https://github.com/parcheesime>

…

On Mon, Oct 13, 2025, 6:21 PM ANDREW W TAYLOR ***@***.***> wrote: *chinaexpert1* left a comment (hackforla/data-science#118) <#118 (comment)> Please provide update @parcheesime <https://github.com/parcheesime> Instructions 1. Progress: "What is the current status of your project? What have you completed and what is left to do?" 2. Blockers: "Difficulties or errors encountered." 3. Availability: "How much time will you have this week to work on this issue?" 4. ETA: "When do you expect this issue to be completed?" 5. Pictures (if necessary): "Add any pictures that will help illustrate what you are working on." You can use this template 1. Progress: 2. Blockers: 3. Availability: 4. ETA: 5. Pictures (if necessary): — Reply to this email directly, view it on GitHub <#118 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AJDAJOBDJYJFHBIVSEUTBNT3XRF2DAVCNFSM6AAAAAB2K4FBH6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTGOJZGYYTMOBRG4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

I was unable to work on the district data collection tool this week as I am still displaced due to repairs of a water leak. That said, this issue is ready for automated updates and has a csv file set to add meta data about each update. I hope to be all settled in by next week and will provide another update. website: portfolio <https://parcheesime.github.io/portfolio-site/> <https://sites.google.com/view/aletiatrepte/home> linkedin: aletia-trepte <https://www.linkedin.com/in/aletia-trepte/> github: parcheesime <https://github.com/parcheesime>

…

On Tue, Oct 14, 2025, 3:46 AM aletia trepte ***@***.***> wrote: I will give a full update next week. There was a wayer leak in my place and have been displaced and going through repairs. website: portfolio <https://parcheesime.github.io/portfolio-site/> <https://sites.google.com/view/aletiatrepte/home> linkedin: aletia-trepte <https://www.linkedin.com/in/aletia-trepte/> github: parcheesime <https://github.com/parcheesime> On Mon, Oct 13, 2025, 6:21 PM ANDREW W TAYLOR ***@***.***> wrote: > *chinaexpert1* left a comment (hackforla/data-science#118) > <#118 (comment)> > > Please provide update @parcheesime <https://github.com/parcheesime> > Instructions > > 1. Progress: "What is the current status of your project? What have > you completed and what is left to do?" > 2. Blockers: "Difficulties or errors encountered." > 3. Availability: "How much time will you have this week to work on > this issue?" > 4. ETA: "When do you expect this issue to be completed?" > 5. Pictures (if necessary): "Add any pictures that will help > illustrate what you are working on." > > You can use this template > > 1. Progress: > 2. Blockers: > 3. Availability: > 4. ETA: > 5. Pictures (if necessary): > > — > Reply to this email directly, view it on GitHub > <#118 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AJDAJOBDJYJFHBIVSEUTBNT3XRF2DAVCNFSM6AAAAAB2K4FBH6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTGOJZGYYTMOBRG4> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> >

Progress:
Set up and tested the new check_drive_folder_access helper to verify Google Drive connectivity. Created a dedicated service account for the project and confirmed the local script runs and logs successfully. The workflow correctly saves shapefiles and logs locally.
Blockers:
The service account can’t be added to the top-level Shared Drive, which prevents access to the project folder. This access is required so the workflow can upload the weekly API call runs (CSV logs and updated shapefiles) to the shared Google Drive where everyone can view them. Without it, the automated Drive upload step can’t complete.
Availability:
Available to proceed once the service account has access to the Shared Drive or an alternative shared location is provided.
ETA:
Within a week after Drive access is granted (to confirm upload, test the workflow end-to-end, and finalize automation).

Hi Aletia, Can you make it to the Monday meetings? I approved the share files request. Hope that works.

…

On Mon, Oct 27, 2025 at 4:49 PM Aletia Trepte ***@***.***> wrote: *parcheesime* left a comment (hackforla/data-science#118) <#118 (comment)> 1. Progress: Set up and tested the new check_drive_folder_access helper to verify Google Drive connectivity. Created a dedicated service account for the project and confirmed the local script runs and logs successfully. The workflow correctly saves shapefiles and logs locally. 2. Blockers: The service account can’t be added to the top-level Shared Drive, which prevents access to the project folder. This access is required so the workflow can upload the weekly API call runs (CSV logs and updated shapefiles) to the shared Google Drive where everyone can view them. Without it, the automated Drive upload step can’t complete. 3. Availability: Available to proceed once the service account has access to the Shared Drive or an alternative shared location is provided. 4. ETA: Within a week after Drive access is granted (to confirm upload, test the workflow end-to-end, and finalize automation). — Reply to this email directly, view it on GitHub <#118 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA6SRHZMRFTVYGP5X6GHE4D3Z2VQLAVCNFSM6AAAAAB2K4FBH6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTINJTHAZTINRSGA> . You are receiving this because you commented.Message ID: ***@***.***>

@chinaexpert1

@chinaexpert1
Update:

I need my service account added to dscop drive so that I can write the district files log data into our shared Data Science COP drive/folder. The service account is one I am using for the district files issue and I shared in Slack in a private message.

I AM able to share the service account with folders within the shared drive. :done:
I am NOT able to share the service account with the Shared Drive. This is needed for the csv log files to access/update.

Can you please add it to the shared drive?

I just updated it.

…

On Mon, Nov 3, 2025, 5:17 PM Aletia Trepte ***@***.***> wrote: *parcheesime* left a comment (hackforla/data-science#118) <#118 (comment)> @chinaexpert1 <https://github.com/chinaexpert1> Update: I need my service account added so that I can write the district files log data into our shared Data Science COP drive/folder. The service account is one I am using for the district files issue and I shared in Slack in a private message. - I AM able to share the service account with folders within the shared drive. :done: - I am NOT able to share the service account with the Shared Drive. This is needed for the csv log files to access/update. Can you please add it to the shared drive? — Reply to this email directly, view it on GitHub <#118 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA6SRH7ER42OMCSRRSNGRY3327WCDAVCNFSM6AAAAAB2K4FBH6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTIOBTGI4DSNRXGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

Progress:
Tested the Google Drive API connection using the new check_drive_folder_access helper. Successfully connected to the Shared Drive and verified access to the target “Shape Files” folder. Next step is to set up automated log uploads to Drive.
Blockers:
No known blockers — service account access confirmed and Drive API working correctly.
Availability:
Available to continue implementation and testing this week.
ETA:
Complete Drive upload integration and testing within 1 week.

Here’s the update:

1. Progress:

Identified and confirmed district types (Assembly, Senate, Neighborhood Councils, Supervisors, Congressional, City Clerk/BIDs, City Council).
Implemented automated data collection using ArcGIS REST endpoints (no auth required).
Built a Python pipeline that fetches district boundaries and outputs both JSON and GeoJSON.
Set up a GitHub Actions workflow that runs automatically on the 1st of every month and can also be triggered manually.
Configured a Google Drive shared folder as the primary distribution location for outputs, with a separate folder for run logs.
Verified end-to-end execution: workflow runs → data processed → files uploaded to shared Drive.

2. Blockers:

None at this time. Initial issues with Drive permissions, folder IDs, and workflow configuration have been resolved.

3. Availability:

Available for follow-ups, documentation updates, or coordination on downstream usage of the Drive-hosted data.

4. ETA:

Pipeline is complete and operational.
Final documentation cleanup and observation of the next scheduled run: ~4 weeks.

5. Links:

Folder with json and geojson files

Folder with log showing latest run and status

She has completed the issue and now just waiting for it to run on the first of the month.

Navigation Menu

Repository navigation

CoP: Data Science: Create district types reusable tool (API, single dataset, etc.) #118

Overview

Action Items

Resources

Initial Identification of Large Groups/Districts

41 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

CoP: Data Science: Create district types reusable tool (API, single dataset, etc.) #118

Description

Overview

Action Items

Resources

Initial Identification of Large Groups/Districts

Activity

ExperimentsInHonesty commented on Dec 7, 2021

ExperimentsInHonesty commented on Dec 7, 2021

akhaleghi commented on Dec 10, 2021

akhaleghi commented on Mar 18, 2022

41 remaining items

parcheesime commented on Sep 10, 2025

parcheesime commented on Sep 23, 2025

parcheesime commented on Sep 30, 2025

chinaexpert1 commented on Oct 14, 2025

parcheesime commented on Oct 14, 2025

parcheesime commented on Oct 21, 2025

parcheesime commented on Oct 27, 2025

chinaexpert1 commented on Oct 28, 2025

parcheesime commented on Nov 4, 2025

chinaexpert1 commented on Nov 4, 2025

parcheesime commented on Nov 11, 2025

parcheesime commented on Jan 3, 2026

chinaexpert1 commented on Jan 6, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions