Repository navigation
CoP: Data Science: SEIE Survey Analysis #26
Description
Overview
We need to report on our progress for our partner at the Department of Neighborhood Empowerment
Action Items
Phase 1--Completed by Sathwik:
- NLP engineering/analysis on a large survey featuring free text columns.
- Run TF-IDF analysis
- create word clouds
- Create a presentation to show data to DONE and neighborhood council members. Tooling could include scikit-learn, spacy, pandas. First version here
Phase 2--Picked up by Henry (8.2.2021):
- Create filtering system on google sheets (labels)
- Revisit preprocess script and apply to dataset (ex. remove stopwords, puncuation, duplicates, etc)
- Frequency counts on most common unigrams, bigrams and trigrams for One Question
- [x] On the entire dataset
- [x] By Region - Create clear view of responses containing key phrases
- Number of council members who mention the "top" topics
- Present to Julien
Phase 3
- Expand to the rest of the questionnaire
- Create dashboard with results of quantitative analysis, showing comparisons of themes topic by region, and comparisons of regions grouped by topic. (Dashboard link)
- Develop presentation/write-up for final delivery to Julien
Resources/Instructions
- HFLA Final Writeup
- Supporting Datapack for Final Writeup
- Final Slide Deck
- Spreadsheet provide by DONE
- SEIE Survey Summary (So. LA) provided by DONE
Currently Underway:
@henrykaplan
Metadata
Metadata
Assignees
Labels
Type
Projects
- Status
Activity
ExperimentsInHonesty commentedon Feb 26, 2021
@RinkalAgarwal can self assign when she has accepted the github invite.
ExperimentsInHonesty commentedon Feb 26, 2021
@sathwikkes Please provide process update
sathwikkes commentedon Feb 26, 2021
screenshots of sample work are provided below
ExperimentsInHonesty commentedon Mar 26, 2021
@RinkalAgarwal @sathwikkes
Please provide update
sathwikkes commentedon Mar 26, 2021
sathwikkes commentedon Apr 28, 2021
snooravi commentedon Aug 11, 2021
Presented updated to Julien here based on progress made in Collab notebook
snooravi commentedon Sep 28, 2021
Affinity Map here
21 remaining items
henrykaplan commentedon Jun 3, 2022
@akhaleghi Yes, let's plan on that.
akhaleghi commentedon Jun 30, 2022
@henrykaplan are there any recent updates on this issue?
henrykaplan commentedon Aug 17, 2022
A working draft of the dashboard is here: https://datastudio.google.com/u/0/reporting/45c01785-e1ac-447f-8442-d617e0193d7a/page/Y8wrC
I'll be looking at it again and to edit the text and tweak the visauls, but the calculations and analysis are complete.
akhaleghi commentedon Aug 31, 2022
Hey @henrykaplan, I think there is just one action item left on this issue (developing a presentation). Is that something you wanted to work on or we can assign that to another volunteer?
henrykaplan commentedon Aug 31, 2022
Hey @akhaleghi,
Last I heard, the project leads were leaning towards a write-up rather than a presentation as a final deliverable, but were still undecided about what it should look like. I'm planning to step off this project and on to something else, so I created the google data studio dashboard as the final writeup and results for everything I've personally worked on or know closely. That's mainly the natural language analysis side of this project.
My goal was that the dashboard could either be the final deliverable, or, should you want to continue changing or adding to the analysis, a way to hand off the work I've done so far to the next volunteer. So I think the project leads should take a look at the draft dashboard and decide where they want to take it from here. And then I'll do whatever remains to be done to package up the analysis I've done, whether for the client or for another volunteer.
akhaleghi commentedon Jan 26, 2023
@henrykaplan are there any loose ends that need to be tied up on this issue or is the project complete?
[-]SEIE Survey Analysis[/-][+]CoP: Data Science: SEIE Survey Analysis[/+]