Data analyst intern
China Unicom, Inner Mongolia Branch
Hohhot, Hohhot
06.2024 - 08.2024
- Developed a Python script using libraries such as Pandas and Regular Expressions to filter a raw dataset of over 4,000 entries. The script identified and removed expired notifications (e.g., holiday and tourist attraction announcements) as well as government service responses containing sensitive data (e.g., phone numbers, addresses), reducing the dataset to over 2,000 valid records.
- Ensured data compliance by completely deleting any sensitive information prior to feeding the sanitized data into an AI-driven Q&A model.
- Overcame challenges associated with inconsistent data formats and varying content quality by designing robust data cleaning and validation routines.
- Successfully imported the refined dataset into a pre-tuned intelligent Q&A system and performed thorough tests to verify its normal operation and data compatibility.
