Instructor:

• Anna Monreale
  • KDDLab, Università di Pisa
  • anna [dot] monreale [at] unipi [dot] it

Teaching Assistant:

• Francesca Naretto
  • KDDLab, SNS, Pisa
  • francesca [dot] naretto [at] sns [dot] it

• [28.10.2021] Lecture of Friday 29.10.2021 will be canceled
• [23.09.2021] Please, fill this document: Student-Lists anf Project groups. On Teams you can find instructions for GroupID
• [06.09.2021] The first lecture of this course will take place on Thursday, 16 Sept 2021.
• [08.09.2021]People that intend to attend the course online should use this link: https://teams.microsoft.com/l/team/19%3aWKvq4kg0XbKZ5pEeiZcarbBXPCYsTvTwMkKZs2PWiHA1%40thread.tacv2/conversations?groupId=aea1385b-6721-4d90-a169-c97f7d066eca&tenantId=c7456b31-a220-47f5-be52-473828670aa1

• Fundamental concepts of data knowledge and discovery.
• Data understanding
• Data preparation
• Clustering
• Classification
• Pattern Mining and Association Rules
• Outlier Detection
• Time Series Analysis
• Sequential Pattern Mining
• Ethical Issues

Classes

Office hours - Ricevimento: Anna Monreale: Wednesday: 11:00-13:00 online using Teams (Appointment by email) Francesca Naretto: Monday: 15:00-18:00 online using Teams (Appointment by email)

• Pang-Ning Tan, Michael Steinbach, Vipin Kumar. Introduction to Data Mining. Addison Wesley, ISBN 0-321-32136-7, 2006
  • http://www-users.cs.umn.edu/~kumar/dmbook/index.php
  • Chapters 4,6 and 8 are also available at the publisher's Web site.
• Berthold, M.R., Borgelt, C., Höppner, F., Klawonn, F. GUIDE TO INTELLIGENT DATA ANALYSIS. Springer Verlag, 1st Edition., 2010. ISBN 978-1-84882-259-7
• Laura Igual et al. Introduction to Data Science: A Python Approach to Concepts, Techniques and Applications. 1st ed. 2017 Edition.
• Jake VanderPlas. Python Data Science Handbook: Essential Tools for Working with Data. 1st Edition.
• For Python Notions: Very basic notions on Python

• The slides used in the course will be inserted in the calendar after each class. Most of them are part of the slides provided by the textbook's authors Slides per "Introduction to Data Mining".

• Python - Anaconda (3.7 version!!!): Anaconda is the leading open data science platform powered by Python. Download page (the following libraries are already included)
• Scikit-learn: python library with tools for data mining and data analysis Documentation page
• Pandas: pandas is an open source, BSD-licensed library providing high-performance, easy-to-use data structures and data analysis tools for the Python programming language. Documentation page

Mid-term Project

A project consists in data analyses based on the use of data mining tools. The project has to be performed by a team of 2/3 students. It has to be performed by using Python. The guidelines require to address specific tasks. Results must be reported in a unique paper. The total length of this paper must be max 25 pages of text including figures. The students must deliver both: paper (single column) and well commented Python Notebooks.

• First part of the project consists in the assignments described here: Project Description

1. Dataset: Dataset
2. Deadline: the fist part has to be delivered within November, 5th 2021 15th 2021.Send an email to: [email protected] and [email protected]

• Second part of the project consists in the assignment Task 3 described here: Updated Project Description
  1. Deadline: 5th January 2022

• Third part of the project consists in the assignment Task 4 described here: Updated Project Description

1. Note that the document contains also rules for the delivery and final exam!
2. Data for time series analysis: CityTemp
3. Deadline: 5th January 2022

Students who did not deliver the above project within 5th Jan 2022 need to ask by email a new project to the teachers. The project that will be assigned will require about 2 weeks of work and after the delivery it will be discussed during the oral exam.

Paper Presentation (OPTIONAL)

Students need to present a research paper (made available by the teacher) during the last week of the course. This presentation is OPTIONAL: Students that decide to do the paper presentation can avoid the oral exam with open questions. They only need to present the project (see next point). The paper presentation can be done by the group or by a single person.

Oral Exam

• Project presentation (with slides) – 10 minutes: mandatory for all the students
• Open questions on the entire program: optional only for students opting for paper presentation.

… a new kind of professional has emerged, the data scientist, who combines the skills of software programmer, statistician and storyteller/artist to extract the nuggets of gold hidden under mountains of data. Hal Varian, Google’s chief economist, predicts that the job of statistician will become the “sexiest” around. Data, he explains, are widely available; what is scarce is the ability to extract wisdom from them.

Data, data everywhere. The Economist, Special Report on Big Data, Feb. 2010.

• Data, data everywhere. The Economist, Feb. 2010 download
• Data scientist: The hot new gig in tech, CNN & Fortune, Sept. 2011 link
• Welcome to the yotta world. The Economist, Sept. 2011 download
• Data Scientist: The Sexiest Job of the 21st Century. Harvard Business Review, Sept 2012 link
• Il futuro è già scritto in Big Data. Il SOle 24 Ore, Sept 2012 link
• Special issue of Crossroads - The ACM Magazine for Students - on Big Data Analytics download
• Peter Sondergaard, Gartner, Says Big Data Creates Big Jobs. Oct 22, 2012: YouTube video

• Towards Effective Decision-Making Through Data Visualization: Six World-Class Enterprises Show The Way. White paper at FusionCharts.com. download

Data Mining (309AA) - 9 CFU A.Y. 2020/2021

DM-2019/20