# Crystal Ball

<table data-card-size="large" data-view="cards"><thead><tr><th></th><th></th></tr></thead><tbody><tr><td></td><td><div><figure><img src="https://2679854115-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F0ExkNOXtHgt64fkjqn3Z%2Fuploads%2FS2J3GZnMv19KTJ83dEnh%2FScreen%20Shot%202025-10-16%20at%209.47.11%20PM.png?alt=media&#x26;token=649962d9-55e5-477d-adc5-9a7ae4323587" alt=""><figcaption></figcaption></figure></div></td></tr><tr><td><p>Now let's enter the cave to see Lady H.'s crystal ball. Could you imagine a cave could have such tranquil scenery inside? I mean, witnessing flowers, a lake, and a gazebo bathed in moonlight, all within a cave?! Lady H. made this arrangement because the crystal ball is constantly too bright, and yes, the crystal ball is that "moon".</p><p>Lady H. acquired this magical crystal ball from a witch who had used it for decades. Originally, the ball would glow and dim, but one day it became permanently bright and couldn't be turned off. The witch decided to get a replacement and brought the old crystal ball to a flea market, giving it away for free. Coincidentally, Lady H. was at the same market searching for a light to adorn her cave. After exploring the entire market without finding anything suitable, she finally came across the glowing crystal ball carried by the witch.</p></td><td></td></tr></tbody></table>

It turned out to be quite the bargain! Lady H. got it for free, and it's still remarkably powerful. Although she can't wield it quite like a witch, the crystal ball's default ability is to gather data from any corner of the universe. Over the years, Lady H. has also discovered a few more magical powers it possesses, which we are excited to reveal to you now!

## Gear Up

* [Follow this guidance](https://lady-h-s-applied-data-science-wo.gitbook.io/applied-data-science-in-my-garden/rainbow-moss/python-virtual-environment) to create a virtual environment, it can simplify python packages' installation and their version control. The required Python version is Python 3.9. Set up required python packages:
  * [required requirements.txt](https://github.com/lady-h-world/My_Garden/blob/main/code/crystal_ball/requirements.txt)
  * [required setup.sh](https://github.com/lady-h-world/My_Garden/blob/main/code/crystal_ball/setup.sh)
* If there will be any package version problem, you can check the full list of Python packages [here](https://github.com/lady-h-world/My_Garden/blob/main/code/crystal_ball/full_requirements_bk.txt).

## Data List

### Collected Data

The crystal ball has the ability to reach every corner of the universe and collect data, making it the primary data source for Lady H.'s experiments. However, she hasn't extensively explored distant locations. Some of the data is conveniently collected from our garden, while some is sourced from a planet called "Kaggle". On that planet, numerous individuals dedicate sleepless nights to competitive pursuits, a phenomenon Lady H. finds difficult to comprehend, why so many people take such competitions seriously and work so diligently is a mystery to her 🤔.

* [30 categories of leaves](https://github.com/lady-h-world/My_Garden/blob/main/code/crystal_ball/data_collector/generate_leaf.ipynb)
* [100 categories of leaves](https://github.com/lady-h-world/My_Garden/blob/main/code/crystal_ball/data_collector/generate_100leaves.ipynb)
* [banking campaign data](https://github.com/lady-h-world/My_Garden/blob/main/code/crystal_ball/data_collector/generate_campaign.ipynb)
* [102 categories of flower images](https://github.com/lady-h-world/My_Garden/blob/main/code/crystal_ball/data_collector/generate_flowers.ipynb)
* [room occupancy, multivariate time series](https://github.com/lady-h-world/My_Garden/blob/main/code/crystal_ball/data_collector/generate_multivariate_ts.ipynb)
* [sales data](https://github.com/lady-h-world/My_Garden/blob/main/code/crystal_ball/data_collector/generate_sales.ipynb)

### Simulated Data

The crystal ball has the ability to generate data seemingly out of thin air too!

* [clusters simulation](https://github.com/lady-h-world/My_Garden/blob/main/code/crystal_ball/data_collector/simulate_clusters.ipynb)

## Crystal Magic

Besides getting the data, Lady H. has found a few more power this crystal ball carries!

* [Dimension Reduction](https://lady-h-s-applied-data-science-wo.gitbook.io/applied-data-science-in-my-garden/crystal-ball/dimensional-reduction)
