Real-time Big Data Analytics Solution – 1

To create a real-time data analytics dashboard is definitely a change, especially in big data environment. It includes several tricky steps: real-time data ingestion, real-time data cleaning/merging and real-time data visualization. We did a satisfactory solutions evaluation on a near real-time data streaming/analyzing environment by using some cutting-edge techniques. This series of articles will explain …

Real-time Big Data Analytics Solution – 1 Read More »

Migrate Data to ClickHouse

ClickHouse (CH) definitely has data source integration pipeline to other data sources: RDBS, Cloud storage (for example S3), Kafka, JDBC-based, or Hive etc.. but one of our projects require to migrate historical data from Hive(HDFS) based data warehouse into ClickHouse. Reference: ClickHouse migration documents (https://clickhouse.com/docs/en/cloud/migration/clickhouse-local) Step 1, Create ClickHouse Target Table. Unlike some RDBS systems, …

Migrate Data to ClickHouse Read More »

Covid-19 Map – Canada

A interactive Covid19 data for Canada. Updated to D3.js v7. Date range filtered by ‘brush’ pan, multiple axis and colored data. Data source can be JSON or csv. (Original updated daily). (Source Data from Google or CIHI, for data visualization only, data accuracy was not verified).