windy/my-vault
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
Files
86c608881927b69dfdcf4dfe695ecdc217910bd7
my-vault/01_Projects/Home-Automation/Storage.md
T

15 KiB
Raw Blame History 

CREATE DATABASE hass;
CREATE USER hass WITH PASSWORD 'hass';
GRANT ALL PRIVILEGES ON DATABASE hass TO hass;

recorder:
  db_url: postgresql://hass:hass@store.local/hass

Migrating your Home Assistant instance from SQLite to PostgreSQL involves a few steps. The process ensures all your historical state and event data from the existing SQLite database is preserved.

Step 1: Backup Your Current Home Assistant Instance

  1. Stop Home Assistant:

    sudo systemctl stop home-assistant

  2. Create a Backup of Your SQLite Database:

    • The database is typically located in the Home Assistant configuration directory (e.g., /config/ or /home/homeassistant/.homeassistant).
    cp home-assistant_v2.db home-assistant_v2.db.backup

  3. Backup Your Configuration Files:

    tar -czvf home_assistant_config_backup.tar.gz /path/to/home-assistant/config

Step 2: Install and Configure PostgreSQL

  1. Install PostgreSQL:

    sudo apt update
sudo apt install postgresql

  2. Create a Database for Home Assistant:

    • Switch to the postgres user:

      sudo -i -u postgres

    • Create the database and user:

      psql
CREATE DATABASE hass;
CREATE USER hass WITH PASSWORD 'hass';
GRANT ALL PRIVILEGES ON DATABASE hass TO hass;
\q

    • Exit the postgres user:

      exit

  3. Test the Connection: Use the psql client to connect:

    psql -h localhost -U hass -d hass

    Enter the password you set earlier. If successful, you're ready to proceed.

Step 3: Install Required Tools

  1. Install SQLite and PostgreSQL Clients:

    sudo apt install sqlite3 postgresql-client

  2. Install pgloader: pgloader is a tool for migrating data between SQLite and PostgreSQL.

    sudo apt install pgloader

Step 4: Migrate Data from SQLite to PostgreSQL

  1. Prepare the pgloader Command: Create a file called migrate.load with the following content:

LOAD DATABASE FROM sqlite://./home-assistant_v2.db.bak INTO postgresql://hass:hass@localhost/hass

WITH data only, drop indexes, reset sequences, truncate;

ALTER SCHEMA "main" RENAME TO "public";

```


Replace `/path/to/home-assistant_v2.db` with the actual path to your SQLite database file.

  1. Run the Migration:

    pgloader migrate.load

  2. Verify the Data in PostgreSQL:

    • Log in to PostgreSQL:

      psql -h localhost -U hass -d homeassistant

    • Check the tables:

      \dt

Step 5: Configure Home Assistant to Use PostgreSQL

  1. Edit configuration.yaml: Add the PostgreSQL database URL:

    recorder:
  db_url: postgresql://hass:hass@192.168.55.53/hass

    Replace yourpassword and localhost as needed.

  2. Restart Home Assistant:

    sudo systemctl start home-assistant

  3. Verify the Integration:

    • Check the logs in Home Assistant for any database-related errors.

    • Confirm new data is being written to PostgreSQL by querying the states table:

      SELECT * FROM states ORDER BY last_updated DESC LIMIT 10;

Step 6: Clean Up

  1. Remove Old SQLite Database: Once you confirm PostgreSQL is working, you can safely remove the SQLite database:

    rm home-assistant_v2.db

  2. Optimize PostgreSQL:

    • Configure PostgreSQL to improve performance: Edit /etc/postgresql/<version>/main/postgresql.conf:

      shared_buffers = 256MB
work_mem = 16MB
maintenance_work_mem = 64MB

    • Restart PostgreSQL:

      sudo systemctl restart postgresql

Final Notes

  • Keep monitoring Home Assistant's logs during the first few days after migration to ensure the PostgreSQL setup is stable.

  • If needed, adjust the recorder settings in configuration.yaml to exclude entities or domains that generate excessive data:

    recorder:
  include:
    domains:
      - sensor
      - switch
  exclude:
    entities:
      - sensor.unnecessary_metric

Let me know if you need assistance with any specific step!

```sql
LOAD DATABASE
 FROM mysql://root:数据库密码@localhost:3306/homeassistant
 INTO pgsql://homeassistant:数据库密码@localhost:5432/homeassistant
 WITH data only, workers = 8, concurrency = 1
CAST type datetime to timestamp drop default drop not null using zero-dates-to-null
;

sqlite3 home-assistant_v2.db.bak .dump > ha_dump.sql

sed -i 's/DATETIME/TIMESTAMP/g' ha_dump.sql

sed -i 's/BLOB/BYTEA/g' ha_dump.sql

psql -h localhost -U hass -d hass -f ha_dump.sql -W > load.log 2>&1

pgloader sqlite://./home-assistant_v2.db.bak postgresql://hass:hass@localhost/hass

CREATE SEQUENCE event_types_event_type_id_seq;
CREATE SEQUENCE state_attributes_attributes_id_seq;
CREATE SEQUENCE event_data_data_id_seq;
CREATE SEQUENCE states_meta_metadata_id_seq;
CREATE SEQUENCE statistics_meta_id_seq;
CREATE SEQUENCE events_event_id_seq;
CREATE SEQUENCE recorder_runs_run_id_seq;
CREATE SEQUENCE schema_changes_change_id_seq;
CREATE SEQUENCE statistics_runs_run_id_seq;
CREATE SEQUENCE states_state_id_seq;
CREATE SEQUENCE statistics_id_seq;
CREATE SEQUENCE statistics_short_term_id_seq;


SELECT setval('event_types_event_type_id_seq', MAX(event_type_id)) FROM event_types;
SELECT setval('state_attributes_attributes_id_seq', MAX(attributes_id)) FROM state_attributes;
SELECT setval('event_data_data_id_seq', MAX(data_id)) FROM event_data;
SELECT setval('states_meta_metadata_id_seq', MAX(metadata_id)) FROM states_meta;
SELECT setval('statistics_meta_id_seq', MAX(id)) FROM statistics_meta;
SELECT setval('events_event_id_seq', MAX(event_id)) FROM events;
SELECT setval('recorder_runs_run_id_seq', MAX(run_id)) FROM recorder_runs;
SELECT setval('schema_changes_change_id_seq', MAX(change_id)) FROM schema_changes;
SELECT setval('statistics_runs_run_id_seq', MAX(run_id)) FROM statistics_runs;
SELECT setval('states_state_id_seq', MAX(state_id)) FROM states;
SELECT setval('statistics_id_seq', MAX(id)) FROM statistics;
SELECT setval('statistics_short_term_id_seq', MAX(id)) FROM statistics_short_term;


recorder:
  db_url: postgresql://hass:hass@192.168.55.53/hass

influxdb: 
  host: 192.168.55.53
  port: 8428
  database: hass 
  default_measurement: state

mysql:

CREATE DATABASE hass;
CREATE USER 'hass'@'%' IDENTIFIED BY 'hass';
GRANT ALL PRIVILEGES ON homeassistant.* TO 'hass'@'%';
FLUSH PRIVILEGES;


sqlite3mysql --sqlite-file home-assistant_v2.db --mysql-user hass --mysql-password hass --mysql-database hass

recorder:
  db_url: mysql://hass:hass@192.168.55.53/hass?charset=utf8mb4


influxdb:
  api_version: 1
  host: 192.168.55.53
  port: 8428
  max_retries: 3
  measurement_attr: entity_id
  tags_attributes:
    - friendly_name
    - unit_of_measurement
  ignore_attributes:
    - icon
    - source
    - options
    - editable
    - min
    - max
    - step
    - mode
    - marker_type
    - preset_modes
    - supported_features
    - supported_color_modes
    - effect_list
    - attribution
    - assumed_state
    - state_open
    - state_closed
    - writable
    - stateExtra
    - event
    - friendly_name
    - device_class
    - state_class
    - ip_address
    - device_file
    - unit_of_measurement
    - unitOfMeasure
  include:
    domains:
      - sensor
      - binary_sensor
      - light
      - switch
      - cover
      - climate
      - input_boolean
      - input_select
      - number
      - lock
      - weather
  exclude:
    entity_globs:
      - sensor.clock*
      - sensor.date*
      - sensor.glances*
      - sensor.time*
      - sensor.uptime*
      - sensor.dwd_weather_warnings_*
      - weather.weatherstation
      - binary_sensor.*_smartphone_*
      - sensor.*_smartphone_*
      - sensor.adguard_home_*
      - binary_sensor.*_internet_access

get sqlite schema

sqlite3 home-assistant_v2.db <<EOF
.output sqlite-schema.sql
.schema
.exit
EOF

postgresql create:

-- Drop all tables and references to ensure a clean slate

DROP TABLE IF EXISTS event_data CASCADE;

DROP TABLE IF EXISTS event_types CASCADE;

DROP TABLE IF EXISTS state_attributes CASCADE;

DROP TABLE IF EXISTS states_meta CASCADE;

DROP TABLE IF EXISTS statistics_meta CASCADE;

DROP TABLE IF EXISTS recorder_runs CASCADE;

DROP TABLE IF EXISTS migration_changes CASCADE;

DROP TABLE IF EXISTS schema_changes CASCADE;

DROP TABLE IF EXISTS statistics_runs CASCADE;

DROP TABLE IF EXISTS events CASCADE;

DROP TABLE IF EXISTS states CASCADE;

DROP TABLE IF EXISTS statistics CASCADE;

DROP TABLE IF EXISTS statistics_short_term CASCADE;

DROP TABLE IF EXISTS sqlite_stat1 CASCADE;

  

-- Enable TimescaleDB extension

CREATE EXTENSION IF NOT EXISTS timescaledb;

  

-- Create tables with appropriate data types and constraints

CREATE TABLE event_data (

data_id SERIAL PRIMARY KEY,

hash BIGINT,

shared_data TEXT

);

CREATE INDEX ix_event_data_hash ON event_data (hash);

  

CREATE TABLE event_types (

event_type_id SERIAL PRIMARY KEY,

event_type VARCHAR(64) UNIQUE

);

  

CREATE TABLE state_attributes (

attributes_id SERIAL PRIMARY KEY,

hash BIGINT,

shared_attrs TEXT

);

CREATE INDEX ix_state_attributes_hash ON state_attributes (hash);

  

CREATE TABLE states_meta (

metadata_id SERIAL PRIMARY KEY,

entity_id VARCHAR(255) UNIQUE

);

  

CREATE TABLE statistics_meta (

id SERIAL PRIMARY KEY,

statistic_id VARCHAR(255),

source VARCHAR(32),

unit_of_measurement VARCHAR(255),

has_mean BOOLEAN,

has_sum BOOLEAN,

name VARCHAR(255)

);

  

CREATE TABLE recorder_runs (

run_id SERIAL PRIMARY KEY,

start TIMESTAMPTZ NOT NULL,

"end" TIMESTAMPTZ,

closed_incorrect BOOLEAN NOT NULL,

created TIMESTAMPTZ NOT NULL

);

CREATE INDEX ix_recorder_runs_start_end ON recorder_runs (start, "end");

  

CREATE TABLE migration_changes (

migration_id VARCHAR(255) PRIMARY KEY,

version SMALLINT NOT NULL

);

  

CREATE TABLE schema_changes (

change_id SERIAL PRIMARY KEY,

schema_version INTEGER,

changed TIMESTAMPTZ NOT NULL

);

  

CREATE TABLE statistics_runs (

run_id SERIAL PRIMARY KEY,

start TIMESTAMPTZ NOT NULL

);

CREATE INDEX ix_statistics_runs_start ON statistics_runs (start);

  

-- Create hypertable for time-series data

CREATE TABLE events (

event_id SERIAL NOT NULL,

event_type VARCHAR(64),

event_data TEXT,

origin VARCHAR(64),

origin_idx SMALLINT,

time_fired TIMESTAMPTZ NOT NULL, -- Partitioning column

time_fired_ts DOUBLE PRECISION,

context_id UUID,

context_user_id UUID,

context_parent_id UUID,

data_id INTEGER,

context_id_bin BYTEA,

context_user_id_bin BYTEA,

context_parent_id_bin BYTEA,

event_type_id INTEGER,

PRIMARY KEY (event_id, time_fired), -- Composite primary key

FOREIGN KEY (data_id) REFERENCES event_data (data_id),

FOREIGN KEY (event_type_id) REFERENCES event_types (event_type_id)

);

  

SELECT create_hypertable('events', 'time_fired');

  

-- Add time_fired to all unique indexes on hypertables

CREATE INDEX ix_events_data_id ON events (data_id);

CREATE UNIQUE INDEX ix_events_event_type_id_time_fired_ts ON events (event_type_id, time_fired, time_fired_ts);

CREATE INDEX ix_events_time_fired_ts ON events (time_fired_ts);

CREATE INDEX ix_events_context_id_bin ON events (context_id_bin);

  

CREATE TABLE states (

state_id SERIAL PRIMARY KEY,

entity_id VARCHAR(255),

state VARCHAR(255),

attributes TEXT,

event_id INTEGER,

last_changed TIMESTAMPTZ,

last_changed_ts DOUBLE PRECISION,

last_reported_ts DOUBLE PRECISION,

last_updated TIMESTAMPTZ,

last_updated_ts DOUBLE PRECISION,

old_state_id INTEGER,

attributes_id INTEGER,

context_id UUID,

context_user_id UUID,

context_parent_id UUID,

origin_idx SMALLINT,

context_id_bin BYTEA,

context_user_id_bin BYTEA,

context_parent_id_bin BYTEA,

metadata_id INTEGER,

FOREIGN KEY(old_state_id) REFERENCES states (state_id),

FOREIGN KEY(attributes_id) REFERENCES state_attributes (attributes_id),

FOREIGN KEY(metadata_id) REFERENCES states_meta (metadata_id)

);

CREATE INDEX ix_states_last_updated_ts ON states (last_updated_ts);

CREATE INDEX ix_states_context_id_bin ON states (context_id_bin);

CREATE INDEX ix_states_attributes_id ON states (attributes_id);

CREATE INDEX ix_states_old_state_id ON states (old_state_id);

CREATE INDEX ix_states_metadata_id_last_updated_ts ON states (metadata_id, last_updated_ts);

  

-- Drop the existing table if necessary

DROP TABLE IF EXISTS statistics CASCADE;

  

-- Create the statistics table

CREATE TABLE statistics (

id SERIAL not NULL, -- Simple primary key

created TIMESTAMPTZ,

created_ts DOUBLE PRECISION,

metadata_id INTEGER,

start TIMESTAMPTZ NOT NULL, -- Partitioning column

start_ts DOUBLE PRECISION,

mean DOUBLE PRECISION,

min DOUBLE PRECISION,

max DOUBLE PRECISION,

last_reset TIMESTAMPTZ,

last_reset_ts DOUBLE PRECISION,

state DOUBLE PRECISION,

sum DOUBLE PRECISION,

PRIMARY KEY (id, start),

FOREIGN KEY(metadata_id) REFERENCES statistics_meta (id) ON DELETE CASCADE

);

  

-- Create the hypertable with 'start' as the partitioning column

SELECT create_hypertable('statistics', 'start');

  

-- Create a unique index that includes the partitioning column

CREATE UNIQUE INDEX ix_statistics_statistic_id_start_ts ON statistics (metadata_id, start_ts, start);

  

-- Additional non-unique index for querying by start_ts

CREATE INDEX ix_statistics_start_ts ON statistics (start_ts);

  

  

CREATE TABLE statistics_short_term (

id SERIAL not null,

created TIMESTAMPTZ,

created_ts DOUBLE PRECISION,

metadata_id INTEGER,

start TIMESTAMPTZ NOT NULL, -- Partitioning column

start_ts DOUBLE PRECISION,

mean DOUBLE PRECISION,

min DOUBLE PRECISION,

max DOUBLE PRECISION,

last_reset TIMESTAMPTZ,

last_reset_ts DOUBLE PRECISION,

state DOUBLE PRECISION,

sum DOUBLE PRECISION,

primary key(id, start),

FOREIGN KEY(metadata_id) REFERENCES statistics_meta (id) ON DELETE CASCADE

);

SELECT create_hypertable('statistics_short_term', 'start');

CREATE UNIQUE INDEX ix_statistics_short_term_statistic_id_start_ts ON statistics_short_term (metadata_id, start_ts, start);

CREATE INDEX ix_statistics_short_term_start_ts ON statistics_short_term (start_ts);

  

-- Drop unsupported SQLite-specific table

DROP TABLE IF EXISTS sqlite_stat1 CASCADE;
Reference in New Issue View Git Blame Copy Permalink