vault-para/100-project/Personal/Home-Automation/Storage.md

```sql
CREATE DATABASE hass;
CREATE USER hass WITH PASSWORD 'hass';
GRANT ALL PRIVILEGES ON DATABASE hass TO hass;
```


```
recorder:
  db_url: postgresql://hass:hass@store.local/hass
```



Migrating your Home Assistant instance from SQLite to PostgreSQL involves a few steps. The process ensures all your historical state and event data from the existing SQLite database is preserved.

---

### **Step 1: Backup Your Current Home Assistant Instance**

1. **Stop Home Assistant**:
    
    ```bash
    sudo systemctl stop home-assistant
    ```
    
2. **Create a Backup of Your SQLite Database**:
    
    - The database is typically located in the Home Assistant configuration directory (e.g., `/config/` or `/home/homeassistant/.homeassistant`).
    
    ```bash
    cp home-assistant_v2.db home-assistant_v2.db.backup
    ```
    
3. **Backup Your Configuration Files**:
    
    ```bash
    tar -czvf home_assistant_config_backup.tar.gz /path/to/home-assistant/config
    ```
    

---

### **Step 2: Install and Configure PostgreSQL**

1. **Install PostgreSQL**:
    
    ```bash
    sudo apt update
    sudo apt install postgresql
    ```
    
2. **Create a Database for Home Assistant**:
    
    - Switch to the `postgres` user:
        
        ```bash
        sudo -i -u postgres
        ```
        
    - Create the database and user:
        
        ```bash
        psql
        CREATE DATABASE hass;
        CREATE USER hass WITH PASSWORD 'hass';
        GRANT ALL PRIVILEGES ON DATABASE hass TO hass;
        \q
        ```
        
    - Exit the `postgres` user:
        
        ```bash
        exit
        ```
        
3. **Test the Connection**: Use the `psql` client to connect:
    
    ```bash
    psql -h localhost -U hass -d hass
    ```
    
    Enter the password you set earlier. If successful, you're ready to proceed.
    

---

### **Step 3: Install Required Tools**

1. **Install SQLite and PostgreSQL Clients**:
    
    ```bash
    sudo apt install sqlite3 postgresql-client
    ```
    
2. **Install `pgloader`**: `pgloader` is a tool for migrating data between SQLite and PostgreSQL.
    
    ```bash
    sudo apt install pgloader
    ```
    

---

### **Step 4: Migrate Data from SQLite to PostgreSQL**

1. **Prepare the `pgloader` Command**: Create a file called `migrate.load` with the following content:
    
    ```lisp
LOAD DATABASE
     FROM sqlite://./home-assistant_v2.db.bak
     INTO postgresql://hass:hass@localhost/hass

 WITH data only,
      drop indexes,
      reset sequences,
      truncate;

ALTER SCHEMA "main" RENAME TO "public";

    ```
    

    Replace `/path/to/home-assistant_v2.db` with the actual path to your SQLite database file.
    
2. **Run the Migration**:
    
    ```bash
    pgloader migrate.load
    ```
    
3. **Verify the Data in PostgreSQL**:
    
    - Log in to PostgreSQL:
        
        ```bash
        psql -h localhost -U hass -d homeassistant
        ```
        
    - Check the tables:
        
        ```sql
        \dt
        ```
        

---

### **Step 5: Configure Home Assistant to Use PostgreSQL**

1. **Edit `configuration.yaml`**: Add the PostgreSQL database URL:
    
    ```yaml
    recorder:
      db_url: postgresql://hass:hass@192.168.55.53/hass
    ```
    
    Replace `yourpassword` and `localhost` as needed.
    
2. **Restart Home Assistant**:
    
    ```bash
    sudo systemctl start home-assistant
    ```
    
3. **Verify the Integration**:
    
    - Check the logs in Home Assistant for any database-related errors.
    - Confirm new data is being written to PostgreSQL by querying the `states` table:
        
        ```sql
        SELECT * FROM states ORDER BY last_updated DESC LIMIT 10;
        ```
        

---

### **Step 6: Clean Up**

1. **Remove Old SQLite Database**: Once you confirm PostgreSQL is working, you can safely remove the SQLite database:
    
    ```bash
    rm home-assistant_v2.db
    ```
    
2. **Optimize PostgreSQL**:
    
    - Configure PostgreSQL to improve performance: Edit `/etc/postgresql/<version>/main/postgresql.conf`:
        
        ```plaintext
        shared_buffers = 256MB
        work_mem = 16MB
        maintenance_work_mem = 64MB
        ```
        
    - Restart PostgreSQL:
        
        ```bash
        sudo systemctl restart postgresql
        ```
        

---

### **Final Notes**

- Keep monitoring Home Assistant's logs during the first few days after migration to ensure the PostgreSQL setup is stable.
- If needed, adjust the recorder settings in `configuration.yaml` to exclude entities or domains that generate excessive data:
    
    ```yaml
    recorder:
      include:
        domains:
          - sensor
          - switch
      exclude:
        entities:
          - sensor.unnecessary_metric
    ```
    

Let me know if you need assistance with any specific step!




```
```sql
LOAD DATABASE
 FROM mysql://root:数据库密码@localhost:3306/homeassistant
 INTO pgsql://homeassistant:数据库密码@localhost:5432/homeassistant
 WITH data only, workers = 8, concurrency = 1
CAST type datetime to timestamp drop default drop not null using zero-dates-to-null
;
```



```bash
sqlite3 home-assistant_v2.db.bak .dump > ha_dump.sql
```


```bash
sed -i 's/DATETIME/TIMESTAMP/g' ha_dump.sql
```

```bash
sed -i 's/BLOB/BYTEA/g' ha_dump.sql
```


```bash
psql -h localhost -U hass -d hass -f ha_dump.sql -W > load.log 2>&1
```


```
pgloader sqlite://./home-assistant_v2.db.bak postgresql://hass:hass@localhost/hass
```


```sql
CREATE SEQUENCE event_types_event_type_id_seq;
CREATE SEQUENCE state_attributes_attributes_id_seq;
CREATE SEQUENCE event_data_data_id_seq;
CREATE SEQUENCE states_meta_metadata_id_seq;
CREATE SEQUENCE statistics_meta_id_seq;
CREATE SEQUENCE events_event_id_seq;
CREATE SEQUENCE recorder_runs_run_id_seq;
CREATE SEQUENCE schema_changes_change_id_seq;
CREATE SEQUENCE statistics_runs_run_id_seq;
CREATE SEQUENCE states_state_id_seq;
CREATE SEQUENCE statistics_id_seq;
CREATE SEQUENCE statistics_short_term_id_seq;


SELECT setval('event_types_event_type_id_seq', MAX(event_type_id)) FROM event_types;
SELECT setval('state_attributes_attributes_id_seq', MAX(attributes_id)) FROM state_attributes;
SELECT setval('event_data_data_id_seq', MAX(data_id)) FROM event_data;
SELECT setval('states_meta_metadata_id_seq', MAX(metadata_id)) FROM states_meta;
SELECT setval('statistics_meta_id_seq', MAX(id)) FROM statistics_meta;
SELECT setval('events_event_id_seq', MAX(event_id)) FROM events;
SELECT setval('recorder_runs_run_id_seq', MAX(run_id)) FROM recorder_runs;
SELECT setval('schema_changes_change_id_seq', MAX(change_id)) FROM schema_changes;
SELECT setval('statistics_runs_run_id_seq', MAX(run_id)) FROM statistics_runs;
SELECT setval('states_state_id_seq', MAX(state_id)) FROM states;
SELECT setval('statistics_id_seq', MAX(id)) FROM statistics;
SELECT setval('statistics_short_term_id_seq', MAX(id)) FROM statistics_short_term;

```



```
recorder:
  db_url: postgresql://hass:hass@192.168.55.53/hass
```



```
influxdb: 
  host: 192.168.55.53
  port: 8428
  database: hass 
  default_measurement: state
  
```


mysql:

```sql
CREATE DATABASE hass;
CREATE USER 'hass'@'%' IDENTIFIED BY 'hass';
GRANT ALL PRIVILEGES ON homeassistant.* TO 'hass'@'%';
FLUSH PRIVILEGES;

```


```
sqlite3mysql --sqlite-file home-assistant_v2.db --mysql-user hass --mysql-password hass --mysql-database hass
```


```
recorder:
  db_url: mysql://hass:hass@192.168.55.53/hass?charset=utf8mb4

```



```
influxdb:
  api_version: 1
  host: 192.168.55.53
  port: 8428
  max_retries: 3
  measurement_attr: entity_id
  tags_attributes:
    - friendly_name
    - unit_of_measurement
  ignore_attributes:
    - icon
    - source
    - options
    - editable
    - min
    - max
    - step
    - mode
    - marker_type
    - preset_modes
    - supported_features
    - supported_color_modes
    - effect_list
    - attribution
    - assumed_state
    - state_open
    - state_closed
    - writable
    - stateExtra
    - event
    - friendly_name
    - device_class
    - state_class
    - ip_address
    - device_file
    - unit_of_measurement
    - unitOfMeasure
  include:
    domains:
      - sensor
      - binary_sensor
      - light
      - switch
      - cover
      - climate
      - input_boolean
      - input_select
      - number
      - lock
      - weather
  exclude:
    entity_globs:
      - sensor.clock*
      - sensor.date*
      - sensor.glances*
      - sensor.time*
      - sensor.uptime*
      - sensor.dwd_weather_warnings_*
      - weather.weatherstation
      - binary_sensor.*_smartphone_*
      - sensor.*_smartphone_*
      - sensor.adguard_home_*
      - binary_sensor.*_internet_access
      
```





get sqlite schema
```
sqlite3 home-assistant_v2.db <<EOF
.output sqlite-schema.sql
.schema
.exit
EOF

```



postgresql create:

```sql
-- Drop all tables and references to ensure a clean slate

DROP TABLE IF EXISTS event_data CASCADE;

DROP TABLE IF EXISTS event_types CASCADE;

DROP TABLE IF EXISTS state_attributes CASCADE;

DROP TABLE IF EXISTS states_meta CASCADE;

DROP TABLE IF EXISTS statistics_meta CASCADE;

DROP TABLE IF EXISTS recorder_runs CASCADE;

DROP TABLE IF EXISTS migration_changes CASCADE;

DROP TABLE IF EXISTS schema_changes CASCADE;

DROP TABLE IF EXISTS statistics_runs CASCADE;

DROP TABLE IF EXISTS events CASCADE;

DROP TABLE IF EXISTS states CASCADE;

DROP TABLE IF EXISTS statistics CASCADE;

DROP TABLE IF EXISTS statistics_short_term CASCADE;

DROP TABLE IF EXISTS sqlite_stat1 CASCADE;

  

-- Enable TimescaleDB extension

CREATE EXTENSION IF NOT EXISTS timescaledb;

  

-- Create tables with appropriate data types and constraints

CREATE TABLE event_data (

data_id SERIAL PRIMARY KEY,

hash BIGINT,

shared_data TEXT

);

CREATE INDEX ix_event_data_hash ON event_data (hash);

  

CREATE TABLE event_types (

event_type_id SERIAL PRIMARY KEY,

event_type VARCHAR(64) UNIQUE

);

  

CREATE TABLE state_attributes (

attributes_id SERIAL PRIMARY KEY,

hash BIGINT,

shared_attrs TEXT

);

CREATE INDEX ix_state_attributes_hash ON state_attributes (hash);

  

CREATE TABLE states_meta (

metadata_id SERIAL PRIMARY KEY,

entity_id VARCHAR(255) UNIQUE

);

  

CREATE TABLE statistics_meta (

id SERIAL PRIMARY KEY,

statistic_id VARCHAR(255),

source VARCHAR(32),

unit_of_measurement VARCHAR(255),

has_mean BOOLEAN,

has_sum BOOLEAN,

name VARCHAR(255)

);

  

CREATE TABLE recorder_runs (

run_id SERIAL PRIMARY KEY,

start TIMESTAMPTZ NOT NULL,

"end" TIMESTAMPTZ,

closed_incorrect BOOLEAN NOT NULL,

created TIMESTAMPTZ NOT NULL

);

CREATE INDEX ix_recorder_runs_start_end ON recorder_runs (start, "end");

  

CREATE TABLE migration_changes (

migration_id VARCHAR(255) PRIMARY KEY,

version SMALLINT NOT NULL

);

  

CREATE TABLE schema_changes (

change_id SERIAL PRIMARY KEY,

schema_version INTEGER,

changed TIMESTAMPTZ NOT NULL

);

  

CREATE TABLE statistics_runs (

run_id SERIAL PRIMARY KEY,

start TIMESTAMPTZ NOT NULL

);

CREATE INDEX ix_statistics_runs_start ON statistics_runs (start);

  

-- Create hypertable for time-series data

CREATE TABLE events (

event_id SERIAL NOT NULL,

event_type VARCHAR(64),

event_data TEXT,

origin VARCHAR(64),

origin_idx SMALLINT,

time_fired TIMESTAMPTZ NOT NULL, -- Partitioning column

time_fired_ts DOUBLE PRECISION,

context_id UUID,

context_user_id UUID,

context_parent_id UUID,

data_id INTEGER,

context_id_bin BYTEA,

context_user_id_bin BYTEA,

context_parent_id_bin BYTEA,

event_type_id INTEGER,

PRIMARY KEY (event_id, time_fired), -- Composite primary key

FOREIGN KEY (data_id) REFERENCES event_data (data_id),

FOREIGN KEY (event_type_id) REFERENCES event_types (event_type_id)

);

  

SELECT create_hypertable('events', 'time_fired');

  

-- Add time_fired to all unique indexes on hypertables

CREATE INDEX ix_events_data_id ON events (data_id);

CREATE UNIQUE INDEX ix_events_event_type_id_time_fired_ts ON events (event_type_id, time_fired, time_fired_ts);

CREATE INDEX ix_events_time_fired_ts ON events (time_fired_ts);

CREATE INDEX ix_events_context_id_bin ON events (context_id_bin);

  

CREATE TABLE states (

state_id SERIAL PRIMARY KEY,

entity_id VARCHAR(255),

state VARCHAR(255),

attributes TEXT,

event_id INTEGER,

last_changed TIMESTAMPTZ,

last_changed_ts DOUBLE PRECISION,

last_reported_ts DOUBLE PRECISION,

last_updated TIMESTAMPTZ,

last_updated_ts DOUBLE PRECISION,

old_state_id INTEGER,

attributes_id INTEGER,

context_id UUID,

context_user_id UUID,

context_parent_id UUID,

origin_idx SMALLINT,

context_id_bin BYTEA,

context_user_id_bin BYTEA,

context_parent_id_bin BYTEA,

metadata_id INTEGER,

FOREIGN KEY(old_state_id) REFERENCES states (state_id),

FOREIGN KEY(attributes_id) REFERENCES state_attributes (attributes_id),

FOREIGN KEY(metadata_id) REFERENCES states_meta (metadata_id)

);

CREATE INDEX ix_states_last_updated_ts ON states (last_updated_ts);

CREATE INDEX ix_states_context_id_bin ON states (context_id_bin);

CREATE INDEX ix_states_attributes_id ON states (attributes_id);

CREATE INDEX ix_states_old_state_id ON states (old_state_id);

CREATE INDEX ix_states_metadata_id_last_updated_ts ON states (metadata_id, last_updated_ts);

  

-- Drop the existing table if necessary

DROP TABLE IF EXISTS statistics CASCADE;

  

-- Create the statistics table

CREATE TABLE statistics (

id SERIAL not NULL, -- Simple primary key

created TIMESTAMPTZ,

created_ts DOUBLE PRECISION,

metadata_id INTEGER,

start TIMESTAMPTZ NOT NULL, -- Partitioning column

start_ts DOUBLE PRECISION,

mean DOUBLE PRECISION,

min DOUBLE PRECISION,

max DOUBLE PRECISION,

last_reset TIMESTAMPTZ,

last_reset_ts DOUBLE PRECISION,

state DOUBLE PRECISION,

sum DOUBLE PRECISION,

PRIMARY KEY (id, start),

FOREIGN KEY(metadata_id) REFERENCES statistics_meta (id) ON DELETE CASCADE

);

  

-- Create the hypertable with 'start' as the partitioning column

SELECT create_hypertable('statistics', 'start');

  

-- Create a unique index that includes the partitioning column

CREATE UNIQUE INDEX ix_statistics_statistic_id_start_ts ON statistics (metadata_id, start_ts, start);

  

-- Additional non-unique index for querying by start_ts

CREATE INDEX ix_statistics_start_ts ON statistics (start_ts);

  

  

CREATE TABLE statistics_short_term (

id SERIAL not null,

created TIMESTAMPTZ,

created_ts DOUBLE PRECISION,

metadata_id INTEGER,

start TIMESTAMPTZ NOT NULL, -- Partitioning column

start_ts DOUBLE PRECISION,

mean DOUBLE PRECISION,

min DOUBLE PRECISION,

max DOUBLE PRECISION,

last_reset TIMESTAMPTZ,

last_reset_ts DOUBLE PRECISION,

state DOUBLE PRECISION,

sum DOUBLE PRECISION,

primary key(id, start),

FOREIGN KEY(metadata_id) REFERENCES statistics_meta (id) ON DELETE CASCADE

);

SELECT create_hypertable('statistics_short_term', 'start');

CREATE UNIQUE INDEX ix_statistics_short_term_statistic_id_start_ts ON statistics_short_term (metadata_id, start_ts, start);

CREATE INDEX ix_statistics_short_term_start_ts ON statistics_short_term (start_ts);

  

-- Drop unsupported SQLite-specific table

DROP TABLE IF EXISTS sqlite_stat1 CASCADE;
```