my-vault/01_Projects/Work/Government-Projects/Municipal-Development-Reform/ref/minio.md

Apologies for the confusion caused by the deprecated commands in the MinIO Client (mc). The MinIO team periodically updates mc to enhance functionality and improve usability, which sometimes leads to changes in command syntax. To ensure compatibility and take advantage of the latest features, it's essential to use the updated commands.

Below is the updated guide for creating an API user with read and write permissions equivalent to the root user using the latest mc commands: mc admin policy create and mc admin policy attach.

---

Table of Contents

1. Prerequisites
2. Understanding MinIO Users and Policies
3. Installing and Configuring MinIO Client (mc)
4. Creating a Full Access Policy
5. Creating the API User and Assigning the Policy
6. Verifying the API User
7. Best Practices
8. Example: Creating an API User with Full Access
9. Additional Resources

---

1. Prerequisites

Ensure you have the following before proceeding:

• MinIO Server: Installed and running in production.
• Root Access: Administrative privileges to manage MinIO users and policies.
• MinIO Client (mc): Installed on your local machine or a management server.
• Network Access: Ability to connect to the MinIO server from where mc is installed.

---

2. Understanding MinIO Users and Policies

a. Users

In MinIO, users are entities (applications, services, or individuals) that interact with the MinIO server. Each user has a unique Access Key and Secret Key used for authentication.

b. Policies

Policies define the permissions associated with users. They determine what actions a user can perform and on which resources (buckets or objects). Policies can be predefined or custom.

• Read-Only: Allows users to read objects but not modify or delete them.
• Write-Only: Allows users to upload objects but not read or delete them.
• Full Access: Grants all permissions, including read, write, and delete.

---

3. Installing and Configuring MinIO Client (mc)

The MinIO Client (mc) is a command-line tool that simplifies managing MinIO servers and performing administrative tasks.

a. Download and Install mc

1. Download the Latest Release:

   wget https://dl.min.io/client/mc/release/linux-amd64/mc
   

2. Make the Binary Executable:

   chmod +x mc
   

3. Move mc to a Directory in Your PATH:

   sudo mv mc /usr/local/bin/
   

4. Verify Installation:

   mc --version
   

   Expected Output:

   mc version RELEASE.2023-07-24T16-40-29Z
   

b. Configure mc to Connect to Your MinIO Server

1. Set Up an Alias for Your MinIO Server:

   Replace <ALIAS>, <MINIO_ENDPOINT>, <YOUR-ACCESS-KEY>, and <YOUR-SECRET-KEY> with your actual details.

   mc alias set <ALIAS> <MINIO_ENDPOINT> <YOUR-ACCESS-KEY> <YOUR-SECRET-KEY> --api S3v4
   

   Example:

   mc alias set myminio https://minio.example.com Ab3dE6fG9hJkLmN0 Pq8Rs5Tu7Vw9Xy1Z2a3Bc4De5Fg6Hi7J --api S3v4
   

2. Verify Connection:

   mc ls myminio
   

   Expected Output:

   [2024-04-25 10:00:00 UTC]     Bucket1
   [2024-04-25 10:00:00 UTC]     Bucket2
   

   If you encounter errors, ensure that:

   • The MinIO server is accessible from your machine.
   • The access and secret keys are correct.
   • Network firewalls or security groups allow traffic on MinIO ports (default: 9000 for S3 API).

---

4. Creating a Full Access Policy

To replicate the root user's permissions, you'll need to create a policy that grants full access to all resources.

a. Define the Policy JSON

1. Create a File Named full-access.json:

   nano full-access.json
   

2. Add the Following Content:

   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Effect": "Allow",
         "Action": [
           "s3:*"
         ],
         "Resource": [
           "arn:aws:s3:::*"
         ]
       }
     ]
   }
   

   Policy Breakdown:

   • s3:*: Grants all S3 actions (create, read, update, delete).
   • arn:aws:s3:::*: Applies to all buckets and objects.

   Caution: This policy grants full access. Ensure that it's only assigned to trusted users.

3. Save and Exit:

   • Press CTRL + O, then ENTER to save.
   • Press CTRL + X to exit.

b. Validate the Policy JSON

Ensure that the JSON syntax is correct. You can use tools like JSONLint or run:

jq empty full-access.json

If the command executes without errors, the JSON is valid.

c. Create the Policy in MinIO

Use the updated mc admin policy create command instead of the deprecated add command.

mc admin policy create <ALIAS> <POLICY_NAME> <POLICY_FILE>

Example:

mc admin policy create myminio full-access full-access.json

Expected Output:

Policy full-access created successfully

---

5. Creating the API User and Assigning the Policy

Now that the full access policy is defined, you can create a new API user and assign this policy to them.

a. Create the API User

Use the mc client to create a new user. You will need to generate an Access Key and Secret Key for the user.

Method 1: Manual Key Generation

1. Generate an Access Key:

   ACCESS_KEY=$(openssl rand -hex 8)
   echo "Access Key: $ACCESS_KEY"
   

2. Generate a Secret Key:

   SECRET_KEY=$(openssl rand -hex 16)
   echo "Secret Key: $SECRET_KEY"
   

   Sample Output:

   Access Key: a1b2c3d4e5f6g7h8
   Secret Key: i9j0k1l2m3n4o5p6q7r8s9t0u1v2w3x4
   

Method 2: Using MinIO Client (mc) to Create User with Keys

Alternatively, you can let mc generate the keys for you.

mc admin user add <ALIAS> <USERNAME> <SECRET_KEY>

Example:

mc admin user add myminio apiuser1 i9j0k1l2m3n4o5p6q7r8s9t0u1v2w3x4

Note: Replace <USERNAME> and <SECRET_KEY> with your desired username and a strong secret key.

Recommendation: Use Method 1 to generate strong, random keys.

b. Attach the Policy to the User

Assign the previously created full-access policy to the new user.

mc admin policy attach <ALIAS> <POLICY_NAME> user=<USERNAME>

Example:

mc admin policy attach myminio full-access user=apiuser1

Expected Output:

Policy full-access attached to user apiuser1 successfully

---

6. Verifying the API User

Ensure that the new user has been created and has the correct permissions.

a. List Users

mc admin user list <ALIAS>

Example:

mc admin user list myminio

Expected Output:

ACCESSKEY               USERNAME
Ab3dE6fG9hJkLmN0        minioadmin
a1b2c3d4e5f6g7h8        apiuser1

b. Test Access with API User Credentials

To confirm that the user has the correct permissions:

1. Configure mc with the New User:

   mc alias set apiuser myminio https://minio.example.com <ACCESS_KEY> <SECRET_KEY> --api S3v4
   

   Example:

   mc alias set apiuser myminio https://minio.example.com a1b2c3d4e5f6g7h8 i9j0k1l2m3n4o5p6q7r8s9t0u1v2w3x4 --api S3v4
   

2. List All Buckets:

   mc ls apiuser
   

   Expected Output:

   [2024-04-25 10:00:00 UTC]     Bucket1
   [2024-04-25 10:00:00 UTC]     Bucket2
   

3. Create a New Bucket:

   mc mb apiuser/new-bucket
   

   Expected Output:

   Bucket created successfully `new-bucket`.
   

4. Upload an Object:

   mc cp example.txt apiuser/new-bucket/
   

   Expected Output:

   Upload Success:  example.txt to new-bucket/example.txt
   

5. Download an Object:

   mc cp apiuser/new-bucket/example.txt ./example.txt
   

   Expected Output:

   Download Success:  new-bucket/example.txt to example.txt
   

6. Delete an Object:

   mc rm apiuser/new-bucket/example.txt
   

   Expected Output:

   Removed `new-bucket/example.txt`
   

7. Remove the Bucket:

   mc rb apiuser/new-bucket
   

   Expected Output:

   Removed `new-bucket`.
   

Note: Since the API user has full access, all these operations should succeed, mirroring the root user's capabilities.

---

7. Best Practices

To maintain a secure and efficient MinIO environment, adhere to the following best practices when managing API users:

a. Principle of Least Privilege

• Assign Minimal Permissions: Only grant users the permissions they strictly need.
• Avoid Over-Permissioning: Do not assign full access unless absolutely necessary.

Note: While you're setting up an API user with full access, ensure that this is truly required and that the credentials are handled securely.

b. Regularly Rotate Credentials

• Change Access and Secret Keys Periodically: Minimizes the risk of compromised credentials.
• Update Dependent Applications: Ensure that applications using these keys are updated accordingly.

c. Use Strong, Unique Credentials

• High Entropy: Use long and randomly generated keys.
• Avoid Reuse: Ensure that each user has unique credentials.

d. Monitor and Audit User Activities

• Enable Logging: Keep track of user actions for auditing purposes.
• Set Up Alerts: Notify administrators of unusual activities or access patterns.

e. Secure Storage of Credentials

• Use Secret Management Tools: Tools like HashiCorp Vault, AWS Secrets Manager, or Kubernetes Secrets can securely store and manage credentials.
• Avoid Hardcoding Credentials: Do not embed access keys in application code or configuration files.

f. Limit User Lifespans

• Temporary Access: For users that need temporary access, set expiration policies or regularly review and deactivate unused users.

---

8. Example: Creating an API User with Full Access

Let's walk through a practical example where we create an API user named apiuser1 with full access to all buckets and objects, mirroring the root user's permissions.

Step 1: Define the Full Access Policy

1. Create full-access.json:

   nano full-access.json
   

2. Add the Following Content:

   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Effect": "Allow",
         "Action": [
           "s3:*"
         ],
         "Resource": [
           "arn:aws:s3:::*"
         ]
       }
     ]
   }
   

3. Save and Exit:

   • Press CTRL + O, then ENTER to save.
   • Press CTRL + X to exit.

4. Validate the Policy JSON:

   jq empty full-access.json
   

   No output means the JSON is valid.

Step 2: Add the Policy to MinIO

mc admin policy create myminio full-access full-access.json

Expected Output:

Policy full-access created successfully

Step 3: Create the API User

1. Generate Access and Secret Keys:

   ACCESS_KEY=$(openssl rand -hex 8)
   SECRET_KEY=$(openssl rand -hex 16)
   echo "Access Key: $ACCESS_KEY"
   echo "Secret Key: $SECRET_KEY"
   

   Sample Output:

   Access Key: a1b2c3d4e5f6g7h8
   Secret Key: i9j0k1l2m3n4o5p6q7r8s9t0u1v2w3x4
   

2. Add the User with the Policy:

   mc admin user add myminio apiuser1 i9j0k1l2m3n4o5p6q7r8s9t0u1v2w3x4
   mc admin policy attach myminio full-access user=apiuser1
   

   Expected Output:

   User apiuser1 added successfully
   Policy full-access attached to user apiuser1 successfully
   

Step 4: Test the API User

1. Configure mc with the New User:

   mc alias set apiuser1 myminio https://minio.example.com a1b2c3d4e5f6g7h8 i9j0k1l2m3n4o5p6q7r8s9t0u1v2w3x4 --api S3v4
   

2. List All Buckets:

   mc ls apiuser1
   

   Expected Output:

   [2024-04-25 10:00:00 UTC]     Bucket1
   [2024-04-25 10:00:00 UTC]     Bucket2
   

3. Create a New Bucket:

   mc mb apiuser1/new-bucket
   

   Expected Output:

   Bucket created successfully `new-bucket`.
   

4. Upload an Object:

   mc cp example.txt apiuser1/new-bucket/
   

   Expected Output:

   Upload Success:  example.txt to new-bucket/example.txt
   

5. Download an Object:

   mc cp apiuser1/new-bucket/example.txt ./example.txt
   

   Expected Output:

   Download Success:  new-bucket/example.txt to example.txt
   

6. Delete an Object:

   mc rm apiuser1/new-bucket/example.txt
   

   Expected Output:

   Removed `new-bucket/example.txt`
   

7. Remove the Bucket:

   mc rb apiuser1/new-bucket
   

   Expected Output:

   Removed `new-bucket`.
   

Note: Since the API user has full access, all these operations should succeed, mirroring the root user's capabilities.

---

9. Additional Resources

• MinIO Official Documentation:

  • MinIO Client (mc) Quickstart Guide
  • MinIO Admin API
  • MinIO Policy Documentation

• Security Best Practices:

  • MinIO Security Best Practices

• Secret Management:

  • HashiCorp Vault
  • AWS Secrets Manager
  • Kubernetes Secrets

• MinIO Community and Support:

  • MinIO GitHub Repository
  • MinIO Community Slack

---

Conclusion

By following the updated steps outlined above, you can create an API user in MinIO with read and write permissions equivalent to the root user using the latest mc admin policy create and mc admin policy attach commands. This setup allows your applications or services to interact with MinIO seamlessly while maintaining secure and controlled access.

Important Considerations:

• Security Risks: Granting full access poses significant security risks. Ensure that the API user's credentials are stored securely and are only accessible to authorized applications.

• Audit and Monitoring: Regularly monitor the API user's activities to detect any unauthorized or unusual behaviors.

• Credential Management: Implement robust credential management practices, including regular rotation and secure storage.

Always tailor your MinIO user policies to align with your organization's security policies and operational requirements.

If you encounter any issues or need further assistance, consider reaching out to the MinIO community or consulting the official MinIO documentation.