How to Expand Environment Variables and Create Folders in Bash Using a File

Описание: Learn how to use bash scripting with `envsubst` and `awk` to expand environment variables and create directories based on a file input.
---
This video is based on the question https://stackoverflow.com/q/65536183/ asked by the user 'IMTheNachoMan' ( https://stackoverflow.com/u/3383907/ ) and on the answer https://stackoverflow.com/a/65536247/ provided by the user 'anubhava' ( https://stackoverflow.com/u/548225/ ) at 'Stack Overflow' website. Thanks to these great users and Stackexchange community for their contributions.

Visit these links for original content and any more details, such as alternate solutions, latest updates/developments on topic, comments, revision history etc. For example, the original title of the Question was: echo strings with envrionment variables from lines pulled from a file in bash

Also, Content (except music) licensed under CC BY-SA https://meta.stackexchange.com/help/l...
The original Question post is licensed under the 'CC BY-SA 4.0' ( https://creativecommons.org/licenses/... ) license, and the original Answer post is licensed under the 'CC BY-SA 4.0' ( https://creativecommons.org/licenses/... ) license.

If anything seems off to you, please feel free to write me at vlogize [AT] gmail [DOT] com.
---
How to Expand Environment Variables and Create Folders in Bash Using a File

Bash scripting can simplify your workflow and help automate repetitive tasks. One common problem is needing to use environment variables defined in a file to create directories dynamically. If you have a file listing paths that utilize these variables, you might find that directly using them won't work due to the way shell variable expansion works. In this guide, we'll show you how to extract the required folder paths from your file and create those folders using bash scripting.

Understanding the Problem

Imagine you have a file, test.txt, that includes paths with environment variables like so:

[[See Video to Reveal this Text or Code Snippet]]

Your goal is to create these directories based on the defined environment variables. However, when you try to process this file with simple commands, you'll notice that the variables don't expand as expected. Instead of getting the actual directory names, you'll see the variable syntax itself ("${VAR1}", "${VAR2}", etc.).

Steps to Solve the Problem

1. Using envsubst to Expand Variables

One of the most straightforward ways to accomplish the task is to utilize the envsubst command. This command substitutes the environment variables in a text file with their corresponding values. Here's a simple bash script to achieve this:

[[See Video to Reveal this Text or Code Snippet]]

Explanation:

export VAR1 VAR2 VAR3: This sets up the environment variables necessary for processing.

IFS=' -:': This sets the Internal Field Separator to delimit the input on spaces, dashes, or colons.

while read -r: We read each line of the variable-expanded content from test.txt.

mkdir -p "$d": This creates the directory, and the -p flag allows it to create parent directories as needed.

2. Alternative Solution with envsubst, awk, and xargs

If you prefer a more streamlined approach, you can combine several commands using awk and xargs as follows:

[[See Video to Reveal this Text or Code Snippet]]

Breakdown of the Chain:

envsubst test.txt: This expands the environment variables in the contents of test.txt.

awk -F '[-:[:blank:]]+ ': It processes the output, using -, :, and space as delimiters to isolate directory paths.

xargs -0 mkdir -p: Finally, it takes the extracted paths and feeds them into mkdir, creating the directories all at once.

Conclusion

By using the above methods, you can easily expand environment variables from a file in bash and create directories dynamically. Whether you choose the envsubst method or the combination with awk and xargs, both will help streamline your workflow. The next time you need to handle environment variable paths in bash, these techniques will serve you well!