How To Copy Entire Directory In Linux – Recursive Directory Copying

by

Backing up an entire project folder requires a command that replicates the folder and all of its subdirectories. If you’ve ever wondered how to copy entire directory in linux, you’re in the right place. This guide will walk you through the most efficient methods, from basic commands to advanced options, ensuring you never lose a file again.

Copying a directory in Linux isn’t as simple as dragging and dropping. You need the right command to preserve permissions, timestamps, and hidden files. Don’t worry—it’s easier than it sounds. Let’s dive in.

How To Copy Entire Directory In Linux

The core command you’ll use is cp with the -r (recursive) flag. This tells Linux to copy the folder and everything inside it. Here’s the basic syntax:

cp -r source_directory destination_directory

For example, to copy a folder named project into /backup, you’d run:

cp -r project /backup

This creates a copy of project inside /backup. If /backup doesn’t exist, it creates it. Simple, right?

Understanding The Recursive Flag

The -r flag is crucial. Without it, cp will refuse to copy directories. It stands for “recursive,” meaning it copies all subdirectories and files. Think of it as the “include everything” switch.

  • Without -r: cp project /backup → Error: “omitting directory”
  • With -r: cp -r project /backup → Success

Always double-check you’ve included this flag. It’s a common mistake for beginners.

Preserving File Attributes

By default, cp -r doesn’t preserve permissions, ownership, or timestamps. To keep these intact, add the -p flag:

cp -rp project /backup

This is useful for backups or when copying configuration files. The -p flag stands for “preserve.” It copies modification times, access times, and file modes.

Copying With Verbose Output

Want to see what’s being copied? Add the -v flag for verbose output:

cp -rv project /backup

This prints each file name as it’s copied. It’s helpful for large directories to monitor progress.

Using Rsync For Advanced Copying

While cp works well, rsync is more powerful for large or frequent copies. It’s ideal for backups and syncing directories. Here’s the basic command:

rsync -av source_directory/ destination_directory/

Note the trailing slash on the source. This copies the contents of the source directory into the destination, not the source folder itself.

Rsync Advantages

  • Incremental copying: Only copies changed files
  • Preserves attributes: Permissions, timestamps, ownership
  • Compression: Use -z to compress data during transfer
  • Progress display: Use --progress to see transfer stats

Rsync Examples

Copy a directory locally:

rsync -av /home/user/project /backup/

Copy over SSH to a remote server:

rsync -avz -e ssh /home/user/project user@remote:/backup/

This is perfect for off-site backups. The -z flag enables compression, speeding up transfers over slow connections.

Copying With Tar And Pipe

Another method uses tar to archive and copy directories. This is useful when you need to preserve permissions and ownership across filesystems:

tar cf - source_directory | tar xf - -C destination_directory

This creates a tar archive of the source and extracts it at the destination. It’s efficient for large directories.

When To Use Tar

  • Copying across different filesystems (e.g., ext4 to NTFS)
  • Preserving hard links and symbolic links
  • Copying to a remote system without rsync

Handling Hidden Files And Dotfiles

Hidden files (those starting with a dot) are included by default with cp -r and rsync -a. But if you’re using a wildcard, be careful. For example:

cp -r project/* /backup/

This misses hidden files. To include them, use:

shopt -s dotglob
cp -r project/* /backup/

Or better, just copy the whole directory without a wildcard.

Common Mistakes And How To Avoid Them

Even experienced users make errors. Here are pitfalls to watch for:

Forgetting The Trailing Slash

With rsync, a trailing slash on the source changes behavior. Compare:

rsync -av source/ dest/   # Copies contents of source into dest
rsync -av source dest/    # Copies source folder itself into dest

This is a common source of confusion. Test with --dry-run first.

Overwriting Files Unintentionally

By default, cp overwrites existing files without warning. Use the -i flag for interactive mode:

cp -ri project /backup

This prompts before overwriting each file. It’s safer for critical data.

Running Out Of Disk Space

Always check available space before copying large directories. Use:

df -h

This shows disk usage. If space is tight, consider using rsync --partial to resume interrupted transfers.

Copying Across Different Filesystems

When copying between different filesystems (e.g., from ext4 to NTFS), you may lose permissions or metadata. Use rsync -a to preserve what it can, or tar for maximum compatibility.

For example, copying to a FAT32 USB drive:

rsync -rltDv project /mnt/usb/

The -rltD flags copy recursively, preserving timestamps, and handling devices. It’s a safe bet for cross-filesystem transfers.

Automating Directory Copies With Scripts

If you copy directories regularly, write a simple script. Here’s a bash example:

#!/bin/bash
SOURCE="/home/user/project"
DEST="/backup/$(date +%Y%m%d)"
rsync -av --delete "$SOURCE/" "$DEST/"

This creates a dated backup each time. The --delete flag removes files in the destination that no longer exist in the source, keeping it in sync.

Scheduling With Cron

To run the script daily, add a cron job:

crontab -e

Then add:

0 2 * * * /path/to/backup.sh

This runs the backup at 2 AM every day. Adjust the time as needed.

Copying Large Directories Efficiently

For huge directories (hundreds of GB), use rsync with compression and partial transfers:

rsync -avz --partial --progress source/ dest/

The --partial flag keeps partially transferred files if the connection drops. --progress shows real-time stats.

Using Screen Or Tmux

Long copies can tie up your terminal. Use screen or tmux to run the command in the background:

screen -S backup
rsync -avz --progress source/ dest/
# Press Ctrl+A, then D to detach

This way, the copy continues even if you log out. Reattach later with screen -r backup.

Verifying The Copy

After copying, verify integrity. Use diff to compare directories:

diff -r source/ dest/

If there’s no output, the directories are identical. For large directories, this can be slow. Alternatively, use rsync -av --dry-run to check for differences.

Checking File Counts

Compare file counts with:

find source -type f | wc -l
find dest -type f | wc -l

If they match, you’re likely good. But diff is more thorough.

Copying With Sudo

Some directories require root permissions. Use sudo with caution:

sudo cp -r /etc /backup/etc_backup

Be careful not to overwrite system files. Always double-check the destination path.

Changing Ownership After Copy

If you copy as root, the files may be owned by root. Change ownership with:

sudo chown -R user:user /backup/etc_backup

Replace user with your username. The -R flag applies recursively.

Copying Symbolic Links

By default, cp -r copies the target of symbolic links, not the links themselves. To copy the links, use -a (archive mode):

cp -a project /backup

The -a flag preserves links, permissions, and timestamps. It’s equivalent to -dR --preserve=all.

Rsync And Symlinks

Rsync also preserves symlinks with -a. If you want to copy the link targets instead, use --copy-links:

rsync -av --copy-links source/ dest/

This resolves symlinks and copies the actual files.

Copying To A Remote Server

Use scp or rsync over SSH. Here’s scp:

scp -r project user@remote:/backup/

For better performance, use rsync:

rsync -avz -e ssh project/ user@remote:/backup/

This compresses data and only transfers changes. It’s ideal for frequent backups.

Setting Up SSH Keys

To avoid typing passwords, set up SSH keys:

ssh-keygen -t rsa
ssh-copy-id user@remote

Then you can run rsync without a password prompt. This is essential for automated scripts.

Copying With Progress Bars

Both cp and rsync can show progress. For cp, use the -v flag for verbose output. For rsync, use --progress:

rsync -av --progress source/ dest/

This shows each file and a percentage. For a cleaner view, use pv (pipe viewer) with tar:

tar cf - source | pv | tar xf - -C dest

This gives a real-time speed and ETA.

Recovering From Interrupted Copies

If a copy fails mid-way, rsync can resume. Use the --partial flag:

rsync -av --partial --progress source/ dest/

This keeps partially transferred files. When you re-run the command, it picks up where it left off.

Using Rsync’s –Append

For very large files, use --append to resume without re-checking the entire file:

rsync -av --append source/largefile dest/

This is faster than --partial for single files.

Copying Directories With Exclusions

Sometimes you want to skip certain files or folders. Use --exclude with rsync:

rsync -av --exclude='*.tmp' --exclude='node_modules' source/ dest/

This excludes temporary files and node_modules. You can also use --exclude-from with a file listing patterns.

Exclude Patterns

  • --exclude='*.log' — Exclude all log files
  • --exclude='dir1/' — Exclude a specific directory
  • --exclude='.*' — Exclude hidden files

Test with --dry-run to see what will be copied.

Copying Across Networks With Limited Bandwidth

Use --bwlimit to throttle rsync’s bandwidth:

rsync -av --bwlimit=1000 source/ dest/

This limits speed to 1000 KB/s. It’s useful for shared connections.

Compression Trade-offs

Compression (-z) speeds up transfers over slow links but uses CPU. For local copies, skip compression. For remote, it’s usually beneficial.

Copying To Multiple Destinations

To copy a directory to several locations, use a loop:

for dest in /backup1 /backup2 /mnt/usb; do
    rsync -av source/ "$dest/"
done

This is useful for redundant backups. Ensure each destination has enough space.

Testing Your Copy Command

Always test with --dry-run before executing:

rsync -av --dry-run source/ dest/

This shows what would be copied without making changes. It’s a lifesaver for complex commands.

Using Echo For Debugging

If you’re scripting, echo the command first:

echo "rsync -av source/ dest/"
# Then run it

This catches typos before they cause damage.

Common Use Cases

Backing Up A Website

rsync -avz --delete /var/www/html/ user@backup:/backups/website/

This syncs the web root to a remote server, deleting old files.

Copying A User’s Home Directory

cp -rp /home/user /mnt/backup/

Preserves permissions and timestamps. Use sudo if needed.

Migrating To A New Server

rsync -avz -e ssh /data/ user@newserver:/data/

This transfers all data with compression. Verify with diff