Comprehensive .env File Parsing Improvements

[mehrancodes]

Problem

Severity: Enhancement - Functionality & Compatibility
Impact: Improve robustness and compatibility with diverse .env file formats used in Laravel projects

Description

The current MergeEnvironmentVariables implementation uses basic string manipulation that works for simple cases but fails with advanced .env file patterns commonly used in Laravel applications. This leads to inconsistent behavior and potential data corruption.

Current Limitations

1. Quote Handling Issues

• Hash symbols in quoted passwords treated as comments
• Escaped quotes not properly handled
• Mixed quote types (single vs double) inconsistencies

2. Advanced Syntax Not Supported

• Variable interpolation (${VAR} syntax)
• Export statements (export VAR=value)
• Inline comments (VAR=value # comment)

3. Whitespace Inconsistencies

• Spaces around equals signs handled inconsistently
• Mixed line endings (CRLF vs LF) issues
• Unicode characters in values

4. Value Type Ambiguities

• Empty values vs null values not distinguished
• Boolean representations inconsistent
• Numeric values treated as strings

Real-World Examples

Laravel Applications Commonly Use:

1. Complex Passwords with Special Characters

DB_PASSWORD="p@ssw0rd#123!$pecial"
REDIS_PASSWORD='my-password-with-#-symbol'

2. URLs with Hash Fragments

WEBHOOK_URL="https://api.example.com/webhook#section"
OAUTH_REDIRECT="https://app.com/auth/callback#state=xyz"

3. Variable Interpolation

DB_HOST=localhost
DB_PORT=3306
DATABASE_URL="mysql://user:pass@${DB_HOST}:${DB_PORT}/mydb"

4. Export Statements (from shell scripts)

export APP_ENV=production
export DB_CONNECTION=mysql

5. Inline Documentation

APP_NAME=Laravel # Application name
DB_HOST=localhost # Database server
APP_DEBUG=false # Never true in production

6. JSON Configuration

LOGGING_CHANNELS='{"single":{"driver":"single","path":"storage/logs/laravel.log"}}'
MAIL_CONFIG='{"driver":"smtp","host":"smtp.gmail.com","port":587}'

Current Behavior vs Expected

Hash in Quoted Values

# Input
DB_PASSWORD="secret#123"

# Current: Treats everything after # as comment
DB_PASSWORD="secret

# Expected: Preserves full password
DB_PASSWORD="secret#123"

Variable Interpolation

# Input  
API_URL="https://api.${APP_DOMAIN}/v1"

# Current: Treats as literal string
API_URL="https://api.${APP_DOMAIN}/v1"

# Expected: Could expand or preserve (depending on requirements)
API_URL="https://api.${APP_DOMAIN}/v1"

Export Statements

# Input
export APP_ENV=production

# Current: Treats "export APP_ENV" as key
export APP_ENV=production

# Expected: Extracts APP_ENV=production
APP_ENV=production

Proposed Solution

Phase 1: Leverage Existing dotenv Library

Use the already-included vlucas/phpdotenv for proper parsing:

use Dotenv\Dotenv;
use Dotenv\Exception\InvalidPathException;

class ImprovedMergeEnvironmentVariables
{
    public function handle(string $source, array $newVariables): string
    {
        // Parse existing variables properly
        $existingVars = $this->parseEnvironmentString($source);
        
        // Merge with new variables
        $mergedVars = array_merge($existingVars, $newVariables);
        
        // Format back to .env string
        return $this->formatEnvironmentString($mergedVars);
    }
    
    protected function parseEnvironmentString(string $source): array
    {
        // Use dotenv parser for proper handling
        $tempFile = tempnam(sys_get_temp_dir(), 'env_parse');
        file_put_contents($tempFile, $source);
        
        try {
            $dotenv = Dotenv::createMutable(dirname($tempFile), basename($tempFile));
            $parsed = $dotenv->safeLoad();
            return $parsed;
        } finally {
            unlink($tempFile);
        }
    }
}

Phase 2: Enhanced Formatting

Preserve original formatting and comments:

protected function formatEnvironmentString(array $variables): string
{
    $output = '';
    
    foreach ($variables as $key => $value) {
        // Determine if value needs quoting
        if ($this->needsQuoting($value)) {
            $value = '"' . addslashes($value) . '"';
        }
        
        $output .= "$key=$value\n";
    }
    
    return $output;
}

protected function needsQuoting(string $value): bool
{
    return str_contains($value, ' ') 
        || str_contains($value, '#')
        || str_contains($value, '"')
        || str_contains($value, "'")
        || str_contains($value, "\n");
}

Test Cases Required

1. Quote Handling

// Hash in quoted values
$source = 'PASSWORD="secret#123"';
$expected = 'PASSWORD="secret#123"';

// Escaped quotes
$source = 'MESSAGE="He said \"Hello\""';
$expected = 'MESSAGE="He said \"Hello\""';

// Mixed quotes
$source = "MIXED='single \"double\" quotes'";
$expected = "MIXED='single \"double\" quotes'";

2. Advanced Syntax

// Variable interpolation
$source = 'URL="https://api.${DOMAIN}/v1"';
$expected = 'URL="https://api.${DOMAIN}/v1"';

// Export statements
$source = 'export APP_ENV=production';
$expected = 'APP_ENV=production';

// Inline comments
$source = 'DEBUG=false # Production setting';
$expected = 'DEBUG=false';

3. Edge Cases

// Unicode characters
$source = 'TITLE="Café ñoño"';
$expected = 'TITLE="Café ñoño"';

// Empty vs null
$source = "EMPTY=\nNULL=null\nQUOTED=\"\"";
$expected = "EMPTY=\nNULL=null\nQUOTED=\"\"";

Implementation Plan

Phase 1: Research & Design

• Analyze vlucas/phpdotenv capabilities
• Design new parsing architecture
• Create comprehensive test suite

Phase 2: Core Implementation

• Implement proper .env parsing
• Add quote-aware value handling
• Preserve formatting where possible

Phase 3: Advanced Features

• Variable interpolation support
• Export statement handling
• Inline comment preservation

Phase 4: Testing & Validation

• Comprehensive test coverage
• Performance benchmarking
• Backward compatibility verification

Benefits

Improved Reliability

• Handles complex real-world .env files
• Reduces deployment failures
• Better error messages and debugging

Enhanced Security

• Proper handling of passwords with special characters
• No accidental data corruption
• Secure credential management

Better Developer Experience

• Works with any Laravel .env format
• Predictable behavior
• Clear documentation and examples

Future-Proof

• Leverages industry-standard parsing
• Extensible architecture
• Easy to maintain and update

Acceptance Criteria

• All existing functionality preserved (backward compatibility)
• Proper handling of quoted values with special characters
• Support for common Laravel .env patterns
• Comprehensive test coverage (>95%)
• Performance impact < 10% vs current implementation
• Clear documentation with examples
• Migration guide for any breaking changes

Related Issues

• Unable to use FORGE_ENV_KEYS when .env.example contains empty lines #127 - Environment variable parsing crashes (fixed)
• Previous issue - Multiline value corruption (critical)
• This issue - Comprehensive parsing improvements

---

Labels: enhancement, environment-variables, parsing, laravel-compatibility