This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Verification & Testing

Confirm your deployment and run first commands

    Verification & Testing

    Congratulations on deploying Ohlala SmartOps! Let’s verify everything is working and explore the capabilities.

    ✅ Deployment Checklist

    Before testing commands, verify each component:

    AWS Infrastructure

    • CloudFormation stack shows CREATE_COMPLETE
    • ECS service has 1 running task
    • API Gateway endpoint responds to health check
    • CloudWatch logs are being created
    • Secrets Manager contains credentials

    Azure & Teams

    • Azure Bot shows Teams channel as Running
    • Webhook URL is configured correctly
    • Teams app is installed
    • Bot appears in Teams chat/channel

    Bedrock

    • Claude Sonnet 4 shows Access granted
    • Model is available in deployment region

    🎯 Your First Commands

    1. Test Connection

    @Ohlala SmartOps hello

    Expected Response: Friendly greeting confirming the bot is working

    2. Get Help

    @Ohlala SmartOps help

    Expected Response: Interactive card with available commands and examples

    3. Check Instance Status

    @Ohlala SmartOps show me my EC2 instances

    Expected Response: List of your EC2 instances with status information

    4. Health Report

    @Ohlala SmartOps /health

    Expected Response: Detailed health metrics for your instances

    5. Natural Language Query

    @Ohlala SmartOps which instances are running in us-east-1?

    Expected Response: Filtered list based on your query

    🔍 Advanced Testing

    Test SSM Integration

    @Ohlala SmartOps check disk space on i-1234567890abcdef0
    • Verifies SSM command execution
    • Returns disk usage information

    Test Cost Analysis

    @Ohlala SmartOps analyze my EC2 costs
    • Checks CloudWatch metrics access
    • Provides cost optimization suggestions

    Test Multi-Instance Commands

    @Ohlala SmartOps show me all stopped instances
    • Tests filtering and analysis capabilities
    • Demonstrates natural language understanding

    📊 Monitoring Your Deployment

    CloudWatch Metrics

    Monitor key metrics in CloudWatch:

    1. ECS Service

      • CPU utilization (should be <50%)
      • Memory utilization (should be <70%)
      • Task count (should be 1)

    2. API Gateway

      • Request count
      • 4XX/5XX errors (should be minimal)
      • Latency (should be <3 seconds)

    3. Bedrock Usage

      • Token consumption
      • API throttling events
      • Model invocation errors

    🚨 Common Issues & Solutions

    Issue: Bot Not Responding

    Quick Diagnosis:

    # Check health endpoint
curl https://your-api.execute-api.region.amazonaws.com/prod-stackname/health

    Solutions:

    1. Check ECS task is running
    2. Verify webhook URL in Azure
    3. Ensure Teams app is installed
    4. Review CloudWatch logs

    Issue: “Model Access Required” Error

    Symptom: Bot responds but shows Bedrock error

    Solution:

    1. Go to Bedrock Console → Model access
    2. Enable Claude Sonnet 4
    3. Wait for “Access granted”
    4. Retry command (no restart needed)

    Issue: No Instances Found

    Symptom: Bot works but doesn’t see EC2 instances

    Checks:

    1. Instances are in same region as deployment
    2. Instances have SSM agent installed
    3. IAM permissions are correct
    4. Try: @Ohlala SmartOps list all instances in all regions

    Issue: Commands Timeout

    Symptom: Bot shows “thinking” but never responds

    Solutions:

    1. Check ECS task memory/CPU
    2. Look for Bedrock throttling
    3. Verify network connectivity
    4. Scale ECS service if needed

    Issue: Authentication Failures

    Symptom: 401/403 errors in logs

    Solutions:

    1. Regenerate Azure Bot credentials
    2. Update Secrets Manager
    3. Restart ECS service
    4. Check tenant ID is correct

    Best Practices

    1. Start simple: Use basic commands first
    2. Be specific: Include instance IDs for targeted actions
    3. Use natural language: The bot understands context
    4. Review suggestions: Always verify before applying changes

    🎉 Success Indicators

    Your deployment is successful when:

    • ✅ Bot responds within 2-3 seconds
    • ✅ All test commands work
    • ✅ No errors in CloudWatch logs
    • ✅ Costs align with expectations
    • ✅ Team members can use the bot

    📚 Next Steps

    Now that your bot is working:

    1. Explore Features

      • Try advanced commands
      • Experiment with natural language queries
      • Review health and cost reports

    2. Train Your Team

      • Share the bot with team members
      • Create usage guidelines
      • Document common workflows

    🆘 Getting Help

    If you encounter issues:

    1. Check Documentation

    2. Contact Support

    3. Community Resources

    🎊 Congratulations!

    You’ve successfully deployed Ohlala SmartOps! Your AI-powered infrastructure assistant is ready to help manage your AWS environment through natural language conversations in Microsoft Teams.

    Happy automating! 🤖