Verification & Testing

Confirm your deployment and run first commands

Verification & Testing

Congratulations on deploying Ohlala SmartOps! Let’s verify everything is working and explore the capabilities.

✅ Deployment Checklist

Before testing commands, verify each component:

AWS Infrastructure

  • CloudFormation stack shows CREATE_COMPLETE
  • ECS service has 1 running task
  • API Gateway endpoint responds to health check
  • CloudWatch logs are being created
  • Secrets Manager contains credentials

Azure & Teams

  • Azure Bot shows Teams channel as Running
  • Webhook URL is configured correctly
  • Teams app is installed
  • Bot appears in Teams chat/channel

Bedrock

  • Claude Sonnet 4 shows Access granted
  • Model is available in deployment region

🎯 Your First Commands

1. Test Connection

@Ohlala SmartOps hello

Expected Response: Friendly greeting confirming the bot is working

2. Get Help

@Ohlala SmartOps help

Expected Response: Interactive card with available commands and examples

3. Check Instance Status

@Ohlala SmartOps show me my EC2 instances

Expected Response: List of your EC2 instances with status information

4. Health Report

@Ohlala SmartOps /health

Expected Response: Detailed health metrics for your instances

5. Natural Language Query

@Ohlala SmartOps which instances are running in us-east-1?

Expected Response: Filtered list based on your query

🔍 Advanced Testing

Test SSM Integration

@Ohlala SmartOps check disk space on i-1234567890abcdef0
  • Verifies SSM command execution
  • Returns disk usage information

Test Cost Analysis

@Ohlala SmartOps analyze my EC2 costs
  • Checks CloudWatch metrics access
  • Provides cost optimization suggestions

Test Multi-Instance Commands

@Ohlala SmartOps show me all stopped instances
  • Tests filtering and analysis capabilities
  • Demonstrates natural language understanding

📊 Monitoring Your Deployment

CloudWatch Metrics

Monitor key metrics in CloudWatch:

  1. ECS Service

    • CPU utilization (should be <50%)
    • Memory utilization (should be <70%)
    • Task count (should be 1)

  2. API Gateway

    • Request count
    • 4XX/5XX errors (should be minimal)
    • Latency (should be <3 seconds)

  3. Bedrock Usage

    • Token consumption
    • API throttling events
    • Model invocation errors

🚨 Common Issues & Solutions

Issue: Bot Not Responding

Quick Diagnosis:

# Check health endpoint
curl https://your-api.execute-api.region.amazonaws.com/prod-stackname/health

Solutions:

  1. Check ECS task is running
  2. Verify webhook URL in Azure
  3. Ensure Teams app is installed
  4. Review CloudWatch logs

Issue: “Model Access Required” Error

Symptom: Bot responds but shows Bedrock error

Solution:

  1. Go to Bedrock Console → Model access
  2. Enable Claude Sonnet 4
  3. Wait for “Access granted”
  4. Retry command (no restart needed)

Issue: No Instances Found

Symptom: Bot works but doesn’t see EC2 instances

Checks:

  1. Instances are in same region as deployment
  2. Instances have SSM agent installed
  3. IAM permissions are correct
  4. Try: @Ohlala SmartOps list all instances in all regions

Issue: Commands Timeout

Symptom: Bot shows “thinking” but never responds

Solutions:

  1. Check ECS task memory/CPU
  2. Look for Bedrock throttling
  3. Verify network connectivity
  4. Scale ECS service if needed

Issue: Authentication Failures

Symptom: 401/403 errors in logs

Solutions:

  1. Regenerate Azure Bot credentials
  2. Update Secrets Manager
  3. Restart ECS service
  4. Check tenant ID is correct

Best Practices

  1. Start simple: Use basic commands first
  2. Be specific: Include instance IDs for targeted actions
  3. Use natural language: The bot understands context
  4. Review suggestions: Always verify before applying changes

🎉 Success Indicators

Your deployment is successful when:

  • ✅ Bot responds within 2-3 seconds
  • ✅ All test commands work
  • ✅ No errors in CloudWatch logs
  • ✅ Costs align with expectations
  • ✅ Team members can use the bot

📚 Next Steps

Now that your bot is working:

  1. Explore Features

    • Try advanced commands
    • Experiment with natural language queries
    • Review health and cost reports

  2. Train Your Team

    • Share the bot with team members
    • Create usage guidelines
    • Document common workflows

🆘 Getting Help

If you encounter issues:

  1. Check Documentation

  2. Contact Support

  3. Community Resources

🎊 Congratulations!

You’ve successfully deployed Ohlala SmartOps! Your AI-powered infrastructure assistant is ready to help manage your AWS environment through natural language conversations in Microsoft Teams.

Happy automating! 🤖