Azure Storage

1. Azure Storage is a service that you can use to store files, messages, tables and other types of information.
2. Refer following link for Azure Storage update

                  https://azure.microsoft.com/en-gb/updates/?product=storage 

3. Think of Azure Storage in following 3 areas
1. Storage for Virtual machines
2. Unstructured Data storage
3. Structured Data Storage

4. General purpose and BLOB storage accounts. These are 2 kind of storage accounts

 

 

1. A general-purpose storage account gives you access to Storage services such as Tables, Queues, Files, Blobs and Azure VM disks under a single account. This type of storage account has two performance tiers:
• A standard which allows you to store Tables, Queues, Files, Blobs and Azure VM disks.
• A premium  which currently only supports Azure VM disks.

 

2. A Blob storage account is a specialized storage account for storing your unstructured data as blobs (objects) in Azure Storage. Blob storage has two tiers:
• A Hot access tier which indicates that the objects in the storage account will be more frequently accessed. This allows you to store data at a lower access cost.
• A Cool access tier which indicates that the objects in the storage account will be less frequently accessed. This allows you to store data at a lower data storage cost.
• https://docs.microsoft.com/en-gb/azure/storage/storage-blob-storage-tiers

 

5. Standard and Premium storage Accounts

1. Standard Storage Account are backed by Magnetic drives (HDD) and provide the lowest cost per GB.
2. Premium storage account are backed by SSD and offer consistent low-latency performance

 

Note:

It is not possible to convert standard storage to Premium Storage account or vice versa. You must create a new account with the desired type and copy data to a new account, if applicable. 

6. Replication Options: The Data in Storage account is always replicated to ensure durability and HA. Following 4 replication options are available during the creation of the Storage account

 

 

Note:

You can change how your data is replicated after your storage account has been created, unless you specified ZRS when you created the account. However, you may incur an additional one-time data transfer cost if you switch from LRS to GRS or RA-GRS. 

7. There are 4 type of storage Services
1. BLOB Storage:  VM in particular always use BLOB storage. BLOB basically is any large files.
2. Table: is a table structure. Its NoSQL i.e. Key value pair. This is used for like log files, diagnostic information etc. 
3. Queue Storage: Provides messaging for workflows and communication among components and Cloud Services
4. File Storage: Used to provide storage for legacy application that need access to file shares using the standard SMB 3.0 Protocol.

 

8. Planning for Azure Standard Storage
1. BloB Storage
1. Use Block BLOBs for applications that require large files
2. Use page blobs for applications that read random I/O patterns - For ex. Azure VM
2. Table Storage
1. Use tables as a NoSQL store for Key/Value record
3. Queue Storage
1. Use for applications that needs to store a backlot of Sync tasks or to enable async service to service messages
2. Many app developers use the new Azure Service Bus instead of Azure Queue storage
4. File Storage
1. Use the Azure File services to create file shared in Azure Storage 

9. Storage Billing
1. Storage Cost Comprise
1. Bandwidth-amount of data transferred
1. Billed only for data egress (Data exiting from Azure)
2. Transfer of data between Azure services in the same region not billed
2. Transactions - request against a storage account
3. Storage use- amount of storage used
10. standard vs Premium Storage
1. Premium storage uses a different billing model
1. Pay for provisioned, rather than used
2. Premium storage does not incur separate charges for transactional volumes
11. All About Azure Storage Pricing.https://azure.microsoft.com/en-gb/pricing/details/storage/blobs/ 
12. Storage Account: An Azure Storage Account provides a unique namespace to store and access your Azure storage data objects
13. 
14. Creating a Storage Account

All About Storage Accounts

1. Specify a unique name
1. 3 - 24 characters
2. Name should be globally unique
3.  Contains only lowercase letters and digits 

2. Choose a redundancy level and type
1. LRS
2. ZRS
3. GRS
4. RAGRS
3. Generates
1. Primary and Secondary access keys
2. BLOB, queue, table and file service endpoints 

Storage Access Tools 

1. Storage Access Tools
1. Rest API and Client libraries
2. Azure PowerShell
3. Azure Storage Explorer
4. AzCopy
5. Visual Studio 

2. Microsoft Azure Storage explorer:

is a standalone app from Microsoft that allows you to easily work with Azure Storage data n Windows, Mac and Linux. Click http://storageexplorer.com/ to download Storage explorer. Click All About storage explorer 

Storage explorer is supported on MAC, Windows and Linux OS 

Storage explorer can be used to

1. Connect to an Azure subscription - Manage storage resources belonging to your Azure subscription.
2. Work with local development storage - Manage local storage using the Azure Storage Emulator.
3. Attach to external storage - Manage storage resources belonging to another subscription using the storage account's account name and key.
4. Attach storage account using Shared Access Signature* (SAS) - Manage storage resources belonging to another subscription using a SAS.
5. Attach a service using SAS - Manage a specific storage service (blob container, queue, or table) belonging to another Azure subscription using a SAS.

Note * A SAS (Shared Access Signature) gives the admin of an Azure subscription the ability to grant access to a storage account on a temporary basis without having to provide their Azure subscription credentials.  

AZCopy

is a Windows command-line utility designed for copying data to and from Azure Blob, File, and Table storage using commands with optimal performance. You can copy data from one object to another within your storage account, or between storage accounts. More Detail about AzCopy.

Download the latest version of AzCopy

 Refer following to see the comparison of various Storage Client tools  

• https://docs.microsoft.com/en-us/azure/storage/storage-use-azcopy
• http://storagetools.azurewebsites.net/

 

 

Advanced-Azure Virtual Machines

1. IaaS (infrastructure As A Service)
1. Why do you need Azure VMs
2. All underlying infrastructure is managed by MS
3. VMs managed by you

2. Creating Azure VM
1. Azure Management Portal
2. Azure Portal
3. Windows PowerShell
4. Cross-Platform  CLI
5. Visual Studio 

3. Construct of a VM
1. Cloud Service (Domain name)- Container for a VM(s)

Note: Domain name here refers to Cloud Service and not the AD

4. Storage- where the OS/Data VHD of the VM is stored
1. Image-The OS/Underlying image of the VM
2. Networking
3. OS Related configuration 

5. VM Storage
1. Storage Account
2. Storage Endpoints
3. Storage Containers
4. Storage URL -Media Locations

 

6. Some facts about Azure Storage
1. Windows - OS Disk - C: - Max Size is 1 TB-Dynamic disk
2. Linux - OS Disk /dev/sda1 -Partition of root directory
3. Data Disk each max upto 1 TB
4. All VMs have physical temporary disk i.e D Drive
5. Disk IOPS - Max 500 IOPS. To increase the IOPS consider stripping the disk of using storage spaces 

7. By default, Azure VM gets created with a default storage account. In this configuration,  each machine gets created in an individual storage account. In this it created the random storage account. However in production we have to create storage account with the designated process

 

8. Concept-Physical Temporary disk
1. Login to one of the Azure VM
2. Each Azure VM will have a D:\ Drive i.e. Temporary storage
3. Inside the D Drive there will be a text file
4. This is a free drive and a physical disk on the server
5. This should be used only for temp storage
6. As soon as the VM restarts the data will gets deleted
7. THIS IS NOT A DATA DRIVE

 

9. Azure VM Tier and Sizes
1. Basic Tier
2. Standard Tier 

10. Difference in the Tier
1. Load Balancing is available in standard
2. HA is available in Standard
3. Auto Scale is there in standard

Note: This tier information is shown/available when we try to create the VM from the portal

 

11. OS Sizes
1. These are the pre-built sizes for the VM
2. These are available in form of SKU series. For example A, D, G, F, H, N, L
3. Useful Powershell Commandlet
1. Get-AzureRoleSize
2. (Get-AzureRoleSize).InstanceSize. Note: Basic Tier sizes will show with Basic Prefix.
3. Get-AzureRoleSize -InstanceSize ExtraSmall
4. Visit following link for more detailed information on Azure VM and Cloud service sizes https://docs.microsoft.com/en-us/azure/cloud-services/cloud-services-sizes-specs 

 

12. Azure VM Images
1. There are following 2  type of images
1. Azure Provided
2. Custom Images
2. Trial Subscription doesn’t shows Images for Client OS. However MSDN subscription does
3. In Azure, for custom image you cant clone the image. You have to capture the image to create a custom image
4. Get-AzureVMImage shows the complete list of images
5. These are not just OS Images. Azure provides images for example SQL, Sharepoint, Visual studio, Firewall etc.
6. Get-AzureVMImage | Select Imagefamily -unique
7. Concept of Version Released Date.

 

13. Azure VM Advanced features
1. Load balanced sets
2. Availability Sets
3. Scalability. Note: Azure Takes approx. about an hour for Scale In and Scale Out. 

14. Following 2 PowerShell commands can be used for connecting/ Saving RDP on the Azure VM
1. Get-azureremotedesktopfile -servicename <CS name> -name <VM name> -launch
2. Get-azureremotedesktopfile -servicename <CS name> -name <VM name> -localpath "c:\temp" 

15. Following steps on how to access Azure VM using PowerShell Endpoint
1. Launch Chrome
2. Go to the Azure portal
3. Copy the Respective machine's  DNS Name
4. Put the DNS name in Chrome with : <Port Number>
5. Download/copy the certificate locally on the machine
6. Right Click on Certificate and Click on install on the local machine
7. Put this certificate in Trusted Root Certificate
8. Run following command
1. Enter-PSSession -ComputerName "VM Name" -port <Port Number> -Credential <UserName> -UseSSL
2. This will prompt for Password
3. Type in the password. 

16. Ways to create VM
1. Quick Create
2. Create VM from Gallery (Gives more control)
3. Azure VHD Storage
4. Attach a data disk to an existing VM
5. Create a new VM from an existing OS Disk 

17. Certain facts about VM
1. Every Azure VM is associated with a Cloud Service
2. Every Azure VM is associated with respective Storage Account
3. Go under Storage Account > Click Containers Tab > Click VHD folder
4. There  will be 2 files one if the .VHD and .status file
5. .VHD is the actual Virtual Machine.
6. If we create another VM in the same Storage Account then respective .VHD and .status file will be shown under the same VHD folder
7. By default RDP and PowerShell endpoints get created during VM creation
8. VM Agent also gets deployed while creating the Virtual Machines. VM Agent is responsible for managing the extensions which got installed on the Virtual Machine(s)
9. From the VM > Configure tab we can change the VM size
10. By default: VM just have OS disk and a temporary disk.
11. Can we change disk label for D Drive i.e. a temporary Drive? Answer is Yes 

 

Azure VM Data Disks

 

1. There are 2 type of VHDs in Azure
1. Disks- OS and Data Disks. Both OS and Data disks can be upto 1 TB
2. Images - A disk from which many VMs can be provisioned 

2. VHDs can be uploaded from On-Prem to Azure. Both OS and Data VHDs can be uploaded 

3. Important Points-Azure VM Data disks
1. Max size for OS and Data VHDs - 1 TB
2. VHDX are not supported
3. Only support fixed size disk (Dynamic gets converted to fixed during upload, but during the download)
4. VM Size determines the MAX number of data disk that can be attached 

4. Demo
1. Create/Attach new data disk from Management Portal and Powershell
2. Associate Newly created Data disk to a VM
3. Create a local Data VHD
4. Upload a Data VHD from On-Prem to Azure
5. Register/Associate/Create uploaded VHD as an Azure Disk
6. Attached Azure Disk with VM from Management Portal and PowerShell

 

5. When we go under Virtual Machine properties > Disk tab. Disk Name is not necessarily to be same as the corresponding VHD file name in Storage account. Get-AzureDisk Commandlet can be used to see the disk Name and its respective association with the VM

VM in Azure

1. Azure Virtual Machines (VM) are the building block of Cloud IaaS 

2. Following is the list of certain key Business Scenario for IaaS
1. Test and Development
2. Website Hosting
3. Storage, Backup and Recovery
4. Web Apps
5. High Performance Computing
6. Big Data Analysis 

3. Below screenshot provide Comparison for On-Premise and Azure VM

 

 

4. Following PS Commandlet can be used to convert VHDX to VHD. Refer Click Here

Convert-VHD –Path c:\test\MY-VM.vhdx –DestinationPath c:\test\MY-NEW-VM.vhd

 

5. Workloads that are not suitable for  Azure IaaS VM
1. Low Volume or Limited Growth
2. Regulated environment Workloads 

6. The Microsoft server software support for Microsoft Azure virtual machines page lists the current supported products and versions 

7. Following 2 tools can be used to check if your On-Prem Infra is ready for Move to Cloud


1. Azure VM Readiness Assessment tool
2. Azure VM Optimisation Assessment tool 

8. Azure Cost Estimator tool is replaced by Azure Pricing Calculator. However, If you are an EA, Partner or Cloud solution provider then you can obtain access to the Azure Channel pricing calculator. 

9. When planning for Azure VM, there are range of factors to take into Account
1. Storage: How much, where and what configuration
2. Disk: Sizing
3. Compute: Capacity required
4. Availability: Uptime requirements, Geo-distribution, SLA and accessibility
5. Cost: Azure Services, such as storage and compute 

10. VM Sizing consideration
1. The size of the VM affects the pricing and the tier affects of some capabilities
2. A1 is the smallest size recommended for Production
3. When Deploying a VM for SQL Enterprise, Select a VM with at least 4 CPU cores
4. Each Cloud Service can only contain a max of 50 VM 

11. Methods of creating VM
1. Azure Portal
2. ARM template
3. Azure Powershell
4. Visual Studio

12. Considerations for migrating from On-Prem to Azure VM
1. VHD files can be uploaded from On-Prem DC
1. Must be from Hyper-V Virtual machines? Cant we upload VHD generated from other source
2. .VHDX files not supported
3. On-Pre VM must be generalised using sysprep.exe
4. .VHD file must be a fixed size Virtual Disk 

13. Demo on creating a VM through the portal using quick create option. There are 2 options of creating VM from portal
1. Quick Create.
2. Gallery 

14. When we create VM in azure the price includes the licensing as well. This can be validated from My Computer properties which shows that Image is licensed. 

15. Difference between Quick Create and create from gallery is the level of control available while creating the VM. 

16. Green Check mark next to the DNS name denotes that you have unique name which can be publically available 

17. If we go under the settings of the VM following tabs will get reflected
1. Dashboard
• Status
• DNS Name
• Host Name
• Public VIP ( This is the public VIP . This is the internal IP Address of the machine)
• Internal IP
• Size
• RDP Certificate Thumbprint
• Location
• Subscription Name/ID 
2. Monitor

• CPU Percentage
• Disk Read Bytes/Sec
• Disk Write Bytes /Sec
• Network In
• Network Out 

• Endpoints
• RDP (This allow us to remotely connect to the VM)
• Powershell 

4. Configure
• VM Tier
• VM Size
• Availability set 

18. Demo on accessing the VM in Azure
1. When you try to connect to a VM it will download the RDP file for the respective connection
2. BGINFO is deployed when we create a VM which gives basic information on the machine's desktop, BGINFO is deployed as part of the image
3. We don’t  have control on the public IP Address that’s given by azure 

19. Following notifications will appear once the VM build is completed
1. Successfully created Virtual Machine <VM_Name>
2. Successfully provisioned Virtual Machine <VM_Name>
3. Successfully installed extensions in Virtual Machine <VM_Name>

 

Virtual Machine Provisioning using PowerShell:  

20. There are 2 main methods to deploy VM using Powershell.
1. Quick VM Provisioning Mode
1. With this command a Single VM can be provisioned using an Image

New-AzureQuickVM –Windows –ServiceName "MySvc1" –name "MyWinVM1" –ImageName $image –Password $adminPasswd

2. Advanced Provisioning configuration mode:
1. This use combination of commands to build a configuration object for VM. Command line to use is New-AzureVMConfig
21. Come features are Only available from the Advanced Configuration mode
1. Create a VM using OS disk
2. Specify AD domain join information
3. Create new or attach existing disks
4. Disable windows update
5. Specify time zone
6. Specify Static IP Address
7. Specify reserved IP Address of the Cloud Service/Domain Name

 

Virtual Machine Provisioning using Templates

22. Following Process is used for creating VM using ARM templates 

1. User can create their own template using the information found in Authoring Azure Resource Manager templates. Can also deploy templates that have been created from Azure Quickstarts Templates. The template file can be created with a simple text editor and the file type is JSON.
2. To specify values for the resource parameters that were defined in the template, you create a parameters file. This is also a JSON file type.
3. All resources, like virtual machines, must be part of a resource group.
4. Once the template and resource files are ready, you can use the New-AzureRmResourceGroupDeployment command to implement your resource.
5. Command: New-AzureRmResourceGroupDeployment -Name <DeploymentName> -ResourceGroupName <ResourceGroupName -TemplateURI <TemplateURI>>

 

 

Virtual Machine Provisioning using Custom Image 

Following Process used to create VM using custom Image

 

24. Image Sources: Following 2 sources can be used to source image
1. Azure Market Place
2. VM Depot

 

Virtual Machine IP Addressing

25. There are following two type of address which can be used on Azure VM
1. Public IP Address
2. Private IP Address

 

Virtual Machine Availability Set 

26. Availability Sets: Is one way to prepare for planned or unplanned failures. It’s a logical grouping of 2 or more Virtual Machines 

27. Following are few key principles for Availability sets
1. For redundancy, configure multiple VM in Availability Sets
2. Configure each application tier into separate Availability sets
3. Combine a load balancer with availability sets 

28. What are Update domain (UD) and Fault domain (FD) in Availability Sets

Note: Detailed article on UD and FD Detail-UD and FD 

Each VM in an Availability Set is automatically assigned an update domain and a fault domain. This assignment helps Azure maintain High Availability and fault tolerance when deploying and upgrading Applications 

1. Update Domain

An update domain allows Azure to perform incremental or rolling upgrades across a deployment. Each update domain contains a set of VMs and associated physical Hardware that can be updated and rebooted at the same time. During planned maintenance, only one update domain is rebooted at a time. By default there are five update domains, but you configure up to twenty update domains

 

2. Fault Domain (Can be think of racks)

A fault domain defines a group of Virtual Machines that share a common set of hardware, switches, and more that share a single point of failure. For example, A server rack services by a set of power or networking switches. VMs in a availability set are placed in at least 2 fault domains. This mitigates against the effects of the hardware failures, network outages, power interruptions or software updates

 

Virtual Machine Scalability (Scaleset) 

29. Scale Set are an Azure compute resources you can deploy and manage a set of identical VMs. With all VMs configured the same, VM scale set are designed to support true auto scale - no Pre-provisioning of VM is required- and as such makes it easier to build large-scale services targeting big compute, big data, and containerized workloads 

             Scale Set allow you to automatically grow and shrink the application based on the demand.

             Scale Set can only be implemented for v2 VMs that means this can't be done in classic portal. 

30. Related Links
1. https://docs.microsoft.com/en-us/azure/virtual-machine-scale-sets/virtual-machine-scale-sets-overview
2. https://docs.microsoft.com/en-us/azure/virtual-machine-scale-sets/virtual-machine-scale-sets-windows-create

 

Virtual Machine Disks

 

31. Azure VM disks

 

32. Virtual Machines Disk types

VMs in Azure also use disks as a place to store OS, Application and data. All Azure VMs have atleast 2 disks - OS Disk and temporary disk. VM can also have one or more data disks/ All disks are stored as VHDs and the maximum capacity is 1023 GB

 

33. Operating System Disk

Every VM has one attached OS disk. Its registered as a SATA drive and labelled as the C:\ Drive by default

 

34. Temporary Disk

Every VM has a temporary disk that is automatically created. On Windows VMs, this disk is labelled as D: Drive by default and it is used for storing pagefile.sys 

Don’t store any data on Temporary disks. It provides temporary storage for applications and processes and is intended to ONLY store data such as page or swap files.

 

35. Data Disks

Every VM can have data disks to store application data, or other data you need to keep. Data disks are registered as SCSI drives and labelled with a letter  that you choose

The size of the virtual machine determines how the size of the temporary disk and the maximum number of disks you can attach. Data disks are stored in a BLOB in an Azure storage account. OR Size of the VM dictates how many data disk we can have

 

Importing and exporting disks 

36. If you want to move on-premises data to Azure Storage (or vice versa), there are a variety of ways to do this. One way is the Azure import and export service. The Azure Import/Export Service allows you to:

• Import. Securely transfer large amounts of data to Azure blob storage by shipping hard disk drives to an Azure data center.
• Export. Transfer data from Azure blob storage to hard disk drives and ship to your on-premises site.  

This service is suitable in situations where you want to transfer several TBs of data to or from Azure, but uploading or downloading over the network is not feasible due to limited bandwidth or high network costs. Scenarios where this would be useful include:

 

• Migrating data to the cloud. Move large amounts of data to Azure quickly and cost effectively.
• Content distribution. Quickly send data to your customer sites.
• Backup. Take backups of your on-premises data to store in Azure blob storage.
• Data recovery. Recover large amount of data stored in blob storage and have it delivered to your on-premises location. 

• General Guidance
1. Can create both Linux and Windows VM scale sets from the Azure portal. These Scale sets are automatically created with load balancer NAT rules to enable SSH or RDP connections
2. A Scale Set can vary between 0 and 100 VMs and it is a simple call to change the number of VMs
3. You can set the Maximum, minimum and default number of VMs and define triggers- action rules based on resource consumption
4. When you increase the number of VMs in a scale set, VMs are balanced across update and fault domain to ensure, maximum availability. Similarly when you scale in, VMs are removed with maximum availability in mind.  

• Azure Resource Explorer

This is a great tool to view and modify resources you have created in your subscription. The tool is web based and uses your Azure portal logon credentials. This tool is particularly useful in viewing Azure Scale sets. With the tool you can see the individual VM  and their properties

 

Managing Virtual Machines

37. There are various options for managing azure Virtual machines
1. VM Agents and extensions

Extensions are software components that extend the VM functionality and management operations. Multiple extensions can be installed on a VM. Currently available ones are DSC, Chef, puppet. VM agent is a light weight process intended to bootstrap these additional extensions. These can be installed during or post VM installation

2. Azure XPLAT-CLI

This provide set of open source, cross platform commands for working with azure. XPLAT-CLI are good for Linux based VM as for windows Powershell is a good option

3. RDP
4. SSH

 

39. Configuration Management tools
1. Deploying and maintaining the desired state of your VM and application resources can be tedious and error prone. Azure Supports 2 configuration management systems
• DSC
• Chef and Puppet (Similar to DSC. Popular in Linux environment. Can be used in Windows)

Note: Puppet is a Client Server app. You would require Puppet Management server

 

Monitoring Virtual Machines

40. In Azure, Administrator can enable diagnostic logging for
1. Basic Metrics
2. Network and web metrics
3. .Net Metrics
4. Windows Event System logs
5. Windows event Security logs
6. Windows Event Application logs
7. Diagnostic infrastructure logs

 

41. Alerts: can also be configured 

42. Single Instance SLA-22nd Nov 2016

https://blogs.technet.microsoft.com/uspartner_ts2team/2016/11/22/azure-single-instance-virtual-machine-sla/

Over the last few months, we have done extensive work to improve availability of the Azure infrastructure, including innovative machine-learning to predict failing hardware early and offering premium storage to help improve reliability and performance of attached disks. Today, we are announcing a new 99.9% single-instance availability SLA to better support applications that cannot easily scale beyond single VMs. We hope this enables you to move even more workloads into Azure and take advantage of the agility of the cloud without compromising on your expectations of availability. 

To qualify for the single instance virtual machine SLA, all storage disks attached to the VM must be using premium storage, which offers this high level of availability and performance with up to 80,000 IOPS and 2,000 MBps of disk throughput. In addition to this new availability commitment, customers can continue to build for multi-machine high availability by having two or more VMs deployed in the same Availability Set or by utilizing VM Scale Sets which both provide machine isolation, network isolation, and power unit isolation across multiple virtual machines.

 

1. When you running a VM following charges will be applied














1. Will be charged for running the VM
2. Will be charged for Storage consumed by VM
3. Will be charged for Virtual Network(s)

2. When we stop the VM from the RDP Session the status will be stopped, However when we do the shutdown from portal VM will go through the proper shutdown. There is big difference in two shutdown. If you shutdown from portal then only you wont be charged for the time you are not using the VM. However even if the VM is shutdown from RDP session we will be charged for storage.