azureazure-blob-storageazure-data-lakecloud
I have a question about the use cases of the different Azure storage services:
what is the difference between these services? and when to use them since they all provide the same functionality (storage) on Azure's cloud platform.
The easiest way to think of Data Lake is to think of this large container that has like a real lake with rivers coming into the river you never know where the rivers are coming from (or what "type" of river). Azure Data Lake was introduced to make big data easy for developers, data scientists, and analysts to store data of any size. It removes the complexities of ingesting and storing all your data while making it faster to get up and running with big data. Data Lake is able to stored the mass different types of data (Structured data, unstructured data, log files, real-time, images, etc. ) and to blend that together, to correlate many different data types. The key thing here is as we are moving from traditional way to the modern tools (like Hadoop, Cassandra, NoSQL DB, etc). Azure Data Lake includes three services:
Azure Data Lake Store is like a cloud-based file service or file system that is pretty much unlimited in size. We can run services on top of the data that's in that store. So you could use Hadoop or Spark in an HDInsight cluster, or you could use the Azure Data Lake analytic service, which is a complement to the Azure Data Lake Store. And what that service will let you do is to run jobs that effectively query the data you have stored in the Azure Data Lake store and generate output results.
Basically, think of gen2 as a superset of gen1 plus all of the best parts of blob storage: tiers, HDFS and object store API's and presumably the ability to efficiently handle the management of over 35K files and efficiently dealing with many small sizes and more trickle write type operations.. plus its cheaper.
I'm trying to get some clarity on a few specifics but not finding much in the meantime try these links:
https://azure.microsoft.com/en-us/blog/a-closer-look-at-azure-data-lake-storage-gen2/
https://docs.microsoft.com/en-us/azure/storage/data-lake-storage/introduction
Best Answer
You can take a look at this article: https://docs.microsoft.com/en-us/azure/data-lake-store/data-lake-store-comparison-with-blob-storage
I'd say main differences between Data Lake and Azure Storage Blob is scale and permissions model.
It really makes no sense to paste the whole article here. But you might want to look at Data Lake v2, which (as MS claims) is a mesh or Data Lake v1 and Azure Storage Blob: https://docs.microsoft.com/azure/storage/blobs/data-lake-storage-introduction
As for the Azure File storage its just an SMB share over HTTPS (and it not really fast due to being 1 "stream" only).