今日推薦開源項目:《深度學習可視化框架VisualDL》
推薦理由:目前,大部分深度學習框架都提供了 Python 的用戶界面,其訓練過程的狀態通常以日記的形式被記錄下來,這種方式可以觀察短期內的訓練狀態,但是難以從全局把握訓練過程中的變化趨勢,導致提取信息時受到較多限制。反觀 Visual DL,它可以使得深度學習任務變得生動形象,實現可視分析,改變了傳統的日記式記錄形態,便於用戶將訓練過程可視化,幫助更好地把控全局。
首先,它的「Scalar」功能支持 Scalar 打點數據展示,可將訓練信息以折線圖的形式展現出來,方便觀察整體趨勢,還能在同一個可視化視圖中呈現多條折線,方便用戶對比分析。
其次,Visual DL 的「Image」功能支持圖片展示,用戶可輕鬆查看數據樣本的質量,也可以方便地查看訓練的中間結果,例如卷積層的輸出或者 GAN 生成的圖片。
同時,Visual DL 還具有 Histogram 參數分布展示功能,方便用戶查看參數矩陣中數值的分布曲線,並隨時觀察參數數值分布的變化趨勢。
最後,Visual DL 中的「Graph」還能幫助用戶查看深度神經網路的模型結構。據悉,Graph 支持直接對 ONNX 的模型進行預覽,由於 MXNet、Caffe2、Pytorch 和 CNTK 都支持轉成 ONNX 的模型,這意味著 Graph 可間接支持不同框架的模型可視化功能,讓用戶便於排查網路配置的錯誤,幫助理解網路結構。
Visual DL 除了功能全面以外,還具有易集成、易使用等優勢。
它可提供獨立的 Python SDK,若用戶的訓練任務基於 Python,可直接安裝 Visual DL 的 WHL 軟體包,隨後輸入到項目中進行使用,使用方式簡單便捷。
為了滿足用戶的不同操作需求,用戶在其 Python 代碼中可加入 Visual DL 日誌記錄邏輯,啟動 Visual DL 後即可通過瀏覽器查看日誌的可視化結果。
此外,Visual DL 在底層使用 C++ 編寫,提供原生的 C++ SDK,用戶可將其深入集成到自己 C++ 的項目,以實現更高效的性能。
值得一提的是,Visual DL 現已完全開放,同時支持大部分的深度學習框架。其 SDK 層面可輕鬆集成到 Python 或者 C++ 項目中,此外, Graph 通過 ONNX 還可直接支持 PaddlePaddle、TensorFlow、MxNet、PyTorch 和 Caffe2 等流行的深度學習框架。對於開發者來說,Visual DL 可以將深度學習任務的訓練過程可視化,減少用戶的觀察比對時間,讓整個訓練過程更高效。
使用
VisualDL 同時提供了 Python SDK 和 C++ SDK ,我們可以快速地將數據可視化。
以如何用Python創建一個簡單的可視化標量為例:
import random from visualdl import LogWriter logdir = "./tmp" logger = LogWriter(dir, sync_cycle=10) # mark the components with 'train' label. with logger.mode("train"): # create a scalar component called 'scalars/scalar0' scalar0 = logger.scalar("scalars/scalar0") # add some records during DL model running, lets start from another block. with logger.mode("train"): # add scalars for step in range(100): scalar0.add_record(step, random.random())
在訓練過程生成了一些日誌後,我們可以登錄控制面板並查看實時的數據可視化效果,登陸方式只需要在命令窗口輸入以下命令:
visuaIDL --logdir <some log dir>
相關信息
VisualDL 是由百度團隊PaddlePaddle & ECharts 團隊開發的深度學習可視化工具。
PaddlePaddle是一個易學易用的分散式深度學習平台,提供豐富的演算法服務,致力於機器視覺和自然語言理解的研究。
官方網站:http://www.paddlepaddle.org/;
今日推薦英文原文:《3 tests for NOT moving to blockchain 》作者:
原文鏈接:https://aliceevebob.com/2018/02/13/3-tests-for-not-moving-to-blockchain/
推薦理由:這兩年來,區塊鏈技術火得不得了,人人都在討論區塊鏈,hmmm,小編覺得這不是什麼好事情,
3 tests for NOT moving to blockchain
So, there』s this thing called 「blockchain」 which is quite popular…
You know that already, of course. I keep wondering if we』ve hit 「peak hype」 for blockchain and related technologies yet, but so far there』s no sign of it. As usual for this blog, when I』m talking about blockchain, I』m going to include DLTs – Distributed Ledger Technologies – which are, by some tight definitions of the term, not really blockchains at all. I』m particularly interested, from a professional point of view, in permissioned blockchains. You can read more about how that』s defined in my previous post Is blockchain a security topic? – the key point here is that I』m interested in business applications of blockchain beyond cryptocurrency[1].
And, if the hype is to be believed – and some of it probably should be[2] – then there is an almost infinite set of applications for blockchain. That』s probably correct, but that doesn』t mean that they』re all good applications for blockchain. Some, in fact, are likely to be very bad applications for blockchain.
The hype associated with blockchain, however, means that businesses are rushing to embrace this new technology[3] without really understanding what they』re doing. The drivers towards this move are arguably three-fold:
- you can, if you try, make almost any application with multiple users which stores data into a blockchain-enable application;
- there are lots of conferences and 「gurus」 telling people that if they don』t embrace blockchain now, they』ll go out of business within six months[4];
- it』s not easy technology to understand fully, and lots of the proponents 「on-the-ground」 within organisations are techies.
I want to unpack that last statement before I get a hail of trolls flaming me[5]. I have nothing against techies – I』m one myself – but one of our characteristics tends to be enormous enthusiasm about new things (「shinies」) that we understand, but whose impact on the business we don』t always fully grok[6]. That』s not always a positive for business leaders.
The danger, then, is that the confluence of those three drivers may lead to businesses deciding to start moving to blockchain applications without fully understanding whether that』s a good idea. I wrote in another previous post (Blockchain: should we all play?) about some tests that you can apply to decide whether a process is a good fit for blockchain and when it』s not. They were useful, but the more I think about it, the more I』m convinced that we need some simple tests to tell us when we should definitely not move a process or an application to a blockchain. I present my three tests. If your answer any of these questions is 「yes」, then you almost certainly don』t need a blockchain.
Test 1 – does it have a centralised controller or authority?
If the answer is 「yes」, then you don』t need a blockchain.
If, for instance, you』re selling, I don』t know, futons, and you have a single ordering system, then you have single authority for deciding when to send out a futon. You almost certainly don』t need to make this a blockchain. If you are a purveyor of content that has to pass through a single editorial and publishing process, they you almost certainly don』t need to make this a blockchain.
The lesson is: blockchains really don』t make sense unless the tasks required in the process execution – and the trust associated with those tasks – is distributed between multiple entities.
Test 2 – could it work fine with a standard database?
If the answer to this question is 「yes」, then you don』t need a blockchain.
This question and the previous one are somewhat intertwined, but don』t need to be. There are applications where you have distributed processes, but need to store information centrally, or centralised authorities but distributed data, where one may be yes, but the other 「no」. But if this is question is a 「yes」, then use a standard database.
Databases are good at what they do, they are cheaper in terms of design and operation than running a blockchain or distributed ledger, and we know how to make them work. Blockchains are about letting everybody[8] see and hold data, but the overheads can be high, and the implications costly.
Test 3 – is adoption going to be costly, or annoying, to some stakeholders?
If the answer to this question is 「yes」, then you don』t need a blockchain.
I』ve heard assertions that blockchains always benefit all users. This is a patently false. If you are creating an application for a process, and changing the way that your stakeholders interact with you and it, you need to consider whether that change is in their best interests. It』s very easy to create and introduce an application, blockchain or not, which reduces business friction for the owner of the process, but increases it for other stakeholders.
If I make engine parts for the automotive industry, it may benefit me immensely to be able to track and manage the parts on a blockchain. I may be able to see at a glance who』s supplied what, when, and the quality of the steel used in the ball-bearings. On the other hand, if I』m a ball-bearing producer, and I have an established process which works for the forty companies to whom I sell ball-bearings, then adopting a new process for just one of them, with associated changes to my method of work, new systems and new storage and security requirements is unlikely to be in my best interests: it』s going to be both costly and annoying.
Conclusion
Tests are guidelines: they』re not fixed in stone. One of these tests looks like a technical test (the database one), but is really as much about business roles and responsibilities as the other two. All of them, hopefully, can be used as a counter-balance to the three drivers I mentioned.
1 – which, don』t get me wrong, is definitely interesting and a business application – it』s just not what I』m going to talk about in this post.
2 – the trick is knowing which bits. Let me know if you work out how, OK?
3 – it』s actually quite a large set of technologies, to be honest.
4 – which is patently untrue, unless the word 「they」 refers there to the conferences and gurus, in which case it』s probably correct.
5 – which may happen anyway due to my egregious mixing of metaphors.
6 – there』s a word to love. I』ve put it in to exhibit my techie credentials[7].
7 – and before you doubt them, yes, I』ve read the book, in both cut and uncut versions.
8 – within reason.
每天推薦一個 GitHub 優質開源項目和一篇精選英文科技或編程文章原文,歡迎關注開源日報。交流QQ群:202790710;電報群 https://t.me/OpeningSourceOrg