标签:headers manager form getc 文件的 listen ogre verify public
提到Netroid也许非常多人不知道这个框架,但我假设说Volley想必没有人不知道吧。
Netroid是一个基于Volley实现的Android Http库。提供运行网络请求、缓存返回结果、批量图片载入、大文件断点下载的常见Http交互功能,关于网络请求,图片载入没什么好说的,Volley已经有非常多人解析过了,这里来说一下大文件断点下载。
关于大文件断点下载,网上也有非常多实现的demo,为什么要单单说Netroid呢?由于Netroid断点续传不依赖数据库,我在网上看到过非常多的断点续传的样例,无一例外都是依赖于数据库。包含DownloadManager,大名鼎鼎的xutils,可是这两个都有一定的问题。
1.DownloadManager在三星手机上必须打开下载管理才干应用,而打开这个管理必须须要手动打开,普通情况下无伤大雅。视情况而定
2.xutils这个框架别的不知道。文件下载这块慎用
好了。进入正题,Netroid的地址:https://github.com/vince-styling/,以下简单的说一下这个框架文件下载的实现和原理,
// 1 RequestQueue queue = Netroid.newRequestQueue(getApplicationContext(), null); // 2 mDownloder = new FileDownloader(queue, 1) { @Override public FileDownloadRequest buildRequest(String storeFilePath, String url) { return new FileDownloadRequest(storeFilePath, url) { @Override public void prepare() { addHeader("Accept-Encoding", "identity"); super.prepare(); } }; } }; // 3 task.controller = mDownloder.add(mSaveDirPath + task.storeFileName, task.url, new Listener<Void>() { @Override public void onPreExecute() { task.invalidate(); } @Override public void onSuccess(Void response) { showToast(task.storeFileName + " Success!"); } @Override public void onError(NetroidError error) { NetroidLog.e(error.getMessage()); } @Override public void onFinish() { NetroidLog.e("onFinish size : " + Formatter.formatFileSize( FileDownloadActivity.this, new File(mSaveDirPath + task.storeFileName).length())); task.invalidate(); } @Override public void onProgressChange(long fileSize, long downloadedSize) { task.onProgressChange(fileSize, downloadedSize); // NetroidLog.e("---- fileSize : " + fileSize + " downloadedSize : " + downloadedSize); } });实现的话非常easy,主要分为三步就能够了
1.创建一个请求队列
2.构建一个文件下载管理器
3.将下载任务加入到队列
如今依据上面的三步来看一下它的实现原理:
第一步:创建一个请求队列:RequestQueue queue = Netroid.newRequestQueue(getApplicationContext(), null);
/** * Creates a default instance of the worker pool and calls {@link RequestQueue#start()} on it. * @param context A {@link Context} to use for creating the cache dir. * @return A started {@link RequestQueue} instance. */ public static RequestQueue newRequestQueue(Context context, DiskCache cache) { int poolSize = RequestQueue.DEFAULT_NETWORK_THREAD_POOL_SIZE; HttpStack stack; String userAgent = "netroid/0"; try { String packageName = context.getPackageName(); PackageInfo info = context.getPackageManager().getPackageInfo(packageName, 0); userAgent = packageName + "/" + info.versionCode; } catch (NameNotFoundException e) { } if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.GINGERBREAD) { stack = new HurlStack(userAgent, null); } else { // Prior to Gingerbread, HttpUrlConnection was unreliable. // See: http://android-developers.blogspot.com/2011/09/androids-http-clients.html stack = new HttpClientStack(userAgent); } //实例化BasicNetwork,主要用于运行下载请求 Network network = new BasicNetwork(stack, HTTP.UTF_8); //创建请求队列 RequestQueue queue = new RequestQueue(network, poolSize, cache); //非常重要的一步 queue.start(); return queue; }
/** * Starts the dispatchers in this queue. */ public void start() { stop(); // Make sure any currently running dispatchers are stopped. // Create the cache dispatcher and start it. mCacheDispatcher = new CacheDispatcher(mCacheQueue, mNetworkQueue, mCache, mDelivery); mCacheDispatcher.start(); // Create network dispatchers (and corresponding threads) up to the pool size. for (int i = 0; i < mDispatchers.length; i++) { //一个线程,从请求队列中获取任务并运行 NetworkDispatcher networkDispatcher = new NetworkDispatcher(mNetworkQueue, mNetwork, mCache, mDelivery); mDispatchers[i] = networkDispatcher; //Thread run() networkDispatcher.start(); } } /** * Stops the cache and network dispatchers. */ public void stop() { if (mCacheDispatcher != null) { mCacheDispatcher.quit(); } for (NetworkDispatcher mDispatcher : mDispatchers) { //Thread interrupt()线程中断 if (mDispatcher != null) mDispatcher.quit(); } }
框架中对于文件是没有缓存机制的。所以mCacheDispatcher能够不用理它。看一下NetworkDispatcher这个线程做了什么:com.duowan.mobile.netroid.NetworkDispatcher
public class NetworkDispatcher extends Thread { @Override public void run() { //设置线程优先级 Process.setThreadPriority(Process.THREAD_PRIORITY_BACKGROUND); Request request; while (true) { try { // Take a request from the queue.假设队列为空,则堵塞 request = mQueue.take(); } catch (InterruptedException e) { // We may have been interrupted because it was time to quit.唯有线程中断的时候mQuit才为true,InterruptedException为中断异常 //mQueue.take()假设队列为null,仅仅会堵塞,不会跑出异常 if (mQuit) return; continue; } try { request.addMarker("network-queue-take"); //准备运行 mDelivery.postPreExecute(request); // If the request was cancelled already, // do not perform the network request. if (request.isCanceled()) { request.finish("network-discard-cancelled"); mDelivery.postCancel(request); mDelivery.postFinish(request); continue; } // Perform the network request.最重要一步。Netroid实例化的BasicNetwork在这里运行网络请求 NetworkResponse networkResponse = mNetwork.performRequest(request); request.addMarker("network-http-complete"); // Parse the response here on the worker thread.重命名一下。没做什么 Response<?> response = request.parseNetworkResponse(networkResponse); request.addMarker("network-parse-complete"); // Write to cache if applicable. if (mCache != null && request.shouldCache() && response.cacheEntry != null) { response.cacheEntry.expireTime = request.getCacheExpireTime(); mCache.putEntry(request.getCacheKey(), response.cacheEntry); request.addMarker("network-cache-written"); } // Post the response back. request.markDelivered(); mDelivery.postResponse(request, response); } catch (NetroidError netroidError) { mDelivery.postError(request, request.parseNetworkError(netroidError)); } catch (Exception e) { NetroidLog.e(e, "Unhandled exception %s", e.toString()); mDelivery.postError(request, new NetroidError(e)); } } } }
这里最重要的一步就是NetworkResponse networkResponse = mNetwork.performRequest(request);运行网络请求,可是我们不要忘记我们的mQueue还是空的,mQueue.take()正在堵塞着呢,所以,如今还没有办法进行网络请求,因此我们须要在mQueue中填充任务,才干进行我们的网络请求。
不要忘记这里哦。由于我们还会回到这里!
第二步:创建一个文件下载管理器:new FileDownloader(queue, 1)
mDownloder = new FileDownloader(queue, 1) { @Override public FileDownloadRequest buildRequest(String storeFilePath, String url) { return new FileDownloadRequest(storeFilePath, url) { @Override public void prepare() { addHeader("Accept-Encoding", "identity"); super.prepare(); } }; } };这里有没有看着非常吓人,我起初看的时候也吓了一跳,事实上就是实例化的时候,顺手override了一下
/** The parallel task count, recommend less than 3. */ private final int mParallelTaskCount; /** The linked Task Queue. */ private final LinkedList<DownloadController> mTaskQueue; /** * Construct Downloader and init the Task Queue. * @param queue The RequestQueue for dispatching Download task. * @param parallelTaskCount * Allows parallel task count, * don't forget the value must less than ThreadPoolSize of the RequestQueue. */ public FileDownloader(RequestQueue queue, int parallelTaskCount) { if (parallelTaskCount >= queue.getThreadPoolSize()) { throw new IllegalArgumentException("parallelTaskCount[" + parallelTaskCount + "] must less than threadPoolSize[" + queue.getThreadPoolSize() + "] of the RequestQueue."); } mTaskQueue = new LinkedList<DownloadController>(); mParallelTaskCount = parallelTaskCount; mRequestQueue = queue; }这里是须要注意的一点,mParallelTaskCount并发的数量最好<3.
第三步:将下载任务加入到队列,task.controller = mDownloder.add(mSaveDirPath + task.storeFileName, task.url, new Listener<Void>():
/** * Create a new download request, this request might not run immediately because the parallel task limitation, * you can check the status by the {@link DownloadController} which you got after invoke this method. * * Note: don't perform this method twice or more with same parameters, because we didn't check for * duplicate tasks, it rely on developer done. * * Note: this method should invoke in the main thread. * * @param storeFilePath Once download successed, we'll find it by the store file path. * @param url The download url. * @param listener The event callback by status; * @return The task controller allows pause or resume or discard operation. */ public DownloadController add(String storeFilePath, String url, Listener<Void> listener) { // only fulfill requests that were initiated from the main thread.(reason for the Delivery?) //看名字就知道 throwIfNotOnMainThread(); //创建一个下载控制器 DownloadController controller = new DownloadController(storeFilePath, url, listener); synchronized (mTaskQueue) { //这可不是mQueue,这里仅仅是一个DownloadController的LinkedList集合 mTaskQueue.add(controller); } //重点来了 schedule(); return controller; }
/** * Traverse the Task Queue, count the running task then deploy more if it can be. */ private void schedule() { // make sure only one thread can manipulate the Task Queue. synchronized (mTaskQueue) { // counting ran task. int parallelTaskCount = 0; for (DownloadController controller : mTaskQueue) { //累计队列中正在下载的的任务数 if (controller.isDownloading()) parallelTaskCount++; } //当正在下载的个数大于并行任务数的时候,不在运行下载任务 /* * 这里举个样例说明一下:我们默认mParallelTaskCount=1 * 当我们加入第一个任务的时候。这个的controller.isDownloading()肯定是false * 所以parallelTaskCount >= mParallelTaskCount是不成立的,当我们再加入一个任务的时候,如今mTaskQueue.size是2了 * 且第一个isDownloading,为了保证并发数量为1,会return,说的有点乱。不知道说明确了没有 */ if (parallelTaskCount >= mParallelTaskCount) return; // try to deploy all Task if they're await. for (DownloadController controller : mTaskQueue) { //deploy(),将任务加入到队列中 if (controller.deploy() && ++parallelTaskCount == mParallelTaskCount) return; } } }
/** * For the parallel reason, only the {@link FileDownloader#schedule()} can call this method. * @return true if deploy is successed. */ private boolean deploy() { if (mStatus != STATUS_WAITING) return false; //第二步我说非常吓人那个地方 mRequest = buildRequest(mStoreFilePath, mUrl); // we create a Listener to wrapping that Listener which developer specified, // for the onFinish(), onSuccess(), onError() won't call when request was cancel reason. mRequest.setListener(new Listener<Void>() { boolean isCanceled; @Override public void onPreExecute() { mListener.onPreExecute(); } @Override public void onFinish() { // we don't inform FINISH when it was cancel. if (!isCanceled) { mStatus = STATUS_PAUSE; mListener.onFinish(); // when request was FINISH, remove the task and re-schedule Task Queue. // remove(DownloadController.this); } } @Override public void onSuccess(Void response) { // we don't inform SUCCESS when it was cancel. if (!isCanceled) { mListener.onSuccess(response); mStatus = STATUS_SUCCESS; remove(DownloadController.this); } } @Override public void onError(NetroidError error) { // we don't inform ERROR when it was cancel. if (!isCanceled) mListener.onError(error); } @Override public void onCancel() { mListener.onCancel(); isCanceled = true; } @Override public void onProgressChange(long fileSize, long downloadedSize) { mListener.onProgressChange(fileSize, downloadedSize); } }); mStatus = STATUS_DOWNLOADING; //我擦,最终把任务加到队列中了 mRequestQueue.add(mRequest); return true; }mRequestQueue.add(mRequest);任务加到队列中了,都到了这里了看一下怎么加的吧
public Request add(Request request) { // Tag the request as belonging to this queue and add it to the set of current requests. request.setRequestQueue(this); synchronized (mCurrentRequests) { mCurrentRequests.add(request); } // Process requests in the order they are added. request.setSequence(getSequenceNumber()); request.addMarker("add-to-queue"); // If the request is uncacheable or forceUpdate, skip the cache queue and go straight to the network. if (request.isForceUpdate() || !request.shouldCache()) { mDelivery.postNetworking(request); mNetworkQueue.add(request); return request; } }
request.shouldCache()有兴趣的能够自己去看一下。这里说明了文件下载没有缓存机制,这里就不多说了,由于假设你还没有忘记的话。mQueue.take()还在堵塞着呢。好了让我们回到第一步,运行网络请求
NetworkResponse networkResponse = mNetwork.performRequest(request);
@Override public NetworkResponse performRequest(Request<?> request) throws NetroidError { // Determine if request had non-http perform. NetworkResponse networkResponse = request.perform(); if (networkResponse != null) return networkResponse; long requestStart = SystemClock.elapsedRealtime(); while (true) { // If the request was cancelled already, // do not perform the network request. if (request.isCanceled()) { request.finish("perform-discard-cancelled"); mDelivery.postCancel(request); throw new NetworkError(networkResponse); } HttpResponse httpResponse = null; byte[] responseContents = null; try { // prepare to perform this request, normally is reset the request headers. request.prepare(); httpResponse = mHttpStack.performRequest(request); StatusLine statusLine = httpResponse.getStatusLine(); int statusCode = statusLine.getStatusCode(); responseContents = request.handleResponse(httpResponse, mDelivery); if (statusCode < 200 || statusCode > 299) throw new IOException(); // if the request is slow, log it. long requestLifetime = SystemClock.elapsedRealtime() - requestStart; logSlowRequests(requestLifetime, request, responseContents, statusLine); return new NetworkResponse(statusCode, responseContents, parseCharset(httpResponse)); } catch (SocketTimeoutException e) { attemptRetryOnException("socket", request, new TimeoutError()); } catch (ConnectTimeoutException e) { attemptRetryOnException("connection", request, new TimeoutError()); } catch (MalformedURLException e) { throw new RuntimeException("Bad URL " + request.getUrl(), e); } catch (IOException e) { if (httpResponse == null) throw new NoConnectionError(e); int statusCode = httpResponse.getStatusLine().getStatusCode(); NetroidLog.e("Unexpected response code %d for %s", statusCode, request.getUrl()); if (responseContents != null) { networkResponse = new NetworkResponse(statusCode, responseContents, parseCharset(httpResponse)); if (statusCode == HttpStatus.SC_UNAUTHORIZED || statusCode == HttpStatus.SC_FORBIDDEN) { attemptRetryOnException("auth", request, new AuthFailureError(networkResponse)); } else { // TODO: Only throw ServerError for 5xx status codes. throw new ServerError(networkResponse); } } else { throw new NetworkError(networkResponse); } } } }
responseContents = request.handleResponse(httpResponse, mDelivery);。写文件,断点续传的原理
/** * In this method, we got the Content-Length, with the TemporaryFile length, * we can calculate the actually size of the whole file, if TemporaryFile not exists, * we'll take the store file length then compare to actually size, and if equals, * we consider this download was already done. * We used {@link RandomAccessFile} to continue download, when download success, * the TemporaryFile will be rename to StoreFile. */ @Override public byte[] handleResponse(HttpResponse response, Delivery delivery) throws IOException, ServerError { // Content-Length might be negative when use HttpURLConnection because it default header Accept-Encoding is gzip, // we can force set the Accept-Encoding as identity in prepare() method to slove this problem but also disable gzip response. HttpEntity entity = response.getEntity(); //获取文件的总大小 long fileSize = entity.getContentLength(); if (fileSize <= 0) { NetroidLog.d("Response doesn't present Content-Length!"); } long downloadedSize = mTemporaryFile.length(); /* * 是否支持断点续传 * * client每次提交下载请求时。服务端都要加入这两个响应头,以保证client和服务端将此下载识别为能够断点续传的下载: * Accept-Ranges:告知下载client这是一个能够恢复续传的下载,存放本次下载的開始字节位置、文件的字节大小; * ETag:保存文件的唯一标识(我在用的文件名称+文件最后改动时间,以便续传请求时对文件进行验证)。 * Last-Modified:可选响应头,存放服务端文件的最后改动时间,用于验证 */ boolean isSupportRange = HttpUtils.isSupportRange(response); if (isSupportRange) { fileSize += downloadedSize; // Verify the Content-Range Header, to ensure temporary file is part of the whole file. // Sometime, temporary file length add response content-length might greater than actual file length, // in this situation, we consider the temporary file is invalid, then throw an exception. String realRangeValue = HttpUtils.getHeader(response, "Content-Range"); // response Content-Range may be null when "Range=bytes=0-" if (!TextUtils.isEmpty(realRangeValue)) { String assumeRangeValue = "bytes " + downloadedSize + "-" + (fileSize - 1); if (TextUtils.indexOf(realRangeValue, assumeRangeValue) == -1) { throw new IllegalStateException( "The Content-Range Header is invalid Assume[" + assumeRangeValue + "] vs Real[" + realRangeValue + "], " + "please remove the temporary file [" + mTemporaryFile + "]."); } } } // Compare the store file size(after download successes have) to server-side Content-Length. // temporary file will rename to store file after download success, so we compare the // Content-Length to ensure this request already download or not. if (fileSize > 0 && mStoreFile.length() == fileSize) { // Rename the store file to temporary file, mock the download success. ^_^ mStoreFile.renameTo(mTemporaryFile); // Deliver download progress. delivery.postDownloadProgress(this, fileSize, fileSize); return null; } //之所以能够实现断点续传的原因所在 RandomAccessFile tmpFileRaf = new RandomAccessFile(mTemporaryFile, "rw"); // If server-side support range download, we seek to last point of the temporary file. if (isSupportRange) { //移动文件读写指针位置 tmpFileRaf.seek(downloadedSize); } else { // If not, truncate the temporary file then start download from beginning. tmpFileRaf.setLength(0); downloadedSize = 0; } try { InputStream in = entity.getContent(); // Determine the response gzip encoding, support for HttpClientStack download. if (HttpUtils.isGzipContent(response) && !(in instanceof GZIPInputStream)) { in = new GZIPInputStream(in); } byte[] buffer = new byte[6 * 1024]; // 6K buffer int offset; while ((offset = in.read(buffer)) != -1) { //写文件 tmpFileRaf.write(buffer, 0, offset); downloadedSize += offset; long currTime = SystemClock.uptimeMillis(); //控制下载进度的速度 if (currTime - lastUpdateTime >= DEFAULT_TIME) { lastUpdateTime = currTime; delivery.postDownloadProgress(this, fileSize, downloadedSize); } if (isCanceled()) { delivery.postCancel(this); break; } } } finally { try { // Close the InputStream and release the resources by "consuming the content". if (entity != null) entity.consumeContent(); } catch (Exception e) { // This can happen if there was an exception above that left the entity in // an invalid state. NetroidLog.v("Error occured when calling consumingContent"); } tmpFileRaf.close(); } return null; }实现断点续传主要靠的RandomAccessFile,你假设对c语言不陌生的话tmpFileRaf.seek(downloadedSize)和int fseek(FILE *stream, long offset, int fromwhere);是不是有点眼熟,仅仅与RandomAccessFile就不说了。
好了,Netroid的原理基本上就是这些了,讲一下我用的时候遇到的两个问题:
1.下载进度的速度太快。你假设用notifition来显示,会出现ANR,所以我们要控制一下它的速度,详细方法在上面
//控制下载进度的速度 if (currTime - lastUpdateTime >= DEFAULT_TIME) { lastUpdateTime = currTime; delivery.postDownloadProgress(this, fileSize, downloadedSize); }
@Override public void onFinish() { // we don't inform FINISH when it was cancel. if (!isCanceled) { mStatus = STATUS_PAUSE; mListener.onFinish(); // when request was FINISH, remove the task and re-schedule Task Queue. // remove(DownloadController.this); } } @Override public void onSuccess(Void response) { // we don't inform SUCCESS when it was cancel. if (!isCanceled) { mListener.onSuccess(response); mStatus = STATUS_SUCCESS; remove(DownloadController.this); } }
把onFinish的status改成STATUS_PAUSE。并去掉remove(DownloadController.this);。在onSuccess中再将status改动为STATUS_SUCCESS,并remove,当然这个办法治标不治本。假设有谁知道请告之,谢谢。
Android Netroid解析之——断点续传下载及问题修正
标签:headers manager form getc 文件的 listen ogre verify public
原文地址:http://www.cnblogs.com/gccbuaa/p/7027091.html