鉴黄是我的事业,我会一直做到死为止!——唐马儒
大家都知道,我们的网站内容安全是很重要的
比如今天张三发布一个李四的照片,然后贴上一夜八百量大从优
如果我们的内容未经审核就发布到网站上,那么网站的安全就得不到保障,这对祖国未来的花朵影响是很大的
但如果我们每一条都人工审核的话,会占用大量人力资源,造就更多的就业岗位 造成更多的人力成本
为了解决这个问题,阿里云带着我们的产品来啦!它就是《阿里云内容安全》,简称IAcs
那么如何使用呢?首先,打开符文页,阿不,打开阿里云官网。。。
- 访问阿里云官网,注册阿里云账号。如果已有阿里云账号,请跳过此步骤。
- 访问内容安全产品试用页面,单击立即开通,正式开通服务。
- 访问AccessKey管理页面,管理您的AccessKey ID和AccessKey Secret。
复制依赖
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
| <dependency> <groupId>com.aliyun</groupId> <artifactId>aliyun-java-sdk-core</artifactId> <version>4.1.1</version> </dependency> <dependency> <groupId>com.aliyun</groupId> <artifactId>aliyun-java-sdk-green</artifactId> <version>3.6.1</version> </dependency> <dependency> <groupId>com.alibaba</groupId> <artifactId>fastjson</artifactId> <version>1.2.51</version> </dependency> <dependency> <groupId>com.aliyun.oss</groupId> <artifactId>aliyun-sdk-oss</artifactId> <version>2.8.3</version> </dependency> <dependency> <groupId>commons-io</groupId> <artifactId>commons-io</artifactId> <version>2.4</version> </dependency> <dependency> <groupId>commons-codec</groupId> <artifactId>commons-codec</artifactId> <version>1.10</version> </dependency>
|
那么一开始是图片审核(传URL进行检测)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116
| import com.alibaba.fastjson.JSON; import com.alibaba.fastjson.JSONArray; import com.alibaba.fastjson.JSONObject; import com.aliyuncs.DefaultAcsClient; import com.aliyuncs.IAcsClient; import com.aliyuncs.green.model.v20180509.ImageSyncScanRequest; import com.aliyuncs.http.FormatType; import com.aliyuncs.http.HttpResponse; import com.aliyuncs.http.MethodType; import com.aliyuncs.http.ProtocolType; import com.aliyuncs.profile.DefaultProfile; import com.aliyuncs.profile.IClientProfile;
import java.util.*;
public class Main {
public static void main(String[] args) throws Exception { IClientProfile profile = DefaultProfile .getProfile("cn-shanghai", "请填写您的accessKeyId", "请填写您的accessKeySecret"); DefaultProfile .addEndpoint("cn-shanghai", "cn-shanghai", "Green", "green.cn-shanghai.aliyuncs.com"); IAcsClient client = new DefaultAcsClient(profile);
ImageSyncScanRequest imageSyncScanRequest = new ImageSyncScanRequest(); imageSyncScanRequest.setAcceptFormat(FormatType.JSON); imageSyncScanRequest.setMethod(MethodType.POST); imageSyncScanRequest.setEncoding("utf-8"); imageSyncScanRequest.setProtocol(ProtocolType.HTTP);
JSONObject httpBody = new JSONObject();
httpBody.put("scenes", Arrays.asList("porn"));
JSONObject task = new JSONObject(); task.put("dataId", UUID.randomUUID().toString());
task.put("url", "http://xxx.test.jpg"); task.put("time", new Date()); httpBody.put("tasks", Arrays.asList(task));
imageSyncScanRequest.setHttpContent(org.apache.commons.codec.binary.StringUtils.getBytesUtf8(httpBody.toJSONString()), "UTF-8", FormatType.JSON);
imageSyncScanRequest.setConnectTimeout(3000); imageSyncScanRequest.setReadTimeout(10000); HttpResponse httpResponse = null; try { httpResponse = client.doAction(imageSyncScanRequest); } catch (Exception e) { e.printStackTrace(); }
if (httpResponse != null && httpResponse.isSuccess()) { JSONObject scrResponse = JSON.parseObject(org.apache.commons.codec.binary.StringUtils.newStringUtf8(httpResponse.getHttpContent())); System.out.println(JSON.toJSONString(scrResponse, true)); int requestCode = scrResponse.getIntValue("code"); JSONArray taskResults = scrResponse.getJSONArray("data"); if (200 == requestCode) { for (Object taskResult : taskResults) { int taskCode = ((JSONObject) taskResult).getIntValue("code"); JSONArray sceneResults = ((JSONObject) taskResult).getJSONArray("results"); if (200 == taskCode) { for (Object sceneResult : sceneResults) { String scene = ((JSONObject) sceneResult).getString("scene"); String suggestion = ((JSONObject) sceneResult).getString("suggestion"); System.out.println("scene = [" + scene + "]"); System.out.println("suggestion = [" + suggestion + "]"); } } else { System.out.println("task process fail. task response:" + JSON.toJSONString(taskResult)); } } } else {
System.out.println("the whole image scan request failed. response:" + JSON.toJSONString(scrResponse)); } } }
}
|
如果配置没问题我们会得到这样一个结果
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
| { "msg":"OK", "code":200, "data":[ { "msg":"OK", "code":200, "dataId":"8918dff8-701f-49f2-896b-cadc4829d149", "extras":{}, "results":[ { "rate":99.71, "suggestion":"pass", "label":"normal", "scene":"porn" } ], "taskId":"img3C7DXwtRX$m5$em9b8HkPe-1sZ50L", "url":"http://p18.qhimg.com/bdm/960_593_0/t0177a850faae7b42f4.jpg" } ], "requestId":"88780A46-586A-4555-B562-F6E18F81BFB1" }
|
状态码表示含义、建议内容、检测范围放在最下面
然后是视频检测(传URL进行检测)
因为视频是做的截帧进行检测,所以需要异步处理,我们这里只是提交检测单
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71
| import com.alibaba.fastjson.JSON; import com.alibaba.fastjson.JSONObject; import com.aliyuncs.DefaultAcsClient; import com.aliyuncs.IAcsClient; import com.aliyuncs.exceptions.ClientException; import com.aliyuncs.exceptions.ServerException; import com.aliyuncs.green.model.v20180509.VideoAsyncScanRequest; import com.aliyuncs.http.FormatType; import com.aliyuncs.http.HttpResponse; import com.aliyuncs.profile.DefaultProfile; import com.aliyuncs.profile.IClientProfile;
import java.util.*;
public class Main {
public static void main(String[] args) throws Exception { IClientProfile profile = DefaultProfile.getProfile("cn-shanghai", "请填写您的accessKeyId", "请填写您的accessKeySecret"); DefaultProfile.addEndpoint("cn-shanghai", "cn-shanghai", "Green", "green.cn-shanghai.aliyuncs.com"); IAcsClient client = new DefaultAcsClient(profile);
VideoAsyncScanRequest videoAsyncScanRequest = new VideoAsyncScanRequest(); videoAsyncScanRequest.setAcceptFormat(FormatType.JSON); videoAsyncScanRequest.setMethod(com.aliyuncs.http.MethodType.POST);
List<Map<String, Object>> tasks = new ArrayList<Map<String, Object>>(); Map<String, Object> task = new LinkedHashMap<String, Object>(); task.put("dataId", UUID.randomUUID().toString()); task.put("url", "请填写公网可访问的视频HTTP/HTTPS URL地址");
tasks.add(task);
JSONObject data = new JSONObject(); data.put("scenes", Arrays.asList("porn", "terrorism")); data.put("tasks", tasks); data.put("callback", "http://xxx.xxx.xx/xxx.json"); data.put("seed", "yourPersonalSeed");
videoAsyncScanRequest.setHttpContent(data.toJSONString().getBytes("UTF-8"), "UTF-8", FormatType.JSON);
videoAsyncScanRequest.setConnectTimeout(3000); videoAsyncScanRequest.setReadTimeout(6000); try { HttpResponse httpResponse = client.doAction(videoAsyncScanRequest);
if(httpResponse.isSuccess()){ JSONObject jsonObject = JSON.parseObject(new String(httpResponse.getHttpContent(), "UTF-8")); System.out.println(JSON.toJSONString(jsonObject, true)); }else{ System.out.println("response not success. status:" + httpResponse.getStatus()); } } catch (ServerException e) { e.printStackTrace(); } catch (ClientException e) { e.printStackTrace(); } } }
|
得到的返回值为
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
| { "msg":"OK", "code":200, "data":[ { "msg":"OK", "code":200, "dataId":"41f9476b-087a-42bd-b5a2-37eb174bf4ff", "taskId":"vi44NAEgkcXN64h3sDNZXRsh-1sZbev", "url":"/imgs/oss/QQ%E8%A7%86%E9%A2%91_0670c7201774824ee71c7e2fe4c0dc4d1586480746.mp4" }, { "msg":"OK", "code":200, "dataId":"e57e1583-602c-48b8-ba67-3ec28cd9e651", "taskId":"viUB1uskhXAW5Fay2a@ci5V-1sZbev", "url":"/imgs/oss/QQ%E8%A7%86%E9%A2%91_b5ca71298bf92a778ef1ce94259dcdce1580718352.mp4" } ], "requestId":"EB12BC2A-456E-4554-9705-EEF951A4142C" }
|
然后我们可以拿着taskId
去查询进度taskId
最大保存4小时
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57
| import com.alibaba.fastjson.JSON; import com.alibaba.fastjson.JSONObject; import com.aliyuncs.DefaultAcsClient; import com.aliyuncs.IAcsClient; import com.aliyuncs.exceptions.ClientException; import com.aliyuncs.exceptions.ServerException; import com.aliyuncs.green.model.v20180509.VideoAsyncScanResultsRequest; import com.aliyuncs.http.FormatType; import com.aliyuncs.http.HttpResponse; import com.aliyuncs.profile.DefaultProfile; import com.aliyuncs.profile.IClientProfile;
import java.util.*;
public class Main {
public static void main(String[] args) throws Exception { IClientProfile profile = DefaultProfile.getProfile("cn-shanghai", "请填写您的accessKeyId", "请填写您的accessKeySecret"); DefaultProfile.addEndpoint("cn-shanghai", "cn-shanghai", "Green", "green.cn-shanghai.aliyuncs.com"); IAcsClient client = new DefaultAcsClient(profile);
VideoAsyncScanResultsRequest videoAsyncScanResultsRequest = new VideoAsyncScanResultsRequest(); videoAsyncScanResultsRequest.setAcceptFormat(FormatType.JSON);
List<String> taskList = new ArrayList<String>(); taskList.add("viUB1uskhXAW5Fay2a@ci5V-1sZbev");
videoAsyncScanResultsRequest.setHttpContent(JSON.toJSONString(taskList).getBytes("UTF-8"), "UTF-8", FormatType.JSON);
videoAsyncScanResultsRequest.setConnectTimeout(3000); videoAsyncScanResultsRequest.setReadTimeout(6000); try { HttpResponse httpResponse = client.doAction(videoAsyncScanResultsRequest); if(httpResponse.isSuccess()){ JSONObject jsonObject = JSON .parseObject(new String(httpResponse.getHttpContent(), "UTF-8")); System.out.println(JSON.toJSONString(jsonObject, true)); }else{ System.out.println("response not success. status:" + httpResponse.getStatus()); } } catch (ServerException e) { e.printStackTrace(); } catch (ClientException e) { e.printStackTrace(); } } }
|
未完成时,返回值是这样的
1 2 3 4 5 6 7 8 9 10 11 12 13 14
| { "msg":"OK", "code":200, "data":[ { "msg":"PROCESSING - scanning", "code":280, "dataId":"a4705a67-7c73-40f4-948d-5b197d14f0aa", "taskId":"vi4O50Nxv46L7M878A6jFo1-1sZ8FL", "url":"/imgs/oss/QQ%E8%A7%86%E9%A2%91_b5ca71298bf92a778ef1ce94259dcdce1580718352.mp4" } ], "requestId":"3E8EC910-CE4C-4931-B06D-CC0DEB936E09" }
|
完成后是这样
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
| { "msg":"OK", "code":200, "data":[ { "msg":"OK", "code":200, "dataId":"a4705a67-7c73-40f4-948d-5b197d14f0aa", "results":[ { "rate":99.9, "suggestion":"pass", "label":"normal", "scene":"porn" }, { "rate":99.9, "suggestion":"pass", "label":"normal", "scene":"terrorism" } ], "taskId":"vi4O50Nxv46L7M878A6jFo1-1sZ8FL", "url":"/imgs/oss/QQ%E8%A7%86%E9%A2%91_b5ca71298bf92a778ef1ce94259dcdce1580718352.mp4" } ], "requestId":"1EF03EF2-C59A-40C1-A45D-BB26C2FA85FD" }
|
如果我们配置了异步回调处理,那么阿里云在检测完成后会给我们发送一个回调,回调的参数是这样的
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
| { "scenes": [ "porn" ], "audioScenes": [ "antispam" ], "tasks": [ { "dataId": "videoId xxx", "url": "http://www.foo.bak/a.mp4", "interval": 1, "maxFrames": 200 } ] }
|
那么接下来是我们的文本反垃圾
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87
| import com.alibaba.fastjson.JSON; import com.alibaba.fastjson.JSONArray; import com.alibaba.fastjson.JSONObject; import com.aliyun.oss.ClientException; import com.aliyuncs.DefaultAcsClient; import com.aliyuncs.IAcsClient; import com.aliyuncs.exceptions.ServerException; import com.aliyuncs.green.model.v20180509.TextScanRequest; import com.aliyuncs.http.FormatType; import com.aliyuncs.http.HttpResponse; import com.aliyuncs.profile.DefaultProfile; import com.aliyuncs.profile.IClientProfile;
import java.util.*;
public class Main {
public static void main(String[] args) throws Exception { IClientProfile profile = DefaultProfile .getProfile("cn-shanghai", "您的AccessKeyId", "您的AccessKeySecret"); DefaultProfile .addEndpoint("cn-shanghai", "cn-shanghai", "Green", "green.cn-shanghai.aliyuncs.com"); IAcsClient client = new DefaultAcsClient(profile); TextScanRequest textScanRequest = new TextScanRequest(); textScanRequest.setAcceptFormat(FormatType.JSON); textScanRequest.setHttpContentType(FormatType.JSON); textScanRequest.setMethod(com.aliyuncs.http.MethodType.POST); textScanRequest.setEncoding("UTF-8"); textScanRequest.setRegionId("cn-shanghai"); List<Map<String, Object>> tasks = new ArrayList<Map<String, Object>>(); Map<String, Object> task1 = new LinkedHashMap<String, Object>(); task1.put("dataId", UUID.randomUUID().toString());
task1.put("content", "test content"); tasks.add(task1); JSONObject data = new JSONObject();
data.put("scenes", Arrays.asList("antispam")); data.put("tasks", tasks); System.out.println(JSON.toJSONString(data, true)); textScanRequest.setHttpContent(data.toJSONString().getBytes("UTF-8"), "UTF-8", FormatType.JSON); textScanRequest.setConnectTimeout(3000); textScanRequest.setReadTimeout(6000); try { HttpResponse httpResponse = client.doAction(textScanRequest); if(httpResponse.isSuccess()){ JSONObject scrResponse = JSON.parseObject(new String(httpResponse.getHttpContent(), "UTF-8")); System.out.println(JSON.toJSONString(scrResponse, true)); if (200 == scrResponse.getInteger("code")) { JSONArray taskResults = scrResponse.getJSONArray("data"); for (Object taskResult : taskResults) { if(200 == ((JSONObject)taskResult).getInteger("code")){ JSONArray sceneResults = ((JSONObject)taskResult).getJSONArray("results"); for (Object sceneResult : sceneResults) { String scene = ((JSONObject)sceneResult).getString("scene"); String suggestion = ((JSONObject)sceneResult).getString("suggestion"); System.out.println("args = [" + scene + "]"); System.out.println("args = [" + suggestion + "]"); } }else{ System.out.println("task process fail:" + ((JSONObject)taskResult).getInteger("code")); } } } else { System.out.println("detect not success. code:" + scrResponse.getInteger("code")); } }else{ System.out.println("response not success. status:" + httpResponse.getStatus()); } } catch (ServerException e) { e.printStackTrace(); } catch (ClientException e) { e.printStackTrace(); } catch (Exception e) { e.printStackTrace(); } }
}
|
返回值
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
| { "msg":"OK", "code":200, "data":[ { "msg":"OK", "code":200, "dataId":"d826ad34-8150-4ea3-8e2a-93e3841f681b", "results":[ { "rate":99.91, "suggestion":"pass", "label":"normal", "scene":"antispam" } ], "content":"搞快点", "taskId":"txt28tO0JTEXEl5zkeClNwR@n-1sZbjU" } ], "requestId":"91DF1A2C-192F-4C45-BE9C-B092DBDA8A01" }
|
最终我们需要的返回值是两个,label
表示命中场景,suggestion
表示检测结果
场景 |
描述 |
scene |
label |
图片智能鉴黄 |
识别图片中的色情内容。 |
porn |
normal:正常图片,无色情内容sexy:性感图片porn:色情图片 |
图片暴恐涉政识别 |
识别图片中的暴恐涉政内容。 |
terrorism |
normal:正常图片bloody:血腥explosion:爆炸烟光outfit:特殊装束logo:特殊标识weapon:武器politics:涉政violence : 打斗crowd:聚众parade:游行carcrash:车祸现场flag:旗帜location:地标others:其他 |
图文违规识别 |
识别图片中的广告以及文字违规信息。 |
ad |
normal:正常图片politics:文字含涉政内容porn:文字含涉黄内容abuse:文字含辱骂内容terrorism:文字含暴恐内容contraband:文字含违禁内容spam:文字含其他垃圾内容npx:牛皮藓广告qrcode:包含二维码programCode:包含小程序码ad:其他广告说明 默认只会返回normal或者ad,如需其他label,请通过工单联系我们进行配置。 |
图片二维码识别 |
识别图片中的二维码。 |
qrcode |
normal:正常图片qrcode:含二维码的图片programCode:含小程序码的图片说明 默认不识别小程序码。如果需要识别,请通过工单联系我们调整。 |
图片不良场景识别 |
识别图片中出现的黑屏、黑边、昏暗画面、画中画、抽烟、打架等不良场景。 |
live |
normal:正常图片meaningless:无意义图片PIP:画中画smoking:吸烟drivelive:车内直播 |
图片logo识别 |
识别图片中的logo信息,例如台标,商标等。 |
logo |
normal:正常图片TV:带有管控logo的图片trademark:商标 |
然后是状态码
OK |
200 |
请求成功。 |
PROCESSING |
280 |
任务正在执行中,建议您等待一段时间(例如5s)后再查询结果。 |
BAD_REQUEST |
400 |
请求有误,通常由于请求参数不正确导致,请仔细检查请求参数。 |
NOT_ALLOWED |
401 |
请求失败,通常是由于使用了不安全的图片、视频、语音链接地址。 |
FORBIDDEN |
403 |
请求访问失败,通常由于您的图片、视频、语音链接无法访问导致,请确认公网是否可访问,并且无防盗链策略。 |
NOT_FOUND |
404 |
待检测内容未找到,通常是由于您的图片、视频、语音内容无法下载导致,请确认内容可通过公网访问到。 |
DOWNLOAD_FAILED |
480 |
下载失败,请确认待检测内容的大小、分辨率(如果有)在API的限制范围内。 |
GENERAL_ERROR |
500 |
一般是服务端临时出错。建议重试,若持续返回该错误码,请通过工单联系我们。 |
DB_FAILED |
580 |
数据库操作失败。建议重试,若持续返回该错误码,请通过工单联系我们。 |
TIMEOUT |
581 |
超时。建议重试,若持续返回该错误码,请通过工单联系我们。 |
CACHE_FAILED |
585 |
缓存出错。建议重试,若持续返回该错误码,请通过工单联系我们。 |
ALGO_FAILED |
586 |
算法出错。请通过工单联系我们。 |
MQ_FAILED |
587 |
中间件出错。请通过工单联系我们。 |
EXCEED_QUOTA |
588 |
请求频率超出配额。默认配额:图片检测50张/秒,视频检测20路/秒,语音检测20路/秒,文本检测100条/秒。如果需要调整配额,请通过工单联系我们。说明 关于价格说明,请参见内容安全产品定价。 |
TOO_LARGE |
589 |
待检测内容过大,请确保检测的内容在API的限制范围内。建议重试,若持续返回该错误码,请通过工单联系我们。 |
BAD_FORMAT |
590 |
待检测内容格式错误,请确保检测的内容在API的限制范围内。 |
CONNECTION_POOL_FULL |
591 |
连接池满。请通过工单联系我们。 |
DOWNLOAD_TIMEOUT |
592 |
下载超时,下载时间限制为3s,请确保检测的内容大小在API的限制范围内。 |
EXPIRED |
594 |
任务过期,如taskId过期。 |
CATCH_FRAME_FAILED |
595 |
截帧失败,请通过工单联系我们。 |
PERMISSION_DENY |
596 |
账号未授权、账号欠费、账号未开通、账号被禁等原因,具体可以参考返回的msg。 |
我自己写了个demo
放进昨天创建的springboot
项目里了
git
地址