使用字节的 AWS Rekognition JavaScript 开发工具包

Posted 2023-03-07

【中文标题】使用字节的 AWS Rekognition JavaScript 开发工具包

【英文标题】：AWS Rekognition JavaScript SDK using Bytes

【发布时间】：2017-09-15 14:59:21

【问题描述】：

AWS Rekognition javascript API 声明对于rekognition.compareFaces(params,...) 方法，SourceImage 和TargetImage 可以采用Bytes 或S3Object。我想使用Bytes 可以是

“字节数——（缓冲区、类型化数组、Blob、字符串）”

图像字节的 Blob，最大为 5 MB。

当我传递图像的Base64 编码字符串时，JS SDK 再次重新编码（即双重编码）。因此服务器响应错误提示

"__type":"InvalidImageFormatException","Message":"无效图像 编码”

有没有人设法使用 base64 编码图像（不是S3Object）使用compareFaces JS SDK API？或任何使用 Bytes 参数的 JavaScript 示例都会有所帮助。

【问题讨论】：

我也在寻找如何做到这一点的答案，因为我一直无法做到这一点。我尝试使用 Base64 编码的字符串，结果与上述相同，也尝试将其转换为 Uint8array、Blob 和 ArrayBuffer，但仍然没有成功。有没有人有一个有效的例子？我的源数据位于 html5 Canvas 对象中。

相关并已解决：***.com/questions/43599556/…

【参考方案1】：

来自 AWS Rekognition JS SDK Invalid image encoding error 线程的技术奏效了。

将base64图像编码转换为ArrayBuffer：

function getBinary(base64Image) 
  var binaryImg = atob(base64Image);
  var length = binaryImg.length;
  var ab = new ArrayBuffer(length);
  var ua = new Uint8Array(ab);
  for (var i = 0; i < length; i++) 
    ua[i] = binaryImg.charCodeAt(i);
  

  return ab;

作为Bytes参数传入rekognition：

var data = canvas.toDataURL('image/jpeg');
var base64Image = data.replace(/^data:image\/(png|jpeg|jpg);base64,/, '');
var imageBytes = getBinary(base64Image);

var rekognitionRequest = 
  CollectionId: collectionId,
  Image: 
    Bytes: imageBytes
  
;

【参考方案2】：

我在 Node 中读取文件作为字节数组缓冲区并将其发送到 Rekognition 时遇到了类似的问题。

我通过读取 base64 表示来解决它，然后将其转换为如下所示的缓冲区：

const aws = require('aws-sdk');
const fs = require('fs');

var rekognition = new aws.Rekognition(
  apiVersion: '2016-06-27'
);

// pull base64 representation of image from file system (or somewhere else)
fs.readFile('./test.jpg', 'base64', (err, data) => 

  // create a new buffer out of the string passed to us by fs.readFile()
  const buffer = Buffer.from(data, 'base64');

  // now that we have things in the right type, send it to rekognition
  rekognition.detectLabels(
      Image: 
        Bytes: buffer
      
    ).promise()
    .then((res) => 

      // print out the labels that rekognition sent back
      console.log(res);

    );
    
);

这也可能与人们收到：Expected params.Image.Bytes to be a string, Buffer, Stream, Blob, or typed array object 消息有关。

【参考方案3】：

似乎将字符串转换为缓冲区的工作方式更加一致，但很难找到有关它的文档。

对于 Node，您可以使用它来转换字符串中的参数（确保您将数据删除......直到“，”。

var params = 
    CollectionId: collectionId,
    Image: 
        Bytes: new Buffer(imageBytes, 'base64')
    
;

在普通 JS 中，您希望可以使用 atob 进行转换并使用以下代码传递数组缓冲区：

function getBinary(base64Image) 

    var binaryImg = Buffer.from(base64Image, 'base64').toString();
    var length = binaryImg.length;
    var ab = new ArrayBuffer(length);
    var ua = new Uint8Array(ab);
    for (var i = 0; i < length; i++) 
        ua[i] = binaryImg.charCodeAt(i);
    

    return ab;

【参考方案4】：

我遇到了同样的问题，我会告诉你我是如何解决的。

Amazon Rekognition 支持的图片类型为 JPEG and PNG

这意味着如果你输入的图像文件使用其他格式如 webp 编码，你总是会得到同样的错误。

将不使用 jpeg 或 png 编码的图像格式更改为 jpeg 后，我可以解决这个问题。

希望你能解决这个问题！

【参考方案5】：

根据@Sean 提供的答案，我想添加另一种方法来使用axios 从URL 请求中获取字节并传递给rekognition.detectLabels()——或Amazon Rekognition 的其他各种检测方法。

我继续为 fs.readFile 创建一个承诺，它应该与 async/await 结构一起使用。然后一些regex 来确定您是否需要获取 URL 或读取文件作为后备。

我还为标签添加了Gray 和World Of Warcraft 的检查。不确定是否有其他人经历过这种情况，但lorempixel 似乎每隔一段时间就会抛出这些标签。我以前也见过它们显示在全黑图像上。

/* jshint esversion: 6, node:true, devel: true, undef: true, unused: true */

const AWS = require('aws-sdk'),
  axios = require('axios'),
  fs = require('fs'),
  path = require('path');

// Get credentials from environmental variables.
const S3_ACCESS_KEY, S3_SECRET_ACCESS_KEY, S3_REGION = process.env;

// Set AWS credentials.
AWS.config.update(
  accessKeyId: S3_ACCESS_KEY,
  secretAccessKey: S3_SECRET_ACCESS_KEY,
  region: S3_REGION
);

const rekognition = new AWS.Rekognition(
  apiVersion: '2016-06-27'
);

startDetection();

// ----------------

async function startDetection() 
    let found = ;

    found = await detectFromPath(path.join(__dirname, 'test.jpg'));
    console.log(found);

    found = await detectFromPath('https://upload.wikimedia.org/wikipedia/commons/9/96/Bill_Nye%2C_Barack_Obama_and_Neil_deGrasse_Tyson_selfie_2014.jpg');
    console.log(found);

    found = await detectFromPath('http://placekitten.com/g/200/300');
    console.log(found);

    found = await detectFromPath('https://loremflickr.com/g/320/240/text');
    console.log(found);

    found = await detectFromPath('http://lorempixel.com/400/200/sports/');
    console.log(found);

    // Sometimes 'Grey' and 'World Of Warcraft' are the only labels...
    if (found && found.labels.length === 2 && found.labels.some(i => i.Name === 'Gray') && found.labels.some(i => i.Name === 'World Of Warcraft')) 
        console.log('⚠️', '\n\tMaybe this is a bad image...`Gray` and `World Of Warcraft`???\n');
    


// ----------------

/**
 * @param string path URL or filepath on your local machine.
 * @param Number maxLabels 
 * @param Number minConfidence 
 * @param array attributes 
 */
async function detectFromPath(path, maxLabels, minConfidence, attributes) 

    // Convert path to base64 Buffer data.
    const bytes = (/^https?:\/\//gm.exec(path)) ?
        await getBase64BufferFromURL(path) :
        await getBase64BufferFromFile(path);

    // Invalid data.
    if (!bytes)
        return 
            path,
            faces: [],
            labels: [],
            text: [],
            celebs: [],
            moderation: []
        ;

    // Pass buffer to rekognition methods.
    let labels = await detectLabelsFromBytes(bytes, maxLabels, minConfidence),
        text = await detectTextFromBytes(bytes),
        faces = await detectFacesFromBytes(bytes, attributes),
        celebs = await recognizeCelebritiesFromBytes(bytes),
        moderation = await detectModerationLabelsFromBytes(bytes, minConfidence);

    // Filter out specific values.
    labels = labels && labels.Labels ? labels.Labels : [];
    faces = faces && faces.FaceDetails ? faces.FaceDetails : [];
    text = text && text.TextDetections ? text.TextDetections.map(i => i.DetectedText) : [];

    celebs = celebs && celebs.CelebrityFaces ? celebs.CelebrityFaces.map(i => (
        Name: i.Name,
        MatchConfidence: i.MatchConfidence
    )) : [];

    moderation = moderation && moderation.ModerationLabels ? moderation.ModerationLabels.map(i => (
        Name: i.Name,
        Confidence: i.Confidence
    )) : [];

    // Return collection.
    return 
        path,
        faces,
        labels,
        text,
        celebs,
        moderation
    ;


/**
 * https://nodejs.org/api/fs.html#fs_fs_readfile_path_options_callback
 * 
 * @param string filename 
 */
function getBase64BufferFromFile(filename) 
    return (new Promise(function(resolve, reject) 
        fs.readFile(filename, 'base64', (err, data) => 
            if (err) return reject(err);
            resolve(new Buffer(data, 'base64'));
        );
    )).catch(error => 
        console.log('[ERROR]', error);
    );


/**
 * https://github.com/axios/axios
 *
 * @param string url
 */
function getBase64BufferFromURL(url) 
    return axios
        .get(url, 
            responseType: 'arraybuffer'
        )
        .then(response => new Buffer(response.data, 'base64'))
        .catch(error => 
            console.log('[ERROR]', error);
        );


/**
 * https://docs.aws.amazon.com/rekognition/latest/dg/labels.html
 * https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Rekognition.html#detectLabels-property
 *
 * @param Buffer bytes
 * @param Number maxLabels
 * @param Number minConfidence
 */
function detectLabelsFromBytes(bytes, maxLabels, minConfidence) 
    return rekognition
        .detectLabels(
            Image: 
                Bytes: bytes
            ,
            MaxLabels: typeof maxLabels !== 'undefined' ? maxLabels : 1000,
            MinConfidence: typeof minConfidence !== 'undefined' ? minConfidence : 50.0
        )
        .promise()
        .catch(error => 
            console.error('[ERROR]', error);
        );


/**
 * https://docs.aws.amazon.com/rekognition/latest/dg/text-detection.html
 * https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Rekognition.html#detectText-property
 *
 * @param Buffer bytes
 */
function detectTextFromBytes(bytes) 
    return rekognition
        .detectText(
            Image: 
                Bytes: bytes
            
        )
        .promise()
        .catch(error => 
            console.error('[ERROR]', error);
        );


/**
 * https://docs.aws.amazon.com/rekognition/latest/dg/celebrities.html
 * https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Rekognition.html#recognizeCelebrities-property
 *
 * @param Buffer bytes
 */
function recognizeCelebritiesFromBytes(bytes) 
    return rekognition
        .recognizeCelebrities(
            Image: 
                Bytes: bytes
            
        )
        .promise()
        .catch(error => 
            console.error('[ERROR]', error);
        );


/**
 * https://docs.aws.amazon.com/rekognition/latest/dg/moderation.html
 * https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Rekognition.html#detectModerationLabels-property
 *
 * @param Buffer bytes
 * @param Number minConfidence
 */
function detectModerationLabelsFromBytes(bytes, minConfidence) 
    return rekognition
        .detectModerationLabels(
            Image: 
                Bytes: bytes
            ,
            MinConfidence: typeof minConfidence !== 'undefined' ? minConfidence : 60.0
        )
        .promise()
        .catch(error => 
            console.error('[ERROR]', error);
        );


/**
 * https://docs.aws.amazon.com/rekognition/latest/dg/faces.html
 * https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Rekognition.html#detectFaces-property
 *
 * @param Buffer bytes
 * @param array attributes Attributes can be "ALL" or "DEFAULT". "DEFAULT" includes: BoundingBox, Confidence, Landmarks, Pose, and Quality.
 */
function detectFacesFromBytes(bytes, attributes) 
    return rekognition
        .detectFaces(
            Image: 
                Bytes: bytes
            ,
            Attributes: typeof attributes !== 'undefined' ? attributes : ['ALL']
        )
        .promise()
        .catch(error => 
            console.error('[ERROR]', error);
        );


/**
 * https://docs.aws.amazon.com/rekognition/latest/dg/API_CompareFaces.html
 * https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Rekognition.html#compareFaces-property
 *
 * @param Buffer sourceBytes
 * @param Buffer targetBytes
 * @param Number similarityThreshold
 */
function compareFaces(sourceBytes, targetBytes, similarityThreshold) 
    return rekognition
        .detectModerationLabels(
            SourceImage: 
                Bytes: sourceBytes
            ,
            TargetImage: 
                Bytes: targetBytes
            ,
            SimilarityThreshold: typeof similarityThreshold !== 'undefined' ? similarityThreshold : 0.0
        )
        .promise()
        .catch(error => 
            console.error('[ERROR]', error);
        );

---

资源：

https://github.com/axios/axios https://docs.aws.amazon.com/rekognition/latest/dg/labels.html https://docs.aws.amazon.com/rekognition/latest/dg/text-detection.html https://docs.aws.amazon.com/rekognition/latest/dg/celebrities.html https://docs.aws.amazon.com/rekognition/latest/dg/moderation.html https://docs.aws.amazon.com/rekognition/latest/dg/faces.html https://docs.aws.amazon.com/rekognition/latest/dg/API_CompareFaces.html https://docs.aws.amazon.com/rekognition/latest/dg/image-bytes-javascript.html

AWS JavaScript 开发工具包参考：

detectLabels detectText recognizeCelebrities detectModerationLabels detectFaces compareFaces

参考：

Download an image using Axios and convert it to base64 Upload a binary file to S3 using AWS SDK for Node.js AWS Rekognition JS SDK Invalid image encoding error Pipe a stream to s3.upload() untarring files to S3 fails, not sure why Using Promises with fs.readFile in a loop How do I return the response from an asynchronous call? NodeJS UnhandledPromiseRejectionWarning How do I check whether an array contains a string in TypeScript? Do you need to use path.join in node.js?