For this week’s assignment, I thought I could try combining 2 models… object detection and sentiment analysis.

Users can upload an image via the file input element. After some processing, the image appears on the canvas, where object detection labels each identifies object. Sentiment analysis then assesses these labels, coloring each bounding box based on its positive or negative sentiment.

Link to sketch: https://editor.p5js.org/khz2004/sketches/PBwwTv1mj

I tried using different models but they gave the exact same results? Not sure if I did this correctly or if this is expected..

model: "facebook/detr-resnet-50"

model: "Xenova/detr-resnet-50"

I also tried playing around with cartoon/clip art/animated images which didn’t detect that well.

Compare and Contrast: Transformers.js vs ml5.js.

• Transformers.js runs much slower and crashes a lot… I had a lot of trouble running the text generation and depth estimation models.
• I find ml5.js easier to understand/use… I think that it abstracts a lot of the complexity and is very easy to integrate with p5.js. Also might just be because I’ve used it more and am more familiar…
• I find Transformers.js less intuitive- the setup is more complex. Going through the code line by line to merge the 2 sketches this week was super helpful in building my understanding of how it all works. Next, I’m hoping to experiment with different models.