Aug 04 15:06:16.414679 ip-10-0-132-155 systemd[1]: Starting Kubernetes Kubelet... Aug 04 15:06:16.853371 ip-10-0-132-155 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 04 15:06:16.853371 ip-10-0-132-155 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Aug 04 15:06:16.853371 ip-10-0-132-155 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 04 15:06:16.853371 ip-10-0-132-155 kubenswrapper[2574]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Aug 04 15:06:16.853371 ip-10-0-132-155 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Aug 04 15:06:16.854413 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.854294 2574 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Aug 04 15:06:16.859329 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859315 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 15:06:16.859329 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859328 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 15:06:16.859409 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859332 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 15:06:16.859409 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859335 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 15:06:16.859409 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859339 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 15:06:16.859409 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859342 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 15:06:16.859409 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859345 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 15:06:16.859409 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859348 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 15:06:16.859409 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859351 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 15:06:16.859409 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859355 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 15:06:16.859409 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859357 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 15:06:16.859409 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859360 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 15:06:16.859409 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859364 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 15:06:16.859409 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859366 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 15:06:16.859409 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859369 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 15:06:16.859409 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859372 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 15:06:16.859409 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859375 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 15:06:16.859409 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859377 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 15:06:16.859409 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859380 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 15:06:16.859409 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859382 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 15:06:16.859409 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859385 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 15:06:16.859409 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859402 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 15:06:16.859884 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859405 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 15:06:16.859884 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859407 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 15:06:16.859884 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859412 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 15:06:16.859884 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859416 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 15:06:16.859884 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859419 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 15:06:16.859884 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859422 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 15:06:16.859884 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859425 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 15:06:16.859884 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859428 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 15:06:16.859884 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859430 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 15:06:16.859884 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859433 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 15:06:16.859884 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859436 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 15:06:16.859884 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859439 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 15:06:16.859884 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859441 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 15:06:16.859884 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859444 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 15:06:16.859884 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859447 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 15:06:16.859884 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859449 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 15:06:16.859884 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859453 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 15:06:16.859884 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859455 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 15:06:16.859884 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859458 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 15:06:16.860334 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859462 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 15:06:16.860334 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859466 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 15:06:16.860334 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859468 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 15:06:16.860334 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859471 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 15:06:16.860334 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859474 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 15:06:16.860334 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859477 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 15:06:16.860334 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859480 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 15:06:16.860334 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859482 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 15:06:16.860334 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859486 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 15:06:16.860334 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859489 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 15:06:16.860334 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859491 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 15:06:16.860334 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859494 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 15:06:16.860334 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859497 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 15:06:16.860334 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859499 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 15:06:16.860334 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859503 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 15:06:16.860334 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859506 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 15:06:16.860334 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859510 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 15:06:16.860334 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859514 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 15:06:16.860334 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859517 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 15:06:16.860840 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859520 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 15:06:16.860840 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859523 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 15:06:16.860840 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859527 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 15:06:16.860840 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859530 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 15:06:16.860840 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859533 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 15:06:16.860840 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859535 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 15:06:16.860840 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859539 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 15:06:16.860840 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859541 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 15:06:16.860840 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859544 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 15:06:16.860840 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859546 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 15:06:16.860840 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859549 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 15:06:16.860840 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859552 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 15:06:16.860840 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859554 2574 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 15:06:16.860840 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859557 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 15:06:16.860840 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859559 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 15:06:16.860840 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859562 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 15:06:16.860840 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859565 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 15:06:16.860840 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859567 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 15:06:16.860840 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859570 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 15:06:16.861303 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859572 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 15:06:16.861303 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859575 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 15:06:16.861303 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859578 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 15:06:16.861303 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859581 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 15:06:16.861303 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859583 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 15:06:16.861303 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859585 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 15:06:16.861303 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859588 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 15:06:16.861303 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859590 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 15:06:16.861303 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859594 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 15:06:16.861303 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859597 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 15:06:16.861303 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859600 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 15:06:16.861303 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859602 2574 feature_gate.go:328] unrecognized feature gate: Example Aug 04 15:06:16.861303 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859605 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 15:06:16.861303 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859608 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 15:06:16.861303 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859611 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 15:06:16.861303 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859614 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 15:06:16.861303 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859616 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 15:06:16.861303 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859619 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 15:06:16.861303 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859622 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 15:06:16.861303 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.859625 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 15:06:16.861791 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860374 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 15:06:16.861791 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860383 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 15:06:16.861791 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860400 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 15:06:16.861791 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860404 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 15:06:16.861791 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860407 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 15:06:16.861791 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860411 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 15:06:16.861791 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860414 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 15:06:16.861791 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860418 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 15:06:16.861791 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860422 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 15:06:16.861791 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860425 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 15:06:16.861791 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860428 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 15:06:16.861791 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860431 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 15:06:16.861791 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860433 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 15:06:16.861791 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860436 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 15:06:16.861791 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860439 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 15:06:16.861791 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860441 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 15:06:16.861791 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860444 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 15:06:16.861791 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860446 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 15:06:16.861791 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860449 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 15:06:16.861791 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860451 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 15:06:16.862279 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860455 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 15:06:16.862279 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860458 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 15:06:16.862279 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860461 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 15:06:16.862279 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860464 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 15:06:16.862279 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860467 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 15:06:16.862279 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860470 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 15:06:16.862279 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860473 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 15:06:16.862279 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860476 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 15:06:16.862279 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860478 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 15:06:16.862279 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860481 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 15:06:16.862279 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860483 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 15:06:16.862279 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860486 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 15:06:16.862279 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860488 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 15:06:16.862279 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860491 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 15:06:16.862279 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860494 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 15:06:16.862279 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860497 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 15:06:16.862279 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860499 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 15:06:16.862279 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860502 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 15:06:16.862279 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860506 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 15:06:16.862279 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860508 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 15:06:16.862784 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860511 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 15:06:16.862784 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860514 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 15:06:16.862784 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860517 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 15:06:16.862784 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860519 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 15:06:16.862784 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860522 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 15:06:16.862784 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860524 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 15:06:16.862784 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860527 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 15:06:16.862784 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860530 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 15:06:16.862784 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860532 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 15:06:16.862784 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860535 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 15:06:16.862784 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860537 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 15:06:16.862784 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860540 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 15:06:16.862784 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860543 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 15:06:16.862784 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860546 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 15:06:16.862784 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860548 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 15:06:16.862784 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860552 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 15:06:16.862784 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860554 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 15:06:16.862784 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860557 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 15:06:16.862784 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860560 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 15:06:16.863251 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860564 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 15:06:16.863251 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860567 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 15:06:16.863251 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860571 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 15:06:16.863251 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860574 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 15:06:16.863251 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860577 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 15:06:16.863251 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860580 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 15:06:16.863251 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860584 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 15:06:16.863251 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860587 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 15:06:16.863251 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860589 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 15:06:16.863251 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860593 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 15:06:16.863251 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860597 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 15:06:16.863251 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860600 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 15:06:16.863251 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860604 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 15:06:16.863251 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860607 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 15:06:16.863251 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860610 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 15:06:16.863251 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860612 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 15:06:16.863251 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860615 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 15:06:16.863251 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860617 2574 feature_gate.go:328] unrecognized feature gate: Example Aug 04 15:06:16.863251 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860620 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 15:06:16.863722 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860622 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 15:06:16.863722 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860625 2574 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 15:06:16.863722 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860627 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 15:06:16.863722 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860630 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 15:06:16.863722 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860632 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 15:06:16.863722 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860635 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 15:06:16.863722 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860638 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 15:06:16.863722 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860641 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 15:06:16.863722 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860644 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 15:06:16.863722 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860647 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 15:06:16.863722 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860649 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 15:06:16.863722 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860652 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 15:06:16.863722 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860655 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 15:06:16.863722 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860657 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 15:06:16.863722 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860660 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 15:06:16.863722 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860663 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 15:06:16.863722 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860665 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 15:06:16.863722 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860668 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 15:06:16.863722 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860671 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 15:06:16.863722 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860673 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 15:06:16.864226 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.860676 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 15:06:16.864226 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861094 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 15:06:16.864226 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861100 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 15:06:16.864226 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861104 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 15:06:16.864226 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861108 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 15:06:16.864226 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861112 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 15:06:16.864226 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861115 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 15:06:16.864226 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861118 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 15:06:16.864226 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861124 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 15:06:16.864226 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861126 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 15:06:16.864226 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861129 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 15:06:16.864226 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861132 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 15:06:16.864226 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861135 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 15:06:16.864226 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861137 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 15:06:16.864226 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861140 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 15:06:16.864226 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861143 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 15:06:16.864226 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861145 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 15:06:16.864226 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861148 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 15:06:16.864226 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861151 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 15:06:16.864778 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861154 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 15:06:16.864778 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861157 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 15:06:16.864778 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861159 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 15:06:16.864778 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861162 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 15:06:16.864778 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861165 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 15:06:16.864778 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861167 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 15:06:16.864778 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861170 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 15:06:16.864778 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861172 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 15:06:16.864778 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861175 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 15:06:16.864778 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861178 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 15:06:16.864778 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861180 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 15:06:16.864778 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861183 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 15:06:16.864778 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861185 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 15:06:16.864778 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861188 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 15:06:16.864778 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861190 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 15:06:16.864778 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861193 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 15:06:16.864778 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861196 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 15:06:16.864778 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861199 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 15:06:16.864778 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861203 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 15:06:16.864778 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861206 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 15:06:16.865263 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861210 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 15:06:16.865263 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861214 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 15:06:16.865263 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861218 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 15:06:16.865263 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861221 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 15:06:16.865263 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861225 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 15:06:16.865263 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861228 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 15:06:16.865263 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861230 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 15:06:16.865263 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861233 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 15:06:16.865263 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861236 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 15:06:16.865263 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861238 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 15:06:16.865263 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861243 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 15:06:16.865263 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861245 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 15:06:16.865263 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861248 2574 feature_gate.go:328] unrecognized feature gate: Example Aug 04 15:06:16.865263 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861251 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 15:06:16.865263 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861253 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 15:06:16.865263 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861256 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 15:06:16.865263 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861259 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 15:06:16.865263 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861261 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 15:06:16.865263 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861264 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 15:06:16.865263 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861267 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 15:06:16.865800 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861269 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 15:06:16.865800 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861272 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 15:06:16.865800 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861275 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 15:06:16.865800 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861277 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 15:06:16.865800 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861280 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 15:06:16.865800 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861284 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 15:06:16.865800 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861287 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 15:06:16.865800 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861290 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 15:06:16.865800 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861293 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 15:06:16.865800 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861296 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 15:06:16.865800 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861298 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 15:06:16.865800 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861301 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 15:06:16.865800 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861305 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 15:06:16.865800 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861310 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 15:06:16.865800 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861313 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 15:06:16.865800 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861316 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 15:06:16.865800 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861318 2574 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 15:06:16.865800 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861321 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 15:06:16.865800 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861323 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 15:06:16.866261 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861326 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 15:06:16.866261 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861329 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 15:06:16.866261 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861331 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 15:06:16.866261 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861335 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 15:06:16.866261 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861337 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 15:06:16.866261 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861340 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 15:06:16.866261 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861343 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 15:06:16.866261 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861345 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 15:06:16.866261 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861348 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 15:06:16.866261 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861352 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 15:06:16.866261 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861356 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 15:06:16.866261 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861358 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 15:06:16.866261 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861361 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 15:06:16.866261 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861364 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 15:06:16.866261 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861375 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 15:06:16.866261 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861377 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 15:06:16.866261 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861380 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 15:06:16.866261 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861382 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 15:06:16.866261 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861397 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 15:06:16.866261 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861400 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 15:06:16.866759 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861403 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 15:06:16.866759 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.861406 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 15:06:16.866759 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861467 2574 flags.go:64] FLAG: --address="0.0.0.0" Aug 04 15:06:16.866759 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861479 2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Aug 04 15:06:16.866759 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861485 2574 flags.go:64] FLAG: --anonymous-auth="true" Aug 04 15:06:16.866759 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861489 2574 flags.go:64] FLAG: --application-metrics-count-limit="100" Aug 04 15:06:16.866759 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861498 2574 flags.go:64] FLAG: --authentication-token-webhook="false" Aug 04 15:06:16.866759 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861501 2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Aug 04 15:06:16.866759 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861505 2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Aug 04 15:06:16.866759 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861509 2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Aug 04 15:06:16.866759 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861512 2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Aug 04 15:06:16.866759 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861515 2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Aug 04 15:06:16.866759 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861519 2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Aug 04 15:06:16.866759 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861522 2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Aug 04 15:06:16.866759 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861525 2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Aug 04 15:06:16.866759 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861528 2574 flags.go:64] FLAG: --cgroup-root="" Aug 04 15:06:16.866759 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861531 2574 flags.go:64] FLAG: --cgroups-per-qos="true" Aug 04 15:06:16.866759 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861535 2574 flags.go:64] FLAG: --client-ca-file="" Aug 04 15:06:16.866759 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861538 2574 flags.go:64] FLAG: --cloud-provider="external" Aug 04 15:06:16.866759 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861540 2574 flags.go:64] FLAG: --cluster-dns="[]" Aug 04 15:06:16.866759 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861545 2574 flags.go:64] FLAG: --cluster-domain="" Aug 04 15:06:16.866759 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861548 2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Aug 04 15:06:16.866759 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861551 2574 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Aug 04 15:06:16.867316 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861554 2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Aug 04 15:06:16.867316 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861557 2574 flags.go:64] FLAG: --container-log-max-files="5" Aug 04 15:06:16.867316 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861561 2574 flags.go:64] FLAG: --container-log-max-size="10Mi" Aug 04 15:06:16.867316 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861564 2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Aug 04 15:06:16.867316 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861567 2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Aug 04 15:06:16.867316 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861571 2574 flags.go:64] FLAG: --containerd-namespace="k8s.io" Aug 04 15:06:16.867316 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861574 2574 flags.go:64] FLAG: --contention-profiling="false" Aug 04 15:06:16.867316 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861576 2574 flags.go:64] FLAG: --cpu-cfs-quota="true" Aug 04 15:06:16.867316 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861579 2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Aug 04 15:06:16.867316 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861583 2574 flags.go:64] FLAG: --cpu-manager-policy="none" Aug 04 15:06:16.867316 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861586 2574 flags.go:64] FLAG: --cpu-manager-policy-options="" Aug 04 15:06:16.867316 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861590 2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Aug 04 15:06:16.867316 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861593 2574 flags.go:64] FLAG: --enable-controller-attach-detach="true" Aug 04 15:06:16.867316 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861596 2574 flags.go:64] FLAG: --enable-debugging-handlers="true" Aug 04 15:06:16.867316 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861599 2574 flags.go:64] FLAG: --enable-load-reader="false" Aug 04 15:06:16.867316 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861603 2574 flags.go:64] FLAG: --enable-server="true" Aug 04 15:06:16.867316 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861607 2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Aug 04 15:06:16.867316 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861611 2574 flags.go:64] FLAG: --event-burst="100" Aug 04 15:06:16.867316 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861614 2574 flags.go:64] FLAG: --event-qps="50" Aug 04 15:06:16.867316 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861617 2574 flags.go:64] FLAG: --event-storage-age-limit="default=0" Aug 04 15:06:16.867316 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861620 2574 flags.go:64] FLAG: --event-storage-event-limit="default=0" Aug 04 15:06:16.867316 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861623 2574 flags.go:64] FLAG: --eviction-hard="" Aug 04 15:06:16.867316 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861627 2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Aug 04 15:06:16.867316 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861630 2574 flags.go:64] FLAG: --eviction-minimum-reclaim="" Aug 04 15:06:16.867941 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861633 2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Aug 04 15:06:16.867941 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861637 2574 flags.go:64] FLAG: --eviction-soft="" Aug 04 15:06:16.867941 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861639 2574 flags.go:64] FLAG: --eviction-soft-grace-period="" Aug 04 15:06:16.867941 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861642 2574 flags.go:64] FLAG: --exit-on-lock-contention="false" Aug 04 15:06:16.867941 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861645 2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Aug 04 15:06:16.867941 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861648 2574 flags.go:64] FLAG: --experimental-mounter-path="" Aug 04 15:06:16.867941 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861651 2574 flags.go:64] FLAG: --fail-cgroupv1="false" Aug 04 15:06:16.867941 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861654 2574 flags.go:64] FLAG: --fail-swap-on="true" Aug 04 15:06:16.867941 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861657 2574 flags.go:64] FLAG: --feature-gates="" Aug 04 15:06:16.867941 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861661 2574 flags.go:64] FLAG: --file-check-frequency="20s" Aug 04 15:06:16.867941 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861664 2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Aug 04 15:06:16.867941 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861667 2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Aug 04 15:06:16.867941 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861670 2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Aug 04 15:06:16.867941 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861673 2574 flags.go:64] FLAG: --healthz-port="10248" Aug 04 15:06:16.867941 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861676 2574 flags.go:64] FLAG: --help="false" Aug 04 15:06:16.867941 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861679 2574 flags.go:64] FLAG: --hostname-override="ip-10-0-132-155.ec2.internal" Aug 04 15:06:16.867941 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861682 2574 flags.go:64] FLAG: --housekeeping-interval="10s" Aug 04 15:06:16.867941 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861685 2574 flags.go:64] FLAG: --http-check-frequency="20s" Aug 04 15:06:16.867941 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861687 2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Aug 04 15:06:16.867941 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861690 2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Aug 04 15:06:16.867941 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861700 2574 flags.go:64] FLAG: --image-gc-high-threshold="85" Aug 04 15:06:16.867941 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861703 2574 flags.go:64] FLAG: --image-gc-low-threshold="80" Aug 04 15:06:16.867941 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861706 2574 flags.go:64] FLAG: --image-service-endpoint="" Aug 04 15:06:16.868580 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861710 2574 flags.go:64] FLAG: --kernel-memcg-notification="false" Aug 04 15:06:16.868580 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861713 2574 flags.go:64] FLAG: --kube-api-burst="100" Aug 04 15:06:16.868580 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861716 2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Aug 04 15:06:16.868580 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861720 2574 flags.go:64] FLAG: --kube-api-qps="50" Aug 04 15:06:16.868580 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861723 2574 flags.go:64] FLAG: --kube-reserved="" Aug 04 15:06:16.868580 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861726 2574 flags.go:64] FLAG: --kube-reserved-cgroup="" Aug 04 15:06:16.868580 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861729 2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Aug 04 15:06:16.868580 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861732 2574 flags.go:64] FLAG: --kubelet-cgroups="" Aug 04 15:06:16.868580 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861735 2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Aug 04 15:06:16.868580 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861738 2574 flags.go:64] FLAG: --lock-file="" Aug 04 15:06:16.868580 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861741 2574 flags.go:64] FLAG: --log-cadvisor-usage="false" Aug 04 15:06:16.868580 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861744 2574 flags.go:64] FLAG: --log-flush-frequency="5s" Aug 04 15:06:16.868580 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861747 2574 flags.go:64] FLAG: --log-json-info-buffer-size="0" Aug 04 15:06:16.868580 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861752 2574 flags.go:64] FLAG: --log-json-split-stream="false" Aug 04 15:06:16.868580 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861755 2574 flags.go:64] FLAG: --log-text-info-buffer-size="0" Aug 04 15:06:16.868580 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861758 2574 flags.go:64] FLAG: --log-text-split-stream="false" Aug 04 15:06:16.868580 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861760 2574 flags.go:64] FLAG: --logging-format="text" Aug 04 15:06:16.868580 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861763 2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Aug 04 15:06:16.868580 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861766 2574 flags.go:64] FLAG: --make-iptables-util-chains="true" Aug 04 15:06:16.868580 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861769 2574 flags.go:64] FLAG: --manifest-url="" Aug 04 15:06:16.868580 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861772 2574 flags.go:64] FLAG: --manifest-url-header="" Aug 04 15:06:16.868580 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861776 2574 flags.go:64] FLAG: --max-housekeeping-interval="15s" Aug 04 15:06:16.868580 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861779 2574 flags.go:64] FLAG: --max-open-files="1000000" Aug 04 15:06:16.868580 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861783 2574 flags.go:64] FLAG: --max-pods="110" Aug 04 15:06:16.868580 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861786 2574 flags.go:64] FLAG: --maximum-dead-containers="-1" Aug 04 15:06:16.869161 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861789 2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Aug 04 15:06:16.869161 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861792 2574 flags.go:64] FLAG: --memory-manager-policy="None" Aug 04 15:06:16.869161 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861795 2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Aug 04 15:06:16.869161 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861798 2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Aug 04 15:06:16.869161 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861801 2574 flags.go:64] FLAG: --node-ip="0.0.0.0" Aug 04 15:06:16.869161 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861804 2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Aug 04 15:06:16.869161 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861811 2574 flags.go:64] FLAG: --node-status-max-images="50" Aug 04 15:06:16.869161 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861815 2574 flags.go:64] FLAG: --node-status-update-frequency="10s" Aug 04 15:06:16.869161 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861818 2574 flags.go:64] FLAG: --oom-score-adj="-999" Aug 04 15:06:16.869161 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861820 2574 flags.go:64] FLAG: --pod-cidr="" Aug 04 15:06:16.869161 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861824 2574 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8" Aug 04 15:06:16.869161 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861829 2574 flags.go:64] FLAG: --pod-manifest-path="" Aug 04 15:06:16.869161 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861832 2574 flags.go:64] FLAG: --pod-max-pids="-1" Aug 04 15:06:16.869161 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861835 2574 flags.go:64] FLAG: --pods-per-core="0" Aug 04 15:06:16.869161 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861838 2574 flags.go:64] FLAG: --port="10250" Aug 04 15:06:16.869161 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861841 2574 flags.go:64] FLAG: --protect-kernel-defaults="false" Aug 04 15:06:16.869161 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861845 2574 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f877676d267655a7" Aug 04 15:06:16.869161 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861848 2574 flags.go:64] FLAG: --qos-reserved="" Aug 04 15:06:16.869161 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861851 2574 flags.go:64] FLAG: --read-only-port="10255" Aug 04 15:06:16.869161 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861854 2574 flags.go:64] FLAG: --register-node="true" Aug 04 15:06:16.869161 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861857 2574 flags.go:64] FLAG: --register-with-taints="" Aug 04 15:06:16.869161 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861861 2574 flags.go:64] FLAG: --registry-burst="10" Aug 04 15:06:16.869161 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861864 2574 flags.go:64] FLAG: --registry-qps="5" Aug 04 15:06:16.869161 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861866 2574 flags.go:64] FLAG: --reserved-cpus="" Aug 04 15:06:16.869791 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861869 2574 flags.go:64] FLAG: --reserved-memory="" Aug 04 15:06:16.869791 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861873 2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Aug 04 15:06:16.869791 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861876 2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Aug 04 15:06:16.869791 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861879 2574 flags.go:64] FLAG: --rotate-certificates="false" Aug 04 15:06:16.869791 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861882 2574 flags.go:64] FLAG: --rotate-server-certificates="false" Aug 04 15:06:16.869791 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861885 2574 flags.go:64] FLAG: --runonce="false" Aug 04 15:06:16.869791 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861887 2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Aug 04 15:06:16.869791 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861890 2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Aug 04 15:06:16.869791 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861893 2574 flags.go:64] FLAG: --seccomp-default="false" Aug 04 15:06:16.869791 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861896 2574 flags.go:64] FLAG: --serialize-image-pulls="true" Aug 04 15:06:16.869791 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861899 2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Aug 04 15:06:16.869791 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861902 2574 flags.go:64] FLAG: --storage-driver-db="cadvisor" Aug 04 15:06:16.869791 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861905 2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Aug 04 15:06:16.869791 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861908 2574 flags.go:64] FLAG: --storage-driver-password="root" Aug 04 15:06:16.869791 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861911 2574 flags.go:64] FLAG: --storage-driver-secure="false" Aug 04 15:06:16.869791 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861914 2574 flags.go:64] FLAG: --storage-driver-table="stats" Aug 04 15:06:16.869791 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861917 2574 flags.go:64] FLAG: --storage-driver-user="root" Aug 04 15:06:16.869791 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861920 2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Aug 04 15:06:16.869791 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861936 2574 flags.go:64] FLAG: --sync-frequency="1m0s" Aug 04 15:06:16.869791 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861940 2574 flags.go:64] FLAG: --system-cgroups="" Aug 04 15:06:16.869791 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861944 2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Aug 04 15:06:16.869791 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861950 2574 flags.go:64] FLAG: --system-reserved-cgroup="" Aug 04 15:06:16.869791 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861953 2574 flags.go:64] FLAG: --tls-cert-file="" Aug 04 15:06:16.869791 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861956 2574 flags.go:64] FLAG: --tls-cipher-suites="[]" Aug 04 15:06:16.869791 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861962 2574 flags.go:64] FLAG: --tls-min-version="" Aug 04 15:06:16.870413 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861964 2574 flags.go:64] FLAG: --tls-private-key-file="" Aug 04 15:06:16.870413 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861967 2574 flags.go:64] FLAG: --topology-manager-policy="none" Aug 04 15:06:16.870413 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861970 2574 flags.go:64] FLAG: --topology-manager-policy-options="" Aug 04 15:06:16.870413 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861973 2574 flags.go:64] FLAG: --topology-manager-scope="container" Aug 04 15:06:16.870413 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861976 2574 flags.go:64] FLAG: --v="2" Aug 04 15:06:16.870413 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861980 2574 flags.go:64] FLAG: --version="false" Aug 04 15:06:16.870413 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861984 2574 flags.go:64] FLAG: --vmodule="" Aug 04 15:06:16.870413 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861988 2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Aug 04 15:06:16.870413 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.861991 2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Aug 04 15:06:16.870413 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862098 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 15:06:16.870413 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862102 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 15:06:16.870413 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862105 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 15:06:16.870413 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862108 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 15:06:16.870413 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862111 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 15:06:16.870413 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862114 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 15:06:16.870413 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862117 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 15:06:16.870413 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862120 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 15:06:16.870413 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862122 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 15:06:16.870413 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862125 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 15:06:16.870413 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862128 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 15:06:16.870413 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862130 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 15:06:16.870413 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862133 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 15:06:16.870941 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862135 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 15:06:16.870941 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862138 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 15:06:16.870941 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862141 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 15:06:16.870941 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862144 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 15:06:16.870941 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862147 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 15:06:16.870941 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862150 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 15:06:16.870941 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862153 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 15:06:16.870941 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862155 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 15:06:16.870941 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862158 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 15:06:16.870941 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862163 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 15:06:16.870941 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862165 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 15:06:16.870941 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862168 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 15:06:16.870941 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862170 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 15:06:16.870941 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862173 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 15:06:16.870941 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862175 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 15:06:16.870941 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862178 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 15:06:16.870941 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862181 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 15:06:16.870941 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862183 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 15:06:16.870941 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862186 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 15:06:16.870941 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862188 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 15:06:16.871440 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862191 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 15:06:16.871440 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862194 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 15:06:16.871440 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862198 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 15:06:16.871440 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862202 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 15:06:16.871440 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862205 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 15:06:16.871440 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862207 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 15:06:16.871440 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862210 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 15:06:16.871440 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862213 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 15:06:16.871440 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862215 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 15:06:16.871440 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862218 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 15:06:16.871440 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862221 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 15:06:16.871440 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862223 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 15:06:16.871440 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862226 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 15:06:16.871440 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862229 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 15:06:16.871440 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862232 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 15:06:16.871440 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862236 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 15:06:16.871440 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862238 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 15:06:16.871440 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862241 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 15:06:16.871440 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862244 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 15:06:16.871915 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862247 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 15:06:16.871915 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862249 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 15:06:16.871915 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862253 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 15:06:16.871915 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862256 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 15:06:16.871915 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862259 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 15:06:16.871915 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862261 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 15:06:16.871915 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862264 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 15:06:16.871915 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862266 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 15:06:16.871915 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862269 2574 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 15:06:16.871915 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862272 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 15:06:16.871915 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862274 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 15:06:16.871915 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862277 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 15:06:16.871915 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862279 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 15:06:16.871915 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862282 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 15:06:16.871915 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862285 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 15:06:16.871915 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862287 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 15:06:16.871915 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862291 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 15:06:16.871915 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862294 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 15:06:16.871915 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862296 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 15:06:16.871915 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862299 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 15:06:16.872410 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862301 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 15:06:16.872410 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862304 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 15:06:16.872410 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862307 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 15:06:16.872410 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862309 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 15:06:16.872410 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862312 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 15:06:16.872410 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862314 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 15:06:16.872410 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862318 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 15:06:16.872410 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862321 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 15:06:16.872410 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862324 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 15:06:16.872410 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862326 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 15:06:16.872410 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862329 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 15:06:16.872410 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862331 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 15:06:16.872410 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862334 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 15:06:16.872410 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862337 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 15:06:16.872410 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862341 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 15:06:16.872410 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862344 2574 feature_gate.go:328] unrecognized feature gate: Example Aug 04 15:06:16.872410 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862346 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 15:06:16.872410 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862349 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 15:06:16.872410 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862351 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 15:06:16.872410 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862354 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 15:06:16.873246 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862357 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 15:06:16.873246 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862361 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 15:06:16.873246 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862364 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 15:06:16.873246 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862367 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 15:06:16.873246 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862369 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 15:06:16.873246 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862372 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 15:06:16.873246 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.862374 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 15:06:16.873246 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.863064 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 04 15:06:16.873246 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.869941 2574 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Aug 04 15:06:16.873246 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.869954 2574 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Aug 04 15:06:16.873246 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.869973 2574 watchdog_linux.go:95] "Systemd watchdog is not enabled" Aug 04 15:06:16.873246 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.869978 2574 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Aug 04 15:06:16.873246 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870041 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 15:06:16.873246 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870047 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 15:06:16.873826 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870050 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 15:06:16.873826 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870053 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 15:06:16.873826 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870056 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 15:06:16.873826 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870058 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 15:06:16.873826 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870062 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 15:06:16.873826 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870064 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 15:06:16.873826 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870067 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 15:06:16.873826 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870070 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 15:06:16.873826 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870073 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 15:06:16.873826 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870076 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 15:06:16.873826 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870085 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 15:06:16.873826 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870088 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 15:06:16.873826 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870091 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 15:06:16.873826 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870093 2574 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 15:06:16.873826 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870096 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 15:06:16.873826 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870099 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 15:06:16.873826 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870102 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 15:06:16.873826 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870105 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 15:06:16.873826 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870107 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 15:06:16.873826 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870110 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 15:06:16.874310 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870112 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 15:06:16.874310 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870115 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 15:06:16.874310 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870118 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 15:06:16.874310 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870120 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 15:06:16.874310 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870123 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 15:06:16.874310 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870126 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 15:06:16.874310 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870129 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 15:06:16.874310 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870132 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 15:06:16.874310 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870134 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 15:06:16.874310 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870137 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 15:06:16.874310 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870140 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 15:06:16.874310 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870143 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 15:06:16.874310 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870145 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 15:06:16.874310 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870148 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 15:06:16.874310 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870150 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 15:06:16.874310 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870153 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 15:06:16.874310 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870155 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 15:06:16.874310 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870158 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 15:06:16.874310 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870160 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 15:06:16.874310 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870163 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 15:06:16.874891 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870165 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 15:06:16.874891 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870168 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 15:06:16.874891 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870170 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 15:06:16.874891 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870183 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 15:06:16.874891 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870187 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 15:06:16.874891 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870190 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 15:06:16.874891 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870193 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 15:06:16.874891 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870196 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 15:06:16.874891 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870200 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 15:06:16.874891 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870203 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 15:06:16.874891 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870206 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 15:06:16.874891 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870209 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 15:06:16.874891 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870212 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 15:06:16.874891 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870214 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 15:06:16.874891 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870217 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 15:06:16.874891 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870220 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 15:06:16.874891 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870222 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 15:06:16.874891 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870225 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 15:06:16.874891 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870228 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 15:06:16.875343 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870231 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 15:06:16.875343 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870234 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 15:06:16.875343 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870236 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 15:06:16.875343 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870239 2574 feature_gate.go:328] unrecognized feature gate: Example Aug 04 15:06:16.875343 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870241 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 15:06:16.875343 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870244 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 15:06:16.875343 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870246 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 15:06:16.875343 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870249 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 15:06:16.875343 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870252 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 15:06:16.875343 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870254 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 15:06:16.875343 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870256 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 15:06:16.875343 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870259 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 15:06:16.875343 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870261 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 15:06:16.875343 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870264 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 15:06:16.875343 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870266 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 15:06:16.875343 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870269 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 15:06:16.875343 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870271 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 15:06:16.875343 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870279 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 15:06:16.875343 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870282 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 15:06:16.875343 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870285 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 15:06:16.875894 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870287 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 15:06:16.875894 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870290 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 15:06:16.875894 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870293 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 15:06:16.875894 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870297 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 15:06:16.875894 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870299 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 15:06:16.875894 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870302 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 15:06:16.875894 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870305 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 15:06:16.875894 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870308 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 15:06:16.875894 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870310 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 15:06:16.875894 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870313 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 15:06:16.875894 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870316 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 15:06:16.875894 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870319 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 15:06:16.875894 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870322 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 15:06:16.875894 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870324 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 15:06:16.875894 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870327 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 15:06:16.875894 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870329 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 15:06:16.875894 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870332 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 15:06:16.875894 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870334 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 15:06:16.876347 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.870339 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 04 15:06:16.876347 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870496 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Aug 04 15:06:16.876347 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870501 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Aug 04 15:06:16.876347 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870504 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Aug 04 15:06:16.876347 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870507 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Aug 04 15:06:16.876347 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870509 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Aug 04 15:06:16.876347 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870512 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Aug 04 15:06:16.876347 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870514 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Aug 04 15:06:16.876347 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870517 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Aug 04 15:06:16.876347 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870519 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Aug 04 15:06:16.876347 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870522 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Aug 04 15:06:16.876347 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870524 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Aug 04 15:06:16.876347 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870532 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Aug 04 15:06:16.876347 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870535 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Aug 04 15:06:16.876347 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870537 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Aug 04 15:06:16.876729 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870540 2574 feature_gate.go:328] unrecognized feature gate: Example2 Aug 04 15:06:16.876729 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870542 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Aug 04 15:06:16.876729 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870545 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Aug 04 15:06:16.876729 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870547 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Aug 04 15:06:16.876729 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870550 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Aug 04 15:06:16.876729 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870553 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Aug 04 15:06:16.876729 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870555 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Aug 04 15:06:16.876729 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870557 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Aug 04 15:06:16.876729 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870560 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Aug 04 15:06:16.876729 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870562 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Aug 04 15:06:16.876729 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870565 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Aug 04 15:06:16.876729 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870568 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Aug 04 15:06:16.876729 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870570 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Aug 04 15:06:16.876729 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870573 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Aug 04 15:06:16.876729 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870575 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Aug 04 15:06:16.876729 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870578 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Aug 04 15:06:16.876729 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870581 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Aug 04 15:06:16.876729 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870583 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Aug 04 15:06:16.876729 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870586 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Aug 04 15:06:16.876729 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870588 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Aug 04 15:06:16.877202 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870591 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Aug 04 15:06:16.877202 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870593 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Aug 04 15:06:16.877202 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870596 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Aug 04 15:06:16.877202 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870598 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Aug 04 15:06:16.877202 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870600 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Aug 04 15:06:16.877202 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870603 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Aug 04 15:06:16.877202 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870605 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Aug 04 15:06:16.877202 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870608 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Aug 04 15:06:16.877202 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870611 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Aug 04 15:06:16.877202 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870613 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Aug 04 15:06:16.877202 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870621 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Aug 04 15:06:16.877202 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870624 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Aug 04 15:06:16.877202 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870626 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Aug 04 15:06:16.877202 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870630 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Aug 04 15:06:16.877202 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870634 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Aug 04 15:06:16.877202 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870637 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Aug 04 15:06:16.877202 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870639 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Aug 04 15:06:16.877202 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870642 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Aug 04 15:06:16.877202 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870644 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Aug 04 15:06:16.877202 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870647 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Aug 04 15:06:16.877700 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870649 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Aug 04 15:06:16.877700 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870652 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Aug 04 15:06:16.877700 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870654 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Aug 04 15:06:16.877700 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870657 2574 feature_gate.go:328] unrecognized feature gate: Example Aug 04 15:06:16.877700 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870659 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Aug 04 15:06:16.877700 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870661 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Aug 04 15:06:16.877700 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870664 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Aug 04 15:06:16.877700 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870666 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Aug 04 15:06:16.877700 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870674 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Aug 04 15:06:16.877700 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870677 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Aug 04 15:06:16.877700 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870680 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Aug 04 15:06:16.877700 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870683 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Aug 04 15:06:16.877700 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870685 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Aug 04 15:06:16.877700 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870687 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Aug 04 15:06:16.877700 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870690 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Aug 04 15:06:16.877700 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870692 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Aug 04 15:06:16.877700 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870695 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Aug 04 15:06:16.877700 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870697 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Aug 04 15:06:16.877700 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870700 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Aug 04 15:06:16.877700 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870702 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Aug 04 15:06:16.878166 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870705 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Aug 04 15:06:16.878166 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870707 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Aug 04 15:06:16.878166 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870710 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Aug 04 15:06:16.878166 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870718 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Aug 04 15:06:16.878166 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870721 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Aug 04 15:06:16.878166 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870723 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Aug 04 15:06:16.878166 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870726 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Aug 04 15:06:16.878166 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870728 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Aug 04 15:06:16.878166 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870731 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Aug 04 15:06:16.878166 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870733 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Aug 04 15:06:16.878166 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870736 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Aug 04 15:06:16.878166 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870738 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Aug 04 15:06:16.878166 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870740 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Aug 04 15:06:16.878166 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870743 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Aug 04 15:06:16.878166 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870745 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Aug 04 15:06:16.878166 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870748 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Aug 04 15:06:16.878166 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870750 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Aug 04 15:06:16.878166 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870753 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Aug 04 15:06:16.878166 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870755 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Aug 04 15:06:16.878644 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870759 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Aug 04 15:06:16.878644 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870762 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Aug 04 15:06:16.878644 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870765 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Aug 04 15:06:16.878644 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870767 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Aug 04 15:06:16.878644 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870770 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Aug 04 15:06:16.878644 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:16.870773 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Aug 04 15:06:16.878644 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.870778 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Aug 04 15:06:16.878644 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.870868 2574 server.go:962] "Client rotation is on, will bootstrap in background" Aug 04 15:06:16.878644 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.874422 2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Aug 04 15:06:16.878644 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.875888 2574 server.go:1019] "Starting client certificate rotation" Aug 04 15:06:16.878644 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.875986 2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Aug 04 15:06:16.878644 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.876024 2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Aug 04 15:06:16.878971 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.878654 2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Aug 04 15:06:16.879816 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.879802 2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Aug 04 15:06:16.893811 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.893792 2574 log.go:25] "Validated CRI v1 runtime API" Aug 04 15:06:16.895881 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.895868 2574 log.go:25] "Validated CRI v1 image API" Aug 04 15:06:16.896625 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.896604 2574 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Aug 04 15:06:16.899196 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.899168 2574 fs.go:135] Filesystem UUIDs: map[1a7c5bed-b0ff-4ff8-a2fe-a23d69eddc6e:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 8c3d063c-e776-47da-bf34-991e5f09f17b:/dev/nvme0n1p4] Aug 04 15:06:16.899267 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.899195 2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Aug 04 15:06:16.905924 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.905807 2574 manager.go:216] Machine: {Timestamp:2026-08-04 15:06:16.904353532 +0000 UTC m=+0.364225872 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3107610 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2adccb6d6ef31d2ce64518fc0b9907 SystemUUID:ec2adccb-6d6e-f31d-2ce6-4518fc0b9907 BootID:663baa88-a959-42ab-b4df-ddabf568439c Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ee:36:28:35:17 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ee:36:28:35:17 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:42:0d:29:66:a2:d1 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Aug 04 15:06:16.905924 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.905921 2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Aug 04 15:06:16.906035 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.906021 2574 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Aug 04 15:06:16.909443 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.909407 2574 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Aug 04 15:06:16.909594 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.909452 2574 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-155.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Aug 04 15:06:16.910611 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.910600 2574 topology_manager.go:138] "Creating topology manager with none policy" Aug 04 15:06:16.910647 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.910614 2574 container_manager_linux.go:309] "Creating device plugin manager" Aug 04 15:06:16.910647 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.910622 2574 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Aug 04 15:06:16.911596 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.911585 2574 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Aug 04 15:06:16.912652 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.912640 2574 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Aug 04 15:06:16.915204 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.915185 2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Aug 04 15:06:16.916201 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.916189 2574 state_mem.go:36] "Initialized new in-memory state store" Aug 04 15:06:16.916324 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.916313 2574 server.go:1240] "Using root directory" path="/var/lib/kubelet" Aug 04 15:06:16.916362 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.916354 2574 kubelet.go:486] "Attempting to sync node with API server" Aug 04 15:06:16.916408 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.916380 2574 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Aug 04 15:06:16.916443 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.916409 2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Aug 04 15:06:16.916443 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.916421 2574 kubelet.go:398] "Adding apiserver pod source" Aug 04 15:06:16.916443 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.916433 2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Aug 04 15:06:16.916905 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.916893 2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Aug 04 15:06:16.916959 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.916913 2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Aug 04 15:06:16.920301 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.920275 2574 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Aug 04 15:06:16.922121 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.922105 2574 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Aug 04 15:06:16.922181 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.922137 2574 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Aug 04 15:06:16.923588 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.923577 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Aug 04 15:06:16.923635 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.923594 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Aug 04 15:06:16.923635 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.923603 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Aug 04 15:06:16.923635 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.923612 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Aug 04 15:06:16.923635 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.923624 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Aug 04 15:06:16.923635 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.923629 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Aug 04 15:06:16.923635 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.923636 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Aug 04 15:06:16.923791 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.923642 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Aug 04 15:06:16.923791 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.923650 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Aug 04 15:06:16.923791 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.923655 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Aug 04 15:06:16.923791 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.923663 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Aug 04 15:06:16.923791 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.923668 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Aug 04 15:06:16.924657 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.924647 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Aug 04 15:06:16.924657 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.924657 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Aug 04 15:06:16.927647 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.927632 2574 server.go:1268] "Started kubelet" Aug 04 15:06:16.927761 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.927712 2574 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Aug 04 15:06:16.928137 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.927715 2574 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Aug 04 15:06:16.928202 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.928161 2574 server_v1.go:49] "podresources" method="list" useActivePods=true Aug 04 15:06:16.928418 ip-10-0-132-155 systemd[1]: Started Kubernetes Kubelet. Aug 04 15:06:16.929675 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.929655 2574 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Aug 04 15:06:16.930568 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.930552 2574 server.go:310] "Adding debug handlers to kubelet server" Aug 04 15:06:16.934288 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:16.934270 2574 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Aug 04 15:06:16.934522 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.934508 2574 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-155.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Aug 04 15:06:16.935309 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:16.935291 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-155.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Aug 04 15:06:16.935370 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:16.935290 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Aug 04 15:06:16.935606 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:16.934643 2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-155.ec2.internal.18c8a28c94c7257d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-155.ec2.internal,UID:ip-10-0-132-155.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-155.ec2.internal,},FirstTimestamp:2026-08-04 15:06:16.927602045 +0000 UTC m=+0.387474385,LastTimestamp:2026-08-04 15:06:16.927602045 +0000 UTC m=+0.387474385,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-155.ec2.internal,}" Aug 04 15:06:16.937480 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.937465 2574 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Aug 04 15:06:16.937546 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.937481 2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Aug 04 15:06:16.938085 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:16.937998 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-155.ec2.internal\" not found" Aug 04 15:06:16.938211 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.938186 2574 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Aug 04 15:06:16.938286 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.938191 2574 volume_manager.go:311] "The desired_state_of_world populator starts" Aug 04 15:06:16.938286 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.938232 2574 volume_manager.go:313] "Starting Kubelet Volume Manager" Aug 04 15:06:16.938451 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.938398 2574 factory.go:55] Registering systemd factory Aug 04 15:06:16.938451 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.938412 2574 factory.go:223] Registration of the systemd container factory successfully Aug 04 15:06:16.938595 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.938582 2574 factory.go:153] Registering CRI-O factory Aug 04 15:06:16.938660 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.938597 2574 factory.go:223] Registration of the crio container factory successfully Aug 04 15:06:16.938660 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.938612 2574 reconstruct.go:97] "Volume reconstruction finished" Aug 04 15:06:16.938660 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.938623 2574 reconciler.go:29] "Reconciler: start to sync state" Aug 04 15:06:16.938660 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.938646 2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Aug 04 15:06:16.938771 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.938668 2574 factory.go:103] Registering Raw factory Aug 04 15:06:16.938771 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.938682 2574 manager.go:1196] Started watching for new ooms in manager Aug 04 15:06:16.939246 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.939235 2574 manager.go:349] Starting recovery of all containers Aug 04 15:06:16.941206 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:16.941176 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Aug 04 15:06:16.941294 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:16.941250 2574 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-155.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Aug 04 15:06:16.949800 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.949778 2574 manager.go:354] Recovery completed Aug 04 15:06:16.955072 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.954974 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 15:06:16.956872 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.956856 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-155.ec2.internal" event="NodeHasSufficientMemory" Aug 04 15:06:16.956926 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.956884 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-155.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 15:06:16.956926 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.956894 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-155.ec2.internal" event="NodeHasSufficientPID" Aug 04 15:06:16.957441 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.957425 2574 cpu_manager.go:222] "Starting CPU manager" policy="none" Aug 04 15:06:16.957441 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.957440 2574 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Aug 04 15:06:16.957580 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.957443 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 15:06:16.957580 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.957456 2574 state_mem.go:36] "Initialized new in-memory state store" Aug 04 15:06:16.958094 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.958080 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-155.ec2.internal" event="NodeHasSufficientMemory" Aug 04 15:06:16.958136 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.958103 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-155.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 15:06:16.958136 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.958116 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-155.ec2.internal" event="NodeHasSufficientPID" Aug 04 15:06:16.958593 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.958577 2574 policy_none.go:49] "None policy: Start" Aug 04 15:06:16.958659 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.958596 2574 memory_manager.go:187] "Starting memorymanager" policy="None" Aug 04 15:06:16.958659 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.958609 2574 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Aug 04 15:06:16.959537 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.959524 2574 policy_none.go:47] "Start" Aug 04 15:06:16.959779 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:16.959700 2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-155.ec2.internal.18c8a28c9685c3f7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-155.ec2.internal,UID:ip-10-0-132-155.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-155.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-155.ec2.internal,},FirstTimestamp:2026-08-04 15:06:16.956871671 +0000 UTC m=+0.416744014,LastTimestamp:2026-08-04 15:06:16.956871671 +0000 UTC m=+0.416744014,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-155.ec2.internal,}" Aug 04 15:06:16.986674 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:16.986605 2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-155.ec2.internal.18c8a28c96860908 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-155.ec2.internal,UID:ip-10-0-132-155.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-132-155.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-132-155.ec2.internal,},FirstTimestamp:2026-08-04 15:06:16.956889352 +0000 UTC m=+0.416761693,LastTimestamp:2026-08-04 15:06:16.956889352 +0000 UTC m=+0.416761693,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-155.ec2.internal,}" Aug 04 15:06:16.998979 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.998964 2574 manager.go:337] "Starting Device Plugin manager" Aug 04 15:06:17.011606 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:16.999001 2574 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Aug 04 15:06:17.011606 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.999014 2574 server.go:87] "Starting device plugin registration server" Aug 04 15:06:17.011606 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.999243 2574 eviction_manager.go:189] "Eviction manager: starting control loop" Aug 04 15:06:17.011606 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.999255 2574 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Aug 04 15:06:17.011606 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.999366 2574 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Aug 04 15:06:17.011606 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.999449 2574 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Aug 04 15:06:17.011606 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:16.999457 2574 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Aug 04 15:06:17.011606 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:17.000001 2574 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Aug 04 15:06:17.011606 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:17.000053 2574 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-155.ec2.internal\" not found" Aug 04 15:06:17.011606 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:17.008067 2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-155.ec2.internal.18c8a28c96862e9b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-155.ec2.internal,UID:ip-10-0-132-155.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-132-155.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-132-155.ec2.internal,},FirstTimestamp:2026-08-04 15:06:16.956898971 +0000 UTC m=+0.416771311,LastTimestamp:2026-08-04 15:06:16.956898971 +0000 UTC m=+0.416771311,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-155.ec2.internal,}" Aug 04 15:06:17.020995 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:17.020921 2574 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-132-155.ec2.internal.18c8a28c9685c3f7\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-155.ec2.internal.18c8a28c9685c3f7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-155.ec2.internal,UID:ip-10-0-132-155.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-155.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-155.ec2.internal,},FirstTimestamp:2026-08-04 15:06:16.956871671 +0000 UTC m=+0.416744014,LastTimestamp:2026-08-04 15:06:16.958094277 +0000 UTC m=+0.417966617,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-155.ec2.internal,}" Aug 04 15:06:17.029149 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:17.029090 2574 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-132-155.ec2.internal.18c8a28c96860908\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-155.ec2.internal.18c8a28c96860908 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-155.ec2.internal,UID:ip-10-0-132-155.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-132-155.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-132-155.ec2.internal,},FirstTimestamp:2026-08-04 15:06:16.956889352 +0000 UTC m=+0.416761693,LastTimestamp:2026-08-04 15:06:16.958109528 +0000 UTC m=+0.417981869,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-155.ec2.internal,}" Aug 04 15:06:17.036332 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:17.036274 2574 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-132-155.ec2.internal.18c8a28c96862e9b\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-155.ec2.internal.18c8a28c96862e9b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-155.ec2.internal,UID:ip-10-0-132-155.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-132-155.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-132-155.ec2.internal,},FirstTimestamp:2026-08-04 15:06:16.956898971 +0000 UTC m=+0.416771311,LastTimestamp:2026-08-04 15:06:16.958121057 +0000 UTC m=+0.417993399,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-155.ec2.internal,}" Aug 04 15:06:17.043570 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:17.043516 2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-155.ec2.internal.18c8a28c992977cd default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-155.ec2.internal,UID:ip-10-0-132-155.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-132-155.ec2.internal,},FirstTimestamp:2026-08-04 15:06:17.001154509 +0000 UTC m=+0.461026855,LastTimestamp:2026-08-04 15:06:17.001154509 +0000 UTC m=+0.461026855,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-155.ec2.internal,}" Aug 04 15:06:17.045705 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.045677 2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Aug 04 15:06:17.046884 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.046869 2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Aug 04 15:06:17.046969 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.046905 2574 status_manager.go:244] "Starting to sync pod status with apiserver" Aug 04 15:06:17.046969 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.046924 2574 kubelet.go:2448] "Starting kubelet main sync loop" Aug 04 15:06:17.046969 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:17.046956 2574 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Aug 04 15:06:17.051540 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:17.051516 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" Aug 04 15:06:17.094908 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.094877 2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6sg8n" Aug 04 15:06:17.099931 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.099915 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 15:06:17.100793 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.100769 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-155.ec2.internal" event="NodeHasSufficientMemory" Aug 04 15:06:17.100894 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.100802 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-155.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 15:06:17.100894 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.100819 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-155.ec2.internal" event="NodeHasSufficientPID" Aug 04 15:06:17.100894 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.100843 2574 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-155.ec2.internal" Aug 04 15:06:17.103178 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:17.103110 2574 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-132-155.ec2.internal.18c8a28c9685c3f7\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-155.ec2.internal.18c8a28c9685c3f7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-155.ec2.internal,UID:ip-10-0-132-155.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-155.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-155.ec2.internal,},FirstTimestamp:2026-08-04 15:06:16.956871671 +0000 UTC m=+0.416744014,LastTimestamp:2026-08-04 15:06:17.100785574 +0000 UTC m=+0.560657914,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-155.ec2.internal,}" Aug 04 15:06:17.113568 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.113511 2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6sg8n" Aug 04 15:06:17.114253 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:17.114233 2574 kubelet_node_status.go:116] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-132-155.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"nodes\" in API group \"\" at the cluster scope" node="ip-10-0-132-155.ec2.internal" Aug 04 15:06:17.114641 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:17.114555 2574 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-132-155.ec2.internal.18c8a28c96860908\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-155.ec2.internal.18c8a28c96860908 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-155.ec2.internal,UID:ip-10-0-132-155.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-132-155.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-132-155.ec2.internal,},FirstTimestamp:2026-08-04 15:06:16.956889352 +0000 UTC m=+0.416761693,LastTimestamp:2026-08-04 15:06:17.100809834 +0000 UTC m=+0.560682175,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-155.ec2.internal,}" Aug 04 15:06:17.117653 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:17.117594 2574 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-132-155.ec2.internal.18c8a28c96862e9b\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-155.ec2.internal.18c8a28c96862e9b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-155.ec2.internal,UID:ip-10-0-132-155.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-132-155.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-132-155.ec2.internal,},FirstTimestamp:2026-08-04 15:06:16.956898971 +0000 UTC m=+0.416771311,LastTimestamp:2026-08-04 15:06:17.100825834 +0000 UTC m=+0.560698174,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-155.ec2.internal,}" Aug 04 15:06:17.147032 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.147000 2574 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-155.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-155.ec2.internal"] Aug 04 15:06:17.147141 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.147073 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 15:06:17.147973 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.147957 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-155.ec2.internal" event="NodeHasSufficientMemory" Aug 04 15:06:17.148051 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.147986 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-155.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 15:06:17.148051 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.148002 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-155.ec2.internal" event="NodeHasSufficientPID" Aug 04 15:06:17.149144 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.149131 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 15:06:17.149264 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.149251 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-155.ec2.internal" Aug 04 15:06:17.149321 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.149281 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 15:06:17.149910 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:17.149887 2574 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-132-155.ec2.internal\" not found" node="ip-10-0-132-155.ec2.internal" Aug 04 15:06:17.149910 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.149905 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-155.ec2.internal" event="NodeHasSufficientMemory" Aug 04 15:06:17.150021 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.149929 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-155.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 15:06:17.150021 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.149895 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-155.ec2.internal" event="NodeHasSufficientMemory" Aug 04 15:06:17.150021 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.149944 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-155.ec2.internal" event="NodeHasSufficientPID" Aug 04 15:06:17.150021 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.149956 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-155.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 15:06:17.150021 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.149970 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-155.ec2.internal" event="NodeHasSufficientPID" Aug 04 15:06:17.151148 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.151131 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-155.ec2.internal" Aug 04 15:06:17.151219 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.151162 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 15:06:17.151819 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.151802 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-155.ec2.internal" event="NodeHasSufficientMemory" Aug 04 15:06:17.151914 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.151831 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-155.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 15:06:17.151914 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.151848 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-155.ec2.internal" event="NodeHasSufficientPID" Aug 04 15:06:17.180988 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:17.180963 2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-155.ec2.internal\" not found" node="ip-10-0-132-155.ec2.internal" Aug 04 15:06:17.185326 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:17.185309 2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-155.ec2.internal\" not found" node="ip-10-0-132-155.ec2.internal" Aug 04 15:06:17.239785 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.239761 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b47c00155e76867b44c5cd48f4a5b502-config\") pod \"kube-apiserver-proxy-ip-10-0-132-155.ec2.internal\" (UID: \"b47c00155e76867b44c5cd48f4a5b502\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-155.ec2.internal" Aug 04 15:06:17.239854 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.239787 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7f45dc1fb51e13f5f7cb27b85338da6d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-155.ec2.internal\" (UID: \"7f45dc1fb51e13f5f7cb27b85338da6d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-155.ec2.internal" Aug 04 15:06:17.239854 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.239803 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f45dc1fb51e13f5f7cb27b85338da6d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-155.ec2.internal\" (UID: \"7f45dc1fb51e13f5f7cb27b85338da6d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-155.ec2.internal" Aug 04 15:06:17.315090 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.315066 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 15:06:17.315986 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.315970 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-155.ec2.internal" event="NodeHasSufficientMemory" Aug 04 15:06:17.316068 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.316006 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-155.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 15:06:17.316068 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.316015 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-155.ec2.internal" event="NodeHasSufficientPID" Aug 04 15:06:17.316068 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.316040 2574 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-155.ec2.internal" Aug 04 15:06:17.328157 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.328137 2574 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-155.ec2.internal" Aug 04 15:06:17.328157 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:17.328158 2574 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-155.ec2.internal\": node \"ip-10-0-132-155.ec2.internal\" not found" Aug 04 15:06:17.340235 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.340213 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f45dc1fb51e13f5f7cb27b85338da6d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-155.ec2.internal\" (UID: \"7f45dc1fb51e13f5f7cb27b85338da6d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-155.ec2.internal" Aug 04 15:06:17.340325 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.340246 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b47c00155e76867b44c5cd48f4a5b502-config\") pod \"kube-apiserver-proxy-ip-10-0-132-155.ec2.internal\" (UID: \"b47c00155e76867b44c5cd48f4a5b502\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-155.ec2.internal" Aug 04 15:06:17.340325 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.340269 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7f45dc1fb51e13f5f7cb27b85338da6d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-155.ec2.internal\" (UID: \"7f45dc1fb51e13f5f7cb27b85338da6d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-155.ec2.internal" Aug 04 15:06:17.340325 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.340307 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f45dc1fb51e13f5f7cb27b85338da6d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-155.ec2.internal\" (UID: \"7f45dc1fb51e13f5f7cb27b85338da6d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-155.ec2.internal" Aug 04 15:06:17.340451 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.340326 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b47c00155e76867b44c5cd48f4a5b502-config\") pod \"kube-apiserver-proxy-ip-10-0-132-155.ec2.internal\" (UID: \"b47c00155e76867b44c5cd48f4a5b502\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-155.ec2.internal" Aug 04 15:06:17.340451 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.340308 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7f45dc1fb51e13f5f7cb27b85338da6d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-155.ec2.internal\" (UID: \"7f45dc1fb51e13f5f7cb27b85338da6d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-155.ec2.internal" Aug 04 15:06:17.417496 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:17.417434 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-155.ec2.internal\" not found" Aug 04 15:06:17.483780 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.483757 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-155.ec2.internal" Aug 04 15:06:17.487897 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.487749 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-155.ec2.internal" Aug 04 15:06:17.518136 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:17.518109 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-155.ec2.internal\" not found" Aug 04 15:06:17.619127 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:17.619097 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-155.ec2.internal\" not found" Aug 04 15:06:17.723631 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:17.723550 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-155.ec2.internal\" not found" Aug 04 15:06:17.824024 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:17.823987 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-155.ec2.internal\" not found" Aug 04 15:06:17.875654 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:17.875611 2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Aug 04 15:06:17.924880 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:17.924828 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-155.ec2.internal\" not found" Aug 04 15:06:18.025105 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:18.025081 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-155.ec2.internal\" not found" Aug 04 15:06:18.030643 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:18.030615 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb47c00155e76867b44c5cd48f4a5b502.slice/crio-66ddcb7bb0086c2a707ec092284438d900ccff2ceb18ac8fe96738a34b096acd WatchSource:0}: Error finding container 66ddcb7bb0086c2a707ec092284438d900ccff2ceb18ac8fe96738a34b096acd: Status 404 returned error can't find the container with id 66ddcb7bb0086c2a707ec092284438d900ccff2ceb18ac8fe96738a34b096acd Aug 04 15:06:18.035657 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.035641 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 15:06:18.049940 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.049900 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-155.ec2.internal" event={"ID":"b47c00155e76867b44c5cd48f4a5b502","Type":"ContainerStarted","Data":"66ddcb7bb0086c2a707ec092284438d900ccff2ceb18ac8fe96738a34b096acd"} Aug 04 15:06:18.114814 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.114781 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Aug 04 15:06:18.115912 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.115895 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-155.ec2.internal" event="NodeHasSufficientMemory" Aug 04 15:06:18.116001 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.115930 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-155.ec2.internal" event="NodeHasNoDiskPressure" Aug 04 15:06:18.116001 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.115946 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-155.ec2.internal" event="NodeHasSufficientPID" Aug 04 15:06:18.117099 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.117075 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-08-03 15:01:17 +0000 UTC" deadline="2028-02-26 10:18:21.740400097 +0000 UTC" Aug 04 15:06:18.117154 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.117100 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13699h12m3.623303802s" Aug 04 15:06:18.125216 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:18.125197 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-155.ec2.internal\" not found" Aug 04 15:06:18.172122 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.172098 2574 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 15:06:18.193782 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.193763 2574 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 15:06:18.209383 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.209364 2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 15:06:18.226107 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:18.226088 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-155.ec2.internal\" not found" Aug 04 15:06:18.246094 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:18.246073 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f45dc1fb51e13f5f7cb27b85338da6d.slice/crio-b86c34c43473d62519e748590d898ef4ec8b2abc83996af001a30d555d7680c0 WatchSource:0}: Error finding container b86c34c43473d62519e748590d898ef4ec8b2abc83996af001a30d555d7680c0: Status 404 returned error can't find the container with id b86c34c43473d62519e748590d898ef4ec8b2abc83996af001a30d555d7680c0 Aug 04 15:06:18.326505 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:18.326444 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-155.ec2.internal\" not found" Aug 04 15:06:18.427014 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:18.426976 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-155.ec2.internal\" not found" Aug 04 15:06:18.460818 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.460789 2574 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Aug 04 15:06:18.538131 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.538104 2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-155.ec2.internal" Aug 04 15:06:18.557304 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.557280 2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Aug 04 15:06:18.558468 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.558450 2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-155.ec2.internal" Aug 04 15:06:18.576092 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.576069 2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Aug 04 15:06:18.917714 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.917658 2574 apiserver.go:52] "Watching apiserver" Aug 04 15:06:18.933658 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.932722 2574 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Aug 04 15:06:18.933658 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.933152 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-z8h6k","kube-system/global-pull-secret-syncer-xxmsv","kube-system/konnectivity-agent-xmh7f","openshift-cluster-node-tuning-operator/tuned-nlhv9","openshift-dns/node-resolver-xphfh","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-155.ec2.internal","openshift-multus/multus-additional-cni-plugins-4wbvq","openshift-multus/network-metrics-daemon-8w68v","openshift-network-operator/iptables-alerter-7qt6k","kube-system/kube-apiserver-proxy-ip-10-0-132-155.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c47dc","openshift-image-registry/node-ca-ql74b","openshift-multus/multus-f64s8","openshift-network-diagnostics/network-check-target-2vsg4"] Aug 04 15:06:18.934831 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.934801 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4wbvq" Aug 04 15:06:18.936486 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.936114 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-f64s8" Aug 04 15:06:18.937324 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.937287 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xmh7f" Aug 04 15:06:18.938279 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.938258 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:18.938750 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.938730 2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Aug 04 15:06:18.939340 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.939321 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xphfh" Aug 04 15:06:18.940525 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.940505 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vsg4" Aug 04 15:06:18.940620 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:18.940589 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vsg4" podUID="bf59cab4-36dc-431c-869d-896d50d50e7f" Aug 04 15:06:18.941616 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.941597 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7qt6k" Aug 04 15:06:18.942975 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.942955 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:18.944635 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.944615 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxmsv" Aug 04 15:06:18.944711 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:18.944688 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxmsv" podUID="46bd5ac9-b189-4e6c-b2c6-df0ed33641e2" Aug 04 15:06:18.945573 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.945556 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8w68v" Aug 04 15:06:18.945663 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:18.945641 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8w68v" podUID="1b91798e-f721-4056-bc09-6d4765fa6757" Aug 04 15:06:18.946264 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.946071 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-pcw5k\"" Aug 04 15:06:18.946264 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.946079 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Aug 04 15:06:18.946264 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.946137 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-95nvc\"" Aug 04 15:06:18.946264 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.946244 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Aug 04 15:06:18.946539 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.946479 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Aug 04 15:06:18.946539 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.946528 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Aug 04 15:06:18.946757 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.946732 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-m44fs\"" Aug 04 15:06:18.947116 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.947093 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Aug 04 15:06:18.947461 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.947441 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Aug 04 15:06:18.947718 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.947700 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8jdfj\"" Aug 04 15:06:18.948633 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.948611 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Aug 04 15:06:18.948852 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.948832 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Aug 04 15:06:18.949054 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.949035 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Aug 04 15:06:18.949278 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.949234 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Aug 04 15:06:18.949443 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.949421 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d3dd50da-aab3-4299-98b3-ad684ab46ef0-cni-binary-copy\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:18.949540 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.949450 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Aug 04 15:06:18.949540 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.949492 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-etc-kubernetes\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:18.949540 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.949531 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-etc-sysctl-d\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:18.949699 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.949543 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Aug 04 15:06:18.949699 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.949573 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-etc-sysctl-conf\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:18.949699 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.949628 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-sys\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:18.949699 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.949647 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Aug 04 15:06:18.949699 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.949664 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-tmp\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:18.949947 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.949708 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cb3de761-615b-44d1-84f3-c9a0edf4520e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4wbvq\" (UID: \"cb3de761-615b-44d1-84f3-c9a0edf4520e\") " pod="openshift-multus/multus-additional-cni-plugins-4wbvq" Aug 04 15:06:18.949947 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.949752 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cb3de761-615b-44d1-84f3-c9a0edf4520e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4wbvq\" (UID: \"cb3de761-615b-44d1-84f3-c9a0edf4520e\") " pod="openshift-multus/multus-additional-cni-plugins-4wbvq" Aug 04 15:06:18.949947 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.949778 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-cnibin\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:18.949947 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.949817 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-host-run-k8s-cni-cncf-io\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:18.949947 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.949846 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-host-var-lib-cni-multus\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:18.949947 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.949873 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-etc-modprobe-d\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:18.949947 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.949936 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cb3de761-615b-44d1-84f3-c9a0edf4520e-os-release\") pod \"multus-additional-cni-plugins-4wbvq\" (UID: \"cb3de761-615b-44d1-84f3-c9a0edf4520e\") " pod="openshift-multus/multus-additional-cni-plugins-4wbvq" Aug 04 15:06:18.950300 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.949959 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-multus-conf-dir\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:18.950300 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.950016 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d3dd50da-aab3-4299-98b3-ad684ab46ef0-multus-daemon-config\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:18.950300 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.950071 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-host-run-multus-certs\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:18.950300 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.950112 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qwk2\" (UniqueName: \"kubernetes.io/projected/d3dd50da-aab3-4299-98b3-ad684ab46ef0-kube-api-access-6qwk2\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:18.950300 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.950143 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-run\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:18.950300 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.950169 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-etc-tuned\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:18.950300 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.950208 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7ec6b008-bb9e-42e2-a233-a57bc7449804-hosts-file\") pod \"node-resolver-xphfh\" (UID: \"7ec6b008-bb9e-42e2-a233-a57bc7449804\") " pod="openshift-dns/node-resolver-xphfh" Aug 04 15:06:18.950300 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.950238 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-host-var-lib-cni-bin\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:18.950300 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.950262 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/214ec271-9cb1-4e3b-9b10-b7d15c10333f-agent-certs\") pod \"konnectivity-agent-xmh7f\" (UID: \"214ec271-9cb1-4e3b-9b10-b7d15c10333f\") " pod="kube-system/konnectivity-agent-xmh7f" Aug 04 15:06:18.950300 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.950286 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-etc-sysconfig\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:18.950763 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.950309 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-etc-systemd\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:18.950763 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.950360 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rfl2\" (UniqueName: \"kubernetes.io/projected/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-kube-api-access-4rfl2\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:18.950763 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.950385 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb3de761-615b-44d1-84f3-c9a0edf4520e-system-cni-dir\") pod \"multus-additional-cni-plugins-4wbvq\" (UID: \"cb3de761-615b-44d1-84f3-c9a0edf4520e\") " pod="openshift-multus/multus-additional-cni-plugins-4wbvq" Aug 04 15:06:18.950763 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.950455 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-system-cni-dir\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:18.950763 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.950481 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-os-release\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:18.950763 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.950510 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-host-run-netns\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:18.950763 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.950535 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-host-var-lib-kubelet\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:18.950763 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.950561 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ec6b008-bb9e-42e2-a233-a57bc7449804-tmp-dir\") pod \"node-resolver-xphfh\" (UID: \"7ec6b008-bb9e-42e2-a233-a57bc7449804\") " pod="openshift-dns/node-resolver-xphfh" Aug 04 15:06:18.950763 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.950622 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sptsc\" (UniqueName: \"kubernetes.io/projected/7ec6b008-bb9e-42e2-a233-a57bc7449804-kube-api-access-sptsc\") pod \"node-resolver-xphfh\" (UID: \"7ec6b008-bb9e-42e2-a233-a57bc7449804\") " pod="openshift-dns/node-resolver-xphfh" Aug 04 15:06:18.950763 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.950649 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cb3de761-615b-44d1-84f3-c9a0edf4520e-cni-binary-copy\") pod \"multus-additional-cni-plugins-4wbvq\" (UID: \"cb3de761-615b-44d1-84f3-c9a0edf4520e\") " pod="openshift-multus/multus-additional-cni-plugins-4wbvq" Aug 04 15:06:18.950763 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.950676 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wznxw\" (UniqueName: \"kubernetes.io/projected/cb3de761-615b-44d1-84f3-c9a0edf4520e-kube-api-access-wznxw\") pod \"multus-additional-cni-plugins-4wbvq\" (UID: \"cb3de761-615b-44d1-84f3-c9a0edf4520e\") " pod="openshift-multus/multus-additional-cni-plugins-4wbvq" Aug 04 15:06:18.950763 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.950700 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/214ec271-9cb1-4e3b-9b10-b7d15c10333f-konnectivity-ca\") pod \"konnectivity-agent-xmh7f\" (UID: \"214ec271-9cb1-4e3b-9b10-b7d15c10333f\") " pod="kube-system/konnectivity-agent-xmh7f" Aug 04 15:06:18.950763 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.950735 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-lib-modules\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:18.950763 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.950764 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-var-lib-kubelet\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:18.951423 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.950812 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-host\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:18.951423 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.950840 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cb3de761-615b-44d1-84f3-c9a0edf4520e-cnibin\") pod \"multus-additional-cni-plugins-4wbvq\" (UID: \"cb3de761-615b-44d1-84f3-c9a0edf4520e\") " pod="openshift-multus/multus-additional-cni-plugins-4wbvq" Aug 04 15:06:18.951423 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.950886 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cb3de761-615b-44d1-84f3-c9a0edf4520e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4wbvq\" (UID: \"cb3de761-615b-44d1-84f3-c9a0edf4520e\") " pod="openshift-multus/multus-additional-cni-plugins-4wbvq" Aug 04 15:06:18.951423 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.950911 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-multus-cni-dir\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:18.951423 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.950949 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-multus-socket-dir-parent\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:18.951423 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.950985 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-hostroot\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:18.951423 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.951017 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-etc-kubernetes\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:18.951423 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.951144 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Aug 04 15:06:18.951423 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.951184 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c47dc" Aug 04 15:06:18.951863 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.951597 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Aug 04 15:06:18.951863 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.951622 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Aug 04 15:06:18.951863 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.951693 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-bm984\"" Aug 04 15:06:18.951863 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.951715 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Aug 04 15:06:18.952051 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.951892 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-c2z6b\"" Aug 04 15:06:18.952402 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.952358 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ql74b" Aug 04 15:06:18.959324 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.955187 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Aug 04 15:06:18.959324 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.957854 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Aug 04 15:06:18.960256 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.960228 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-86xsr\"" Aug 04 15:06:18.960660 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.960639 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Aug 04 15:06:18.960737 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.960677 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Aug 04 15:06:18.960995 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.960974 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Aug 04 15:06:18.961078 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.961046 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Aug 04 15:06:18.963480 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.963433 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Aug 04 15:06:18.963996 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.963975 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Aug 04 15:06:18.964264 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.964222 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-q8hkz\"" Aug 04 15:06:18.964330 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.964224 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Aug 04 15:06:18.964503 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.964483 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-krpq7\"" Aug 04 15:06:18.964606 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.964589 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Aug 04 15:06:18.975107 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:18.974263 2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Aug 04 15:06:19.039536 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.039494 2574 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Aug 04 15:06:19.051165 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.051139 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cb3de761-615b-44d1-84f3-c9a0edf4520e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4wbvq\" (UID: \"cb3de761-615b-44d1-84f3-c9a0edf4520e\") " pod="openshift-multus/multus-additional-cni-plugins-4wbvq" Aug 04 15:06:19.051307 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.051174 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-host-run-k8s-cni-cncf-io\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:19.051307 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.051194 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-host-var-lib-cni-multus\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:19.051307 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.051215 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wk7xl\" (UniqueName: \"kubernetes.io/projected/93d48c95-3e8a-4cc4-a12c-21f1e93c4963-kube-api-access-wk7xl\") pod \"iptables-alerter-7qt6k\" (UID: \"93d48c95-3e8a-4cc4-a12c-21f1e93c4963\") " pod="openshift-network-operator/iptables-alerter-7qt6k" Aug 04 15:06:19.051307 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.051234 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-host-cni-netd\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.051307 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.051261 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b91798e-f721-4056-bc09-6d4765fa6757-metrics-certs\") pod \"network-metrics-daemon-8w68v\" (UID: \"1b91798e-f721-4056-bc09-6d4765fa6757\") " pod="openshift-multus/network-metrics-daemon-8w68v" Aug 04 15:06:19.051307 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.051283 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-host-run-multus-certs\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:19.051307 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.051288 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-host-run-k8s-cni-cncf-io\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:19.051307 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.051300 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qwk2\" (UniqueName: \"kubernetes.io/projected/d3dd50da-aab3-4299-98b3-ad684ab46ef0-kube-api-access-6qwk2\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:19.051755 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.051323 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.051755 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.051351 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/843c5666-b42e-4249-981a-4c91cd13173f-etc-selinux\") pod \"aws-ebs-csi-driver-node-c47dc\" (UID: \"843c5666-b42e-4249-981a-4c91cd13173f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c47dc" Aug 04 15:06:19.051755 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.051375 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-etc-tuned\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:19.051755 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.051400 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7ec6b008-bb9e-42e2-a233-a57bc7449804-hosts-file\") pod \"node-resolver-xphfh\" (UID: \"7ec6b008-bb9e-42e2-a233-a57bc7449804\") " pod="openshift-dns/node-resolver-xphfh" Aug 04 15:06:19.051755 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.051447 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/93d48c95-3e8a-4cc4-a12c-21f1e93c4963-iptables-alerter-script\") pod \"iptables-alerter-7qt6k\" (UID: \"93d48c95-3e8a-4cc4-a12c-21f1e93c4963\") " pod="openshift-network-operator/iptables-alerter-7qt6k" Aug 04 15:06:19.051755 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.051474 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-host-var-lib-cni-bin\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:19.051755 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.051460 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cb3de761-615b-44d1-84f3-c9a0edf4520e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4wbvq\" (UID: \"cb3de761-615b-44d1-84f3-c9a0edf4520e\") " pod="openshift-multus/multus-additional-cni-plugins-4wbvq" Aug 04 15:06:19.051755 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.051502 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/214ec271-9cb1-4e3b-9b10-b7d15c10333f-agent-certs\") pod \"konnectivity-agent-xmh7f\" (UID: \"214ec271-9cb1-4e3b-9b10-b7d15c10333f\") " pod="kube-system/konnectivity-agent-xmh7f" Aug 04 15:06:19.051755 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.051528 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rfl2\" (UniqueName: \"kubernetes.io/projected/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-kube-api-access-4rfl2\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:19.051755 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.051550 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-host-kubelet\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.051755 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.051578 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-run-systemd\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.051755 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.051608 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/843c5666-b42e-4249-981a-4c91cd13173f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c47dc\" (UID: \"843c5666-b42e-4249-981a-4c91cd13173f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c47dc" Aug 04 15:06:19.051755 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.051646 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/843c5666-b42e-4249-981a-4c91cd13173f-device-dir\") pod \"aws-ebs-csi-driver-node-c47dc\" (UID: \"843c5666-b42e-4249-981a-4c91cd13173f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c47dc" Aug 04 15:06:19.051755 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.051674 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb3de761-615b-44d1-84f3-c9a0edf4520e-system-cni-dir\") pod \"multus-additional-cni-plugins-4wbvq\" (UID: \"cb3de761-615b-44d1-84f3-c9a0edf4520e\") " pod="openshift-multus/multus-additional-cni-plugins-4wbvq" Aug 04 15:06:19.051755 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.051698 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ec6b008-bb9e-42e2-a233-a57bc7449804-tmp-dir\") pod \"node-resolver-xphfh\" (UID: \"7ec6b008-bb9e-42e2-a233-a57bc7449804\") " pod="openshift-dns/node-resolver-xphfh" Aug 04 15:06:19.051755 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.051729 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-var-lib-openvswitch\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.052622 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.051771 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/843c5666-b42e-4249-981a-4c91cd13173f-sys-fs\") pod \"aws-ebs-csi-driver-node-c47dc\" (UID: \"843c5666-b42e-4249-981a-4c91cd13173f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c47dc" Aug 04 15:06:19.052622 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.051795 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-lib-modules\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:19.052622 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.051823 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-host-var-lib-cni-multus\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:19.052622 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.051830 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-var-lib-kubelet\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:19.052622 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.051862 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/843c5666-b42e-4249-981a-4c91cd13173f-socket-dir\") pod \"aws-ebs-csi-driver-node-c47dc\" (UID: \"843c5666-b42e-4249-981a-4c91cd13173f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c47dc" Aug 04 15:06:19.052622 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.051888 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1e62b3d6-309b-4cf1-bb50-c1d15a0672b1-host\") pod \"node-ca-ql74b\" (UID: \"1e62b3d6-309b-4cf1-bb50-c1d15a0672b1\") " pod="openshift-image-registry/node-ca-ql74b" Aug 04 15:06:19.052622 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.051916 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cb3de761-615b-44d1-84f3-c9a0edf4520e-cnibin\") pod \"multus-additional-cni-plugins-4wbvq\" (UID: \"cb3de761-615b-44d1-84f3-c9a0edf4520e\") " pod="openshift-multus/multus-additional-cni-plugins-4wbvq" Aug 04 15:06:19.052622 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.051927 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-host-run-multus-certs\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:19.052622 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.051944 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cb3de761-615b-44d1-84f3-c9a0edf4520e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4wbvq\" (UID: \"cb3de761-615b-44d1-84f3-c9a0edf4520e\") " pod="openshift-multus/multus-additional-cni-plugins-4wbvq" Aug 04 15:06:19.052622 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.051970 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-hostroot\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:19.052622 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.051993 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-log-socket\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.052622 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.052032 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/46bd5ac9-b189-4e6c-b2c6-df0ed33641e2-kubelet-config\") pod \"global-pull-secret-syncer-xxmsv\" (UID: \"46bd5ac9-b189-4e6c-b2c6-df0ed33641e2\") " pod="kube-system/global-pull-secret-syncer-xxmsv" Aug 04 15:06:19.052622 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.052057 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-etc-kubernetes\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:19.052622 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.052082 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-tmp\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:19.052622 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.052132 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-host-run-netns\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.052622 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.052159 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-node-log\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.052622 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.052180 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wfc2\" (UniqueName: \"kubernetes.io/projected/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-kube-api-access-7wfc2\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.053407 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.052198 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1e62b3d6-309b-4cf1-bb50-c1d15a0672b1-serviceca\") pod \"node-ca-ql74b\" (UID: \"1e62b3d6-309b-4cf1-bb50-c1d15a0672b1\") " pod="openshift-image-registry/node-ca-ql74b" Aug 04 15:06:19.053407 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.052227 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cb3de761-615b-44d1-84f3-c9a0edf4520e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4wbvq\" (UID: \"cb3de761-615b-44d1-84f3-c9a0edf4520e\") " pod="openshift-multus/multus-additional-cni-plugins-4wbvq" Aug 04 15:06:19.053407 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.052259 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-cnibin\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:19.053407 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.052284 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-etc-modprobe-d\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:19.053407 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.052310 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-run-ovn\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.053407 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.052335 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cb3de761-615b-44d1-84f3-c9a0edf4520e-os-release\") pod \"multus-additional-cni-plugins-4wbvq\" (UID: \"cb3de761-615b-44d1-84f3-c9a0edf4520e\") " pod="openshift-multus/multus-additional-cni-plugins-4wbvq" Aug 04 15:06:19.053407 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.052362 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-multus-conf-dir\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:19.053407 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.052386 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d3dd50da-aab3-4299-98b3-ad684ab46ef0-multus-daemon-config\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:19.053407 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.052402 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-lib-modules\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:19.053407 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.052412 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-run\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:19.053407 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.052440 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-host-slash\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.053407 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.052482 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmqpz\" (UniqueName: \"kubernetes.io/projected/843c5666-b42e-4249-981a-4c91cd13173f-kube-api-access-vmqpz\") pod \"aws-ebs-csi-driver-node-c47dc\" (UID: \"843c5666-b42e-4249-981a-4c91cd13173f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c47dc" Aug 04 15:06:19.053407 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.052510 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vr87q\" (UniqueName: \"kubernetes.io/projected/1e62b3d6-309b-4cf1-bb50-c1d15a0672b1-kube-api-access-vr87q\") pod \"node-ca-ql74b\" (UID: \"1e62b3d6-309b-4cf1-bb50-c1d15a0672b1\") " pod="openshift-image-registry/node-ca-ql74b" Aug 04 15:06:19.053407 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.052550 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-systemd-units\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.053407 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.052574 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-etc-openvswitch\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.053407 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.052618 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-host-cni-bin\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.053407 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.052647 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46bd5ac9-b189-4e6c-b2c6-df0ed33641e2-original-pull-secret\") pod \"global-pull-secret-syncer-xxmsv\" (UID: \"46bd5ac9-b189-4e6c-b2c6-df0ed33641e2\") " pod="kube-system/global-pull-secret-syncer-xxmsv" Aug 04 15:06:19.054272 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.052678 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fr5dg\" (UniqueName: \"kubernetes.io/projected/1b91798e-f721-4056-bc09-6d4765fa6757-kube-api-access-fr5dg\") pod \"network-metrics-daemon-8w68v\" (UID: \"1b91798e-f721-4056-bc09-6d4765fa6757\") " pod="openshift-multus/network-metrics-daemon-8w68v" Aug 04 15:06:19.054272 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.052700 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-etc-sysconfig\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:19.054272 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.052718 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-etc-systemd\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:19.054272 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.052736 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-ovnkube-config\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.054272 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.052739 2574 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Aug 04 15:06:19.054272 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.052760 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-env-overrides\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.054272 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.052786 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-ovnkube-script-lib\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.054272 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.052814 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-system-cni-dir\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:19.054272 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.052836 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-os-release\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:19.054272 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.052860 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-host-run-netns\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:19.054272 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.052887 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-host-var-lib-kubelet\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:19.054272 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.052919 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sptsc\" (UniqueName: \"kubernetes.io/projected/7ec6b008-bb9e-42e2-a233-a57bc7449804-kube-api-access-sptsc\") pod \"node-resolver-xphfh\" (UID: \"7ec6b008-bb9e-42e2-a233-a57bc7449804\") " pod="openshift-dns/node-resolver-xphfh" Aug 04 15:06:19.054272 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.052947 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-host-run-ovn-kubernetes\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.054272 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.052978 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cb3de761-615b-44d1-84f3-c9a0edf4520e-cni-binary-copy\") pod \"multus-additional-cni-plugins-4wbvq\" (UID: \"cb3de761-615b-44d1-84f3-c9a0edf4520e\") " pod="openshift-multus/multus-additional-cni-plugins-4wbvq" Aug 04 15:06:19.054272 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.053013 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wznxw\" (UniqueName: \"kubernetes.io/projected/cb3de761-615b-44d1-84f3-c9a0edf4520e-kube-api-access-wznxw\") pod \"multus-additional-cni-plugins-4wbvq\" (UID: \"cb3de761-615b-44d1-84f3-c9a0edf4520e\") " pod="openshift-multus/multus-additional-cni-plugins-4wbvq" Aug 04 15:06:19.054272 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.053040 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/214ec271-9cb1-4e3b-9b10-b7d15c10333f-konnectivity-ca\") pod \"konnectivity-agent-xmh7f\" (UID: \"214ec271-9cb1-4e3b-9b10-b7d15c10333f\") " pod="kube-system/konnectivity-agent-xmh7f" Aug 04 15:06:19.054272 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.053066 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-host\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:19.055172 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.053094 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/93d48c95-3e8a-4cc4-a12c-21f1e93c4963-host-slash\") pod \"iptables-alerter-7qt6k\" (UID: \"93d48c95-3e8a-4cc4-a12c-21f1e93c4963\") " pod="openshift-network-operator/iptables-alerter-7qt6k" Aug 04 15:06:19.055172 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.053122 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-run-openvswitch\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.055172 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.053148 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-ovn-node-metrics-cert\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.055172 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.053179 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-multus-cni-dir\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:19.055172 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.053207 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-multus-socket-dir-parent\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:19.055172 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.053236 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-etc-kubernetes\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:19.055172 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.053277 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/843c5666-b42e-4249-981a-4c91cd13173f-registration-dir\") pod \"aws-ebs-csi-driver-node-c47dc\" (UID: \"843c5666-b42e-4249-981a-4c91cd13173f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c47dc" Aug 04 15:06:19.055172 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.053310 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d3dd50da-aab3-4299-98b3-ad684ab46ef0-cni-binary-copy\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:19.055172 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.053337 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-etc-sysctl-d\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:19.055172 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.053366 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-etc-sysctl-conf\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:19.055172 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.053395 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-sys\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:19.055172 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.053424 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/46bd5ac9-b189-4e6c-b2c6-df0ed33641e2-dbus\") pod \"global-pull-secret-syncer-xxmsv\" (UID: \"46bd5ac9-b189-4e6c-b2c6-df0ed33641e2\") " pod="kube-system/global-pull-secret-syncer-xxmsv" Aug 04 15:06:19.055172 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.053452 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vqcn\" (UniqueName: \"kubernetes.io/projected/bf59cab4-36dc-431c-869d-896d50d50e7f-kube-api-access-4vqcn\") pod \"network-check-target-2vsg4\" (UID: \"bf59cab4-36dc-431c-869d-896d50d50e7f\") " pod="openshift-network-diagnostics/network-check-target-2vsg4" Aug 04 15:06:19.055172 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.053585 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-var-lib-kubelet\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:19.055172 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.053659 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cb3de761-615b-44d1-84f3-c9a0edf4520e-cnibin\") pod \"multus-additional-cni-plugins-4wbvq\" (UID: \"cb3de761-615b-44d1-84f3-c9a0edf4520e\") " pod="openshift-multus/multus-additional-cni-plugins-4wbvq" Aug 04 15:06:19.055172 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.053739 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-host-var-lib-cni-bin\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:19.055172 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.054268 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cb3de761-615b-44d1-84f3-c9a0edf4520e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4wbvq\" (UID: \"cb3de761-615b-44d1-84f3-c9a0edf4520e\") " pod="openshift-multus/multus-additional-cni-plugins-4wbvq" Aug 04 15:06:19.056124 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.054364 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7ec6b008-bb9e-42e2-a233-a57bc7449804-hosts-file\") pod \"node-resolver-xphfh\" (UID: \"7ec6b008-bb9e-42e2-a233-a57bc7449804\") " pod="openshift-dns/node-resolver-xphfh" Aug 04 15:06:19.056124 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.054414 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-hostroot\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:19.056124 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.054497 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-etc-kubernetes\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:19.056124 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.054709 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-os-release\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:19.056124 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.055208 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cb3de761-615b-44d1-84f3-c9a0edf4520e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4wbvq\" (UID: \"cb3de761-615b-44d1-84f3-c9a0edf4520e\") " pod="openshift-multus/multus-additional-cni-plugins-4wbvq" Aug 04 15:06:19.056124 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.055237 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-etc-kubernetes\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:19.056124 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.055265 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-host-run-netns\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:19.056124 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.055298 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-host-var-lib-kubelet\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:19.056124 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.055778 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d3dd50da-aab3-4299-98b3-ad684ab46ef0-cni-binary-copy\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:19.056124 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.055903 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-etc-sysctl-d\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:19.056124 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.055951 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cb3de761-615b-44d1-84f3-c9a0edf4520e-cni-binary-copy\") pod \"multus-additional-cni-plugins-4wbvq\" (UID: \"cb3de761-615b-44d1-84f3-c9a0edf4520e\") " pod="openshift-multus/multus-additional-cni-plugins-4wbvq" Aug 04 15:06:19.056124 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.056016 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-etc-sysctl-conf\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:19.056124 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.056057 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-sys\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:19.056124 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.056113 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-etc-sysconfig\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:19.056826 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.056165 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-etc-systemd\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:19.056826 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.056209 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb3de761-615b-44d1-84f3-c9a0edf4520e-system-cni-dir\") pod \"multus-additional-cni-plugins-4wbvq\" (UID: \"cb3de761-615b-44d1-84f3-c9a0edf4520e\") " pod="openshift-multus/multus-additional-cni-plugins-4wbvq" Aug 04 15:06:19.056826 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.056216 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-host\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:19.056826 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.056318 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-multus-cni-dir\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:19.056826 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.056365 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-multus-socket-dir-parent\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:19.056826 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.056408 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-cnibin\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:19.056826 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.056500 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-etc-modprobe-d\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:19.056826 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.056507 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ec6b008-bb9e-42e2-a233-a57bc7449804-tmp-dir\") pod \"node-resolver-xphfh\" (UID: \"7ec6b008-bb9e-42e2-a233-a57bc7449804\") " pod="openshift-dns/node-resolver-xphfh" Aug 04 15:06:19.056826 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.056558 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cb3de761-615b-44d1-84f3-c9a0edf4520e-os-release\") pod \"multus-additional-cni-plugins-4wbvq\" (UID: \"cb3de761-615b-44d1-84f3-c9a0edf4520e\") " pod="openshift-multus/multus-additional-cni-plugins-4wbvq" Aug 04 15:06:19.056826 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.056568 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-system-cni-dir\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:19.056826 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.056592 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d3dd50da-aab3-4299-98b3-ad684ab46ef0-multus-conf-dir\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:19.056826 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.056750 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-run\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:19.056826 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.056827 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-etc-tuned\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:19.057536 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.057036 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d3dd50da-aab3-4299-98b3-ad684ab46ef0-multus-daemon-config\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:19.057536 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.057355 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/214ec271-9cb1-4e3b-9b10-b7d15c10333f-konnectivity-ca\") pod \"konnectivity-agent-xmh7f\" (UID: \"214ec271-9cb1-4e3b-9b10-b7d15c10333f\") " pod="kube-system/konnectivity-agent-xmh7f" Aug 04 15:06:19.057536 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.057427 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-155.ec2.internal" event={"ID":"7f45dc1fb51e13f5f7cb27b85338da6d","Type":"ContainerStarted","Data":"b86c34c43473d62519e748590d898ef4ec8b2abc83996af001a30d555d7680c0"} Aug 04 15:06:19.059591 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.059507 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/214ec271-9cb1-4e3b-9b10-b7d15c10333f-agent-certs\") pod \"konnectivity-agent-xmh7f\" (UID: \"214ec271-9cb1-4e3b-9b10-b7d15c10333f\") " pod="kube-system/konnectivity-agent-xmh7f" Aug 04 15:06:19.060574 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.060553 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-tmp\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:19.075015 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.074991 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wznxw\" (UniqueName: \"kubernetes.io/projected/cb3de761-615b-44d1-84f3-c9a0edf4520e-kube-api-access-wznxw\") pod \"multus-additional-cni-plugins-4wbvq\" (UID: \"cb3de761-615b-44d1-84f3-c9a0edf4520e\") " pod="openshift-multus/multus-additional-cni-plugins-4wbvq" Aug 04 15:06:19.077205 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.077116 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rfl2\" (UniqueName: \"kubernetes.io/projected/5bb7f9d7-3a00-48b8-8538-7ab88dec74cc-kube-api-access-4rfl2\") pod \"tuned-nlhv9\" (UID: \"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc\") " pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:19.079004 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.078892 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sptsc\" (UniqueName: \"kubernetes.io/projected/7ec6b008-bb9e-42e2-a233-a57bc7449804-kube-api-access-sptsc\") pod \"node-resolver-xphfh\" (UID: \"7ec6b008-bb9e-42e2-a233-a57bc7449804\") " pod="openshift-dns/node-resolver-xphfh" Aug 04 15:06:19.080846 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.080820 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qwk2\" (UniqueName: \"kubernetes.io/projected/d3dd50da-aab3-4299-98b3-ad684ab46ef0-kube-api-access-6qwk2\") pod \"multus-f64s8\" (UID: \"d3dd50da-aab3-4299-98b3-ad684ab46ef0\") " pod="openshift-multus/multus-f64s8" Aug 04 15:06:19.110517 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.110476 2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-jr7jg" Aug 04 15:06:19.131737 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.131710 2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-jr7jg" Aug 04 15:06:19.154235 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.154200 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-run-ovn\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.154356 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.154240 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-host-slash\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.154356 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.154285 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmqpz\" (UniqueName: \"kubernetes.io/projected/843c5666-b42e-4249-981a-4c91cd13173f-kube-api-access-vmqpz\") pod \"aws-ebs-csi-driver-node-c47dc\" (UID: \"843c5666-b42e-4249-981a-4c91cd13173f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c47dc" Aug 04 15:06:19.154356 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.154324 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vr87q\" (UniqueName: \"kubernetes.io/projected/1e62b3d6-309b-4cf1-bb50-c1d15a0672b1-kube-api-access-vr87q\") pod \"node-ca-ql74b\" (UID: \"1e62b3d6-309b-4cf1-bb50-c1d15a0672b1\") " pod="openshift-image-registry/node-ca-ql74b" Aug 04 15:06:19.154356 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.154327 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-run-ovn\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.154356 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.154349 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-systemd-units\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.154705 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.154373 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-etc-openvswitch\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.154705 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.154398 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-host-cni-bin\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.154705 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.154424 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46bd5ac9-b189-4e6c-b2c6-df0ed33641e2-original-pull-secret\") pod \"global-pull-secret-syncer-xxmsv\" (UID: \"46bd5ac9-b189-4e6c-b2c6-df0ed33641e2\") " pod="kube-system/global-pull-secret-syncer-xxmsv" Aug 04 15:06:19.154705 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.154465 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fr5dg\" (UniqueName: \"kubernetes.io/projected/1b91798e-f721-4056-bc09-6d4765fa6757-kube-api-access-fr5dg\") pod \"network-metrics-daemon-8w68v\" (UID: \"1b91798e-f721-4056-bc09-6d4765fa6757\") " pod="openshift-multus/network-metrics-daemon-8w68v" Aug 04 15:06:19.154705 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.154514 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-ovnkube-config\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.154705 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.154538 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-env-overrides\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.154705 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.154562 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-ovnkube-script-lib\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.154705 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.154591 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-host-run-ovn-kubernetes\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.154705 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.154627 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/93d48c95-3e8a-4cc4-a12c-21f1e93c4963-host-slash\") pod \"iptables-alerter-7qt6k\" (UID: \"93d48c95-3e8a-4cc4-a12c-21f1e93c4963\") " pod="openshift-network-operator/iptables-alerter-7qt6k" Aug 04 15:06:19.154705 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.154673 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-host-slash\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.154705 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.154678 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-run-openvswitch\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.155150 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.155077 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-systemd-units\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.155150 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.155129 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-etc-openvswitch\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.155249 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.155171 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-host-cni-bin\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.155310 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:19.155254 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 15:06:19.155310 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.155265 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-env-overrides\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.155374 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:19.155319 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46bd5ac9-b189-4e6c-b2c6-df0ed33641e2-original-pull-secret podName:46bd5ac9-b189-4e6c-b2c6-df0ed33641e2 nodeName:}" failed. No retries permitted until 2026-08-04 15:06:19.655296424 +0000 UTC m=+3.115168771 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/46bd5ac9-b189-4e6c-b2c6-df0ed33641e2-original-pull-secret") pod "global-pull-secret-syncer-xxmsv" (UID: "46bd5ac9-b189-4e6c-b2c6-df0ed33641e2") : object "kube-system"/"original-pull-secret" not registered Aug 04 15:06:19.155374 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.155350 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-ovn-node-metrics-cert\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.155511 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.155395 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/843c5666-b42e-4249-981a-4c91cd13173f-registration-dir\") pod \"aws-ebs-csi-driver-node-c47dc\" (UID: \"843c5666-b42e-4249-981a-4c91cd13173f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c47dc" Aug 04 15:06:19.155511 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.155424 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/46bd5ac9-b189-4e6c-b2c6-df0ed33641e2-dbus\") pod \"global-pull-secret-syncer-xxmsv\" (UID: \"46bd5ac9-b189-4e6c-b2c6-df0ed33641e2\") " pod="kube-system/global-pull-secret-syncer-xxmsv" Aug 04 15:06:19.155511 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.155450 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vqcn\" (UniqueName: \"kubernetes.io/projected/bf59cab4-36dc-431c-869d-896d50d50e7f-kube-api-access-4vqcn\") pod \"network-check-target-2vsg4\" (UID: \"bf59cab4-36dc-431c-869d-896d50d50e7f\") " pod="openshift-network-diagnostics/network-check-target-2vsg4" Aug 04 15:06:19.155511 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.155493 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wk7xl\" (UniqueName: \"kubernetes.io/projected/93d48c95-3e8a-4cc4-a12c-21f1e93c4963-kube-api-access-wk7xl\") pod \"iptables-alerter-7qt6k\" (UID: \"93d48c95-3e8a-4cc4-a12c-21f1e93c4963\") " pod="openshift-network-operator/iptables-alerter-7qt6k" Aug 04 15:06:19.155705 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.155519 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-host-cni-netd\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.155705 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.155545 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b91798e-f721-4056-bc09-6d4765fa6757-metrics-certs\") pod \"network-metrics-daemon-8w68v\" (UID: \"1b91798e-f721-4056-bc09-6d4765fa6757\") " pod="openshift-multus/network-metrics-daemon-8w68v" Aug 04 15:06:19.155705 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.155575 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.155705 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.155609 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/843c5666-b42e-4249-981a-4c91cd13173f-etc-selinux\") pod \"aws-ebs-csi-driver-node-c47dc\" (UID: \"843c5666-b42e-4249-981a-4c91cd13173f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c47dc" Aug 04 15:06:19.155705 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.155641 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/93d48c95-3e8a-4cc4-a12c-21f1e93c4963-iptables-alerter-script\") pod \"iptables-alerter-7qt6k\" (UID: \"93d48c95-3e8a-4cc4-a12c-21f1e93c4963\") " pod="openshift-network-operator/iptables-alerter-7qt6k" Aug 04 15:06:19.155705 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.155673 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-host-kubelet\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.155705 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.155699 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-run-systemd\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.155998 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.155726 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/843c5666-b42e-4249-981a-4c91cd13173f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c47dc\" (UID: \"843c5666-b42e-4249-981a-4c91cd13173f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c47dc" Aug 04 15:06:19.155998 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.155752 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/843c5666-b42e-4249-981a-4c91cd13173f-device-dir\") pod \"aws-ebs-csi-driver-node-c47dc\" (UID: \"843c5666-b42e-4249-981a-4c91cd13173f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c47dc" Aug 04 15:06:19.155998 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.155780 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-var-lib-openvswitch\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.155998 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.155809 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/843c5666-b42e-4249-981a-4c91cd13173f-sys-fs\") pod \"aws-ebs-csi-driver-node-c47dc\" (UID: \"843c5666-b42e-4249-981a-4c91cd13173f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c47dc" Aug 04 15:06:19.155998 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.155838 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/843c5666-b42e-4249-981a-4c91cd13173f-socket-dir\") pod \"aws-ebs-csi-driver-node-c47dc\" (UID: \"843c5666-b42e-4249-981a-4c91cd13173f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c47dc" Aug 04 15:06:19.155998 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.155867 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1e62b3d6-309b-4cf1-bb50-c1d15a0672b1-host\") pod \"node-ca-ql74b\" (UID: \"1e62b3d6-309b-4cf1-bb50-c1d15a0672b1\") " pod="openshift-image-registry/node-ca-ql74b" Aug 04 15:06:19.155998 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.155863 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-ovnkube-script-lib\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.155998 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.155899 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-log-socket\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.155998 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.155944 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-host-run-ovn-kubernetes\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.155998 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.155955 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-log-socket\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.156486 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.156082 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/843c5666-b42e-4249-981a-4c91cd13173f-etc-selinux\") pod \"aws-ebs-csi-driver-node-c47dc\" (UID: \"843c5666-b42e-4249-981a-4c91cd13173f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c47dc" Aug 04 15:06:19.156538 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.156496 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-ovnkube-config\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.156587 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.156565 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-run-openvswitch\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.156626 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.156600 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/46bd5ac9-b189-4e6c-b2c6-df0ed33641e2-kubelet-config\") pod \"global-pull-secret-syncer-xxmsv\" (UID: \"46bd5ac9-b189-4e6c-b2c6-df0ed33641e2\") " pod="kube-system/global-pull-secret-syncer-xxmsv" Aug 04 15:06:19.156663 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.156630 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-host-run-netns\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.156663 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.156655 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-node-log\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.156759 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.156683 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wfc2\" (UniqueName: \"kubernetes.io/projected/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-kube-api-access-7wfc2\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.156759 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.156710 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1e62b3d6-309b-4cf1-bb50-c1d15a0672b1-serviceca\") pod \"node-ca-ql74b\" (UID: \"1e62b3d6-309b-4cf1-bb50-c1d15a0672b1\") " pod="openshift-image-registry/node-ca-ql74b" Aug 04 15:06:19.157219 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.157165 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1e62b3d6-309b-4cf1-bb50-c1d15a0672b1-serviceca\") pod \"node-ca-ql74b\" (UID: \"1e62b3d6-309b-4cf1-bb50-c1d15a0672b1\") " pod="openshift-image-registry/node-ca-ql74b" Aug 04 15:06:19.157722 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.157698 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/93d48c95-3e8a-4cc4-a12c-21f1e93c4963-iptables-alerter-script\") pod \"iptables-alerter-7qt6k\" (UID: \"93d48c95-3e8a-4cc4-a12c-21f1e93c4963\") " pod="openshift-network-operator/iptables-alerter-7qt6k" Aug 04 15:06:19.157841 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.157764 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-host-kubelet\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.157841 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.157815 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-run-systemd\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.157968 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.157868 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/843c5666-b42e-4249-981a-4c91cd13173f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c47dc\" (UID: \"843c5666-b42e-4249-981a-4c91cd13173f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c47dc" Aug 04 15:06:19.157968 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.157900 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/843c5666-b42e-4249-981a-4c91cd13173f-registration-dir\") pod \"aws-ebs-csi-driver-node-c47dc\" (UID: \"843c5666-b42e-4249-981a-4c91cd13173f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c47dc" Aug 04 15:06:19.157968 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.157919 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/843c5666-b42e-4249-981a-4c91cd13173f-device-dir\") pod \"aws-ebs-csi-driver-node-c47dc\" (UID: \"843c5666-b42e-4249-981a-4c91cd13173f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c47dc" Aug 04 15:06:19.158119 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.157965 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-var-lib-openvswitch\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.158119 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.158020 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/843c5666-b42e-4249-981a-4c91cd13173f-sys-fs\") pod \"aws-ebs-csi-driver-node-c47dc\" (UID: \"843c5666-b42e-4249-981a-4c91cd13173f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c47dc" Aug 04 15:06:19.158119 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.158049 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/46bd5ac9-b189-4e6c-b2c6-df0ed33641e2-dbus\") pod \"global-pull-secret-syncer-xxmsv\" (UID: \"46bd5ac9-b189-4e6c-b2c6-df0ed33641e2\") " pod="kube-system/global-pull-secret-syncer-xxmsv" Aug 04 15:06:19.158295 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.158121 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/843c5666-b42e-4249-981a-4c91cd13173f-socket-dir\") pod \"aws-ebs-csi-driver-node-c47dc\" (UID: \"843c5666-b42e-4249-981a-4c91cd13173f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c47dc" Aug 04 15:06:19.158295 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.158171 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1e62b3d6-309b-4cf1-bb50-c1d15a0672b1-host\") pod \"node-ca-ql74b\" (UID: \"1e62b3d6-309b-4cf1-bb50-c1d15a0672b1\") " pod="openshift-image-registry/node-ca-ql74b" Aug 04 15:06:19.158295 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.158213 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/93d48c95-3e8a-4cc4-a12c-21f1e93c4963-host-slash\") pod \"iptables-alerter-7qt6k\" (UID: \"93d48c95-3e8a-4cc4-a12c-21f1e93c4963\") " pod="openshift-network-operator/iptables-alerter-7qt6k" Aug 04 15:06:19.158295 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.158258 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/46bd5ac9-b189-4e6c-b2c6-df0ed33641e2-kubelet-config\") pod \"global-pull-secret-syncer-xxmsv\" (UID: \"46bd5ac9-b189-4e6c-b2c6-df0ed33641e2\") " pod="kube-system/global-pull-secret-syncer-xxmsv" Aug 04 15:06:19.158529 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.158303 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-node-log\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.158529 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:19.158311 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 15:06:19.158529 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:19.158371 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b91798e-f721-4056-bc09-6d4765fa6757-metrics-certs podName:1b91798e-f721-4056-bc09-6d4765fa6757 nodeName:}" failed. No retries permitted until 2026-08-04 15:06:19.658352622 +0000 UTC m=+3.118224951 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1b91798e-f721-4056-bc09-6d4765fa6757-metrics-certs") pod "network-metrics-daemon-8w68v" (UID: "1b91798e-f721-4056-bc09-6d4765fa6757") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 15:06:19.158529 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.158411 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-host-cni-netd\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.158529 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.158454 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-host-run-netns\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.158529 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.158510 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.159412 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.159393 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-ovn-node-metrics-cert\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.192872 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.192800 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmqpz\" (UniqueName: \"kubernetes.io/projected/843c5666-b42e-4249-981a-4c91cd13173f-kube-api-access-vmqpz\") pod \"aws-ebs-csi-driver-node-c47dc\" (UID: \"843c5666-b42e-4249-981a-4c91cd13173f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c47dc" Aug 04 15:06:19.192872 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.192828 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wk7xl\" (UniqueName: \"kubernetes.io/projected/93d48c95-3e8a-4cc4-a12c-21f1e93c4963-kube-api-access-wk7xl\") pod \"iptables-alerter-7qt6k\" (UID: \"93d48c95-3e8a-4cc4-a12c-21f1e93c4963\") " pod="openshift-network-operator/iptables-alerter-7qt6k" Aug 04 15:06:19.194811 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:19.194788 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 15:06:19.194910 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:19.194818 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 15:06:19.194910 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:19.194837 2574 projected.go:196] Error preparing data for projected volume kube-api-access-4vqcn for pod openshift-network-diagnostics/network-check-target-2vsg4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 15:06:19.194910 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:19.194902 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bf59cab4-36dc-431c-869d-896d50d50e7f-kube-api-access-4vqcn podName:bf59cab4-36dc-431c-869d-896d50d50e7f nodeName:}" failed. No retries permitted until 2026-08-04 15:06:19.694883747 +0000 UTC m=+3.154756080 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4vqcn" (UniqueName: "kubernetes.io/projected/bf59cab4-36dc-431c-869d-896d50d50e7f-kube-api-access-4vqcn") pod "network-check-target-2vsg4" (UID: "bf59cab4-36dc-431c-869d-896d50d50e7f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 15:06:19.195269 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.195228 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fr5dg\" (UniqueName: \"kubernetes.io/projected/1b91798e-f721-4056-bc09-6d4765fa6757-kube-api-access-fr5dg\") pod \"network-metrics-daemon-8w68v\" (UID: \"1b91798e-f721-4056-bc09-6d4765fa6757\") " pod="openshift-multus/network-metrics-daemon-8w68v" Aug 04 15:06:19.195938 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.195914 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vr87q\" (UniqueName: \"kubernetes.io/projected/1e62b3d6-309b-4cf1-bb50-c1d15a0672b1-kube-api-access-vr87q\") pod \"node-ca-ql74b\" (UID: \"1e62b3d6-309b-4cf1-bb50-c1d15a0672b1\") " pod="openshift-image-registry/node-ca-ql74b" Aug 04 15:06:19.197970 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.197946 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wfc2\" (UniqueName: \"kubernetes.io/projected/9c62e3b1-c2dc-4b2a-a61d-7d80049f355f-kube-api-access-7wfc2\") pod \"ovnkube-node-z8h6k\" (UID: \"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f\") " pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.248487 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.248418 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4wbvq" Aug 04 15:06:19.259418 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:19.259386 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb3de761_615b_44d1_84f3_c9a0edf4520e.slice/crio-12af8c241f28319d6913a5269dc9df1a45c33d4ea1cdd2971b9f2604e9c28426 WatchSource:0}: Error finding container 12af8c241f28319d6913a5269dc9df1a45c33d4ea1cdd2971b9f2604e9c28426: Status 404 returned error can't find the container with id 12af8c241f28319d6913a5269dc9df1a45c33d4ea1cdd2971b9f2604e9c28426 Aug 04 15:06:19.263734 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.263705 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-f64s8" Aug 04 15:06:19.272051 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.272005 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xmh7f" Aug 04 15:06:19.273402 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:19.273377 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3dd50da_aab3_4299_98b3_ad684ab46ef0.slice/crio-3f19fa68e69b2e595f18e22cf949e9c800654117bec03e1be4e26480b5df8103 WatchSource:0}: Error finding container 3f19fa68e69b2e595f18e22cf949e9c800654117bec03e1be4e26480b5df8103: Status 404 returned error can't find the container with id 3f19fa68e69b2e595f18e22cf949e9c800654117bec03e1be4e26480b5df8103 Aug 04 15:06:19.279201 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.279110 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" Aug 04 15:06:19.282836 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:19.282809 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod214ec271_9cb1_4e3b_9b10_b7d15c10333f.slice/crio-13effdbfde1f34985e158aff62f2de31abfe4e5afd26bfcc9dd0cf20d962773e WatchSource:0}: Error finding container 13effdbfde1f34985e158aff62f2de31abfe4e5afd26bfcc9dd0cf20d962773e: Status 404 returned error can't find the container with id 13effdbfde1f34985e158aff62f2de31abfe4e5afd26bfcc9dd0cf20d962773e Aug 04 15:06:19.287970 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:19.287950 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bb7f9d7_3a00_48b8_8538_7ab88dec74cc.slice/crio-2b0ec5962d007a38baf60a9c116bbcc9acaddb45ad3b17cd548377b47bfe53b7 WatchSource:0}: Error finding container 2b0ec5962d007a38baf60a9c116bbcc9acaddb45ad3b17cd548377b47bfe53b7: Status 404 returned error can't find the container with id 2b0ec5962d007a38baf60a9c116bbcc9acaddb45ad3b17cd548377b47bfe53b7 Aug 04 15:06:19.288913 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.288890 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xphfh" Aug 04 15:06:19.298890 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:19.298867 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ec6b008_bb9e_42e2_a233_a57bc7449804.slice/crio-daa7868807329c6738cb9e1d7bed12948a163d69a7d6bf4e72fa7674dd7e64d8 WatchSource:0}: Error finding container daa7868807329c6738cb9e1d7bed12948a163d69a7d6bf4e72fa7674dd7e64d8: Status 404 returned error can't find the container with id daa7868807329c6738cb9e1d7bed12948a163d69a7d6bf4e72fa7674dd7e64d8 Aug 04 15:06:19.299793 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.299768 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7qt6k" Aug 04 15:06:19.308246 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.308225 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:19.308491 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:19.308467 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93d48c95_3e8a_4cc4_a12c_21f1e93c4963.slice/crio-73759312444d3c0737a7d8fe3417e950c9035d9712a7dcb8a331bf263842f394 WatchSource:0}: Error finding container 73759312444d3c0737a7d8fe3417e950c9035d9712a7dcb8a331bf263842f394: Status 404 returned error can't find the container with id 73759312444d3c0737a7d8fe3417e950c9035d9712a7dcb8a331bf263842f394 Aug 04 15:06:19.317673 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.317649 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c47dc" Aug 04 15:06:19.325788 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.325765 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ql74b" Aug 04 15:06:19.660553 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.660484 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46bd5ac9-b189-4e6c-b2c6-df0ed33641e2-original-pull-secret\") pod \"global-pull-secret-syncer-xxmsv\" (UID: \"46bd5ac9-b189-4e6c-b2c6-df0ed33641e2\") " pod="kube-system/global-pull-secret-syncer-xxmsv" Aug 04 15:06:19.660553 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.660533 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b91798e-f721-4056-bc09-6d4765fa6757-metrics-certs\") pod \"network-metrics-daemon-8w68v\" (UID: \"1b91798e-f721-4056-bc09-6d4765fa6757\") " pod="openshift-multus/network-metrics-daemon-8w68v" Aug 04 15:06:19.660754 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:19.660634 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 15:06:19.660754 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:19.660636 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 15:06:19.660754 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:19.660685 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b91798e-f721-4056-bc09-6d4765fa6757-metrics-certs podName:1b91798e-f721-4056-bc09-6d4765fa6757 nodeName:}" failed. No retries permitted until 2026-08-04 15:06:20.660671375 +0000 UTC m=+4.120543704 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1b91798e-f721-4056-bc09-6d4765fa6757-metrics-certs") pod "network-metrics-daemon-8w68v" (UID: "1b91798e-f721-4056-bc09-6d4765fa6757") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 15:06:19.660754 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:19.660711 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46bd5ac9-b189-4e6c-b2c6-df0ed33641e2-original-pull-secret podName:46bd5ac9-b189-4e6c-b2c6-df0ed33641e2 nodeName:}" failed. No retries permitted until 2026-08-04 15:06:20.660692163 +0000 UTC m=+4.120564492 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/46bd5ac9-b189-4e6c-b2c6-df0ed33641e2-original-pull-secret") pod "global-pull-secret-syncer-xxmsv" (UID: "46bd5ac9-b189-4e6c-b2c6-df0ed33641e2") : object "kube-system"/"original-pull-secret" not registered Aug 04 15:06:19.761541 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:19.761507 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vqcn\" (UniqueName: \"kubernetes.io/projected/bf59cab4-36dc-431c-869d-896d50d50e7f-kube-api-access-4vqcn\") pod \"network-check-target-2vsg4\" (UID: \"bf59cab4-36dc-431c-869d-896d50d50e7f\") " pod="openshift-network-diagnostics/network-check-target-2vsg4" Aug 04 15:06:19.761735 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:19.761700 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 15:06:19.761735 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:19.761720 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 15:06:19.761735 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:19.761732 2574 projected.go:196] Error preparing data for projected volume kube-api-access-4vqcn for pod openshift-network-diagnostics/network-check-target-2vsg4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 15:06:19.761854 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:19.761791 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bf59cab4-36dc-431c-869d-896d50d50e7f-kube-api-access-4vqcn podName:bf59cab4-36dc-431c-869d-896d50d50e7f nodeName:}" failed. No retries permitted until 2026-08-04 15:06:20.761771562 +0000 UTC m=+4.221643893 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4vqcn" (UniqueName: "kubernetes.io/projected/bf59cab4-36dc-431c-869d-896d50d50e7f-kube-api-access-4vqcn") pod "network-check-target-2vsg4" (UID: "bf59cab4-36dc-431c-869d-896d50d50e7f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 15:06:20.047567 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:20.047487 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxmsv" Aug 04 15:06:20.048087 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:20.047610 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxmsv" podUID="46bd5ac9-b189-4e6c-b2c6-df0ed33641e2" Aug 04 15:06:20.059675 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:20.059640 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-f64s8" event={"ID":"d3dd50da-aab3-4299-98b3-ad684ab46ef0","Type":"ContainerStarted","Data":"3f19fa68e69b2e595f18e22cf949e9c800654117bec03e1be4e26480b5df8103"} Aug 04 15:06:20.060760 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:20.060739 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4wbvq" event={"ID":"cb3de761-615b-44d1-84f3-c9a0edf4520e","Type":"ContainerStarted","Data":"12af8c241f28319d6913a5269dc9df1a45c33d4ea1cdd2971b9f2604e9c28426"} Aug 04 15:06:20.061984 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:20.061958 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7qt6k" event={"ID":"93d48c95-3e8a-4cc4-a12c-21f1e93c4963","Type":"ContainerStarted","Data":"73759312444d3c0737a7d8fe3417e950c9035d9712a7dcb8a331bf263842f394"} Aug 04 15:06:20.063148 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:20.063128 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xphfh" event={"ID":"7ec6b008-bb9e-42e2-a233-a57bc7449804","Type":"ContainerStarted","Data":"daa7868807329c6738cb9e1d7bed12948a163d69a7d6bf4e72fa7674dd7e64d8"} Aug 04 15:06:20.064416 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:20.064382 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" event={"ID":"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc","Type":"ContainerStarted","Data":"2b0ec5962d007a38baf60a9c116bbcc9acaddb45ad3b17cd548377b47bfe53b7"} Aug 04 15:06:20.065514 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:20.065480 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xmh7f" event={"ID":"214ec271-9cb1-4e3b-9b10-b7d15c10333f","Type":"ContainerStarted","Data":"13effdbfde1f34985e158aff62f2de31abfe4e5afd26bfcc9dd0cf20d962773e"} Aug 04 15:06:20.134824 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:20.134781 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-03 15:01:19 +0000 UTC" deadline="2028-01-02 00:46:28.145670053 +0000 UTC" Aug 04 15:06:20.134824 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:20.134817 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12369h40m8.010857073s" Aug 04 15:06:20.356014 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:20.355976 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c62e3b1_c2dc_4b2a_a61d_7d80049f355f.slice/crio-222d0185f6e69f395161c9d8ac4b253ce74b188acca78e4f49e1df6332fc1d77 WatchSource:0}: Error finding container 222d0185f6e69f395161c9d8ac4b253ce74b188acca78e4f49e1df6332fc1d77: Status 404 returned error can't find the container with id 222d0185f6e69f395161c9d8ac4b253ce74b188acca78e4f49e1df6332fc1d77 Aug 04 15:06:20.363307 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:20.363280 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod843c5666_b42e_4249_981a_4c91cd13173f.slice/crio-214aa11519d2cd280a21a9a4d9fd853ff7c2d944815b1a54fc49c3d3cf819e0e WatchSource:0}: Error finding container 214aa11519d2cd280a21a9a4d9fd853ff7c2d944815b1a54fc49c3d3cf819e0e: Status 404 returned error can't find the container with id 214aa11519d2cd280a21a9a4d9fd853ff7c2d944815b1a54fc49c3d3cf819e0e Aug 04 15:06:20.366176 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:20.366153 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e62b3d6_309b_4cf1_bb50_c1d15a0672b1.slice/crio-1e50c0264e4b27dcc0d10cf364cf9f0d3911146893e3e521416655b2f5ce77aa WatchSource:0}: Error finding container 1e50c0264e4b27dcc0d10cf364cf9f0d3911146893e3e521416655b2f5ce77aa: Status 404 returned error can't find the container with id 1e50c0264e4b27dcc0d10cf364cf9f0d3911146893e3e521416655b2f5ce77aa Aug 04 15:06:20.667197 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:20.666866 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b91798e-f721-4056-bc09-6d4765fa6757-metrics-certs\") pod \"network-metrics-daemon-8w68v\" (UID: \"1b91798e-f721-4056-bc09-6d4765fa6757\") " pod="openshift-multus/network-metrics-daemon-8w68v" Aug 04 15:06:20.667387 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:20.667197 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46bd5ac9-b189-4e6c-b2c6-df0ed33641e2-original-pull-secret\") pod \"global-pull-secret-syncer-xxmsv\" (UID: \"46bd5ac9-b189-4e6c-b2c6-df0ed33641e2\") " pod="kube-system/global-pull-secret-syncer-xxmsv" Aug 04 15:06:20.667387 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:20.667027 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 15:06:20.667542 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:20.667390 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b91798e-f721-4056-bc09-6d4765fa6757-metrics-certs podName:1b91798e-f721-4056-bc09-6d4765fa6757 nodeName:}" failed. No retries permitted until 2026-08-04 15:06:22.667368902 +0000 UTC m=+6.127241237 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1b91798e-f721-4056-bc09-6d4765fa6757-metrics-certs") pod "network-metrics-daemon-8w68v" (UID: "1b91798e-f721-4056-bc09-6d4765fa6757") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 15:06:20.669136 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:20.667322 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 15:06:20.669136 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:20.667804 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46bd5ac9-b189-4e6c-b2c6-df0ed33641e2-original-pull-secret podName:46bd5ac9-b189-4e6c-b2c6-df0ed33641e2 nodeName:}" failed. No retries permitted until 2026-08-04 15:06:22.667786132 +0000 UTC m=+6.127658467 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/46bd5ac9-b189-4e6c-b2c6-df0ed33641e2-original-pull-secret") pod "global-pull-secret-syncer-xxmsv" (UID: "46bd5ac9-b189-4e6c-b2c6-df0ed33641e2") : object "kube-system"/"original-pull-secret" not registered Aug 04 15:06:20.768294 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:20.767637 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vqcn\" (UniqueName: \"kubernetes.io/projected/bf59cab4-36dc-431c-869d-896d50d50e7f-kube-api-access-4vqcn\") pod \"network-check-target-2vsg4\" (UID: \"bf59cab4-36dc-431c-869d-896d50d50e7f\") " pod="openshift-network-diagnostics/network-check-target-2vsg4" Aug 04 15:06:20.768294 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:20.767849 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 15:06:20.768294 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:20.767870 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 15:06:20.768294 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:20.767883 2574 projected.go:196] Error preparing data for projected volume kube-api-access-4vqcn for pod openshift-network-diagnostics/network-check-target-2vsg4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 15:06:20.768294 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:20.767939 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bf59cab4-36dc-431c-869d-896d50d50e7f-kube-api-access-4vqcn podName:bf59cab4-36dc-431c-869d-896d50d50e7f nodeName:}" failed. No retries permitted until 2026-08-04 15:06:22.767920288 +0000 UTC m=+6.227792621 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4vqcn" (UniqueName: "kubernetes.io/projected/bf59cab4-36dc-431c-869d-896d50d50e7f-kube-api-access-4vqcn") pod "network-check-target-2vsg4" (UID: "bf59cab4-36dc-431c-869d-896d50d50e7f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 15:06:21.050663 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:21.049912 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vsg4" Aug 04 15:06:21.050663 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:21.050046 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vsg4" podUID="bf59cab4-36dc-431c-869d-896d50d50e7f" Aug 04 15:06:21.050663 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:21.050137 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8w68v" Aug 04 15:06:21.050663 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:21.050228 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8w68v" podUID="1b91798e-f721-4056-bc09-6d4765fa6757" Aug 04 15:06:21.071405 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:21.071369 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ql74b" event={"ID":"1e62b3d6-309b-4cf1-bb50-c1d15a0672b1","Type":"ContainerStarted","Data":"1e50c0264e4b27dcc0d10cf364cf9f0d3911146893e3e521416655b2f5ce77aa"} Aug 04 15:06:21.074909 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:21.074873 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c47dc" event={"ID":"843c5666-b42e-4249-981a-4c91cd13173f","Type":"ContainerStarted","Data":"214aa11519d2cd280a21a9a4d9fd853ff7c2d944815b1a54fc49c3d3cf819e0e"} Aug 04 15:06:21.091613 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:21.091581 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-155.ec2.internal" event={"ID":"b47c00155e76867b44c5cd48f4a5b502","Type":"ContainerStarted","Data":"80c1201225f66e20baae5eb387d47d49a80b0d9ce8d41a50969724658cbd80de"} Aug 04 15:06:21.095627 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:21.095600 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" event={"ID":"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f","Type":"ContainerStarted","Data":"222d0185f6e69f395161c9d8ac4b253ce74b188acca78e4f49e1df6332fc1d77"} Aug 04 15:06:21.103542 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:21.103513 2574 generic.go:356] "Generic (PLEG): container finished" podID="7f45dc1fb51e13f5f7cb27b85338da6d" containerID="a82c44c56fb536050c1219d3df444bc47cf3e2ecd7a1f5056f8e638d2cfbe45b" exitCode=0 Aug 04 15:06:21.103640 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:21.103558 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-155.ec2.internal" event={"ID":"7f45dc1fb51e13f5f7cb27b85338da6d","Type":"ContainerDied","Data":"a82c44c56fb536050c1219d3df444bc47cf3e2ecd7a1f5056f8e638d2cfbe45b"} Aug 04 15:06:21.135315 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:21.135263 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-03 15:01:19 +0000 UTC" deadline="2028-01-21 21:01:32.803655444 +0000 UTC" Aug 04 15:06:21.135393 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:21.135315 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12845h55m11.668344663s" Aug 04 15:06:21.169439 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:21.169385 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-155.ec2.internal" podStartSLOduration=2.169365566 podStartE2EDuration="2.169365566s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 15:06:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:06:21.13081156 +0000 UTC m=+4.590683913" watchObservedRunningTime="2026-08-04 15:06:21.169365566 +0000 UTC m=+4.629237920" Aug 04 15:06:22.047541 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:22.047504 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxmsv" Aug 04 15:06:22.047742 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:22.047652 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxmsv" podUID="46bd5ac9-b189-4e6c-b2c6-df0ed33641e2" Aug 04 15:06:22.122953 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:22.122598 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-155.ec2.internal" event={"ID":"7f45dc1fb51e13f5f7cb27b85338da6d","Type":"ContainerStarted","Data":"ed3ab36fffb27100269149390d004c9229d9d2dfafa197c4b9478d3a759ea347"} Aug 04 15:06:22.685405 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:22.684617 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b91798e-f721-4056-bc09-6d4765fa6757-metrics-certs\") pod \"network-metrics-daemon-8w68v\" (UID: \"1b91798e-f721-4056-bc09-6d4765fa6757\") " pod="openshift-multus/network-metrics-daemon-8w68v" Aug 04 15:06:22.685405 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:22.684690 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46bd5ac9-b189-4e6c-b2c6-df0ed33641e2-original-pull-secret\") pod \"global-pull-secret-syncer-xxmsv\" (UID: \"46bd5ac9-b189-4e6c-b2c6-df0ed33641e2\") " pod="kube-system/global-pull-secret-syncer-xxmsv" Aug 04 15:06:22.685405 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:22.684832 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 15:06:22.685405 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:22.684895 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46bd5ac9-b189-4e6c-b2c6-df0ed33641e2-original-pull-secret podName:46bd5ac9-b189-4e6c-b2c6-df0ed33641e2 nodeName:}" failed. No retries permitted until 2026-08-04 15:06:26.684877373 +0000 UTC m=+10.144749708 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/46bd5ac9-b189-4e6c-b2c6-df0ed33641e2-original-pull-secret") pod "global-pull-secret-syncer-xxmsv" (UID: "46bd5ac9-b189-4e6c-b2c6-df0ed33641e2") : object "kube-system"/"original-pull-secret" not registered Aug 04 15:06:22.685405 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:22.685303 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 15:06:22.685405 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:22.685352 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b91798e-f721-4056-bc09-6d4765fa6757-metrics-certs podName:1b91798e-f721-4056-bc09-6d4765fa6757 nodeName:}" failed. No retries permitted until 2026-08-04 15:06:26.685335829 +0000 UTC m=+10.145208165 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1b91798e-f721-4056-bc09-6d4765fa6757-metrics-certs") pod "network-metrics-daemon-8w68v" (UID: "1b91798e-f721-4056-bc09-6d4765fa6757") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 15:06:22.786282 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:22.785593 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vqcn\" (UniqueName: \"kubernetes.io/projected/bf59cab4-36dc-431c-869d-896d50d50e7f-kube-api-access-4vqcn\") pod \"network-check-target-2vsg4\" (UID: \"bf59cab4-36dc-431c-869d-896d50d50e7f\") " pod="openshift-network-diagnostics/network-check-target-2vsg4" Aug 04 15:06:22.786282 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:22.785785 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 15:06:22.786282 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:22.785807 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 15:06:22.786282 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:22.785823 2574 projected.go:196] Error preparing data for projected volume kube-api-access-4vqcn for pod openshift-network-diagnostics/network-check-target-2vsg4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 15:06:22.786282 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:22.785894 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bf59cab4-36dc-431c-869d-896d50d50e7f-kube-api-access-4vqcn podName:bf59cab4-36dc-431c-869d-896d50d50e7f nodeName:}" failed. No retries permitted until 2026-08-04 15:06:26.785872364 +0000 UTC m=+10.245744694 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4vqcn" (UniqueName: "kubernetes.io/projected/bf59cab4-36dc-431c-869d-896d50d50e7f-kube-api-access-4vqcn") pod "network-check-target-2vsg4" (UID: "bf59cab4-36dc-431c-869d-896d50d50e7f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 15:06:23.048066 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:23.047421 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8w68v" Aug 04 15:06:23.048066 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:23.047561 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8w68v" podUID="1b91798e-f721-4056-bc09-6d4765fa6757" Aug 04 15:06:23.048280 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:23.048120 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vsg4" Aug 04 15:06:23.048280 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:23.048235 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vsg4" podUID="bf59cab4-36dc-431c-869d-896d50d50e7f" Aug 04 15:06:24.047577 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:24.047537 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxmsv" Aug 04 15:06:24.048103 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:24.047699 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxmsv" podUID="46bd5ac9-b189-4e6c-b2c6-df0ed33641e2" Aug 04 15:06:25.048140 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:25.048104 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8w68v" Aug 04 15:06:25.048611 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:25.048150 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vsg4" Aug 04 15:06:25.048611 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:25.048263 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8w68v" podUID="1b91798e-f721-4056-bc09-6d4765fa6757" Aug 04 15:06:25.048611 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:25.048416 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vsg4" podUID="bf59cab4-36dc-431c-869d-896d50d50e7f" Aug 04 15:06:26.047866 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:26.047754 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxmsv" Aug 04 15:06:26.048143 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:26.047911 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxmsv" podUID="46bd5ac9-b189-4e6c-b2c6-df0ed33641e2" Aug 04 15:06:26.721648 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:26.721612 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46bd5ac9-b189-4e6c-b2c6-df0ed33641e2-original-pull-secret\") pod \"global-pull-secret-syncer-xxmsv\" (UID: \"46bd5ac9-b189-4e6c-b2c6-df0ed33641e2\") " pod="kube-system/global-pull-secret-syncer-xxmsv" Aug 04 15:06:26.722087 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:26.721688 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b91798e-f721-4056-bc09-6d4765fa6757-metrics-certs\") pod \"network-metrics-daemon-8w68v\" (UID: \"1b91798e-f721-4056-bc09-6d4765fa6757\") " pod="openshift-multus/network-metrics-daemon-8w68v" Aug 04 15:06:26.722087 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:26.721827 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 15:06:26.722087 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:26.721831 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 15:06:26.722087 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:26.721895 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b91798e-f721-4056-bc09-6d4765fa6757-metrics-certs podName:1b91798e-f721-4056-bc09-6d4765fa6757 nodeName:}" failed. No retries permitted until 2026-08-04 15:06:34.72187515 +0000 UTC m=+18.181747483 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1b91798e-f721-4056-bc09-6d4765fa6757-metrics-certs") pod "network-metrics-daemon-8w68v" (UID: "1b91798e-f721-4056-bc09-6d4765fa6757") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 15:06:26.722087 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:26.721915 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46bd5ac9-b189-4e6c-b2c6-df0ed33641e2-original-pull-secret podName:46bd5ac9-b189-4e6c-b2c6-df0ed33641e2 nodeName:}" failed. No retries permitted until 2026-08-04 15:06:34.721905018 +0000 UTC m=+18.181777350 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/46bd5ac9-b189-4e6c-b2c6-df0ed33641e2-original-pull-secret") pod "global-pull-secret-syncer-xxmsv" (UID: "46bd5ac9-b189-4e6c-b2c6-df0ed33641e2") : object "kube-system"/"original-pull-secret" not registered Aug 04 15:06:26.822784 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:26.822749 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vqcn\" (UniqueName: \"kubernetes.io/projected/bf59cab4-36dc-431c-869d-896d50d50e7f-kube-api-access-4vqcn\") pod \"network-check-target-2vsg4\" (UID: \"bf59cab4-36dc-431c-869d-896d50d50e7f\") " pod="openshift-network-diagnostics/network-check-target-2vsg4" Aug 04 15:06:26.822947 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:26.822886 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 15:06:26.822947 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:26.822905 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 15:06:26.822947 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:26.822918 2574 projected.go:196] Error preparing data for projected volume kube-api-access-4vqcn for pod openshift-network-diagnostics/network-check-target-2vsg4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 15:06:26.823103 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:26.822977 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bf59cab4-36dc-431c-869d-896d50d50e7f-kube-api-access-4vqcn podName:bf59cab4-36dc-431c-869d-896d50d50e7f nodeName:}" failed. No retries permitted until 2026-08-04 15:06:34.822961326 +0000 UTC m=+18.282833657 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4vqcn" (UniqueName: "kubernetes.io/projected/bf59cab4-36dc-431c-869d-896d50d50e7f-kube-api-access-4vqcn") pod "network-check-target-2vsg4" (UID: "bf59cab4-36dc-431c-869d-896d50d50e7f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 15:06:27.048470 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:27.048380 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8w68v" Aug 04 15:06:27.048628 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:27.048526 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8w68v" podUID="1b91798e-f721-4056-bc09-6d4765fa6757" Aug 04 15:06:27.048869 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:27.048850 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vsg4" Aug 04 15:06:27.048956 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:27.048938 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vsg4" podUID="bf59cab4-36dc-431c-869d-896d50d50e7f" Aug 04 15:06:28.048122 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:28.048082 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxmsv" Aug 04 15:06:28.048556 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:28.048207 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxmsv" podUID="46bd5ac9-b189-4e6c-b2c6-df0ed33641e2" Aug 04 15:06:29.047442 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:29.047408 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vsg4" Aug 04 15:06:29.047634 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:29.047543 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vsg4" podUID="bf59cab4-36dc-431c-869d-896d50d50e7f" Aug 04 15:06:29.047634 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:29.047593 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8w68v" Aug 04 15:06:29.047749 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:29.047686 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8w68v" podUID="1b91798e-f721-4056-bc09-6d4765fa6757" Aug 04 15:06:30.047621 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:30.047587 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxmsv" Aug 04 15:06:30.048109 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:30.047716 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxmsv" podUID="46bd5ac9-b189-4e6c-b2c6-df0ed33641e2" Aug 04 15:06:31.047181 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:31.047135 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vsg4" Aug 04 15:06:31.047358 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:31.047135 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8w68v" Aug 04 15:06:31.047358 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:31.047271 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vsg4" podUID="bf59cab4-36dc-431c-869d-896d50d50e7f" Aug 04 15:06:31.047358 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:31.047336 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8w68v" podUID="1b91798e-f721-4056-bc09-6d4765fa6757" Aug 04 15:06:32.048087 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:32.048053 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxmsv" Aug 04 15:06:32.048490 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:32.048172 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxmsv" podUID="46bd5ac9-b189-4e6c-b2c6-df0ed33641e2" Aug 04 15:06:33.047656 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:33.047616 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8w68v" Aug 04 15:06:33.047856 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:33.047616 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vsg4" Aug 04 15:06:33.047856 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:33.047759 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8w68v" podUID="1b91798e-f721-4056-bc09-6d4765fa6757" Aug 04 15:06:33.047856 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:33.047798 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vsg4" podUID="bf59cab4-36dc-431c-869d-896d50d50e7f" Aug 04 15:06:34.048111 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:34.048075 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxmsv" Aug 04 15:06:34.048621 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:34.048216 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxmsv" podUID="46bd5ac9-b189-4e6c-b2c6-df0ed33641e2" Aug 04 15:06:34.786268 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:34.786235 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b91798e-f721-4056-bc09-6d4765fa6757-metrics-certs\") pod \"network-metrics-daemon-8w68v\" (UID: \"1b91798e-f721-4056-bc09-6d4765fa6757\") " pod="openshift-multus/network-metrics-daemon-8w68v" Aug 04 15:06:34.786451 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:34.786292 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46bd5ac9-b189-4e6c-b2c6-df0ed33641e2-original-pull-secret\") pod \"global-pull-secret-syncer-xxmsv\" (UID: \"46bd5ac9-b189-4e6c-b2c6-df0ed33641e2\") " pod="kube-system/global-pull-secret-syncer-xxmsv" Aug 04 15:06:34.786451 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:34.786410 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 15:06:34.786572 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:34.786467 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b91798e-f721-4056-bc09-6d4765fa6757-metrics-certs podName:1b91798e-f721-4056-bc09-6d4765fa6757 nodeName:}" failed. No retries permitted until 2026-08-04 15:06:50.786451664 +0000 UTC m=+34.246323995 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1b91798e-f721-4056-bc09-6d4765fa6757-metrics-certs") pod "network-metrics-daemon-8w68v" (UID: "1b91798e-f721-4056-bc09-6d4765fa6757") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 15:06:34.786572 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:34.786414 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 15:06:34.786572 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:34.786565 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46bd5ac9-b189-4e6c-b2c6-df0ed33641e2-original-pull-secret podName:46bd5ac9-b189-4e6c-b2c6-df0ed33641e2 nodeName:}" failed. No retries permitted until 2026-08-04 15:06:50.786549114 +0000 UTC m=+34.246421448 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/46bd5ac9-b189-4e6c-b2c6-df0ed33641e2-original-pull-secret") pod "global-pull-secret-syncer-xxmsv" (UID: "46bd5ac9-b189-4e6c-b2c6-df0ed33641e2") : object "kube-system"/"original-pull-secret" not registered Aug 04 15:06:34.887466 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:34.887436 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vqcn\" (UniqueName: \"kubernetes.io/projected/bf59cab4-36dc-431c-869d-896d50d50e7f-kube-api-access-4vqcn\") pod \"network-check-target-2vsg4\" (UID: \"bf59cab4-36dc-431c-869d-896d50d50e7f\") " pod="openshift-network-diagnostics/network-check-target-2vsg4" Aug 04 15:06:34.887615 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:34.887583 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 15:06:34.887615 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:34.887601 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 15:06:34.887615 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:34.887610 2574 projected.go:196] Error preparing data for projected volume kube-api-access-4vqcn for pod openshift-network-diagnostics/network-check-target-2vsg4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 15:06:34.887728 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:34.887657 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bf59cab4-36dc-431c-869d-896d50d50e7f-kube-api-access-4vqcn podName:bf59cab4-36dc-431c-869d-896d50d50e7f nodeName:}" failed. No retries permitted until 2026-08-04 15:06:50.887644781 +0000 UTC m=+34.347517111 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4vqcn" (UniqueName: "kubernetes.io/projected/bf59cab4-36dc-431c-869d-896d50d50e7f-kube-api-access-4vqcn") pod "network-check-target-2vsg4" (UID: "bf59cab4-36dc-431c-869d-896d50d50e7f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 15:06:35.047527 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:35.047438 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8w68v" Aug 04 15:06:35.047684 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:35.047438 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vsg4" Aug 04 15:06:35.047684 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:35.047583 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8w68v" podUID="1b91798e-f721-4056-bc09-6d4765fa6757" Aug 04 15:06:35.047684 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:35.047608 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vsg4" podUID="bf59cab4-36dc-431c-869d-896d50d50e7f" Aug 04 15:06:36.047710 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:36.047676 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxmsv" Aug 04 15:06:36.048101 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:36.047803 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxmsv" podUID="46bd5ac9-b189-4e6c-b2c6-df0ed33641e2" Aug 04 15:06:37.048976 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:37.048945 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8w68v" Aug 04 15:06:37.049455 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:37.049019 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vsg4" Aug 04 15:06:37.049455 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:37.049171 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8w68v" podUID="1b91798e-f721-4056-bc09-6d4765fa6757" Aug 04 15:06:37.049455 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:37.049206 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vsg4" podUID="bf59cab4-36dc-431c-869d-896d50d50e7f" Aug 04 15:06:38.047356 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:38.047318 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxmsv" Aug 04 15:06:38.047559 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:38.047453 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxmsv" podUID="46bd5ac9-b189-4e6c-b2c6-df0ed33641e2" Aug 04 15:06:39.047136 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:39.047107 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8w68v" Aug 04 15:06:39.047542 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:39.047115 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vsg4" Aug 04 15:06:39.047542 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:39.047243 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8w68v" podUID="1b91798e-f721-4056-bc09-6d4765fa6757" Aug 04 15:06:39.047542 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:39.047298 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vsg4" podUID="bf59cab4-36dc-431c-869d-896d50d50e7f" Aug 04 15:06:40.047552 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:40.047203 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxmsv" Aug 04 15:06:40.048258 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:40.047648 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxmsv" podUID="46bd5ac9-b189-4e6c-b2c6-df0ed33641e2" Aug 04 15:06:40.160401 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:40.160357 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xmh7f" event={"ID":"214ec271-9cb1-4e3b-9b10-b7d15c10333f","Type":"ContainerStarted","Data":"b5bc912d46a615f3ea96c646423049444eafac809cfa95aa19b5eb5b84372e11"} Aug 04 15:06:40.162003 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:40.161974 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-f64s8" event={"ID":"d3dd50da-aab3-4299-98b3-ad684ab46ef0","Type":"ContainerStarted","Data":"f9edc67d4ce69ed69573c19b17eacffd4a12f8c51f429eb49791122256ea763a"} Aug 04 15:06:40.163582 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:40.163556 2574 generic.go:356] "Generic (PLEG): container finished" podID="cb3de761-615b-44d1-84f3-c9a0edf4520e" containerID="84511889afd041ef1fa6774e826f61c3976bfea0cc7ede2c9013c525138a5af1" exitCode=0 Aug 04 15:06:40.163680 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:40.163641 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4wbvq" event={"ID":"cb3de761-615b-44d1-84f3-c9a0edf4520e","Type":"ContainerDied","Data":"84511889afd041ef1fa6774e826f61c3976bfea0cc7ede2c9013c525138a5af1"} Aug 04 15:06:40.165323 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:40.165281 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ql74b" event={"ID":"1e62b3d6-309b-4cf1-bb50-c1d15a0672b1","Type":"ContainerStarted","Data":"8e9159bf24362cbc6a71ae60b6542741572b9e63f8c6311a95c7c4abe1c8288b"} Aug 04 15:06:40.166980 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:40.166958 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c47dc" event={"ID":"843c5666-b42e-4249-981a-4c91cd13173f","Type":"ContainerStarted","Data":"0ae53f5e2a5ebac276a5f7762da1812bb8a9e86b8723025432f12e56c139ecac"} Aug 04 15:06:40.168517 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:40.168498 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7qt6k" event={"ID":"93d48c95-3e8a-4cc4-a12c-21f1e93c4963","Type":"ContainerStarted","Data":"c6747492c2c91499165c750368aa5c91515ecf6daf2f72d625b87188e9bf122a"} Aug 04 15:06:40.170072 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:40.170053 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xphfh" event={"ID":"7ec6b008-bb9e-42e2-a233-a57bc7449804","Type":"ContainerStarted","Data":"ba47f304867d2ded5534b80d79d278d2b2f4905e60203b0d64ba9f99499cba52"} Aug 04 15:06:40.173151 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:40.173136 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8h6k_9c62e3b1-c2dc-4b2a-a61d-7d80049f355f/ovn-acl-logging/0.log" Aug 04 15:06:40.173569 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:40.173550 2574 generic.go:356] "Generic (PLEG): container finished" podID="9c62e3b1-c2dc-4b2a-a61d-7d80049f355f" containerID="5f67ad32e731c9b4c92200d3610586be1f7e9b282e9ac816f0da73d9527fb654" exitCode=1 Aug 04 15:06:40.173649 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:40.173612 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" event={"ID":"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f","Type":"ContainerStarted","Data":"f7fb5b81c466f70b15543babecf29df8dc94cdd96c013db266e06107ba2b97cb"} Aug 04 15:06:40.173716 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:40.173681 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" event={"ID":"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f","Type":"ContainerStarted","Data":"8f59f6e773bbb83506ff7e4f45ef95480c19d4ba37032803633206f8fa1a61be"} Aug 04 15:06:40.173767 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:40.173722 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" event={"ID":"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f","Type":"ContainerStarted","Data":"6b4331070588e70e587f9b56915d4d0a4051720bbaba9cf600ccd3da3bfca5f1"} Aug 04 15:06:40.173813 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:40.173768 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" event={"ID":"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f","Type":"ContainerStarted","Data":"9ff404b570235403e81e9844bb8a2056307617a838a39bdbcd9649bd35ae17e2"} Aug 04 15:06:40.173872 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:40.173815 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" event={"ID":"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f","Type":"ContainerDied","Data":"5f67ad32e731c9b4c92200d3610586be1f7e9b282e9ac816f0da73d9527fb654"} Aug 04 15:06:40.173872 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:40.173864 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" event={"ID":"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f","Type":"ContainerStarted","Data":"f3a14d025e63972ca4fd288a807ece3a0764028d66ea03ab2b7dc6a3c53f4dcd"} Aug 04 15:06:40.175061 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:40.175040 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" event={"ID":"5bb7f9d7-3a00-48b8-8538-7ab88dec74cc","Type":"ContainerStarted","Data":"97a3fce93b9072d7c5e6a3d523a1f280897ba38eedf574685486ef9052bde40e"} Aug 04 15:06:40.193524 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:40.193477 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-155.ec2.internal" podStartSLOduration=21.193460417 podStartE2EDuration="21.193460417s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 15:06:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:06:22.157436593 +0000 UTC m=+5.617308943" watchObservedRunningTime="2026-08-04 15:06:40.193460417 +0000 UTC m=+23.653332766" Aug 04 15:06:40.194146 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:40.194120 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-xmh7f" podStartSLOduration=2.467407604 podStartE2EDuration="22.194113253s" totalImagesPullingTime="19.726705649s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:06:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:06:40.193332014 +0000 UTC m=+23.653204364" watchObservedRunningTime="2026-08-04 15:06:40.194113253 +0000 UTC m=+23.653985602" Aug 04 15:06:40.270111 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:40.270052 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-nlhv9" podStartSLOduration=2.526406115 podStartE2EDuration="22.270032939s" totalImagesPullingTime="19.743626824s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 15:06:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:06:40.269919671 +0000 UTC m=+23.729792021" watchObservedRunningTime="2026-08-04 15:06:40.270032939 +0000 UTC m=+23.729905292" Aug 04 15:06:40.270520 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:40.270486 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-ql74b" podStartSLOduration=3.6266642239999998 podStartE2EDuration="22.270478007s" totalImagesPullingTime="18.643813783s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 15:06:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:06:40.22009585 +0000 UTC m=+23.679968201" watchObservedRunningTime="2026-08-04 15:06:40.270478007 +0000 UTC m=+23.730350356" Aug 04 15:06:40.379720 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:40.379632 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-xphfh" podStartSLOduration=6.8827528000000004 podStartE2EDuration="22.379616786s" totalImagesPullingTime="15.496863986s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 15:06:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:06:40.328012042 +0000 UTC m=+23.787884403" watchObservedRunningTime="2026-08-04 15:06:40.379616786 +0000 UTC m=+23.839489135" Aug 04 15:06:40.552552 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:40.552502 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-7qt6k" podStartSLOduration=7.064075221 podStartE2EDuration="22.552486234s" totalImagesPullingTime="15.488411013s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 15:06:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:06:40.379280876 +0000 UTC m=+23.839153226" watchObservedRunningTime="2026-08-04 15:06:40.552486234 +0000 UTC m=+24.012358582" Aug 04 15:06:41.016376 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:41.016348 2574 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Aug 04 15:06:41.048189 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:41.048166 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vsg4" Aug 04 15:06:41.048614 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:41.048175 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8w68v" Aug 04 15:06:41.048614 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:41.048293 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vsg4" podUID="bf59cab4-36dc-431c-869d-896d50d50e7f" Aug 04 15:06:41.048614 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:41.048343 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8w68v" podUID="1b91798e-f721-4056-bc09-6d4765fa6757" Aug 04 15:06:41.179177 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:41.179094 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c47dc" event={"ID":"843c5666-b42e-4249-981a-4c91cd13173f","Type":"ContainerStarted","Data":"322c24ff18dace5a269c11a7696413269a973fa4b36739fe264d641111da9b11"} Aug 04 15:06:42.014291 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:42.014185 2574 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-08-04T15:06:41.01637129Z","UUID":"9cdc9fa1-c259-48ef-a5b7-d22e5c2b96d7","Handler":null,"Name":"","Endpoint":""} Aug 04 15:06:42.015971 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:42.015943 2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Aug 04 15:06:42.016088 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:42.015993 2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Aug 04 15:06:42.047784 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:42.047755 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxmsv" Aug 04 15:06:42.047940 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:42.047889 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxmsv" podUID="46bd5ac9-b189-4e6c-b2c6-df0ed33641e2" Aug 04 15:06:43.047989 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:43.047803 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vsg4" Aug 04 15:06:43.048580 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:43.047800 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8w68v" Aug 04 15:06:43.048580 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:43.048089 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vsg4" podUID="bf59cab4-36dc-431c-869d-896d50d50e7f" Aug 04 15:06:43.048580 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:43.048217 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8w68v" podUID="1b91798e-f721-4056-bc09-6d4765fa6757" Aug 04 15:06:43.188695 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:43.188667 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8h6k_9c62e3b1-c2dc-4b2a-a61d-7d80049f355f/ovn-acl-logging/0.log" Aug 04 15:06:43.189091 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:43.189063 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" event={"ID":"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f","Type":"ContainerStarted","Data":"8eed9295b9eda84dfeb99e317641faeacffe8192a8b3b771df4e4164483844e8"} Aug 04 15:06:43.191251 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:43.191229 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c47dc" event={"ID":"843c5666-b42e-4249-981a-4c91cd13173f","Type":"ContainerStarted","Data":"191258a966e638cdfac467b916c90003ae436c43fa03d1571f5b6fa9bf42c5fd"} Aug 04 15:06:43.235424 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:43.235354 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-f64s8" podStartSLOduration=5.446008212 podStartE2EDuration="25.235337239s" totalImagesPullingTime="19.789329027s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 15:06:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:06:40.601136116 +0000 UTC m=+24.061008464" watchObservedRunningTime="2026-08-04 15:06:43.235337239 +0000 UTC m=+26.695209588" Aug 04 15:06:43.982614 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:43.982580 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-xmh7f" Aug 04 15:06:43.983643 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:43.983623 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-xmh7f" Aug 04 15:06:44.011866 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:44.011818 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c47dc" podStartSLOduration=4.4530061100000005 podStartE2EDuration="26.011792795s" totalImagesPullingTime="21.558786685s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 15:06:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:06:43.235316414 +0000 UTC m=+26.695188763" watchObservedRunningTime="2026-08-04 15:06:44.011792795 +0000 UTC m=+27.471665146" Aug 04 15:06:44.047608 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:44.047586 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxmsv" Aug 04 15:06:44.047734 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:44.047688 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxmsv" podUID="46bd5ac9-b189-4e6c-b2c6-df0ed33641e2" Aug 04 15:06:45.047433 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:45.047400 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vsg4" Aug 04 15:06:45.047991 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:45.047510 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vsg4" podUID="bf59cab4-36dc-431c-869d-896d50d50e7f" Aug 04 15:06:45.047991 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:45.047575 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8w68v" Aug 04 15:06:45.047991 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:45.047680 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8w68v" podUID="1b91798e-f721-4056-bc09-6d4765fa6757" Aug 04 15:06:46.047634 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:46.047478 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxmsv" Aug 04 15:06:46.048201 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:46.047718 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxmsv" podUID="46bd5ac9-b189-4e6c-b2c6-df0ed33641e2" Aug 04 15:06:46.201184 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:46.201158 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8h6k_9c62e3b1-c2dc-4b2a-a61d-7d80049f355f/ovn-acl-logging/0.log" Aug 04 15:06:46.201573 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:46.201551 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" event={"ID":"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f","Type":"ContainerStarted","Data":"2c6f0fbf492f29d9d6a3a83d41eba82c100aff23b4d8a8045dec37cc62ca54fe"} Aug 04 15:06:46.201780 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:46.201753 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:46.201879 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:46.201826 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:46.201926 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:46.201916 2574 scope.go:117] "RemoveContainer" containerID="5f67ad32e731c9b4c92200d3610586be1f7e9b282e9ac816f0da73d9527fb654" Aug 04 15:06:46.203422 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:46.203381 2574 generic.go:356] "Generic (PLEG): container finished" podID="cb3de761-615b-44d1-84f3-c9a0edf4520e" containerID="9b9e70645e1db20ccbcb4f62c94eacf39c7e3c1c2a04df49ce3fdc0cd3137ed4" exitCode=0 Aug 04 15:06:46.203517 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:46.203415 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4wbvq" event={"ID":"cb3de761-615b-44d1-84f3-c9a0edf4520e","Type":"ContainerDied","Data":"9b9e70645e1db20ccbcb4f62c94eacf39c7e3c1c2a04df49ce3fdc0cd3137ed4"} Aug 04 15:06:46.218181 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:46.218162 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:46.413845 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:46.413765 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-xmh7f" Aug 04 15:06:46.413985 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:46.413883 2574 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Aug 04 15:06:46.414242 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:46.414228 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-xmh7f" Aug 04 15:06:47.048142 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:47.048110 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8w68v" Aug 04 15:06:47.048631 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:47.048231 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8w68v" podUID="1b91798e-f721-4056-bc09-6d4765fa6757" Aug 04 15:06:47.048631 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:47.048278 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vsg4" Aug 04 15:06:47.048631 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:47.048374 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vsg4" podUID="bf59cab4-36dc-431c-869d-896d50d50e7f" Aug 04 15:06:47.213249 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:47.213222 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8h6k_9c62e3b1-c2dc-4b2a-a61d-7d80049f355f/ovn-acl-logging/0.log" Aug 04 15:06:47.213698 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:47.213670 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" event={"ID":"9c62e3b1-c2dc-4b2a-a61d-7d80049f355f","Type":"ContainerStarted","Data":"46c3c2473330266a74f8b46b2baab2b8fae4485f8deecf487d7970786ac4dec9"} Aug 04 15:06:47.213961 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:47.213928 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:47.228906 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:47.228882 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:06:47.275505 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:47.274810 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" podStartSLOduration=10.321281558 podStartE2EDuration="29.274787102s" totalImagesPullingTime="18.953505544s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 15:06:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:06:47.274104056 +0000 UTC m=+30.733976405" watchObservedRunningTime="2026-08-04 15:06:47.274787102 +0000 UTC m=+30.734659452" Aug 04 15:06:47.768708 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:47.768674 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8w68v"] Aug 04 15:06:47.768861 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:47.768846 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8w68v" Aug 04 15:06:47.768995 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:47.768973 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8w68v" podUID="1b91798e-f721-4056-bc09-6d4765fa6757" Aug 04 15:06:47.774017 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:47.773982 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2vsg4"] Aug 04 15:06:47.774172 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:47.774118 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vsg4" Aug 04 15:06:47.774261 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:47.774207 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vsg4" podUID="bf59cab4-36dc-431c-869d-896d50d50e7f" Aug 04 15:06:47.774486 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:47.774468 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xxmsv"] Aug 04 15:06:47.774559 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:47.774548 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxmsv" Aug 04 15:06:47.774630 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:47.774615 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxmsv" podUID="46bd5ac9-b189-4e6c-b2c6-df0ed33641e2" Aug 04 15:06:48.217358 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:48.217283 2574 generic.go:356] "Generic (PLEG): container finished" podID="cb3de761-615b-44d1-84f3-c9a0edf4520e" containerID="ff5968a885508bf8fbade473d1f19965fcb5fadbdd79bd93be264c150184cb65" exitCode=0 Aug 04 15:06:48.217741 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:48.217364 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4wbvq" event={"ID":"cb3de761-615b-44d1-84f3-c9a0edf4520e","Type":"ContainerDied","Data":"ff5968a885508bf8fbade473d1f19965fcb5fadbdd79bd93be264c150184cb65"} Aug 04 15:06:49.048286 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:49.048080 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8w68v" Aug 04 15:06:49.048461 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:49.048144 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxmsv" Aug 04 15:06:49.048461 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:49.048348 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8w68v" podUID="1b91798e-f721-4056-bc09-6d4765fa6757" Aug 04 15:06:49.048461 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:49.048401 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxmsv" podUID="46bd5ac9-b189-4e6c-b2c6-df0ed33641e2" Aug 04 15:06:50.047375 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:50.047351 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vsg4" Aug 04 15:06:50.047773 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:50.047456 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2vsg4" podUID="bf59cab4-36dc-431c-869d-896d50d50e7f" Aug 04 15:06:50.224307 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:50.224213 2574 generic.go:356] "Generic (PLEG): container finished" podID="cb3de761-615b-44d1-84f3-c9a0edf4520e" containerID="e36a4b1eb37982a87c7bb1a8459dc54dc164ed714e29aa762d069e58ff01ea89" exitCode=0 Aug 04 15:06:50.224307 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:50.224260 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4wbvq" event={"ID":"cb3de761-615b-44d1-84f3-c9a0edf4520e","Type":"ContainerDied","Data":"e36a4b1eb37982a87c7bb1a8459dc54dc164ed714e29aa762d069e58ff01ea89"} Aug 04 15:06:50.800716 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:50.800688 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46bd5ac9-b189-4e6c-b2c6-df0ed33641e2-original-pull-secret\") pod \"global-pull-secret-syncer-xxmsv\" (UID: \"46bd5ac9-b189-4e6c-b2c6-df0ed33641e2\") " pod="kube-system/global-pull-secret-syncer-xxmsv" Aug 04 15:06:50.800879 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:50.800733 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b91798e-f721-4056-bc09-6d4765fa6757-metrics-certs\") pod \"network-metrics-daemon-8w68v\" (UID: \"1b91798e-f721-4056-bc09-6d4765fa6757\") " pod="openshift-multus/network-metrics-daemon-8w68v" Aug 04 15:06:50.800879 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:50.800831 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 15:06:50.800879 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:50.800853 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Aug 04 15:06:50.800992 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:50.800892 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b91798e-f721-4056-bc09-6d4765fa6757-metrics-certs podName:1b91798e-f721-4056-bc09-6d4765fa6757 nodeName:}" failed. No retries permitted until 2026-08-04 15:07:22.800875283 +0000 UTC m=+66.260747624 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1b91798e-f721-4056-bc09-6d4765fa6757-metrics-certs") pod "network-metrics-daemon-8w68v" (UID: "1b91798e-f721-4056-bc09-6d4765fa6757") : object "openshift-multus"/"metrics-daemon-secret" not registered Aug 04 15:06:50.800992 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:50.800911 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46bd5ac9-b189-4e6c-b2c6-df0ed33641e2-original-pull-secret podName:46bd5ac9-b189-4e6c-b2c6-df0ed33641e2 nodeName:}" failed. No retries permitted until 2026-08-04 15:07:22.80090253 +0000 UTC m=+66.260774864 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/46bd5ac9-b189-4e6c-b2c6-df0ed33641e2-original-pull-secret") pod "global-pull-secret-syncer-xxmsv" (UID: "46bd5ac9-b189-4e6c-b2c6-df0ed33641e2") : object "kube-system"/"original-pull-secret" not registered Aug 04 15:06:50.901985 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:50.901958 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vqcn\" (UniqueName: \"kubernetes.io/projected/bf59cab4-36dc-431c-869d-896d50d50e7f-kube-api-access-4vqcn\") pod \"network-check-target-2vsg4\" (UID: \"bf59cab4-36dc-431c-869d-896d50d50e7f\") " pod="openshift-network-diagnostics/network-check-target-2vsg4" Aug 04 15:06:50.902124 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:50.902110 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Aug 04 15:06:50.902170 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:50.902125 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Aug 04 15:06:50.902170 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:50.902139 2574 projected.go:196] Error preparing data for projected volume kube-api-access-4vqcn for pod openshift-network-diagnostics/network-check-target-2vsg4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 15:06:50.902253 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:50.902199 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bf59cab4-36dc-431c-869d-896d50d50e7f-kube-api-access-4vqcn podName:bf59cab4-36dc-431c-869d-896d50d50e7f nodeName:}" failed. No retries permitted until 2026-08-04 15:07:22.902181985 +0000 UTC m=+66.362054319 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4vqcn" (UniqueName: "kubernetes.io/projected/bf59cab4-36dc-431c-869d-896d50d50e7f-kube-api-access-4vqcn") pod "network-check-target-2vsg4" (UID: "bf59cab4-36dc-431c-869d-896d50d50e7f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Aug 04 15:06:51.047806 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:51.047770 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxmsv" Aug 04 15:06:51.048250 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:51.047876 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xxmsv" podUID="46bd5ac9-b189-4e6c-b2c6-df0ed33641e2" Aug 04 15:06:51.048250 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:51.047901 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8w68v" Aug 04 15:06:51.048250 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:51.047965 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8w68v" podUID="1b91798e-f721-4056-bc09-6d4765fa6757" Aug 04 15:06:51.915497 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:51.915428 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-155.ec2.internal" event="NodeReady" Aug 04 15:06:51.915711 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:51.915550 2574 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Aug 04 15:06:51.970552 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:51.970526 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-bddc5f76d-42zvh"] Aug 04 15:06:51.985577 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:51.985544 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-r6lsj"] Aug 04 15:06:51.986486 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:51.986451 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" Aug 04 15:06:51.990325 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:51.990307 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Aug 04 15:06:51.990429 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:51.990400 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-j4zcx\"" Aug 04 15:06:51.990492 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:51.990400 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Aug 04 15:06:51.992316 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:51.992297 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Aug 04 15:06:52.001019 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.001000 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Aug 04 15:06:52.001147 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.001129 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-tqzqn"] Aug 04 15:06:52.001249 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.001235 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-r6lsj" Aug 04 15:06:52.010577 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.010555 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Aug 04 15:06:52.010828 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.010811 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Aug 04 15:06:52.010985 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.010823 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-j85bf\"" Aug 04 15:06:52.021490 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.021467 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-r8wkp"] Aug 04 15:06:52.021594 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.021575 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-tqzqn" Aug 04 15:06:52.029908 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.029810 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Aug 04 15:06:52.030109 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.030087 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-gm7d7\"" Aug 04 15:06:52.030290 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.030268 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Aug 04 15:06:52.030993 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.030977 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Aug 04 15:06:52.038263 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.038233 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-bddc5f76d-42zvh"] Aug 04 15:06:52.038366 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.038301 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-r6lsj"] Aug 04 15:06:52.038366 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.038328 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-tqzqn"] Aug 04 15:06:52.038366 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.038240 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-5ccd6b445f-r8wkp" Aug 04 15:06:52.038366 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.038347 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-r8wkp"] Aug 04 15:06:52.038366 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.038367 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-68756"] Aug 04 15:06:52.038630 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.038436 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Aug 04 15:06:52.042112 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.042098 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Aug 04 15:06:52.042516 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.042499 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Aug 04 15:06:52.043429 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.043380 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Aug 04 15:06:52.043724 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.043706 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-79qbm\"" Aug 04 15:06:52.045011 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.044992 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Aug 04 15:06:52.051063 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.051039 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-77696685b-kpmh2"] Aug 04 15:06:52.051425 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.051203 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-68756" Aug 04 15:06:52.062741 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.062721 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Aug 04 15:06:52.062880 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.062776 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Aug 04 15:06:52.062949 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.062886 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Aug 04 15:06:52.063065 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.063050 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-ln79b\"" Aug 04 15:06:52.063219 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.063201 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Aug 04 15:06:52.066620 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.066602 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-68756"] Aug 04 15:06:52.066699 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.066672 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vsg4" Aug 04 15:06:52.066699 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.066676 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8w68v" Aug 04 15:06:52.066787 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.066774 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-77696685b-kpmh2" Aug 04 15:06:52.072438 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.072419 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Aug 04 15:06:52.073036 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.073019 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jm4b6\"" Aug 04 15:06:52.073982 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.073961 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Aug 04 15:06:52.074122 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.074103 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Aug 04 15:06:52.074190 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.074106 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8z69l\"" Aug 04 15:06:52.074568 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.074553 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Aug 04 15:06:52.074644 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.074628 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Aug 04 15:06:52.074783 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.074761 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Aug 04 15:06:52.076009 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.075965 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-77696685b-kpmh2"] Aug 04 15:06:52.076341 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.076297 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-8glb6\"" Aug 04 15:06:52.076474 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.076374 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Aug 04 15:06:52.082879 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.082252 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Aug 04 15:06:52.083582 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.083564 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-48zkn"] Aug 04 15:06:52.099422 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.099403 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7fc97bc64b-45b4c"] Aug 04 15:06:52.099531 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.099510 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-f77f48c6d-48zkn" Aug 04 15:06:52.103021 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.103000 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Aug 04 15:06:52.103103 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.103075 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-dl44g\"" Aug 04 15:06:52.103567 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.103549 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Aug 04 15:06:52.113026 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.112716 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1a2285d4-f4ce-47be-80d4-7b4be7f930b1-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-tqzqn\" (UID: \"1a2285d4-f4ce-47be-80d4-7b4be7f930b1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-tqzqn" Aug 04 15:06:52.113026 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.112747 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j56cf\" (UniqueName: \"kubernetes.io/projected/1a2285d4-f4ce-47be-80d4-7b4be7f930b1-kube-api-access-j56cf\") pod \"kube-storage-version-migrator-operator-7878f4b88c-tqzqn\" (UID: \"1a2285d4-f4ce-47be-80d4-7b4be7f930b1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-tqzqn" Aug 04 15:06:52.113026 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.112772 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-registry-certificates\") pod \"image-registry-bddc5f76d-42zvh\" (UID: \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\") " pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" Aug 04 15:06:52.113026 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.112794 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-bound-sa-token\") pod \"image-registry-bddc5f76d-42zvh\" (UID: \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\") " pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" Aug 04 15:06:52.113362 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.113299 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0580fdab-0381-4d8e-be5f-bde5eb44e0e6-trusted-ca\") pod \"console-operator-5ccd6b445f-r8wkp\" (UID: \"0580fdab-0381-4d8e-be5f-bde5eb44e0e6\") " pod="openshift-console-operator/console-operator-5ccd6b445f-r8wkp" Aug 04 15:06:52.113362 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.113339 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0580fdab-0381-4d8e-be5f-bde5eb44e0e6-config\") pod \"console-operator-5ccd6b445f-r8wkp\" (UID: \"0580fdab-0381-4d8e-be5f-bde5eb44e0e6\") " pod="openshift-console-operator/console-operator-5ccd6b445f-r8wkp" Aug 04 15:06:52.113530 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.113372 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59h6n\" (UniqueName: \"kubernetes.io/projected/c4fa9178-411c-42ae-94fe-62d80cf8eb0b-kube-api-access-59h6n\") pod \"cluster-samples-operator-559cbf5745-68756\" (UID: \"c4fa9178-411c-42ae-94fe-62d80cf8eb0b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-68756" Aug 04 15:06:52.113530 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.113466 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2frq6\" (UniqueName: \"kubernetes.io/projected/f1b0f182-034b-4512-9bae-213071366d67-kube-api-access-2frq6\") pod \"volume-data-source-validator-5b8776669-r6lsj\" (UID: \"f1b0f182-034b-4512-9bae-213071366d67\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-r6lsj" Aug 04 15:06:52.113530 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.113496 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v66fh\" (UniqueName: \"kubernetes.io/projected/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-kube-api-access-v66fh\") pod \"image-registry-bddc5f76d-42zvh\" (UID: \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\") " pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" Aug 04 15:06:52.113639 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.113548 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0580fdab-0381-4d8e-be5f-bde5eb44e0e6-serving-cert\") pod \"console-operator-5ccd6b445f-r8wkp\" (UID: \"0580fdab-0381-4d8e-be5f-bde5eb44e0e6\") " pod="openshift-console-operator/console-operator-5ccd6b445f-r8wkp" Aug 04 15:06:52.113639 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.113602 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ld5qp\" (UniqueName: \"kubernetes.io/projected/0580fdab-0381-4d8e-be5f-bde5eb44e0e6-kube-api-access-ld5qp\") pod \"console-operator-5ccd6b445f-r8wkp\" (UID: \"0580fdab-0381-4d8e-be5f-bde5eb44e0e6\") " pod="openshift-console-operator/console-operator-5ccd6b445f-r8wkp" Aug 04 15:06:52.113698 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.113641 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-image-registry-private-configuration\") pod \"image-registry-bddc5f76d-42zvh\" (UID: \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\") " pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" Aug 04 15:06:52.113698 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.113671 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-trusted-ca\") pod \"image-registry-bddc5f76d-42zvh\" (UID: \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\") " pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" Aug 04 15:06:52.113698 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.113688 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-installation-pull-secrets\") pod \"image-registry-bddc5f76d-42zvh\" (UID: \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\") " pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" Aug 04 15:06:52.113786 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.113724 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1a2285d4-f4ce-47be-80d4-7b4be7f930b1-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-tqzqn\" (UID: \"1a2285d4-f4ce-47be-80d4-7b4be7f930b1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-tqzqn" Aug 04 15:06:52.113786 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.113744 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c4fa9178-411c-42ae-94fe-62d80cf8eb0b-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-68756\" (UID: \"c4fa9178-411c-42ae-94fe-62d80cf8eb0b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-68756" Aug 04 15:06:52.113786 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.113775 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-ca-trust-extracted\") pod \"image-registry-bddc5f76d-42zvh\" (UID: \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\") " pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" Aug 04 15:06:52.113874 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.113800 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-registry-tls\") pod \"image-registry-bddc5f76d-42zvh\" (UID: \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\") " pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" Aug 04 15:06:52.113977 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.113964 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-hqwr6"] Aug 04 15:06:52.114092 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.114077 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7fc97bc64b-45b4c" Aug 04 15:06:52.117691 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.117675 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Aug 04 15:06:52.118018 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.118002 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Aug 04 15:06:52.118193 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.118172 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Aug 04 15:06:52.118636 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.118617 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-g4lk6\"" Aug 04 15:06:52.119244 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.119223 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Aug 04 15:06:52.119615 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.119599 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Aug 04 15:06:52.119827 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.119603 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Aug 04 15:06:52.126355 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.126338 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-6qdqp"] Aug 04 15:06:52.126582 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.126565 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hqwr6" Aug 04 15:06:52.129450 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.129433 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Aug 04 15:06:52.130411 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.130365 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kghgl\"" Aug 04 15:06:52.130548 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.130371 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Aug 04 15:06:52.146927 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.146906 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-6qdqp" Aug 04 15:06:52.146927 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.146905 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rcqrj"] Aug 04 15:06:52.149756 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.149735 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Aug 04 15:06:52.150924 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.150903 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-lbk76\"" Aug 04 15:06:52.151972 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.151946 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Aug 04 15:06:52.152151 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.152132 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Aug 04 15:06:52.152211 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.152200 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Aug 04 15:06:52.167002 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.166955 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-vz7vb"] Aug 04 15:06:52.167100 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.167049 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rcqrj" Aug 04 15:06:52.170690 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.170671 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Aug 04 15:06:52.170788 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.170720 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-49dc4\"" Aug 04 15:06:52.170788 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.170780 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Aug 04 15:06:52.170949 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.170932 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Aug 04 15:06:52.189447 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.189419 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-7jlpj"] Aug 04 15:06:52.189581 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.189564 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-vz7vb" Aug 04 15:06:52.193136 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.193116 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-7hhnd\"" Aug 04 15:06:52.211534 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.211517 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-48zkn"] Aug 04 15:06:52.211635 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.211550 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hqwr6"] Aug 04 15:06:52.211635 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.211575 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-7jlpj"] Aug 04 15:06:52.211635 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.211593 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-6qdqp"] Aug 04 15:06:52.211635 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.211615 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7fc97bc64b-45b4c"] Aug 04 15:06:52.211635 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.211629 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-7jlpj" Aug 04 15:06:52.211861 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.211633 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rcqrj"] Aug 04 15:06:52.211861 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.211782 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-vz7vb"] Aug 04 15:06:52.214166 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.214144 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1a2285d4-f4ce-47be-80d4-7b4be7f930b1-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-tqzqn\" (UID: \"1a2285d4-f4ce-47be-80d4-7b4be7f930b1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-tqzqn" Aug 04 15:06:52.214305 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.214289 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c4fa9178-411c-42ae-94fe-62d80cf8eb0b-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-68756\" (UID: \"c4fa9178-411c-42ae-94fe-62d80cf8eb0b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-68756" Aug 04 15:06:52.214381 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.214349 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnbvv\" (UniqueName: \"kubernetes.io/projected/ded4fb33-8f70-4475-9a73-e8fff5c7e638-kube-api-access-xnbvv\") pod \"dns-default-hqwr6\" (UID: \"ded4fb33-8f70-4475-9a73-e8fff5c7e638\") " pod="openshift-dns/dns-default-hqwr6" Aug 04 15:06:52.214444 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.214379 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91c9aec0-0301-40bd-8bbd-ebd389fb4faf-trusted-ca-bundle\") pod \"insights-operator-77696685b-kpmh2\" (UID: \"91c9aec0-0301-40bd-8bbd-ebd389fb4faf\") " pod="openshift-insights/insights-operator-77696685b-kpmh2" Aug 04 15:06:52.214444 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.214428 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e011eeba-5d48-483b-905a-a1f7b7e0ec1f-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-48zkn\" (UID: \"e011eeba-5d48-483b-905a-a1f7b7e0ec1f\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-48zkn" Aug 04 15:06:52.214551 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.214482 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j56cf\" (UniqueName: \"kubernetes.io/projected/1a2285d4-f4ce-47be-80d4-7b4be7f930b1-kube-api-access-j56cf\") pod \"kube-storage-version-migrator-operator-7878f4b88c-tqzqn\" (UID: \"1a2285d4-f4ce-47be-80d4-7b4be7f930b1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-tqzqn" Aug 04 15:06:52.214551 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.214510 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-gbrf7\"" Aug 04 15:06:52.214551 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.214539 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/91c9aec0-0301-40bd-8bbd-ebd389fb4faf-serving-cert\") pod \"insights-operator-77696685b-kpmh2\" (UID: \"91c9aec0-0301-40bd-8bbd-ebd389fb4faf\") " pod="openshift-insights/insights-operator-77696685b-kpmh2" Aug 04 15:06:52.214698 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.214576 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0580fdab-0381-4d8e-be5f-bde5eb44e0e6-trusted-ca\") pod \"console-operator-5ccd6b445f-r8wkp\" (UID: \"0580fdab-0381-4d8e-be5f-bde5eb44e0e6\") " pod="openshift-console-operator/console-operator-5ccd6b445f-r8wkp" Aug 04 15:06:52.214698 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:52.214600 2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Aug 04 15:06:52.214698 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:52.214652 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4fa9178-411c-42ae-94fe-62d80cf8eb0b-samples-operator-tls podName:c4fa9178-411c-42ae-94fe-62d80cf8eb0b nodeName:}" failed. No retries permitted until 2026-08-04 15:06:52.714632893 +0000 UTC m=+36.174505221 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c4fa9178-411c-42ae-94fe-62d80cf8eb0b-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-68756" (UID: "c4fa9178-411c-42ae-94fe-62d80cf8eb0b") : secret "samples-operator-tls" not found Aug 04 15:06:52.214698 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.214513 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Aug 04 15:06:52.214896 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.214738 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Aug 04 15:06:52.214896 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.214787 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Aug 04 15:06:52.214896 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.214601 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0580fdab-0381-4d8e-be5f-bde5eb44e0e6-config\") pod \"console-operator-5ccd6b445f-r8wkp\" (UID: \"0580fdab-0381-4d8e-be5f-bde5eb44e0e6\") " pod="openshift-console-operator/console-operator-5ccd6b445f-r8wkp" Aug 04 15:06:52.215034 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.214999 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2frq6\" (UniqueName: \"kubernetes.io/projected/f1b0f182-034b-4512-9bae-213071366d67-kube-api-access-2frq6\") pod \"volume-data-source-validator-5b8776669-r6lsj\" (UID: \"f1b0f182-034b-4512-9bae-213071366d67\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-r6lsj" Aug 04 15:06:52.215094 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.215030 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v66fh\" (UniqueName: \"kubernetes.io/projected/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-kube-api-access-v66fh\") pod \"image-registry-bddc5f76d-42zvh\" (UID: \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\") " pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" Aug 04 15:06:52.215094 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.215061 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0580fdab-0381-4d8e-be5f-bde5eb44e0e6-serving-cert\") pod \"console-operator-5ccd6b445f-r8wkp\" (UID: \"0580fdab-0381-4d8e-be5f-bde5eb44e0e6\") " pod="openshift-console-operator/console-operator-5ccd6b445f-r8wkp" Aug 04 15:06:52.215184 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.215093 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqhtq\" (UniqueName: \"kubernetes.io/projected/d201490f-504d-4d2a-8b4f-c4eea2023469-kube-api-access-rqhtq\") pod \"router-default-7fc97bc64b-45b4c\" (UID: \"d201490f-504d-4d2a-8b4f-c4eea2023469\") " pod="openshift-ingress/router-default-7fc97bc64b-45b4c" Aug 04 15:06:52.215184 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.215132 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ld5qp\" (UniqueName: \"kubernetes.io/projected/0580fdab-0381-4d8e-be5f-bde5eb44e0e6-kube-api-access-ld5qp\") pod \"console-operator-5ccd6b445f-r8wkp\" (UID: \"0580fdab-0381-4d8e-be5f-bde5eb44e0e6\") " pod="openshift-console-operator/console-operator-5ccd6b445f-r8wkp" Aug 04 15:06:52.215184 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.215171 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d201490f-504d-4d2a-8b4f-c4eea2023469-service-ca-bundle\") pod \"router-default-7fc97bc64b-45b4c\" (UID: \"d201490f-504d-4d2a-8b4f-c4eea2023469\") " pod="openshift-ingress/router-default-7fc97bc64b-45b4c" Aug 04 15:06:52.215332 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.215197 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-image-registry-private-configuration\") pod \"image-registry-bddc5f76d-42zvh\" (UID: \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\") " pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" Aug 04 15:06:52.215332 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.215185 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Aug 04 15:06:52.215332 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.215227 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ded4fb33-8f70-4475-9a73-e8fff5c7e638-metrics-tls\") pod \"dns-default-hqwr6\" (UID: \"ded4fb33-8f70-4475-9a73-e8fff5c7e638\") " pod="openshift-dns/dns-default-hqwr6" Aug 04 15:06:52.215332 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.215264 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0580fdab-0381-4d8e-be5f-bde5eb44e0e6-config\") pod \"console-operator-5ccd6b445f-r8wkp\" (UID: \"0580fdab-0381-4d8e-be5f-bde5eb44e0e6\") " pod="openshift-console-operator/console-operator-5ccd6b445f-r8wkp" Aug 04 15:06:52.215551 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.215425 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-trusted-ca\") pod \"image-registry-bddc5f76d-42zvh\" (UID: \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\") " pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" Aug 04 15:06:52.215551 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.215482 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-installation-pull-secrets\") pod \"image-registry-bddc5f76d-42zvh\" (UID: \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\") " pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" Aug 04 15:06:52.215551 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.215517 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q26l8\" (UniqueName: \"kubernetes.io/projected/91c9aec0-0301-40bd-8bbd-ebd389fb4faf-kube-api-access-q26l8\") pod \"insights-operator-77696685b-kpmh2\" (UID: \"91c9aec0-0301-40bd-8bbd-ebd389fb4faf\") " pod="openshift-insights/insights-operator-77696685b-kpmh2" Aug 04 15:06:52.215698 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.215593 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d201490f-504d-4d2a-8b4f-c4eea2023469-metrics-certs\") pod \"router-default-7fc97bc64b-45b4c\" (UID: \"d201490f-504d-4d2a-8b4f-c4eea2023469\") " pod="openshift-ingress/router-default-7fc97bc64b-45b4c" Aug 04 15:06:52.215698 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.215626 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e011eeba-5d48-483b-905a-a1f7b7e0ec1f-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-48zkn\" (UID: \"e011eeba-5d48-483b-905a-a1f7b7e0ec1f\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-48zkn" Aug 04 15:06:52.215698 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.215651 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ded4fb33-8f70-4475-9a73-e8fff5c7e638-config-volume\") pod \"dns-default-hqwr6\" (UID: \"ded4fb33-8f70-4475-9a73-e8fff5c7e638\") " pod="openshift-dns/dns-default-hqwr6" Aug 04 15:06:52.215698 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.215677 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e46dbe2d-eb92-47cd-9bea-47228f399ba3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-6qdqp\" (UID: \"e46dbe2d-eb92-47cd-9bea-47228f399ba3\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-6qdqp" Aug 04 15:06:52.215902 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.215723 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lmfm\" (UniqueName: \"kubernetes.io/projected/e46dbe2d-eb92-47cd-9bea-47228f399ba3-kube-api-access-5lmfm\") pod \"cluster-monitoring-operator-6f55c4f58d-6qdqp\" (UID: \"e46dbe2d-eb92-47cd-9bea-47228f399ba3\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-6qdqp" Aug 04 15:06:52.215902 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.215750 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/91c9aec0-0301-40bd-8bbd-ebd389fb4faf-snapshots\") pod \"insights-operator-77696685b-kpmh2\" (UID: \"91c9aec0-0301-40bd-8bbd-ebd389fb4faf\") " pod="openshift-insights/insights-operator-77696685b-kpmh2" Aug 04 15:06:52.215902 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.215763 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0580fdab-0381-4d8e-be5f-bde5eb44e0e6-trusted-ca\") pod \"console-operator-5ccd6b445f-r8wkp\" (UID: \"0580fdab-0381-4d8e-be5f-bde5eb44e0e6\") " pod="openshift-console-operator/console-operator-5ccd6b445f-r8wkp" Aug 04 15:06:52.215902 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.215780 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-ca-trust-extracted\") pod \"image-registry-bddc5f76d-42zvh\" (UID: \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\") " pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" Aug 04 15:06:52.215902 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.215818 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-registry-tls\") pod \"image-registry-bddc5f76d-42zvh\" (UID: \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\") " pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" Aug 04 15:06:52.215902 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.215853 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/91c9aec0-0301-40bd-8bbd-ebd389fb4faf-tmp\") pod \"insights-operator-77696685b-kpmh2\" (UID: \"91c9aec0-0301-40bd-8bbd-ebd389fb4faf\") " pod="openshift-insights/insights-operator-77696685b-kpmh2" Aug 04 15:06:52.216163 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:52.215903 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 04 15:06:52.216163 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:52.215920 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bddc5f76d-42zvh: secret "image-registry-tls" not found Aug 04 15:06:52.216163 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:52.215968 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-registry-tls podName:aab2968d-11b7-4174-b0c4-b3ce5a46b90e nodeName:}" failed. No retries permitted until 2026-08-04 15:06:52.715953542 +0000 UTC m=+36.175825868 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-registry-tls") pod "image-registry-bddc5f76d-42zvh" (UID: "aab2968d-11b7-4174-b0c4-b3ce5a46b90e") : secret "image-registry-tls" not found Aug 04 15:06:52.216163 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.216031 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1a2285d4-f4ce-47be-80d4-7b4be7f930b1-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-tqzqn\" (UID: \"1a2285d4-f4ce-47be-80d4-7b4be7f930b1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-tqzqn" Aug 04 15:06:52.216163 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.216079 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-registry-certificates\") pod \"image-registry-bddc5f76d-42zvh\" (UID: \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\") " pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" Aug 04 15:06:52.216432 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.216228 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-ca-trust-extracted\") pod \"image-registry-bddc5f76d-42zvh\" (UID: \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\") " pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" Aug 04 15:06:52.216432 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.216273 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-bound-sa-token\") pod \"image-registry-bddc5f76d-42zvh\" (UID: \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\") " pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" Aug 04 15:06:52.216432 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.216298 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d201490f-504d-4d2a-8b4f-c4eea2023469-default-certificate\") pod \"router-default-7fc97bc64b-45b4c\" (UID: \"d201490f-504d-4d2a-8b4f-c4eea2023469\") " pod="openshift-ingress/router-default-7fc97bc64b-45b4c" Aug 04 15:06:52.216432 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.216330 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e46dbe2d-eb92-47cd-9bea-47228f399ba3-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-6qdqp\" (UID: \"e46dbe2d-eb92-47cd-9bea-47228f399ba3\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-6qdqp" Aug 04 15:06:52.216432 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.216355 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ded4fb33-8f70-4475-9a73-e8fff5c7e638-tmp-dir\") pod \"dns-default-hqwr6\" (UID: \"ded4fb33-8f70-4475-9a73-e8fff5c7e638\") " pod="openshift-dns/dns-default-hqwr6" Aug 04 15:06:52.216689 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.216559 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1a2285d4-f4ce-47be-80d4-7b4be7f930b1-config\") pod \"kube-storage-version-migrator-operator-7878f4b88c-tqzqn\" (UID: \"1a2285d4-f4ce-47be-80d4-7b4be7f930b1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-tqzqn" Aug 04 15:06:52.216689 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.216632 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59h6n\" (UniqueName: \"kubernetes.io/projected/c4fa9178-411c-42ae-94fe-62d80cf8eb0b-kube-api-access-59h6n\") pod \"cluster-samples-operator-559cbf5745-68756\" (UID: \"c4fa9178-411c-42ae-94fe-62d80cf8eb0b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-68756" Aug 04 15:06:52.216800 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.216685 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-trusted-ca\") pod \"image-registry-bddc5f76d-42zvh\" (UID: \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\") " pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" Aug 04 15:06:52.216800 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.216696 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91c9aec0-0301-40bd-8bbd-ebd389fb4faf-service-ca-bundle\") pod \"insights-operator-77696685b-kpmh2\" (UID: \"91c9aec0-0301-40bd-8bbd-ebd389fb4faf\") " pod="openshift-insights/insights-operator-77696685b-kpmh2" Aug 04 15:06:52.216800 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.216762 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d201490f-504d-4d2a-8b4f-c4eea2023469-stats-auth\") pod \"router-default-7fc97bc64b-45b4c\" (UID: \"d201490f-504d-4d2a-8b4f-c4eea2023469\") " pod="openshift-ingress/router-default-7fc97bc64b-45b4c" Aug 04 15:06:52.216951 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.216842 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-registry-certificates\") pod \"image-registry-bddc5f76d-42zvh\" (UID: \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\") " pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" Aug 04 15:06:52.219813 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.219781 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0580fdab-0381-4d8e-be5f-bde5eb44e0e6-serving-cert\") pod \"console-operator-5ccd6b445f-r8wkp\" (UID: \"0580fdab-0381-4d8e-be5f-bde5eb44e0e6\") " pod="openshift-console-operator/console-operator-5ccd6b445f-r8wkp" Aug 04 15:06:52.220443 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.220056 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-image-registry-private-configuration\") pod \"image-registry-bddc5f76d-42zvh\" (UID: \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\") " pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" Aug 04 15:06:52.220443 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.220057 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-installation-pull-secrets\") pod \"image-registry-bddc5f76d-42zvh\" (UID: \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\") " pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" Aug 04 15:06:52.220443 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.220108 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1a2285d4-f4ce-47be-80d4-7b4be7f930b1-serving-cert\") pod \"kube-storage-version-migrator-operator-7878f4b88c-tqzqn\" (UID: \"1a2285d4-f4ce-47be-80d4-7b4be7f930b1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-tqzqn" Aug 04 15:06:52.234863 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.234834 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2frq6\" (UniqueName: \"kubernetes.io/projected/f1b0f182-034b-4512-9bae-213071366d67-kube-api-access-2frq6\") pod \"volume-data-source-validator-5b8776669-r6lsj\" (UID: \"f1b0f182-034b-4512-9bae-213071366d67\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-r6lsj" Aug 04 15:06:52.237123 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.237094 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j56cf\" (UniqueName: \"kubernetes.io/projected/1a2285d4-f4ce-47be-80d4-7b4be7f930b1-kube-api-access-j56cf\") pod \"kube-storage-version-migrator-operator-7878f4b88c-tqzqn\" (UID: \"1a2285d4-f4ce-47be-80d4-7b4be7f930b1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-tqzqn" Aug 04 15:06:52.237477 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.237456 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ld5qp\" (UniqueName: \"kubernetes.io/projected/0580fdab-0381-4d8e-be5f-bde5eb44e0e6-kube-api-access-ld5qp\") pod \"console-operator-5ccd6b445f-r8wkp\" (UID: \"0580fdab-0381-4d8e-be5f-bde5eb44e0e6\") " pod="openshift-console-operator/console-operator-5ccd6b445f-r8wkp" Aug 04 15:06:52.237591 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.237568 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v66fh\" (UniqueName: \"kubernetes.io/projected/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-kube-api-access-v66fh\") pod \"image-registry-bddc5f76d-42zvh\" (UID: \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\") " pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" Aug 04 15:06:52.238402 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.238373 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-bound-sa-token\") pod \"image-registry-bddc5f76d-42zvh\" (UID: \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\") " pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" Aug 04 15:06:52.239689 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.239658 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59h6n\" (UniqueName: \"kubernetes.io/projected/c4fa9178-411c-42ae-94fe-62d80cf8eb0b-kube-api-access-59h6n\") pod \"cluster-samples-operator-559cbf5745-68756\" (UID: \"c4fa9178-411c-42ae-94fe-62d80cf8eb0b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-68756" Aug 04 15:06:52.309654 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.309625 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-r6lsj" Aug 04 15:06:52.317888 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.317857 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91c9aec0-0301-40bd-8bbd-ebd389fb4faf-service-ca-bundle\") pod \"insights-operator-77696685b-kpmh2\" (UID: \"91c9aec0-0301-40bd-8bbd-ebd389fb4faf\") " pod="openshift-insights/insights-operator-77696685b-kpmh2" Aug 04 15:06:52.318024 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.317894 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d201490f-504d-4d2a-8b4f-c4eea2023469-stats-auth\") pod \"router-default-7fc97bc64b-45b4c\" (UID: \"d201490f-504d-4d2a-8b4f-c4eea2023469\") " pod="openshift-ingress/router-default-7fc97bc64b-45b4c" Aug 04 15:06:52.318024 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.317958 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ad87b28c-e7a3-4354-af24-4e0d4aa3f464-config\") pod \"service-ca-operator-784f97bcf6-7jlpj\" (UID: \"ad87b28c-e7a3-4354-af24-4e0d4aa3f464\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-7jlpj" Aug 04 15:06:52.318024 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.317987 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnbvv\" (UniqueName: \"kubernetes.io/projected/ded4fb33-8f70-4475-9a73-e8fff5c7e638-kube-api-access-xnbvv\") pod \"dns-default-hqwr6\" (UID: \"ded4fb33-8f70-4475-9a73-e8fff5c7e638\") " pod="openshift-dns/dns-default-hqwr6" Aug 04 15:06:52.318024 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.318010 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91c9aec0-0301-40bd-8bbd-ebd389fb4faf-trusted-ca-bundle\") pod \"insights-operator-77696685b-kpmh2\" (UID: \"91c9aec0-0301-40bd-8bbd-ebd389fb4faf\") " pod="openshift-insights/insights-operator-77696685b-kpmh2" Aug 04 15:06:52.318255 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.318124 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e011eeba-5d48-483b-905a-a1f7b7e0ec1f-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-48zkn\" (UID: \"e011eeba-5d48-483b-905a-a1f7b7e0ec1f\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-48zkn" Aug 04 15:06:52.318255 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.318171 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/91c9aec0-0301-40bd-8bbd-ebd389fb4faf-serving-cert\") pod \"insights-operator-77696685b-kpmh2\" (UID: \"91c9aec0-0301-40bd-8bbd-ebd389fb4faf\") " pod="openshift-insights/insights-operator-77696685b-kpmh2" Aug 04 15:06:52.318255 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.318201 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88dwq\" (UniqueName: \"kubernetes.io/projected/ad87b28c-e7a3-4354-af24-4e0d4aa3f464-kube-api-access-88dwq\") pod \"service-ca-operator-784f97bcf6-7jlpj\" (UID: \"ad87b28c-e7a3-4354-af24-4e0d4aa3f464\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-7jlpj" Aug 04 15:06:52.318430 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:52.318264 2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Aug 04 15:06:52.318430 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:52.318328 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e011eeba-5d48-483b-905a-a1f7b7e0ec1f-networking-console-plugin-cert podName:e011eeba-5d48-483b-905a-a1f7b7e0ec1f nodeName:}" failed. No retries permitted until 2026-08-04 15:06:52.818309347 +0000 UTC m=+36.278181684 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e011eeba-5d48-483b-905a-a1f7b7e0ec1f-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-48zkn" (UID: "e011eeba-5d48-483b-905a-a1f7b7e0ec1f") : secret "networking-console-plugin-cert" not found Aug 04 15:06:52.318430 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.318346 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqhtq\" (UniqueName: \"kubernetes.io/projected/d201490f-504d-4d2a-8b4f-c4eea2023469-kube-api-access-rqhtq\") pod \"router-default-7fc97bc64b-45b4c\" (UID: \"d201490f-504d-4d2a-8b4f-c4eea2023469\") " pod="openshift-ingress/router-default-7fc97bc64b-45b4c" Aug 04 15:06:52.318430 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.318385 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d201490f-504d-4d2a-8b4f-c4eea2023469-service-ca-bundle\") pod \"router-default-7fc97bc64b-45b4c\" (UID: \"d201490f-504d-4d2a-8b4f-c4eea2023469\") " pod="openshift-ingress/router-default-7fc97bc64b-45b4c" Aug 04 15:06:52.318690 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.318456 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91c9aec0-0301-40bd-8bbd-ebd389fb4faf-service-ca-bundle\") pod \"insights-operator-77696685b-kpmh2\" (UID: \"91c9aec0-0301-40bd-8bbd-ebd389fb4faf\") " pod="openshift-insights/insights-operator-77696685b-kpmh2" Aug 04 15:06:52.318690 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:52.318483 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d201490f-504d-4d2a-8b4f-c4eea2023469-service-ca-bundle podName:d201490f-504d-4d2a-8b4f-c4eea2023469 nodeName:}" failed. No retries permitted until 2026-08-04 15:06:52.818470606 +0000 UTC m=+36.278342950 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d201490f-504d-4d2a-8b4f-c4eea2023469-service-ca-bundle") pod "router-default-7fc97bc64b-45b4c" (UID: "d201490f-504d-4d2a-8b4f-c4eea2023469") : configmap references non-existent config key: service-ca.crt Aug 04 15:06:52.318690 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.318537 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ad87b28c-e7a3-4354-af24-4e0d4aa3f464-serving-cert\") pod \"service-ca-operator-784f97bcf6-7jlpj\" (UID: \"ad87b28c-e7a3-4354-af24-4e0d4aa3f464\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-7jlpj" Aug 04 15:06:52.318690 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.318576 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ded4fb33-8f70-4475-9a73-e8fff5c7e638-metrics-tls\") pod \"dns-default-hqwr6\" (UID: \"ded4fb33-8f70-4475-9a73-e8fff5c7e638\") " pod="openshift-dns/dns-default-hqwr6" Aug 04 15:06:52.318690 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:52.318636 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 15:06:52.318690 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.318656 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q26l8\" (UniqueName: \"kubernetes.io/projected/91c9aec0-0301-40bd-8bbd-ebd389fb4faf-kube-api-access-q26l8\") pod \"insights-operator-77696685b-kpmh2\" (UID: \"91c9aec0-0301-40bd-8bbd-ebd389fb4faf\") " pod="openshift-insights/insights-operator-77696685b-kpmh2" Aug 04 15:06:52.318690 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:52.318689 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ded4fb33-8f70-4475-9a73-e8fff5c7e638-metrics-tls podName:ded4fb33-8f70-4475-9a73-e8fff5c7e638 nodeName:}" failed. No retries permitted until 2026-08-04 15:06:52.818673921 +0000 UTC m=+36.278546261 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ded4fb33-8f70-4475-9a73-e8fff5c7e638-metrics-tls") pod "dns-default-hqwr6" (UID: "ded4fb33-8f70-4475-9a73-e8fff5c7e638") : secret "dns-default-metrics-tls" not found Aug 04 15:06:52.319088 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.318715 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d201490f-504d-4d2a-8b4f-c4eea2023469-metrics-certs\") pod \"router-default-7fc97bc64b-45b4c\" (UID: \"d201490f-504d-4d2a-8b4f-c4eea2023469\") " pod="openshift-ingress/router-default-7fc97bc64b-45b4c" Aug 04 15:06:52.319088 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.318745 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e011eeba-5d48-483b-905a-a1f7b7e0ec1f-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-48zkn\" (UID: \"e011eeba-5d48-483b-905a-a1f7b7e0ec1f\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-48zkn" Aug 04 15:06:52.319088 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.318771 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ded4fb33-8f70-4475-9a73-e8fff5c7e638-config-volume\") pod \"dns-default-hqwr6\" (UID: \"ded4fb33-8f70-4475-9a73-e8fff5c7e638\") " pod="openshift-dns/dns-default-hqwr6" Aug 04 15:06:52.319088 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:52.318835 2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Aug 04 15:06:52.319088 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:52.318903 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d201490f-504d-4d2a-8b4f-c4eea2023469-metrics-certs podName:d201490f-504d-4d2a-8b4f-c4eea2023469 nodeName:}" failed. No retries permitted until 2026-08-04 15:06:52.818865645 +0000 UTC m=+36.278737972 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d201490f-504d-4d2a-8b4f-c4eea2023469-metrics-certs") pod "router-default-7fc97bc64b-45b4c" (UID: "d201490f-504d-4d2a-8b4f-c4eea2023469") : secret "router-metrics-certs-default" not found Aug 04 15:06:52.319088 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.318920 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e46dbe2d-eb92-47cd-9bea-47228f399ba3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-6qdqp\" (UID: \"e46dbe2d-eb92-47cd-9bea-47228f399ba3\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-6qdqp" Aug 04 15:06:52.319088 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.318955 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lmfm\" (UniqueName: \"kubernetes.io/projected/e46dbe2d-eb92-47cd-9bea-47228f399ba3-kube-api-access-5lmfm\") pod \"cluster-monitoring-operator-6f55c4f58d-6qdqp\" (UID: \"e46dbe2d-eb92-47cd-9bea-47228f399ba3\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-6qdqp" Aug 04 15:06:52.319088 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.318981 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/91c9aec0-0301-40bd-8bbd-ebd389fb4faf-snapshots\") pod \"insights-operator-77696685b-kpmh2\" (UID: \"91c9aec0-0301-40bd-8bbd-ebd389fb4faf\") " pod="openshift-insights/insights-operator-77696685b-kpmh2" Aug 04 15:06:52.319088 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.318988 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/91c9aec0-0301-40bd-8bbd-ebd389fb4faf-trusted-ca-bundle\") pod \"insights-operator-77696685b-kpmh2\" (UID: \"91c9aec0-0301-40bd-8bbd-ebd389fb4faf\") " pod="openshift-insights/insights-operator-77696685b-kpmh2" Aug 04 15:06:52.319088 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.319044 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/91c9aec0-0301-40bd-8bbd-ebd389fb4faf-tmp\") pod \"insights-operator-77696685b-kpmh2\" (UID: \"91c9aec0-0301-40bd-8bbd-ebd389fb4faf\") " pod="openshift-insights/insights-operator-77696685b-kpmh2" Aug 04 15:06:52.319088 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:52.319064 2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Aug 04 15:06:52.319088 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.319072 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cnf9\" (UniqueName: \"kubernetes.io/projected/5b4f499c-41e3-4d50-a426-84a3f3926bea-kube-api-access-7cnf9\") pod \"network-check-source-65bdbbff45-vz7vb\" (UID: \"5b4f499c-41e3-4d50-a426-84a3f3926bea\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-vz7vb" Aug 04 15:06:52.319708 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:52.319103 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e46dbe2d-eb92-47cd-9bea-47228f399ba3-cluster-monitoring-operator-tls podName:e46dbe2d-eb92-47cd-9bea-47228f399ba3 nodeName:}" failed. No retries permitted until 2026-08-04 15:06:52.819091292 +0000 UTC m=+36.278963634 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e46dbe2d-eb92-47cd-9bea-47228f399ba3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-6qdqp" (UID: "e46dbe2d-eb92-47cd-9bea-47228f399ba3") : secret "cluster-monitoring-operator-tls" not found Aug 04 15:06:52.319708 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.319123 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d201490f-504d-4d2a-8b4f-c4eea2023469-default-certificate\") pod \"router-default-7fc97bc64b-45b4c\" (UID: \"d201490f-504d-4d2a-8b4f-c4eea2023469\") " pod="openshift-ingress/router-default-7fc97bc64b-45b4c" Aug 04 15:06:52.319708 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.319151 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e46dbe2d-eb92-47cd-9bea-47228f399ba3-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-6qdqp\" (UID: \"e46dbe2d-eb92-47cd-9bea-47228f399ba3\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-6qdqp" Aug 04 15:06:52.319708 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.319177 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5c7c\" (UniqueName: \"kubernetes.io/projected/43e78d75-11c3-457d-aec1-5d4379ab2470-kube-api-access-h5c7c\") pod \"ingress-canary-rcqrj\" (UID: \"43e78d75-11c3-457d-aec1-5d4379ab2470\") " pod="openshift-ingress-canary/ingress-canary-rcqrj" Aug 04 15:06:52.319708 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.319205 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ded4fb33-8f70-4475-9a73-e8fff5c7e638-tmp-dir\") pod \"dns-default-hqwr6\" (UID: \"ded4fb33-8f70-4475-9a73-e8fff5c7e638\") " pod="openshift-dns/dns-default-hqwr6" Aug 04 15:06:52.319708 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.319230 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/43e78d75-11c3-457d-aec1-5d4379ab2470-cert\") pod \"ingress-canary-rcqrj\" (UID: \"43e78d75-11c3-457d-aec1-5d4379ab2470\") " pod="openshift-ingress-canary/ingress-canary-rcqrj" Aug 04 15:06:52.319708 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.319343 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ded4fb33-8f70-4475-9a73-e8fff5c7e638-config-volume\") pod \"dns-default-hqwr6\" (UID: \"ded4fb33-8f70-4475-9a73-e8fff5c7e638\") " pod="openshift-dns/dns-default-hqwr6" Aug 04 15:06:52.319708 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.319453 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e011eeba-5d48-483b-905a-a1f7b7e0ec1f-nginx-conf\") pod \"networking-console-plugin-f77f48c6d-48zkn\" (UID: \"e011eeba-5d48-483b-905a-a1f7b7e0ec1f\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-48zkn" Aug 04 15:06:52.319708 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.319567 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ded4fb33-8f70-4475-9a73-e8fff5c7e638-tmp-dir\") pod \"dns-default-hqwr6\" (UID: \"ded4fb33-8f70-4475-9a73-e8fff5c7e638\") " pod="openshift-dns/dns-default-hqwr6" Aug 04 15:06:52.319708 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.319639 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/91c9aec0-0301-40bd-8bbd-ebd389fb4faf-tmp\") pod \"insights-operator-77696685b-kpmh2\" (UID: \"91c9aec0-0301-40bd-8bbd-ebd389fb4faf\") " pod="openshift-insights/insights-operator-77696685b-kpmh2" Aug 04 15:06:52.320211 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.319901 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/91c9aec0-0301-40bd-8bbd-ebd389fb4faf-snapshots\") pod \"insights-operator-77696685b-kpmh2\" (UID: \"91c9aec0-0301-40bd-8bbd-ebd389fb4faf\") " pod="openshift-insights/insights-operator-77696685b-kpmh2" Aug 04 15:06:52.320211 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.320166 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e46dbe2d-eb92-47cd-9bea-47228f399ba3-telemetry-config\") pod \"cluster-monitoring-operator-6f55c4f58d-6qdqp\" (UID: \"e46dbe2d-eb92-47cd-9bea-47228f399ba3\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-6qdqp" Aug 04 15:06:52.320707 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.320685 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/91c9aec0-0301-40bd-8bbd-ebd389fb4faf-serving-cert\") pod \"insights-operator-77696685b-kpmh2\" (UID: \"91c9aec0-0301-40bd-8bbd-ebd389fb4faf\") " pod="openshift-insights/insights-operator-77696685b-kpmh2" Aug 04 15:06:52.320828 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.320776 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d201490f-504d-4d2a-8b4f-c4eea2023469-stats-auth\") pod \"router-default-7fc97bc64b-45b4c\" (UID: \"d201490f-504d-4d2a-8b4f-c4eea2023469\") " pod="openshift-ingress/router-default-7fc97bc64b-45b4c" Aug 04 15:06:52.321406 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.321375 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d201490f-504d-4d2a-8b4f-c4eea2023469-default-certificate\") pod \"router-default-7fc97bc64b-45b4c\" (UID: \"d201490f-504d-4d2a-8b4f-c4eea2023469\") " pod="openshift-ingress/router-default-7fc97bc64b-45b4c" Aug 04 15:06:52.330757 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.330742 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-tqzqn" Aug 04 15:06:52.331265 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.331242 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnbvv\" (UniqueName: \"kubernetes.io/projected/ded4fb33-8f70-4475-9a73-e8fff5c7e638-kube-api-access-xnbvv\") pod \"dns-default-hqwr6\" (UID: \"ded4fb33-8f70-4475-9a73-e8fff5c7e638\") " pod="openshift-dns/dns-default-hqwr6" Aug 04 15:06:52.331642 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.331621 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lmfm\" (UniqueName: \"kubernetes.io/projected/e46dbe2d-eb92-47cd-9bea-47228f399ba3-kube-api-access-5lmfm\") pod \"cluster-monitoring-operator-6f55c4f58d-6qdqp\" (UID: \"e46dbe2d-eb92-47cd-9bea-47228f399ba3\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-6qdqp" Aug 04 15:06:52.333218 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.333201 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q26l8\" (UniqueName: \"kubernetes.io/projected/91c9aec0-0301-40bd-8bbd-ebd389fb4faf-kube-api-access-q26l8\") pod \"insights-operator-77696685b-kpmh2\" (UID: \"91c9aec0-0301-40bd-8bbd-ebd389fb4faf\") " pod="openshift-insights/insights-operator-77696685b-kpmh2" Aug 04 15:06:52.333639 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.333619 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqhtq\" (UniqueName: \"kubernetes.io/projected/d201490f-504d-4d2a-8b4f-c4eea2023469-kube-api-access-rqhtq\") pod \"router-default-7fc97bc64b-45b4c\" (UID: \"d201490f-504d-4d2a-8b4f-c4eea2023469\") " pod="openshift-ingress/router-default-7fc97bc64b-45b4c" Aug 04 15:06:52.347879 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.347853 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-5ccd6b445f-r8wkp" Aug 04 15:06:52.386421 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.386321 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-77696685b-kpmh2" Aug 04 15:06:52.420546 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.420339 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cnf9\" (UniqueName: \"kubernetes.io/projected/5b4f499c-41e3-4d50-a426-84a3f3926bea-kube-api-access-7cnf9\") pod \"network-check-source-65bdbbff45-vz7vb\" (UID: \"5b4f499c-41e3-4d50-a426-84a3f3926bea\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-vz7vb" Aug 04 15:06:52.420655 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.420576 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5c7c\" (UniqueName: \"kubernetes.io/projected/43e78d75-11c3-457d-aec1-5d4379ab2470-kube-api-access-h5c7c\") pod \"ingress-canary-rcqrj\" (UID: \"43e78d75-11c3-457d-aec1-5d4379ab2470\") " pod="openshift-ingress-canary/ingress-canary-rcqrj" Aug 04 15:06:52.420655 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.420607 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/43e78d75-11c3-457d-aec1-5d4379ab2470-cert\") pod \"ingress-canary-rcqrj\" (UID: \"43e78d75-11c3-457d-aec1-5d4379ab2470\") " pod="openshift-ingress-canary/ingress-canary-rcqrj" Aug 04 15:06:52.420759 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.420690 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ad87b28c-e7a3-4354-af24-4e0d4aa3f464-config\") pod \"service-ca-operator-784f97bcf6-7jlpj\" (UID: \"ad87b28c-e7a3-4354-af24-4e0d4aa3f464\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-7jlpj" Aug 04 15:06:52.420759 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.420737 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88dwq\" (UniqueName: \"kubernetes.io/projected/ad87b28c-e7a3-4354-af24-4e0d4aa3f464-kube-api-access-88dwq\") pod \"service-ca-operator-784f97bcf6-7jlpj\" (UID: \"ad87b28c-e7a3-4354-af24-4e0d4aa3f464\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-7jlpj" Aug 04 15:06:52.420868 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.420805 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ad87b28c-e7a3-4354-af24-4e0d4aa3f464-serving-cert\") pod \"service-ca-operator-784f97bcf6-7jlpj\" (UID: \"ad87b28c-e7a3-4354-af24-4e0d4aa3f464\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-7jlpj" Aug 04 15:06:52.425422 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.425317 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/43e78d75-11c3-457d-aec1-5d4379ab2470-cert\") pod \"ingress-canary-rcqrj\" (UID: \"43e78d75-11c3-457d-aec1-5d4379ab2470\") " pod="openshift-ingress-canary/ingress-canary-rcqrj" Aug 04 15:06:52.427086 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.426543 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ad87b28c-e7a3-4354-af24-4e0d4aa3f464-config\") pod \"service-ca-operator-784f97bcf6-7jlpj\" (UID: \"ad87b28c-e7a3-4354-af24-4e0d4aa3f464\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-7jlpj" Aug 04 15:06:52.427086 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.426841 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ad87b28c-e7a3-4354-af24-4e0d4aa3f464-serving-cert\") pod \"service-ca-operator-784f97bcf6-7jlpj\" (UID: \"ad87b28c-e7a3-4354-af24-4e0d4aa3f464\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-7jlpj" Aug 04 15:06:52.441436 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.441373 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5c7c\" (UniqueName: \"kubernetes.io/projected/43e78d75-11c3-457d-aec1-5d4379ab2470-kube-api-access-h5c7c\") pod \"ingress-canary-rcqrj\" (UID: \"43e78d75-11c3-457d-aec1-5d4379ab2470\") " pod="openshift-ingress-canary/ingress-canary-rcqrj" Aug 04 15:06:52.442230 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.442140 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cnf9\" (UniqueName: \"kubernetes.io/projected/5b4f499c-41e3-4d50-a426-84a3f3926bea-kube-api-access-7cnf9\") pod \"network-check-source-65bdbbff45-vz7vb\" (UID: \"5b4f499c-41e3-4d50-a426-84a3f3926bea\") " pod="openshift-network-diagnostics/network-check-source-65bdbbff45-vz7vb" Aug 04 15:06:52.442928 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.442878 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88dwq\" (UniqueName: \"kubernetes.io/projected/ad87b28c-e7a3-4354-af24-4e0d4aa3f464-kube-api-access-88dwq\") pod \"service-ca-operator-784f97bcf6-7jlpj\" (UID: \"ad87b28c-e7a3-4354-af24-4e0d4aa3f464\") " pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-7jlpj" Aug 04 15:06:52.479366 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.478174 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rcqrj" Aug 04 15:06:52.500722 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.498569 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-vz7vb" Aug 04 15:06:52.530054 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.529855 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-7jlpj" Aug 04 15:06:52.549698 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.549535 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-tqzqn"] Aug 04 15:06:52.549698 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.549598 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-r6lsj"] Aug 04 15:06:52.557015 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:52.556922 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a2285d4_f4ce_47be_80d4_7b4be7f930b1.slice/crio-d963867acaca3c7fac692614f696352220eef5ecf00fea8778fa167f97562732 WatchSource:0}: Error finding container d963867acaca3c7fac692614f696352220eef5ecf00fea8778fa167f97562732: Status 404 returned error can't find the container with id d963867acaca3c7fac692614f696352220eef5ecf00fea8778fa167f97562732 Aug 04 15:06:52.557431 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:52.557327 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1b0f182_034b_4512_9bae_213071366d67.slice/crio-49234e3e6cf79f00ba0fc988c1a0759dfae08be37676cc29ea533edfdd60eb71 WatchSource:0}: Error finding container 49234e3e6cf79f00ba0fc988c1a0759dfae08be37676cc29ea533edfdd60eb71: Status 404 returned error can't find the container with id 49234e3e6cf79f00ba0fc988c1a0759dfae08be37676cc29ea533edfdd60eb71 Aug 04 15:06:52.564485 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.564311 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-5ccd6b445f-r8wkp"] Aug 04 15:06:52.571139 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:52.570909 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0580fdab_0381_4d8e_be5f_bde5eb44e0e6.slice/crio-db80d6661124c7b43cae783bc4fedbc5fd92b11a53d46dd02b82623d30583166 WatchSource:0}: Error finding container db80d6661124c7b43cae783bc4fedbc5fd92b11a53d46dd02b82623d30583166: Status 404 returned error can't find the container with id db80d6661124c7b43cae783bc4fedbc5fd92b11a53d46dd02b82623d30583166 Aug 04 15:06:52.609718 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.609288 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-77696685b-kpmh2"] Aug 04 15:06:52.692956 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.692926 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rcqrj"] Aug 04 15:06:52.696083 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:52.696044 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43e78d75_11c3_457d_aec1_5d4379ab2470.slice/crio-1f50dec482acba70eeaceb327b5abe4d19dd0a0285c460870049e1b3df698ffd WatchSource:0}: Error finding container 1f50dec482acba70eeaceb327b5abe4d19dd0a0285c460870049e1b3df698ffd: Status 404 returned error can't find the container with id 1f50dec482acba70eeaceb327b5abe4d19dd0a0285c460870049e1b3df698ffd Aug 04 15:06:52.707271 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.707199 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-65bdbbff45-vz7vb"] Aug 04 15:06:52.709943 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:52.709916 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b4f499c_41e3_4d50_a426_84a3f3926bea.slice/crio-42b634c7d8a62a7705f68de73cadeecbbd48305097cb684ee4748ee25d4fb848 WatchSource:0}: Error finding container 42b634c7d8a62a7705f68de73cadeecbbd48305097cb684ee4748ee25d4fb848: Status 404 returned error can't find the container with id 42b634c7d8a62a7705f68de73cadeecbbd48305097cb684ee4748ee25d4fb848 Aug 04 15:06:52.725282 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.725260 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c4fa9178-411c-42ae-94fe-62d80cf8eb0b-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-68756\" (UID: \"c4fa9178-411c-42ae-94fe-62d80cf8eb0b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-68756" Aug 04 15:06:52.725461 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:52.725442 2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Aug 04 15:06:52.725527 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:52.725515 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4fa9178-411c-42ae-94fe-62d80cf8eb0b-samples-operator-tls podName:c4fa9178-411c-42ae-94fe-62d80cf8eb0b nodeName:}" failed. No retries permitted until 2026-08-04 15:06:53.725495799 +0000 UTC m=+37.185368146 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c4fa9178-411c-42ae-94fe-62d80cf8eb0b-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-68756" (UID: "c4fa9178-411c-42ae-94fe-62d80cf8eb0b") : secret "samples-operator-tls" not found Aug 04 15:06:52.725589 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.725544 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-registry-tls\") pod \"image-registry-bddc5f76d-42zvh\" (UID: \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\") " pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" Aug 04 15:06:52.725758 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:52.725680 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 04 15:06:52.725758 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:52.725696 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bddc5f76d-42zvh: secret "image-registry-tls" not found Aug 04 15:06:52.725758 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:52.725738 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-registry-tls podName:aab2968d-11b7-4174-b0c4-b3ce5a46b90e nodeName:}" failed. No retries permitted until 2026-08-04 15:06:53.725726145 +0000 UTC m=+37.185598491 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-registry-tls") pod "image-registry-bddc5f76d-42zvh" (UID: "aab2968d-11b7-4174-b0c4-b3ce5a46b90e") : secret "image-registry-tls" not found Aug 04 15:06:52.750263 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.750238 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-784f97bcf6-7jlpj"] Aug 04 15:06:52.753147 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:06:52.753120 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad87b28c_e7a3_4354_af24_4e0d4aa3f464.slice/crio-5b7d7bf47de0a6ee43f78e43a9a64184752728aa74ecf96cd48c62104d830d04 WatchSource:0}: Error finding container 5b7d7bf47de0a6ee43f78e43a9a64184752728aa74ecf96cd48c62104d830d04: Status 404 returned error can't find the container with id 5b7d7bf47de0a6ee43f78e43a9a64184752728aa74ecf96cd48c62104d830d04 Aug 04 15:06:52.826090 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.826056 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e011eeba-5d48-483b-905a-a1f7b7e0ec1f-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-48zkn\" (UID: \"e011eeba-5d48-483b-905a-a1f7b7e0ec1f\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-48zkn" Aug 04 15:06:52.826258 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.826116 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d201490f-504d-4d2a-8b4f-c4eea2023469-service-ca-bundle\") pod \"router-default-7fc97bc64b-45b4c\" (UID: \"d201490f-504d-4d2a-8b4f-c4eea2023469\") " pod="openshift-ingress/router-default-7fc97bc64b-45b4c" Aug 04 15:06:52.826258 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.826141 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ded4fb33-8f70-4475-9a73-e8fff5c7e638-metrics-tls\") pod \"dns-default-hqwr6\" (UID: \"ded4fb33-8f70-4475-9a73-e8fff5c7e638\") " pod="openshift-dns/dns-default-hqwr6" Aug 04 15:06:52.826258 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.826197 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d201490f-504d-4d2a-8b4f-c4eea2023469-metrics-certs\") pod \"router-default-7fc97bc64b-45b4c\" (UID: \"d201490f-504d-4d2a-8b4f-c4eea2023469\") " pod="openshift-ingress/router-default-7fc97bc64b-45b4c" Aug 04 15:06:52.826258 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:52.826210 2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Aug 04 15:06:52.826484 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:52.826275 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 15:06:52.826484 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:52.826289 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e011eeba-5d48-483b-905a-a1f7b7e0ec1f-networking-console-plugin-cert podName:e011eeba-5d48-483b-905a-a1f7b7e0ec1f nodeName:}" failed. No retries permitted until 2026-08-04 15:06:53.826270184 +0000 UTC m=+37.286142528 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e011eeba-5d48-483b-905a-a1f7b7e0ec1f-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-48zkn" (UID: "e011eeba-5d48-483b-905a-a1f7b7e0ec1f") : secret "networking-console-plugin-cert" not found Aug 04 15:06:52.826484 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:52.826305 2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Aug 04 15:06:52.826484 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:52.826330 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ded4fb33-8f70-4475-9a73-e8fff5c7e638-metrics-tls podName:ded4fb33-8f70-4475-9a73-e8fff5c7e638 nodeName:}" failed. No retries permitted until 2026-08-04 15:06:53.826313645 +0000 UTC m=+37.286185987 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ded4fb33-8f70-4475-9a73-e8fff5c7e638-metrics-tls") pod "dns-default-hqwr6" (UID: "ded4fb33-8f70-4475-9a73-e8fff5c7e638") : secret "dns-default-metrics-tls" not found Aug 04 15:06:52.826484 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.826360 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e46dbe2d-eb92-47cd-9bea-47228f399ba3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-6qdqp\" (UID: \"e46dbe2d-eb92-47cd-9bea-47228f399ba3\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-6qdqp" Aug 04 15:06:52.826484 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:52.826455 2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Aug 04 15:06:52.826484 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:52.826488 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e46dbe2d-eb92-47cd-9bea-47228f399ba3-cluster-monitoring-operator-tls podName:e46dbe2d-eb92-47cd-9bea-47228f399ba3 nodeName:}" failed. No retries permitted until 2026-08-04 15:06:53.826474417 +0000 UTC m=+37.286346747 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e46dbe2d-eb92-47cd-9bea-47228f399ba3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-6qdqp" (UID: "e46dbe2d-eb92-47cd-9bea-47228f399ba3") : secret "cluster-monitoring-operator-tls" not found Aug 04 15:06:52.826836 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:52.826503 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d201490f-504d-4d2a-8b4f-c4eea2023469-service-ca-bundle podName:d201490f-504d-4d2a-8b4f-c4eea2023469 nodeName:}" failed. No retries permitted until 2026-08-04 15:06:53.826496116 +0000 UTC m=+37.286368443 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d201490f-504d-4d2a-8b4f-c4eea2023469-service-ca-bundle") pod "router-default-7fc97bc64b-45b4c" (UID: "d201490f-504d-4d2a-8b4f-c4eea2023469") : configmap references non-existent config key: service-ca.crt Aug 04 15:06:52.826836 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:52.826518 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d201490f-504d-4d2a-8b4f-c4eea2023469-metrics-certs podName:d201490f-504d-4d2a-8b4f-c4eea2023469 nodeName:}" failed. No retries permitted until 2026-08-04 15:06:53.82650987 +0000 UTC m=+37.286382229 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d201490f-504d-4d2a-8b4f-c4eea2023469-metrics-certs") pod "router-default-7fc97bc64b-45b4c" (UID: "d201490f-504d-4d2a-8b4f-c4eea2023469") : secret "router-metrics-certs-default" not found Aug 04 15:06:52.882923 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:52.882889 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xmh7f_214ec271-9cb1-4e3b-9b10-b7d15c10333f/konnectivity-agent/0.log" limit=1024 Aug 04 15:06:53.047519 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:53.047490 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxmsv" Aug 04 15:06:53.051093 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:53.051071 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Aug 04 15:06:53.052344 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:53.052319 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xmh7f_214ec271-9cb1-4e3b-9b10-b7d15c10333f/konnectivity-agent/0.log" limit=1024 Aug 04 15:06:53.233050 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:53.233004 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-tqzqn" event={"ID":"1a2285d4-f4ce-47be-80d4-7b4be7f930b1","Type":"ContainerStarted","Data":"d963867acaca3c7fac692614f696352220eef5ecf00fea8778fa167f97562732"} Aug 04 15:06:53.234334 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:53.234300 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-7jlpj" event={"ID":"ad87b28c-e7a3-4354-af24-4e0d4aa3f464","Type":"ContainerStarted","Data":"5b7d7bf47de0a6ee43f78e43a9a64184752728aa74ecf96cd48c62104d830d04"} Aug 04 15:06:53.235450 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:53.235425 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-kpmh2" event={"ID":"91c9aec0-0301-40bd-8bbd-ebd389fb4faf","Type":"ContainerStarted","Data":"c241e75107416f264e7f345603fce6faefad3234e574dff74b572a456a9f1000"} Aug 04 15:06:53.236823 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:53.236777 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-r6lsj" event={"ID":"f1b0f182-034b-4512-9bae-213071366d67","Type":"ContainerStarted","Data":"49234e3e6cf79f00ba0fc988c1a0759dfae08be37676cc29ea533edfdd60eb71"} Aug 04 15:06:53.238197 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:53.238174 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-vz7vb" event={"ID":"5b4f499c-41e3-4d50-a426-84a3f3926bea","Type":"ContainerStarted","Data":"42b634c7d8a62a7705f68de73cadeecbbd48305097cb684ee4748ee25d4fb848"} Aug 04 15:06:53.239535 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:53.239503 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rcqrj" event={"ID":"43e78d75-11c3-457d-aec1-5d4379ab2470","Type":"ContainerStarted","Data":"1f50dec482acba70eeaceb327b5abe4d19dd0a0285c460870049e1b3df698ffd"} Aug 04 15:06:53.240720 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:53.240700 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-r8wkp" event={"ID":"0580fdab-0381-4d8e-be5f-bde5eb44e0e6","Type":"ContainerStarted","Data":"db80d6661124c7b43cae783bc4fedbc5fd92b11a53d46dd02b82623d30583166"} Aug 04 15:06:53.735589 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:53.735547 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-registry-tls\") pod \"image-registry-bddc5f76d-42zvh\" (UID: \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\") " pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" Aug 04 15:06:53.735880 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:53.735684 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c4fa9178-411c-42ae-94fe-62d80cf8eb0b-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-68756\" (UID: \"c4fa9178-411c-42ae-94fe-62d80cf8eb0b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-68756" Aug 04 15:06:53.735954 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:53.735924 2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Aug 04 15:06:53.736009 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:53.735995 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4fa9178-411c-42ae-94fe-62d80cf8eb0b-samples-operator-tls podName:c4fa9178-411c-42ae-94fe-62d80cf8eb0b nodeName:}" failed. No retries permitted until 2026-08-04 15:06:55.735971915 +0000 UTC m=+39.195844268 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c4fa9178-411c-42ae-94fe-62d80cf8eb0b-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-68756" (UID: "c4fa9178-411c-42ae-94fe-62d80cf8eb0b") : secret "samples-operator-tls" not found Aug 04 15:06:53.736215 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:53.736197 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 04 15:06:53.736286 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:53.736219 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bddc5f76d-42zvh: secret "image-registry-tls" not found Aug 04 15:06:53.736286 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:53.736272 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-registry-tls podName:aab2968d-11b7-4174-b0c4-b3ce5a46b90e nodeName:}" failed. No retries permitted until 2026-08-04 15:06:55.736257406 +0000 UTC m=+39.196129734 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-registry-tls") pod "image-registry-bddc5f76d-42zvh" (UID: "aab2968d-11b7-4174-b0c4-b3ce5a46b90e") : secret "image-registry-tls" not found Aug 04 15:06:53.836805 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:53.836767 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d201490f-504d-4d2a-8b4f-c4eea2023469-service-ca-bundle\") pod \"router-default-7fc97bc64b-45b4c\" (UID: \"d201490f-504d-4d2a-8b4f-c4eea2023469\") " pod="openshift-ingress/router-default-7fc97bc64b-45b4c" Aug 04 15:06:53.836983 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:53.836818 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ded4fb33-8f70-4475-9a73-e8fff5c7e638-metrics-tls\") pod \"dns-default-hqwr6\" (UID: \"ded4fb33-8f70-4475-9a73-e8fff5c7e638\") " pod="openshift-dns/dns-default-hqwr6" Aug 04 15:06:53.836983 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:53.836846 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d201490f-504d-4d2a-8b4f-c4eea2023469-metrics-certs\") pod \"router-default-7fc97bc64b-45b4c\" (UID: \"d201490f-504d-4d2a-8b4f-c4eea2023469\") " pod="openshift-ingress/router-default-7fc97bc64b-45b4c" Aug 04 15:06:53.836983 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:53.836874 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e46dbe2d-eb92-47cd-9bea-47228f399ba3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-6qdqp\" (UID: \"e46dbe2d-eb92-47cd-9bea-47228f399ba3\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-6qdqp" Aug 04 15:06:53.837145 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:53.837006 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e011eeba-5d48-483b-905a-a1f7b7e0ec1f-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-48zkn\" (UID: \"e011eeba-5d48-483b-905a-a1f7b7e0ec1f\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-48zkn" Aug 04 15:06:53.837200 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:53.837179 2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Aug 04 15:06:53.837253 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:53.837241 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e011eeba-5d48-483b-905a-a1f7b7e0ec1f-networking-console-plugin-cert podName:e011eeba-5d48-483b-905a-a1f7b7e0ec1f nodeName:}" failed. No retries permitted until 2026-08-04 15:06:55.837222637 +0000 UTC m=+39.297094969 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e011eeba-5d48-483b-905a-a1f7b7e0ec1f-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-48zkn" (UID: "e011eeba-5d48-483b-905a-a1f7b7e0ec1f") : secret "networking-console-plugin-cert" not found Aug 04 15:06:53.837754 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:53.837733 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d201490f-504d-4d2a-8b4f-c4eea2023469-service-ca-bundle podName:d201490f-504d-4d2a-8b4f-c4eea2023469 nodeName:}" failed. No retries permitted until 2026-08-04 15:06:55.837716965 +0000 UTC m=+39.297589295 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d201490f-504d-4d2a-8b4f-c4eea2023469-service-ca-bundle") pod "router-default-7fc97bc64b-45b4c" (UID: "d201490f-504d-4d2a-8b4f-c4eea2023469") : configmap references non-existent config key: service-ca.crt Aug 04 15:06:53.837866 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:53.837812 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 15:06:53.837866 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:53.837844 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ded4fb33-8f70-4475-9a73-e8fff5c7e638-metrics-tls podName:ded4fb33-8f70-4475-9a73-e8fff5c7e638 nodeName:}" failed. No retries permitted until 2026-08-04 15:06:55.837833557 +0000 UTC m=+39.297705890 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ded4fb33-8f70-4475-9a73-e8fff5c7e638-metrics-tls") pod "dns-default-hqwr6" (UID: "ded4fb33-8f70-4475-9a73-e8fff5c7e638") : secret "dns-default-metrics-tls" not found Aug 04 15:06:53.837993 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:53.837891 2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Aug 04 15:06:53.837993 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:53.837918 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d201490f-504d-4d2a-8b4f-c4eea2023469-metrics-certs podName:d201490f-504d-4d2a-8b4f-c4eea2023469 nodeName:}" failed. No retries permitted until 2026-08-04 15:06:55.837908869 +0000 UTC m=+39.297781202 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d201490f-504d-4d2a-8b4f-c4eea2023469-metrics-certs") pod "router-default-7fc97bc64b-45b4c" (UID: "d201490f-504d-4d2a-8b4f-c4eea2023469") : secret "router-metrics-certs-default" not found Aug 04 15:06:53.837993 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:53.837968 2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Aug 04 15:06:53.838140 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:53.837996 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e46dbe2d-eb92-47cd-9bea-47228f399ba3-cluster-monitoring-operator-tls podName:e46dbe2d-eb92-47cd-9bea-47228f399ba3 nodeName:}" failed. No retries permitted until 2026-08-04 15:06:55.837987338 +0000 UTC m=+39.297859665 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e46dbe2d-eb92-47cd-9bea-47228f399ba3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-6qdqp" (UID: "e46dbe2d-eb92-47cd-9bea-47228f399ba3") : secret "cluster-monitoring-operator-tls" not found Aug 04 15:06:55.266258 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:55.265978 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-r6lsj" event={"ID":"f1b0f182-034b-4512-9bae-213071366d67","Type":"ContainerStarted","Data":"38545ee813134c97ed86f2f0d06e1e66074abcf6490fcbd453a92bb49fe12fb5"} Aug 04 15:06:55.289572 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:55.289514 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5b8776669-r6lsj" podStartSLOduration=9.004379882 podStartE2EDuration="11.289493074s" totalImagesPullingTime="2.285113192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:06:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:06:55.28794447 +0000 UTC m=+38.747816819" watchObservedRunningTime="2026-08-04 15:06:55.289493074 +0000 UTC m=+38.749365425" Aug 04 15:06:55.503813 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:55.503728 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xmh7f_214ec271-9cb1-4e3b-9b10-b7d15c10333f/konnectivity-agent/0.log" limit=1024 Aug 04 15:06:55.589381 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:55.589300 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xmh7f_214ec271-9cb1-4e3b-9b10-b7d15c10333f/konnectivity-agent/0.log" limit=1024 Aug 04 15:06:55.760567 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:55.760529 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c4fa9178-411c-42ae-94fe-62d80cf8eb0b-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-68756\" (UID: \"c4fa9178-411c-42ae-94fe-62d80cf8eb0b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-68756" Aug 04 15:06:55.760745 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:55.760691 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-registry-tls\") pod \"image-registry-bddc5f76d-42zvh\" (UID: \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\") " pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" Aug 04 15:06:55.761359 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:55.760832 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 04 15:06:55.761359 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:55.760850 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bddc5f76d-42zvh: secret "image-registry-tls" not found Aug 04 15:06:55.761359 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:55.760905 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-registry-tls podName:aab2968d-11b7-4174-b0c4-b3ce5a46b90e nodeName:}" failed. No retries permitted until 2026-08-04 15:06:59.760888572 +0000 UTC m=+43.220760903 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-registry-tls") pod "image-registry-bddc5f76d-42zvh" (UID: "aab2968d-11b7-4174-b0c4-b3ce5a46b90e") : secret "image-registry-tls" not found Aug 04 15:06:55.761359 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:55.761205 2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Aug 04 15:06:55.761359 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:55.761246 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4fa9178-411c-42ae-94fe-62d80cf8eb0b-samples-operator-tls podName:c4fa9178-411c-42ae-94fe-62d80cf8eb0b nodeName:}" failed. No retries permitted until 2026-08-04 15:06:59.761232835 +0000 UTC m=+43.221105168 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c4fa9178-411c-42ae-94fe-62d80cf8eb0b-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-68756" (UID: "c4fa9178-411c-42ae-94fe-62d80cf8eb0b") : secret "samples-operator-tls" not found Aug 04 15:06:55.862189 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:55.861681 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e011eeba-5d48-483b-905a-a1f7b7e0ec1f-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-48zkn\" (UID: \"e011eeba-5d48-483b-905a-a1f7b7e0ec1f\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-48zkn" Aug 04 15:06:55.862189 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:55.861769 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d201490f-504d-4d2a-8b4f-c4eea2023469-service-ca-bundle\") pod \"router-default-7fc97bc64b-45b4c\" (UID: \"d201490f-504d-4d2a-8b4f-c4eea2023469\") " pod="openshift-ingress/router-default-7fc97bc64b-45b4c" Aug 04 15:06:55.862189 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:55.861799 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ded4fb33-8f70-4475-9a73-e8fff5c7e638-metrics-tls\") pod \"dns-default-hqwr6\" (UID: \"ded4fb33-8f70-4475-9a73-e8fff5c7e638\") " pod="openshift-dns/dns-default-hqwr6" Aug 04 15:06:55.862189 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:55.861818 2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Aug 04 15:06:55.862189 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:55.861833 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d201490f-504d-4d2a-8b4f-c4eea2023469-metrics-certs\") pod \"router-default-7fc97bc64b-45b4c\" (UID: \"d201490f-504d-4d2a-8b4f-c4eea2023469\") " pod="openshift-ingress/router-default-7fc97bc64b-45b4c" Aug 04 15:06:55.862189 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:55.861867 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e46dbe2d-eb92-47cd-9bea-47228f399ba3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-6qdqp\" (UID: \"e46dbe2d-eb92-47cd-9bea-47228f399ba3\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-6qdqp" Aug 04 15:06:55.862189 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:55.861892 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e011eeba-5d48-483b-905a-a1f7b7e0ec1f-networking-console-plugin-cert podName:e011eeba-5d48-483b-905a-a1f7b7e0ec1f nodeName:}" failed. No retries permitted until 2026-08-04 15:06:59.861871235 +0000 UTC m=+43.321743567 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e011eeba-5d48-483b-905a-a1f7b7e0ec1f-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-48zkn" (UID: "e011eeba-5d48-483b-905a-a1f7b7e0ec1f") : secret "networking-console-plugin-cert" not found Aug 04 15:06:55.862189 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:55.861934 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d201490f-504d-4d2a-8b4f-c4eea2023469-service-ca-bundle podName:d201490f-504d-4d2a-8b4f-c4eea2023469 nodeName:}" failed. No retries permitted until 2026-08-04 15:06:59.861923215 +0000 UTC m=+43.321795551 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d201490f-504d-4d2a-8b4f-c4eea2023469-service-ca-bundle") pod "router-default-7fc97bc64b-45b4c" (UID: "d201490f-504d-4d2a-8b4f-c4eea2023469") : configmap references non-existent config key: service-ca.crt Aug 04 15:06:55.862189 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:55.861969 2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Aug 04 15:06:55.862189 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:55.862021 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e46dbe2d-eb92-47cd-9bea-47228f399ba3-cluster-monitoring-operator-tls podName:e46dbe2d-eb92-47cd-9bea-47228f399ba3 nodeName:}" failed. No retries permitted until 2026-08-04 15:06:59.862005316 +0000 UTC m=+43.321877666 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e46dbe2d-eb92-47cd-9bea-47228f399ba3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-6qdqp" (UID: "e46dbe2d-eb92-47cd-9bea-47228f399ba3") : secret "cluster-monitoring-operator-tls" not found Aug 04 15:06:55.862189 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:55.862042 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 15:06:55.862189 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:55.862075 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ded4fb33-8f70-4475-9a73-e8fff5c7e638-metrics-tls podName:ded4fb33-8f70-4475-9a73-e8fff5c7e638 nodeName:}" failed. No retries permitted until 2026-08-04 15:06:59.862065673 +0000 UTC m=+43.321938012 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ded4fb33-8f70-4475-9a73-e8fff5c7e638-metrics-tls") pod "dns-default-hqwr6" (UID: "ded4fb33-8f70-4475-9a73-e8fff5c7e638") : secret "dns-default-metrics-tls" not found Aug 04 15:06:55.862189 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:55.862082 2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Aug 04 15:06:55.862189 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:55.862118 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d201490f-504d-4d2a-8b4f-c4eea2023469-metrics-certs podName:d201490f-504d-4d2a-8b4f-c4eea2023469 nodeName:}" failed. No retries permitted until 2026-08-04 15:06:59.862105507 +0000 UTC m=+43.321977847 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d201490f-504d-4d2a-8b4f-c4eea2023469-metrics-certs") pod "router-default-7fc97bc64b-45b4c" (UID: "d201490f-504d-4d2a-8b4f-c4eea2023469") : secret "router-metrics-certs-default" not found Aug 04 15:06:57.606228 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:57.606195 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xmh7f_214ec271-9cb1-4e3b-9b10-b7d15c10333f/konnectivity-agent/0.log" limit=1024 Aug 04 15:06:57.680295 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:57.680262 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xmh7f_214ec271-9cb1-4e3b-9b10-b7d15c10333f/konnectivity-agent/0.log" limit=1024 Aug 04 15:06:59.797633 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:59.797589 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-registry-tls\") pod \"image-registry-bddc5f76d-42zvh\" (UID: \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\") " pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" Aug 04 15:06:59.798099 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:59.797672 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c4fa9178-411c-42ae-94fe-62d80cf8eb0b-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-68756\" (UID: \"c4fa9178-411c-42ae-94fe-62d80cf8eb0b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-68756" Aug 04 15:06:59.798099 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:59.797763 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 04 15:06:59.798099 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:59.797784 2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Aug 04 15:06:59.798099 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:59.797847 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4fa9178-411c-42ae-94fe-62d80cf8eb0b-samples-operator-tls podName:c4fa9178-411c-42ae-94fe-62d80cf8eb0b nodeName:}" failed. No retries permitted until 2026-08-04 15:07:07.797828583 +0000 UTC m=+51.257700911 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c4fa9178-411c-42ae-94fe-62d80cf8eb0b-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-68756" (UID: "c4fa9178-411c-42ae-94fe-62d80cf8eb0b") : secret "samples-operator-tls" not found Aug 04 15:06:59.798099 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:59.797788 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bddc5f76d-42zvh: secret "image-registry-tls" not found Aug 04 15:06:59.798099 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:59.797912 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-registry-tls podName:aab2968d-11b7-4174-b0c4-b3ce5a46b90e nodeName:}" failed. No retries permitted until 2026-08-04 15:07:07.79789924 +0000 UTC m=+51.257771567 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-registry-tls") pod "image-registry-bddc5f76d-42zvh" (UID: "aab2968d-11b7-4174-b0c4-b3ce5a46b90e") : secret "image-registry-tls" not found Aug 04 15:06:59.898105 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:59.898065 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e011eeba-5d48-483b-905a-a1f7b7e0ec1f-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-48zkn\" (UID: \"e011eeba-5d48-483b-905a-a1f7b7e0ec1f\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-48zkn" Aug 04 15:06:59.898300 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:59.898144 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d201490f-504d-4d2a-8b4f-c4eea2023469-service-ca-bundle\") pod \"router-default-7fc97bc64b-45b4c\" (UID: \"d201490f-504d-4d2a-8b4f-c4eea2023469\") " pod="openshift-ingress/router-default-7fc97bc64b-45b4c" Aug 04 15:06:59.898300 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:59.898171 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ded4fb33-8f70-4475-9a73-e8fff5c7e638-metrics-tls\") pod \"dns-default-hqwr6\" (UID: \"ded4fb33-8f70-4475-9a73-e8fff5c7e638\") " pod="openshift-dns/dns-default-hqwr6" Aug 04 15:06:59.898300 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:59.898204 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d201490f-504d-4d2a-8b4f-c4eea2023469-metrics-certs\") pod \"router-default-7fc97bc64b-45b4c\" (UID: \"d201490f-504d-4d2a-8b4f-c4eea2023469\") " pod="openshift-ingress/router-default-7fc97bc64b-45b4c" Aug 04 15:06:59.898300 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:06:59.898235 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e46dbe2d-eb92-47cd-9bea-47228f399ba3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-6qdqp\" (UID: \"e46dbe2d-eb92-47cd-9bea-47228f399ba3\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-6qdqp" Aug 04 15:06:59.898300 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:59.898244 2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Aug 04 15:06:59.898587 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:59.898322 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e011eeba-5d48-483b-905a-a1f7b7e0ec1f-networking-console-plugin-cert podName:e011eeba-5d48-483b-905a-a1f7b7e0ec1f nodeName:}" failed. No retries permitted until 2026-08-04 15:07:07.898300723 +0000 UTC m=+51.358173053 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e011eeba-5d48-483b-905a-a1f7b7e0ec1f-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-48zkn" (UID: "e011eeba-5d48-483b-905a-a1f7b7e0ec1f") : secret "networking-console-plugin-cert" not found Aug 04 15:06:59.898587 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:59.898342 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 15:06:59.898587 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:59.898344 2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Aug 04 15:06:59.898587 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:59.898345 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d201490f-504d-4d2a-8b4f-c4eea2023469-service-ca-bundle podName:d201490f-504d-4d2a-8b4f-c4eea2023469 nodeName:}" failed. No retries permitted until 2026-08-04 15:07:07.898333603 +0000 UTC m=+51.358205930 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d201490f-504d-4d2a-8b4f-c4eea2023469-service-ca-bundle") pod "router-default-7fc97bc64b-45b4c" (UID: "d201490f-504d-4d2a-8b4f-c4eea2023469") : configmap references non-existent config key: service-ca.crt Aug 04 15:06:59.898587 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:59.898425 2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Aug 04 15:06:59.898587 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:59.898445 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ded4fb33-8f70-4475-9a73-e8fff5c7e638-metrics-tls podName:ded4fb33-8f70-4475-9a73-e8fff5c7e638 nodeName:}" failed. No retries permitted until 2026-08-04 15:07:07.89842789 +0000 UTC m=+51.358300219 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ded4fb33-8f70-4475-9a73-e8fff5c7e638-metrics-tls") pod "dns-default-hqwr6" (UID: "ded4fb33-8f70-4475-9a73-e8fff5c7e638") : secret "dns-default-metrics-tls" not found Aug 04 15:06:59.898587 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:59.898463 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e46dbe2d-eb92-47cd-9bea-47228f399ba3-cluster-monitoring-operator-tls podName:e46dbe2d-eb92-47cd-9bea-47228f399ba3 nodeName:}" failed. No retries permitted until 2026-08-04 15:07:07.898453717 +0000 UTC m=+51.358326045 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e46dbe2d-eb92-47cd-9bea-47228f399ba3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-6qdqp" (UID: "e46dbe2d-eb92-47cd-9bea-47228f399ba3") : secret "cluster-monitoring-operator-tls" not found Aug 04 15:06:59.898587 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:06:59.898480 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d201490f-504d-4d2a-8b4f-c4eea2023469-metrics-certs podName:d201490f-504d-4d2a-8b4f-c4eea2023469 nodeName:}" failed. No retries permitted until 2026-08-04 15:07:07.898474648 +0000 UTC m=+51.358346975 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d201490f-504d-4d2a-8b4f-c4eea2023469-metrics-certs") pod "router-default-7fc97bc64b-45b4c" (UID: "d201490f-504d-4d2a-8b4f-c4eea2023469") : secret "router-metrics-certs-default" not found Aug 04 15:07:01.406874 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:01.406840 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xmh7f_214ec271-9cb1-4e3b-9b10-b7d15c10333f/konnectivity-agent/0.log" limit=1024 Aug 04 15:07:01.503490 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:01.503456 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xmh7f_214ec271-9cb1-4e3b-9b10-b7d15c10333f/konnectivity-agent/0.log" limit=1024 Aug 04 15:07:03.292503 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:03.292466 2574 generic.go:356] "Generic (PLEG): container finished" podID="cb3de761-615b-44d1-84f3-c9a0edf4520e" containerID="03886d33b097d54f0b1b3abb905b521b8a9725d579b518bb04ad1f4e0c209938" exitCode=0 Aug 04 15:07:03.292964 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:03.292550 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4wbvq" event={"ID":"cb3de761-615b-44d1-84f3-c9a0edf4520e","Type":"ContainerDied","Data":"03886d33b097d54f0b1b3abb905b521b8a9725d579b518bb04ad1f4e0c209938"} Aug 04 15:07:03.294668 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:03.294645 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-vz7vb" event={"ID":"5b4f499c-41e3-4d50-a426-84a3f3926bea","Type":"ContainerStarted","Data":"dfe8f8a5815ece9343de033bc587be7654ce4f430b1baec4e1ce352c233a2ee3"} Aug 04 15:07:03.296915 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:03.296859 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rcqrj" event={"ID":"43e78d75-11c3-457d-aec1-5d4379ab2470","Type":"ContainerStarted","Data":"e18efe4cf8d5f0359e6b4c47adaf22d092ecc1c33b0d5691ef49e23df43b7fa3"} Aug 04 15:07:03.298545 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:03.298506 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-r8wkp_0580fdab-0381-4d8e-be5f-bde5eb44e0e6/console-operator/0.log" Aug 04 15:07:03.298545 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:03.298526 2574 generic.go:356] "Generic (PLEG): container finished" podID="0580fdab-0381-4d8e-be5f-bde5eb44e0e6" containerID="794beaded3d2b62ea621c9c24bc19d15c725f04d5cd19de4d15507f871f3552d" exitCode=255 Aug 04 15:07:03.298723 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:03.298704 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-r8wkp" event={"ID":"0580fdab-0381-4d8e-be5f-bde5eb44e0e6","Type":"ContainerDied","Data":"794beaded3d2b62ea621c9c24bc19d15c725f04d5cd19de4d15507f871f3552d"} Aug 04 15:07:03.298802 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:03.298781 2574 scope.go:117] "RemoveContainer" containerID="794beaded3d2b62ea621c9c24bc19d15c725f04d5cd19de4d15507f871f3552d" Aug 04 15:07:03.301696 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:03.301676 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-tqzqn" event={"ID":"1a2285d4-f4ce-47be-80d4-7b4be7f930b1","Type":"ContainerStarted","Data":"cbda7288e1252f8b1f64163a1c4213cd41539f7bf878294fd99f80201e57ee78"} Aug 04 15:07:03.303239 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:03.303216 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-7jlpj" event={"ID":"ad87b28c-e7a3-4354-af24-4e0d4aa3f464","Type":"ContainerStarted","Data":"5bce6b8bae28745bdae6024b35671e687159c673f7bd0af50b016d78b58b8eb4"} Aug 04 15:07:03.304632 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:03.304612 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-kpmh2" event={"ID":"91c9aec0-0301-40bd-8bbd-ebd389fb4faf","Type":"ContainerStarted","Data":"462d2e13500f5049f7053e8325bc78d4f654ea5cd7d1469b5a011a8a216e4029"} Aug 04 15:07:03.354612 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:03.354552 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-7jlpj" podStartSLOduration=8.567649213 podStartE2EDuration="18.354518558s" totalImagesPullingTime="9.786869345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:06:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:07:03.353105923 +0000 UTC m=+46.812978276" watchObservedRunningTime="2026-08-04 15:07:03.354518558 +0000 UTC m=+46.814390908" Aug 04 15:07:03.407429 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:03.407062 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-tqzqn" podStartSLOduration=9.432709598 podStartE2EDuration="19.40704079s" totalImagesPullingTime="9.974331192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:06:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:07:03.381749963 +0000 UTC m=+46.841622313" watchObservedRunningTime="2026-08-04 15:07:03.40704079 +0000 UTC m=+46.866913140" Aug 04 15:07:03.407429 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:03.407169 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-77696685b-kpmh2" podStartSLOduration=8.488031448 podStartE2EDuration="18.407160767s" totalImagesPullingTime="9.919129319s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:06:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:07:03.406047871 +0000 UTC m=+46.865920225" watchObservedRunningTime="2026-08-04 15:07:03.407160767 +0000 UTC m=+46.867033117" Aug 04 15:07:03.499014 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:03.498955 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-65bdbbff45-vz7vb" podStartSLOduration=8.67499358 podStartE2EDuration="18.498937271s" totalImagesPullingTime="9.823943691s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:06:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:07:03.498560054 +0000 UTC m=+46.958432404" watchObservedRunningTime="2026-08-04 15:07:03.498937271 +0000 UTC m=+46.958809622" Aug 04 15:07:04.241634 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:04.241581 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rcqrj" podStartSLOduration=3.405191293 podStartE2EDuration="13.241562613s" totalImagesPullingTime="9.83637132s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:06:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:07:03.53927902 +0000 UTC m=+46.999151370" watchObservedRunningTime="2026-08-04 15:07:04.241562613 +0000 UTC m=+47.701434962" Aug 04 15:07:04.242202 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:04.242186 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-kjjf8"] Aug 04 15:07:04.265321 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:04.265290 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kjjf8"] Aug 04 15:07:04.265467 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:04.265442 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kjjf8" Aug 04 15:07:04.268820 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:04.268793 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-46h4k\"" Aug 04 15:07:04.269294 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:04.269272 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Aug 04 15:07:04.269425 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:04.269354 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Aug 04 15:07:04.281521 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:04.281498 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xmh7f_214ec271-9cb1-4e3b-9b10-b7d15c10333f/konnectivity-agent/0.log" limit=1024 Aug 04 15:07:04.310650 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:04.310620 2574 generic.go:356] "Generic (PLEG): container finished" podID="cb3de761-615b-44d1-84f3-c9a0edf4520e" containerID="2c647a49ea4ff263f567eeabd4a530d5c21a6b5574d7e1b01fe2777d9bea33dd" exitCode=0 Aug 04 15:07:04.311042 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:04.310701 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4wbvq" event={"ID":"cb3de761-615b-44d1-84f3-c9a0edf4520e","Type":"ContainerDied","Data":"2c647a49ea4ff263f567eeabd4a530d5c21a6b5574d7e1b01fe2777d9bea33dd"} Aug 04 15:07:04.312309 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:04.312294 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-r8wkp_0580fdab-0381-4d8e-be5f-bde5eb44e0e6/console-operator/1.log" Aug 04 15:07:04.312730 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:04.312715 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-r8wkp_0580fdab-0381-4d8e-be5f-bde5eb44e0e6/console-operator/0.log" Aug 04 15:07:04.312799 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:04.312751 2574 generic.go:356] "Generic (PLEG): container finished" podID="0580fdab-0381-4d8e-be5f-bde5eb44e0e6" containerID="5f2f234b3df02a5b5491677341f61f90d5655e53040a695f062022300e324f79" exitCode=255 Aug 04 15:07:04.312834 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:04.312816 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-r8wkp" event={"ID":"0580fdab-0381-4d8e-be5f-bde5eb44e0e6","Type":"ContainerDied","Data":"5f2f234b3df02a5b5491677341f61f90d5655e53040a695f062022300e324f79"} Aug 04 15:07:04.312884 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:04.312859 2574 scope.go:117] "RemoveContainer" containerID="794beaded3d2b62ea621c9c24bc19d15c725f04d5cd19de4d15507f871f3552d" Aug 04 15:07:04.313302 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:04.313058 2574 scope.go:117] "RemoveContainer" containerID="5f2f234b3df02a5b5491677341f61f90d5655e53040a695f062022300e324f79" Aug 04 15:07:04.313302 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:07:04.313255 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-r8wkp_openshift-console-operator(0580fdab-0381-4d8e-be5f-bde5eb44e0e6)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-r8wkp" podUID="0580fdab-0381-4d8e-be5f-bde5eb44e0e6" Aug 04 15:07:04.345842 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:04.345730 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lbjj\" (UniqueName: \"kubernetes.io/projected/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-kube-api-access-6lbjj\") pod \"insights-runtime-extractor-kjjf8\" (UID: \"dcee583d-d05a-4b0a-bb25-da3cfe507e2b\") " pod="openshift-insights/insights-runtime-extractor-kjjf8" Aug 04 15:07:04.347448 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:04.346777 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-crio-socket\") pod \"insights-runtime-extractor-kjjf8\" (UID: \"dcee583d-d05a-4b0a-bb25-da3cfe507e2b\") " pod="openshift-insights/insights-runtime-extractor-kjjf8" Aug 04 15:07:04.347448 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:04.346821 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kjjf8\" (UID: \"dcee583d-d05a-4b0a-bb25-da3cfe507e2b\") " pod="openshift-insights/insights-runtime-extractor-kjjf8" Aug 04 15:07:04.347448 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:04.346845 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kjjf8\" (UID: \"dcee583d-d05a-4b0a-bb25-da3cfe507e2b\") " pod="openshift-insights/insights-runtime-extractor-kjjf8" Aug 04 15:07:04.348661 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:04.348104 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-data-volume\") pod \"insights-runtime-extractor-kjjf8\" (UID: \"dcee583d-d05a-4b0a-bb25-da3cfe507e2b\") " pod="openshift-insights/insights-runtime-extractor-kjjf8" Aug 04 15:07:04.374562 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:04.374533 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xmh7f_214ec271-9cb1-4e3b-9b10-b7d15c10333f/konnectivity-agent/0.log" limit=1024 Aug 04 15:07:04.449741 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:04.449690 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-crio-socket\") pod \"insights-runtime-extractor-kjjf8\" (UID: \"dcee583d-d05a-4b0a-bb25-da3cfe507e2b\") " pod="openshift-insights/insights-runtime-extractor-kjjf8" Aug 04 15:07:04.449843 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:04.449751 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kjjf8\" (UID: \"dcee583d-d05a-4b0a-bb25-da3cfe507e2b\") " pod="openshift-insights/insights-runtime-extractor-kjjf8" Aug 04 15:07:04.449843 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:04.449780 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kjjf8\" (UID: \"dcee583d-d05a-4b0a-bb25-da3cfe507e2b\") " pod="openshift-insights/insights-runtime-extractor-kjjf8" Aug 04 15:07:04.449843 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:07:04.449829 2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 04 15:07:04.449967 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:04.449854 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-data-volume\") pod \"insights-runtime-extractor-kjjf8\" (UID: \"dcee583d-d05a-4b0a-bb25-da3cfe507e2b\") " pod="openshift-insights/insights-runtime-extractor-kjjf8" Aug 04 15:07:04.449967 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:07:04.449904 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-kube-rbac-proxy-cm podName:dcee583d-d05a-4b0a-bb25-da3cfe507e2b nodeName:}" failed. No retries permitted until 2026-08-04 15:07:04.949880799 +0000 UTC m=+48.409753126 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kjjf8" (UID: "dcee583d-d05a-4b0a-bb25-da3cfe507e2b") : configmap "kube-rbac-proxy" not found Aug 04 15:07:04.449967 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:07:04.449919 2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 15:07:04.449967 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:04.449914 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-crio-socket\") pod \"insights-runtime-extractor-kjjf8\" (UID: \"dcee583d-d05a-4b0a-bb25-da3cfe507e2b\") " pod="openshift-insights/insights-runtime-extractor-kjjf8" Aug 04 15:07:04.450157 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:07:04.449980 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-insights-runtime-extractor-tls podName:dcee583d-d05a-4b0a-bb25-da3cfe507e2b nodeName:}" failed. No retries permitted until 2026-08-04 15:07:04.949961022 +0000 UTC m=+48.409833369 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kjjf8" (UID: "dcee583d-d05a-4b0a-bb25-da3cfe507e2b") : secret "insights-runtime-extractor-tls" not found Aug 04 15:07:04.450157 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:04.450000 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lbjj\" (UniqueName: \"kubernetes.io/projected/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-kube-api-access-6lbjj\") pod \"insights-runtime-extractor-kjjf8\" (UID: \"dcee583d-d05a-4b0a-bb25-da3cfe507e2b\") " pod="openshift-insights/insights-runtime-extractor-kjjf8" Aug 04 15:07:04.450248 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:04.450218 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-data-volume\") pod \"insights-runtime-extractor-kjjf8\" (UID: \"dcee583d-d05a-4b0a-bb25-da3cfe507e2b\") " pod="openshift-insights/insights-runtime-extractor-kjjf8" Aug 04 15:07:04.469152 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:04.469131 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lbjj\" (UniqueName: \"kubernetes.io/projected/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-kube-api-access-6lbjj\") pod \"insights-runtime-extractor-kjjf8\" (UID: \"dcee583d-d05a-4b0a-bb25-da3cfe507e2b\") " pod="openshift-insights/insights-runtime-extractor-kjjf8" Aug 04 15:07:04.499326 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:04.499269 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xmh7f_214ec271-9cb1-4e3b-9b10-b7d15c10333f/konnectivity-agent/0.log" limit=1024 Aug 04 15:07:04.587458 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:04.587429 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xmh7f_214ec271-9cb1-4e3b-9b10-b7d15c10333f/konnectivity-agent/0.log" limit=1024 Aug 04 15:07:04.954849 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:04.954696 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kjjf8\" (UID: \"dcee583d-d05a-4b0a-bb25-da3cfe507e2b\") " pod="openshift-insights/insights-runtime-extractor-kjjf8" Aug 04 15:07:04.954849 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:04.954748 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kjjf8\" (UID: \"dcee583d-d05a-4b0a-bb25-da3cfe507e2b\") " pod="openshift-insights/insights-runtime-extractor-kjjf8" Aug 04 15:07:04.955142 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:07:04.954888 2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 04 15:07:04.955142 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:07:04.954942 2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 15:07:04.955142 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:07:04.954976 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-kube-rbac-proxy-cm podName:dcee583d-d05a-4b0a-bb25-da3cfe507e2b nodeName:}" failed. No retries permitted until 2026-08-04 15:07:05.954942144 +0000 UTC m=+49.414814486 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kjjf8" (UID: "dcee583d-d05a-4b0a-bb25-da3cfe507e2b") : configmap "kube-rbac-proxy" not found Aug 04 15:07:04.955142 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:07:04.955013 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-insights-runtime-extractor-tls podName:dcee583d-d05a-4b0a-bb25-da3cfe507e2b nodeName:}" failed. No retries permitted until 2026-08-04 15:07:05.954993587 +0000 UTC m=+49.414865915 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kjjf8" (UID: "dcee583d-d05a-4b0a-bb25-da3cfe507e2b") : secret "insights-runtime-extractor-tls" not found Aug 04 15:07:05.116783 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:05.116748 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xmh7f_214ec271-9cb1-4e3b-9b10-b7d15c10333f/konnectivity-agent/0.log" limit=1024 Aug 04 15:07:05.192596 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:05.192565 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xmh7f_214ec271-9cb1-4e3b-9b10-b7d15c10333f/konnectivity-agent/0.log" limit=1024 Aug 04 15:07:05.319520 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:05.319485 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4wbvq" event={"ID":"cb3de761-615b-44d1-84f3-c9a0edf4520e","Type":"ContainerStarted","Data":"bd28d6fdad23da960f0353586d6a3f100094f1380706a2e793a7dc7c9cfafe4d"} Aug 04 15:07:05.321136 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:05.321115 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-r8wkp_0580fdab-0381-4d8e-be5f-bde5eb44e0e6/console-operator/1.log" Aug 04 15:07:05.321397 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:05.321371 2574 scope.go:117] "RemoveContainer" containerID="5f2f234b3df02a5b5491677341f61f90d5655e53040a695f062022300e324f79" Aug 04 15:07:05.321537 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:07:05.321520 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-r8wkp_openshift-console-operator(0580fdab-0381-4d8e-be5f-bde5eb44e0e6)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-r8wkp" podUID="0580fdab-0381-4d8e-be5f-bde5eb44e0e6" Aug 04 15:07:05.361090 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:05.361048 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-4wbvq" podStartSLOduration=7.139594552 podStartE2EDuration="47.361035604s" totalImagesPullingTime="40.221441052s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 15:06:18 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:07:05.359937605 +0000 UTC m=+48.819809953" watchObservedRunningTime="2026-08-04 15:07:05.361035604 +0000 UTC m=+48.820907952" Aug 04 15:07:05.436092 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:05.436063 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xmh7f_214ec271-9cb1-4e3b-9b10-b7d15c10333f/konnectivity-agent/0.log" limit=1024 Aug 04 15:07:05.549372 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:05.549342 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xmh7f_214ec271-9cb1-4e3b-9b10-b7d15c10333f/konnectivity-agent/0.log" limit=1024 Aug 04 15:07:05.908203 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:05.908172 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xmh7f_214ec271-9cb1-4e3b-9b10-b7d15c10333f/konnectivity-agent/0.log" limit=1024 Aug 04 15:07:05.963808 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:05.963762 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kjjf8\" (UID: \"dcee583d-d05a-4b0a-bb25-da3cfe507e2b\") " pod="openshift-insights/insights-runtime-extractor-kjjf8" Aug 04 15:07:05.963808 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:05.963808 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kjjf8\" (UID: \"dcee583d-d05a-4b0a-bb25-da3cfe507e2b\") " pod="openshift-insights/insights-runtime-extractor-kjjf8" Aug 04 15:07:05.964039 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:07:05.963881 2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 04 15:07:05.964039 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:07:05.963910 2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 15:07:05.964039 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:07:05.963951 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-kube-rbac-proxy-cm podName:dcee583d-d05a-4b0a-bb25-da3cfe507e2b nodeName:}" failed. No retries permitted until 2026-08-04 15:07:07.963935965 +0000 UTC m=+51.423808292 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kjjf8" (UID: "dcee583d-d05a-4b0a-bb25-da3cfe507e2b") : configmap "kube-rbac-proxy" not found Aug 04 15:07:05.964039 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:07:05.964026 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-insights-runtime-extractor-tls podName:dcee583d-d05a-4b0a-bb25-da3cfe507e2b nodeName:}" failed. No retries permitted until 2026-08-04 15:07:07.964000615 +0000 UTC m=+51.423872945 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kjjf8" (UID: "dcee583d-d05a-4b0a-bb25-da3cfe507e2b") : secret "insights-runtime-extractor-tls" not found Aug 04 15:07:06.005951 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:06.005928 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xphfh_7ec6b008-bb9e-42e2-a233-a57bc7449804/dns-node-resolver/0.log" Aug 04 15:07:06.030552 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:06.030520 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xmh7f_214ec271-9cb1-4e3b-9b10-b7d15c10333f/konnectivity-agent/0.log" limit=1024 Aug 04 15:07:06.604540 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:06.604516 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ql74b_1e62b3d6-309b-4cf1-bb50-c1d15a0672b1/node-ca/0.log" Aug 04 15:07:07.477988 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:07.477958 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xmh7f_214ec271-9cb1-4e3b-9b10-b7d15c10333f/konnectivity-agent/0.log" limit=1024 Aug 04 15:07:07.554729 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:07.554690 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xmh7f_214ec271-9cb1-4e3b-9b10-b7d15c10333f/konnectivity-agent/0.log" limit=1024 Aug 04 15:07:07.603199 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:07.603167 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rcqrj_43e78d75-11c3-457d-aec1-5d4379ab2470/serve-healthcheck-canary/0.log" Aug 04 15:07:07.879213 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:07.879177 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-registry-tls\") pod \"image-registry-bddc5f76d-42zvh\" (UID: \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\") " pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" Aug 04 15:07:07.879619 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:07.879236 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c4fa9178-411c-42ae-94fe-62d80cf8eb0b-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-68756\" (UID: \"c4fa9178-411c-42ae-94fe-62d80cf8eb0b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-68756" Aug 04 15:07:07.879619 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:07:07.879327 2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Aug 04 15:07:07.879619 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:07:07.879376 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4fa9178-411c-42ae-94fe-62d80cf8eb0b-samples-operator-tls podName:c4fa9178-411c-42ae-94fe-62d80cf8eb0b nodeName:}" failed. No retries permitted until 2026-08-04 15:07:23.879363827 +0000 UTC m=+67.339236154 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c4fa9178-411c-42ae-94fe-62d80cf8eb0b-samples-operator-tls") pod "cluster-samples-operator-559cbf5745-68756" (UID: "c4fa9178-411c-42ae-94fe-62d80cf8eb0b") : secret "samples-operator-tls" not found Aug 04 15:07:07.879619 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:07:07.879327 2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Aug 04 15:07:07.879619 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:07:07.879418 2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-bddc5f76d-42zvh: secret "image-registry-tls" not found Aug 04 15:07:07.879619 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:07:07.879483 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-registry-tls podName:aab2968d-11b7-4174-b0c4-b3ce5a46b90e nodeName:}" failed. No retries permitted until 2026-08-04 15:07:23.879470351 +0000 UTC m=+67.339342677 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-registry-tls") pod "image-registry-bddc5f76d-42zvh" (UID: "aab2968d-11b7-4174-b0c4-b3ce5a46b90e") : secret "image-registry-tls" not found Aug 04 15:07:07.979752 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:07.979718 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e011eeba-5d48-483b-905a-a1f7b7e0ec1f-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-48zkn\" (UID: \"e011eeba-5d48-483b-905a-a1f7b7e0ec1f\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-48zkn" Aug 04 15:07:07.979881 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:07.979769 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d201490f-504d-4d2a-8b4f-c4eea2023469-service-ca-bundle\") pod \"router-default-7fc97bc64b-45b4c\" (UID: \"d201490f-504d-4d2a-8b4f-c4eea2023469\") " pod="openshift-ingress/router-default-7fc97bc64b-45b4c" Aug 04 15:07:07.979881 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:07.979796 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ded4fb33-8f70-4475-9a73-e8fff5c7e638-metrics-tls\") pod \"dns-default-hqwr6\" (UID: \"ded4fb33-8f70-4475-9a73-e8fff5c7e638\") " pod="openshift-dns/dns-default-hqwr6" Aug 04 15:07:07.979881 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:07.979824 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d201490f-504d-4d2a-8b4f-c4eea2023469-metrics-certs\") pod \"router-default-7fc97bc64b-45b4c\" (UID: \"d201490f-504d-4d2a-8b4f-c4eea2023469\") " pod="openshift-ingress/router-default-7fc97bc64b-45b4c" Aug 04 15:07:07.979881 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:07.979844 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e46dbe2d-eb92-47cd-9bea-47228f399ba3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-6qdqp\" (UID: \"e46dbe2d-eb92-47cd-9bea-47228f399ba3\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-6qdqp" Aug 04 15:07:07.979881 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:07.979861 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kjjf8\" (UID: \"dcee583d-d05a-4b0a-bb25-da3cfe507e2b\") " pod="openshift-insights/insights-runtime-extractor-kjjf8" Aug 04 15:07:07.979881 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:07:07.979865 2574 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Aug 04 15:07:07.979881 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:07.979879 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kjjf8\" (UID: \"dcee583d-d05a-4b0a-bb25-da3cfe507e2b\") " pod="openshift-insights/insights-runtime-extractor-kjjf8" Aug 04 15:07:07.980251 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:07:07.979912 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d201490f-504d-4d2a-8b4f-c4eea2023469-service-ca-bundle podName:d201490f-504d-4d2a-8b4f-c4eea2023469 nodeName:}" failed. No retries permitted until 2026-08-04 15:07:23.979893928 +0000 UTC m=+67.439766285 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d201490f-504d-4d2a-8b4f-c4eea2023469-service-ca-bundle") pod "router-default-7fc97bc64b-45b4c" (UID: "d201490f-504d-4d2a-8b4f-c4eea2023469") : configmap references non-existent config key: service-ca.crt Aug 04 15:07:07.980251 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:07:07.979953 2574 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Aug 04 15:07:07.980251 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:07:07.979956 2574 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Aug 04 15:07:07.980251 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:07:07.979977 2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Aug 04 15:07:07.980251 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:07:07.979985 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e46dbe2d-eb92-47cd-9bea-47228f399ba3-cluster-monitoring-operator-tls podName:e46dbe2d-eb92-47cd-9bea-47228f399ba3 nodeName:}" failed. No retries permitted until 2026-08-04 15:07:23.97997432 +0000 UTC m=+67.439846648 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e46dbe2d-eb92-47cd-9bea-47228f399ba3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6f55c4f58d-6qdqp" (UID: "e46dbe2d-eb92-47cd-9bea-47228f399ba3") : secret "cluster-monitoring-operator-tls" not found Aug 04 15:07:07.980251 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:07:07.980003 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e011eeba-5d48-483b-905a-a1f7b7e0ec1f-networking-console-plugin-cert podName:e011eeba-5d48-483b-905a-a1f7b7e0ec1f nodeName:}" failed. No retries permitted until 2026-08-04 15:07:23.979993154 +0000 UTC m=+67.439865495 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e011eeba-5d48-483b-905a-a1f7b7e0ec1f-networking-console-plugin-cert") pod "networking-console-plugin-f77f48c6d-48zkn" (UID: "e011eeba-5d48-483b-905a-a1f7b7e0ec1f") : secret "networking-console-plugin-cert" not found Aug 04 15:07:07.980251 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:07:07.979963 2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 15:07:07.980251 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:07:07.980020 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d201490f-504d-4d2a-8b4f-c4eea2023469-metrics-certs podName:d201490f-504d-4d2a-8b4f-c4eea2023469 nodeName:}" failed. No retries permitted until 2026-08-04 15:07:23.980011528 +0000 UTC m=+67.439883860 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d201490f-504d-4d2a-8b4f-c4eea2023469-metrics-certs") pod "router-default-7fc97bc64b-45b4c" (UID: "d201490f-504d-4d2a-8b4f-c4eea2023469") : secret "router-metrics-certs-default" not found Aug 04 15:07:07.980251 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:07:07.979976 2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Aug 04 15:07:07.980251 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:07:07.980031 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ded4fb33-8f70-4475-9a73-e8fff5c7e638-metrics-tls podName:ded4fb33-8f70-4475-9a73-e8fff5c7e638 nodeName:}" failed. No retries permitted until 2026-08-04 15:07:23.980026354 +0000 UTC m=+67.439898680 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ded4fb33-8f70-4475-9a73-e8fff5c7e638-metrics-tls") pod "dns-default-hqwr6" (UID: "ded4fb33-8f70-4475-9a73-e8fff5c7e638") : secret "dns-default-metrics-tls" not found Aug 04 15:07:07.980251 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:07:07.980045 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-insights-runtime-extractor-tls podName:dcee583d-d05a-4b0a-bb25-da3cfe507e2b nodeName:}" failed. No retries permitted until 2026-08-04 15:07:11.980037173 +0000 UTC m=+55.439909500 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kjjf8" (UID: "dcee583d-d05a-4b0a-bb25-da3cfe507e2b") : secret "insights-runtime-extractor-tls" not found Aug 04 15:07:07.980251 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:07:07.980059 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-kube-rbac-proxy-cm podName:dcee583d-d05a-4b0a-bb25-da3cfe507e2b nodeName:}" failed. No retries permitted until 2026-08-04 15:07:11.98005124 +0000 UTC m=+55.439923567 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kjjf8" (UID: "dcee583d-d05a-4b0a-bb25-da3cfe507e2b") : configmap "kube-rbac-proxy" not found Aug 04 15:07:08.695322 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:08.695286 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xmh7f_214ec271-9cb1-4e3b-9b10-b7d15c10333f/konnectivity-agent/0.log" limit=1024 Aug 04 15:07:09.060615 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:09.060584 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xmh7f_214ec271-9cb1-4e3b-9b10-b7d15c10333f/konnectivity-agent/0.log" limit=1024 Aug 04 15:07:12.011572 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:12.011536 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kjjf8\" (UID: \"dcee583d-d05a-4b0a-bb25-da3cfe507e2b\") " pod="openshift-insights/insights-runtime-extractor-kjjf8" Aug 04 15:07:12.011572 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:12.011575 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kjjf8\" (UID: \"dcee583d-d05a-4b0a-bb25-da3cfe507e2b\") " pod="openshift-insights/insights-runtime-extractor-kjjf8" Aug 04 15:07:12.012174 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:07:12.011719 2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Aug 04 15:07:12.012174 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:07:12.011793 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-insights-runtime-extractor-tls podName:dcee583d-d05a-4b0a-bb25-da3cfe507e2b nodeName:}" failed. No retries permitted until 2026-08-04 15:07:20.01176755 +0000 UTC m=+63.471639877 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kjjf8" (UID: "dcee583d-d05a-4b0a-bb25-da3cfe507e2b") : secret "insights-runtime-extractor-tls" not found Aug 04 15:07:12.012174 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:12.012072 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kjjf8\" (UID: \"dcee583d-d05a-4b0a-bb25-da3cfe507e2b\") " pod="openshift-insights/insights-runtime-extractor-kjjf8" Aug 04 15:07:12.348974 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:12.348904 2574 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-5ccd6b445f-r8wkp" Aug 04 15:07:12.348974 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:12.348952 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-r8wkp" Aug 04 15:07:12.349258 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:12.349246 2574 scope.go:117] "RemoveContainer" containerID="5f2f234b3df02a5b5491677341f61f90d5655e53040a695f062022300e324f79" Aug 04 15:07:12.349414 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:07:12.349379 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-5ccd6b445f-r8wkp_openshift-console-operator(0580fdab-0381-4d8e-be5f-bde5eb44e0e6)\"" pod="openshift-console-operator/console-operator-5ccd6b445f-r8wkp" podUID="0580fdab-0381-4d8e-be5f-bde5eb44e0e6" Aug 04 15:07:19.239530 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:19.239503 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z8h6k" Aug 04 15:07:20.086090 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:20.086055 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kjjf8\" (UID: \"dcee583d-d05a-4b0a-bb25-da3cfe507e2b\") " pod="openshift-insights/insights-runtime-extractor-kjjf8" Aug 04 15:07:20.088411 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:20.088363 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dcee583d-d05a-4b0a-bb25-da3cfe507e2b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kjjf8\" (UID: \"dcee583d-d05a-4b0a-bb25-da3cfe507e2b\") " pod="openshift-insights/insights-runtime-extractor-kjjf8" Aug 04 15:07:20.178631 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:20.178604 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-46h4k\"" Aug 04 15:07:20.185779 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:20.185760 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kjjf8" Aug 04 15:07:20.312578 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:20.312430 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kjjf8"] Aug 04 15:07:20.314895 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:07:20.314869 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcee583d_d05a_4b0a_bb25_da3cfe507e2b.slice/crio-d08de209bc3a0232125945dbfb3412645b120159fceee94042eab408e20c8863 WatchSource:0}: Error finding container d08de209bc3a0232125945dbfb3412645b120159fceee94042eab408e20c8863: Status 404 returned error can't find the container with id d08de209bc3a0232125945dbfb3412645b120159fceee94042eab408e20c8863 Aug 04 15:07:20.371860 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:20.371833 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kjjf8" event={"ID":"dcee583d-d05a-4b0a-bb25-da3cfe507e2b","Type":"ContainerStarted","Data":"d08de209bc3a0232125945dbfb3412645b120159fceee94042eab408e20c8863"} Aug 04 15:07:21.376628 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:21.376598 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kjjf8" event={"ID":"dcee583d-d05a-4b0a-bb25-da3cfe507e2b","Type":"ContainerStarted","Data":"e0d52d9a19e1c37b7880aaca5c2b289023d0feeff024fe28370d7536cb5b14df"} Aug 04 15:07:21.376990 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:21.376650 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kjjf8" event={"ID":"dcee583d-d05a-4b0a-bb25-da3cfe507e2b","Type":"ContainerStarted","Data":"0fa54f36b63593a3dff845f59d5986c8ce4fc784341f49bea81a12ecfa1a5803"} Aug 04 15:07:22.382426 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:22.382379 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kjjf8" event={"ID":"dcee583d-d05a-4b0a-bb25-da3cfe507e2b","Type":"ContainerStarted","Data":"b884e4579c1e3046958fc6acbd104bd70c617982df82f08089760a231aa35b11"} Aug 04 15:07:22.408768 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:22.408726 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-kjjf8" podStartSLOduration=16.569188065 podStartE2EDuration="18.408710436s" totalImagesPullingTime="1.839522371s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 15:07:04 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:07:22.4083612 +0000 UTC m=+65.868233556" watchObservedRunningTime="2026-08-04 15:07:22.408710436 +0000 UTC m=+65.868582785" Aug 04 15:07:22.808327 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:22.808297 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46bd5ac9-b189-4e6c-b2c6-df0ed33641e2-original-pull-secret\") pod \"global-pull-secret-syncer-xxmsv\" (UID: \"46bd5ac9-b189-4e6c-b2c6-df0ed33641e2\") " pod="kube-system/global-pull-secret-syncer-xxmsv" Aug 04 15:07:22.808515 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:22.808442 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b91798e-f721-4056-bc09-6d4765fa6757-metrics-certs\") pod \"network-metrics-daemon-8w68v\" (UID: \"1b91798e-f721-4056-bc09-6d4765fa6757\") " pod="openshift-multus/network-metrics-daemon-8w68v" Aug 04 15:07:22.812532 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:22.812511 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Aug 04 15:07:22.812754 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:22.812739 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Aug 04 15:07:22.821108 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:22.821079 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/46bd5ac9-b189-4e6c-b2c6-df0ed33641e2-original-pull-secret\") pod \"global-pull-secret-syncer-xxmsv\" (UID: \"46bd5ac9-b189-4e6c-b2c6-df0ed33641e2\") " pod="kube-system/global-pull-secret-syncer-xxmsv" Aug 04 15:07:22.821108 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:22.821103 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b91798e-f721-4056-bc09-6d4765fa6757-metrics-certs\") pod \"network-metrics-daemon-8w68v\" (UID: \"1b91798e-f721-4056-bc09-6d4765fa6757\") " pod="openshift-multus/network-metrics-daemon-8w68v" Aug 04 15:07:22.909712 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:22.909674 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vqcn\" (UniqueName: \"kubernetes.io/projected/bf59cab4-36dc-431c-869d-896d50d50e7f-kube-api-access-4vqcn\") pod \"network-check-target-2vsg4\" (UID: \"bf59cab4-36dc-431c-869d-896d50d50e7f\") " pod="openshift-network-diagnostics/network-check-target-2vsg4" Aug 04 15:07:22.912026 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:22.912003 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vqcn\" (UniqueName: \"kubernetes.io/projected/bf59cab4-36dc-431c-869d-896d50d50e7f-kube-api-access-4vqcn\") pod \"network-check-target-2vsg4\" (UID: \"bf59cab4-36dc-431c-869d-896d50d50e7f\") " pod="openshift-network-diagnostics/network-check-target-2vsg4" Aug 04 15:07:22.979975 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:22.979954 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jm4b6\"" Aug 04 15:07:22.987484 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:22.987459 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2vsg4" Aug 04 15:07:22.995713 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:22.995514 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8z69l\"" Aug 04 15:07:23.002406 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:23.002378 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8w68v" Aug 04 15:07:23.047495 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:23.047471 2574 scope.go:117] "RemoveContainer" containerID="5f2f234b3df02a5b5491677341f61f90d5655e53040a695f062022300e324f79" Aug 04 15:07:23.056507 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:23.056480 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xxmsv" Aug 04 15:07:23.129117 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:23.128030 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2vsg4"] Aug 04 15:07:23.131584 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:07:23.131555 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf59cab4_36dc_431c_869d_896d50d50e7f.slice/crio-59a86bf675807efe6d529a4d10d124c2c4a9669fc194287b82de80bd861b724e WatchSource:0}: Error finding container 59a86bf675807efe6d529a4d10d124c2c4a9669fc194287b82de80bd861b724e: Status 404 returned error can't find the container with id 59a86bf675807efe6d529a4d10d124c2c4a9669fc194287b82de80bd861b724e Aug 04 15:07:23.157342 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:23.157286 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8w68v"] Aug 04 15:07:23.170431 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:07:23.170404 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b91798e_f721_4056_bc09_6d4765fa6757.slice/crio-2d6307c7d4cc80b26a3a673f078d997761ae1e228a193c61e8a1d5f78704d6a8 WatchSource:0}: Error finding container 2d6307c7d4cc80b26a3a673f078d997761ae1e228a193c61e8a1d5f78704d6a8: Status 404 returned error can't find the container with id 2d6307c7d4cc80b26a3a673f078d997761ae1e228a193c61e8a1d5f78704d6a8 Aug 04 15:07:23.205716 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:23.205406 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xxmsv"] Aug 04 15:07:23.209065 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:07:23.209042 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46bd5ac9_b189_4e6c_b2c6_df0ed33641e2.slice/crio-42cc94da50d59eddae6da00ebb493946942023a18f8752635529a0b800e1964c WatchSource:0}: Error finding container 42cc94da50d59eddae6da00ebb493946942023a18f8752635529a0b800e1964c: Status 404 returned error can't find the container with id 42cc94da50d59eddae6da00ebb493946942023a18f8752635529a0b800e1964c Aug 04 15:07:23.389232 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:23.389150 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-r8wkp_0580fdab-0381-4d8e-be5f-bde5eb44e0e6/console-operator/1.log" Aug 04 15:07:23.389679 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:23.389260 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-5ccd6b445f-r8wkp" event={"ID":"0580fdab-0381-4d8e-be5f-bde5eb44e0e6","Type":"ContainerStarted","Data":"3a8808cad13ef93c8ceae171561ae8de0bb5332435107cd7e7182b93b280e1ee"} Aug 04 15:07:23.389679 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:23.389640 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-5ccd6b445f-r8wkp" Aug 04 15:07:23.390487 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:23.390458 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xxmsv" event={"ID":"46bd5ac9-b189-4e6c-b2c6-df0ed33641e2","Type":"ContainerStarted","Data":"42cc94da50d59eddae6da00ebb493946942023a18f8752635529a0b800e1964c"} Aug 04 15:07:23.391750 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:23.391726 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8w68v" event={"ID":"1b91798e-f721-4056-bc09-6d4765fa6757","Type":"ContainerStarted","Data":"2d6307c7d4cc80b26a3a673f078d997761ae1e228a193c61e8a1d5f78704d6a8"} Aug 04 15:07:23.393094 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:23.393068 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2vsg4" event={"ID":"bf59cab4-36dc-431c-869d-896d50d50e7f","Type":"ContainerStarted","Data":"1a6ccb04352e908e9baec4dfd2877b0601368b658df2960d56b86b7b63800916"} Aug 04 15:07:23.393199 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:23.393114 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2vsg4" event={"ID":"bf59cab4-36dc-431c-869d-896d50d50e7f","Type":"ContainerStarted","Data":"59a86bf675807efe6d529a4d10d124c2c4a9669fc194287b82de80bd861b724e"} Aug 04 15:07:23.393326 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:23.393308 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2vsg4" Aug 04 15:07:23.411487 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:23.411447 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-5ccd6b445f-r8wkp" podStartSLOduration=29.450661439 podStartE2EDuration="39.411434339s" totalImagesPullingTime="9.9607729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:06:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:07:23.411108687 +0000 UTC m=+66.870981037" watchObservedRunningTime="2026-08-04 15:07:23.411434339 +0000 UTC m=+66.871306688" Aug 04 15:07:23.683766 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:23.683681 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-5ccd6b445f-r8wkp" Aug 04 15:07:23.715696 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:23.715631 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2vsg4" podStartSLOduration=65.715609758 podStartE2EDuration="1m5.715609758s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:06:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:07:23.429601313 +0000 UTC m=+66.889473662" watchObservedRunningTime="2026-08-04 15:07:23.715609758 +0000 UTC m=+67.175482111" Aug 04 15:07:23.917684 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:23.917635 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c4fa9178-411c-42ae-94fe-62d80cf8eb0b-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-68756\" (UID: \"c4fa9178-411c-42ae-94fe-62d80cf8eb0b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-68756" Aug 04 15:07:23.917876 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:23.917776 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-registry-tls\") pod \"image-registry-bddc5f76d-42zvh\" (UID: \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\") " pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" Aug 04 15:07:23.920043 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:23.920019 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-registry-tls\") pod \"image-registry-bddc5f76d-42zvh\" (UID: \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\") " pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" Aug 04 15:07:23.920043 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:23.920028 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c4fa9178-411c-42ae-94fe-62d80cf8eb0b-samples-operator-tls\") pod \"cluster-samples-operator-559cbf5745-68756\" (UID: \"c4fa9178-411c-42ae-94fe-62d80cf8eb0b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-68756" Aug 04 15:07:24.019055 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:24.019027 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e011eeba-5d48-483b-905a-a1f7b7e0ec1f-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-48zkn\" (UID: \"e011eeba-5d48-483b-905a-a1f7b7e0ec1f\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-48zkn" Aug 04 15:07:24.019226 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:24.019070 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d201490f-504d-4d2a-8b4f-c4eea2023469-service-ca-bundle\") pod \"router-default-7fc97bc64b-45b4c\" (UID: \"d201490f-504d-4d2a-8b4f-c4eea2023469\") " pod="openshift-ingress/router-default-7fc97bc64b-45b4c" Aug 04 15:07:24.019226 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:24.019088 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ded4fb33-8f70-4475-9a73-e8fff5c7e638-metrics-tls\") pod \"dns-default-hqwr6\" (UID: \"ded4fb33-8f70-4475-9a73-e8fff5c7e638\") " pod="openshift-dns/dns-default-hqwr6" Aug 04 15:07:24.019226 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:24.019115 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d201490f-504d-4d2a-8b4f-c4eea2023469-metrics-certs\") pod \"router-default-7fc97bc64b-45b4c\" (UID: \"d201490f-504d-4d2a-8b4f-c4eea2023469\") " pod="openshift-ingress/router-default-7fc97bc64b-45b4c" Aug 04 15:07:24.019226 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:24.019143 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e46dbe2d-eb92-47cd-9bea-47228f399ba3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-6qdqp\" (UID: \"e46dbe2d-eb92-47cd-9bea-47228f399ba3\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-6qdqp" Aug 04 15:07:24.019846 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:24.019822 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d201490f-504d-4d2a-8b4f-c4eea2023469-service-ca-bundle\") pod \"router-default-7fc97bc64b-45b4c\" (UID: \"d201490f-504d-4d2a-8b4f-c4eea2023469\") " pod="openshift-ingress/router-default-7fc97bc64b-45b4c" Aug 04 15:07:24.021481 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:24.021456 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ded4fb33-8f70-4475-9a73-e8fff5c7e638-metrics-tls\") pod \"dns-default-hqwr6\" (UID: \"ded4fb33-8f70-4475-9a73-e8fff5c7e638\") " pod="openshift-dns/dns-default-hqwr6" Aug 04 15:07:24.021568 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:24.021534 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e011eeba-5d48-483b-905a-a1f7b7e0ec1f-networking-console-plugin-cert\") pod \"networking-console-plugin-f77f48c6d-48zkn\" (UID: \"e011eeba-5d48-483b-905a-a1f7b7e0ec1f\") " pod="openshift-network-console/networking-console-plugin-f77f48c6d-48zkn" Aug 04 15:07:24.021810 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:24.021795 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d201490f-504d-4d2a-8b4f-c4eea2023469-metrics-certs\") pod \"router-default-7fc97bc64b-45b4c\" (UID: \"d201490f-504d-4d2a-8b4f-c4eea2023469\") " pod="openshift-ingress/router-default-7fc97bc64b-45b4c" Aug 04 15:07:24.021879 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:24.021856 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e46dbe2d-eb92-47cd-9bea-47228f399ba3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6f55c4f58d-6qdqp\" (UID: \"e46dbe2d-eb92-47cd-9bea-47228f399ba3\") " pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-6qdqp" Aug 04 15:07:24.100255 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:24.100232 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-j4zcx\"" Aug 04 15:07:24.107024 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:24.107005 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" Aug 04 15:07:24.162698 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:24.162675 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-ln79b\"" Aug 04 15:07:24.170054 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:24.170028 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-68756" Aug 04 15:07:24.213732 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:24.213701 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-dl44g\"" Aug 04 15:07:24.220961 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:24.220928 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-f77f48c6d-48zkn" Aug 04 15:07:24.227607 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:24.227585 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-g4lk6\"" Aug 04 15:07:24.236025 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:24.235986 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7fc97bc64b-45b4c" Aug 04 15:07:24.241029 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:24.241010 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kghgl\"" Aug 04 15:07:24.246671 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:24.246636 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hqwr6" Aug 04 15:07:24.249705 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:24.249445 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-bddc5f76d-42zvh"] Aug 04 15:07:24.258109 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:07:24.257450 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaab2968d_11b7_4174_b0c4_b3ce5a46b90e.slice/crio-40ade403054082ffb8fa7ef3cdcec04b8915df81f62b5b5d1f7deb80f9337120 WatchSource:0}: Error finding container 40ade403054082ffb8fa7ef3cdcec04b8915df81f62b5b5d1f7deb80f9337120: Status 404 returned error can't find the container with id 40ade403054082ffb8fa7ef3cdcec04b8915df81f62b5b5d1f7deb80f9337120 Aug 04 15:07:24.260834 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:24.260797 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-lbk76\"" Aug 04 15:07:24.267859 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:24.267768 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-6qdqp" Aug 04 15:07:24.332965 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:24.330862 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-68756"] Aug 04 15:07:24.408198 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:24.408126 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" event={"ID":"aab2968d-11b7-4174-b0c4-b3ce5a46b90e","Type":"ContainerStarted","Data":"40ade403054082ffb8fa7ef3cdcec04b8915df81f62b5b5d1f7deb80f9337120"} Aug 04 15:07:24.415939 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:24.415897 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-f77f48c6d-48zkn"] Aug 04 15:07:24.443364 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:24.442262 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7fc97bc64b-45b4c"] Aug 04 15:07:24.447484 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:07:24.447447 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd201490f_504d_4d2a_8b4f_c4eea2023469.slice/crio-db5d9a33ed2d672425338e3f493240c67db4c78d94f4e4a3f3d4e45ae3e1de8c WatchSource:0}: Error finding container db5d9a33ed2d672425338e3f493240c67db4c78d94f4e4a3f3d4e45ae3e1de8c: Status 404 returned error can't find the container with id db5d9a33ed2d672425338e3f493240c67db4c78d94f4e4a3f3d4e45ae3e1de8c Aug 04 15:07:24.455806 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:24.455756 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hqwr6"] Aug 04 15:07:24.460154 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:07:24.460126 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podded4fb33_8f70_4475_9a73_e8fff5c7e638.slice/crio-9f984ad65098be46f748e52930048f6235535a7d61cf8bd151654b72410a6ba4 WatchSource:0}: Error finding container 9f984ad65098be46f748e52930048f6235535a7d61cf8bd151654b72410a6ba4: Status 404 returned error can't find the container with id 9f984ad65098be46f748e52930048f6235535a7d61cf8bd151654b72410a6ba4 Aug 04 15:07:24.489030 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:24.488813 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-6qdqp"] Aug 04 15:07:24.494877 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:07:24.494842 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode46dbe2d_eb92_47cd_9bea_47228f399ba3.slice/crio-a27b8b86692bcd11a1d8ca38e361ce4b03a39105be0bf0e620b28892df2442fa WatchSource:0}: Error finding container a27b8b86692bcd11a1d8ca38e361ce4b03a39105be0bf0e620b28892df2442fa: Status 404 returned error can't find the container with id a27b8b86692bcd11a1d8ca38e361ce4b03a39105be0bf0e620b28892df2442fa Aug 04 15:07:25.417844 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:25.417671 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-68756" event={"ID":"c4fa9178-411c-42ae-94fe-62d80cf8eb0b","Type":"ContainerStarted","Data":"ad30caee193bbe5fbaa3913d487105c69bc2507caee20e58acf4cbb79a8f0a21"} Aug 04 15:07:25.422404 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:25.422359 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8w68v" event={"ID":"1b91798e-f721-4056-bc09-6d4765fa6757","Type":"ContainerStarted","Data":"2ef7b8e31022981d1f31e5fee012d54e59ba76a5d4e0fb4271c7dcf89fcfd1c5"} Aug 04 15:07:25.422536 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:25.422473 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8w68v" event={"ID":"1b91798e-f721-4056-bc09-6d4765fa6757","Type":"ContainerStarted","Data":"4f444368c448e295b92724b041803a2236061f85693de0b7e6fe23e7493cf77e"} Aug 04 15:07:25.426012 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:25.425984 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-6qdqp" event={"ID":"e46dbe2d-eb92-47cd-9bea-47228f399ba3","Type":"ContainerStarted","Data":"a27b8b86692bcd11a1d8ca38e361ce4b03a39105be0bf0e620b28892df2442fa"} Aug 04 15:07:25.428727 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:25.428659 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hqwr6" event={"ID":"ded4fb33-8f70-4475-9a73-e8fff5c7e638","Type":"ContainerStarted","Data":"9f984ad65098be46f748e52930048f6235535a7d61cf8bd151654b72410a6ba4"} Aug 04 15:07:25.432892 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:25.432809 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7fc97bc64b-45b4c" event={"ID":"d201490f-504d-4d2a-8b4f-c4eea2023469","Type":"ContainerStarted","Data":"3f48a1399d3895266c412400ab3b23a5a8be32adfd9891472da849701086dd7a"} Aug 04 15:07:25.432892 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:25.432848 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7fc97bc64b-45b4c" event={"ID":"d201490f-504d-4d2a-8b4f-c4eea2023469","Type":"ContainerStarted","Data":"db5d9a33ed2d672425338e3f493240c67db4c78d94f4e4a3f3d4e45ae3e1de8c"} Aug 04 15:07:25.435042 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:25.435015 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" event={"ID":"aab2968d-11b7-4174-b0c4-b3ce5a46b90e","Type":"ContainerStarted","Data":"0b7c8629f548380ddc854dedc1218029128ceb86a0345708ad4b638242bc5a62"} Aug 04 15:07:25.435516 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:25.435481 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" Aug 04 15:07:25.436926 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:25.436885 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-f77f48c6d-48zkn" event={"ID":"e011eeba-5d48-483b-905a-a1f7b7e0ec1f","Type":"ContainerStarted","Data":"67ce3a43cfbbf55e5cbb34441d4a41d8466a112fef7c08300ac6358670b11028"} Aug 04 15:07:25.451414 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:25.451351 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-8w68v" podStartSLOduration=65.559511433 podStartE2EDuration="1m7.451333237s" totalImagesPullingTime="1.891821804s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:06:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:07:25.448092224 +0000 UTC m=+68.907964574" watchObservedRunningTime="2026-08-04 15:07:25.451333237 +0000 UTC m=+68.911205589" Aug 04 15:07:25.475776 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:25.475103 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" podStartSLOduration=67.475087744 podStartE2EDuration="1m7.475087744s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:06:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:07:25.474690455 +0000 UTC m=+68.934562804" watchObservedRunningTime="2026-08-04 15:07:25.475087744 +0000 UTC m=+68.934960094" Aug 04 15:07:25.508060 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:25.507637 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7fc97bc64b-45b4c" podStartSLOduration=40.507618336 podStartE2EDuration="40.507618336s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:06:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:07:25.50648886 +0000 UTC m=+68.966361211" watchObservedRunningTime="2026-08-04 15:07:25.507618336 +0000 UTC m=+68.967490686" Aug 04 15:07:26.236787 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:26.236673 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7fc97bc64b-45b4c" Aug 04 15:07:26.240432 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:26.240268 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7fc97bc64b-45b4c" Aug 04 15:07:26.444039 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:26.443949 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-f77f48c6d-48zkn" event={"ID":"e011eeba-5d48-483b-905a-a1f7b7e0ec1f","Type":"ContainerStarted","Data":"2dd50fa8dad302bcd82022526924ebd1a2ea4bcf5ec23b60dc5713f3ec86665f"} Aug 04 15:07:26.444841 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:26.444820 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7fc97bc64b-45b4c" Aug 04 15:07:26.446277 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:26.446231 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7fc97bc64b-45b4c" Aug 04 15:07:26.471755 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:26.471678 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-f77f48c6d-48zkn" podStartSLOduration=39.726100278 podStartE2EDuration="41.47165621s" totalImagesPullingTime="1.745555932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:06:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:07:26.471219884 +0000 UTC m=+69.931092236" watchObservedRunningTime="2026-08-04 15:07:26.47165621 +0000 UTC m=+69.931528557" Aug 04 15:07:29.460134 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:29.459935 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-68756" event={"ID":"c4fa9178-411c-42ae-94fe-62d80cf8eb0b","Type":"ContainerStarted","Data":"0915177bc88ae493587a0ff4e32b2e308cbc962437386d0f33721064e680e39e"} Aug 04 15:07:29.462951 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:29.462889 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xxmsv" event={"ID":"46bd5ac9-b189-4e6c-b2c6-df0ed33641e2","Type":"ContainerStarted","Data":"cc4142b47582cc31e277dc20ee0697b2086f89383730eddd9ba9a3b34400b21e"} Aug 04 15:07:29.466306 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:29.465810 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-6qdqp" event={"ID":"e46dbe2d-eb92-47cd-9bea-47228f399ba3","Type":"ContainerStarted","Data":"857813f138c4fdb76e75ee0d89194ccad88752f2e889e664e9b92eaadcda89ac"} Aug 04 15:07:29.469359 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:29.469306 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hqwr6" event={"ID":"ded4fb33-8f70-4475-9a73-e8fff5c7e638","Type":"ContainerStarted","Data":"22392c90625cc1c0dbebbe72fc356a8b179924d30049d286ade8f663f6126f33"} Aug 04 15:07:29.484628 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:29.483704 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xxmsv" podStartSLOduration=65.393758413 podStartE2EDuration="1m11.483685435s" totalImagesPullingTime="6.089927022s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 15:06:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:07:29.483591036 +0000 UTC m=+72.943463386" watchObservedRunningTime="2026-08-04 15:07:29.483685435 +0000 UTC m=+72.943557785" Aug 04 15:07:29.503456 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:29.503319 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6f55c4f58d-6qdqp" podStartSLOduration=39.749234892 podStartE2EDuration="44.503288244s" totalImagesPullingTime="4.754053352s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:06:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:07:29.501055242 +0000 UTC m=+72.960927592" watchObservedRunningTime="2026-08-04 15:07:29.503288244 +0000 UTC m=+72.963160596" Aug 04 15:07:30.474854 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:30.474816 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hqwr6" event={"ID":"ded4fb33-8f70-4475-9a73-e8fff5c7e638","Type":"ContainerStarted","Data":"c2baee486fad6568c963f96446fb709eccca065d4ed93151b6906984606dbd7a"} Aug 04 15:07:30.475276 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:30.474910 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-hqwr6" Aug 04 15:07:30.476534 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:30.476514 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-68756" event={"ID":"c4fa9178-411c-42ae-94fe-62d80cf8eb0b","Type":"ContainerStarted","Data":"4c4062cce05c29dd0bae51d08aaf575c4a9f7347181052ed3a27fb0602914ef1"} Aug 04 15:07:30.498883 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:30.498839 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-hqwr6" podStartSLOduration=34.705652796 podStartE2EDuration="39.498825374s" totalImagesPullingTime="4.793172578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:06:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:07:30.496929347 +0000 UTC m=+73.956801696" watchObservedRunningTime="2026-08-04 15:07:30.498825374 +0000 UTC m=+73.958697722" Aug 04 15:07:30.518063 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:30.518021 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-559cbf5745-68756" podStartSLOduration=41.676537409 podStartE2EDuration="46.518008315s" totalImagesPullingTime="4.841470906s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:06:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:07:30.517500741 +0000 UTC m=+73.977373091" watchObservedRunningTime="2026-08-04 15:07:30.518008315 +0000 UTC m=+73.977880755" Aug 04 15:07:31.462214 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:31.462171 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8668bf47d8-rhrcs"] Aug 04 15:07:31.501668 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:31.501627 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cddcb86f9-s655m"] Aug 04 15:07:31.502200 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:31.502170 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8668bf47d8-rhrcs" Aug 04 15:07:31.507037 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:31.507014 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Aug 04 15:07:31.507186 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:31.507100 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Aug 04 15:07:31.508221 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:31.508197 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-hbgmb\"" Aug 04 15:07:31.508833 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:31.508231 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Aug 04 15:07:31.523122 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:31.523100 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Aug 04 15:07:31.526920 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:31.526902 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8668bf47d8-rhrcs"] Aug 04 15:07:31.527001 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:31.526983 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cddcb86f9-s655m" Aug 04 15:07:31.530508 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:31.530493 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Aug 04 15:07:31.553990 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:31.553968 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cddcb86f9-s655m"] Aug 04 15:07:31.586869 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:31.586838 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5tjc\" (UniqueName: \"kubernetes.io/projected/a1ee2400-c0fe-429d-a439-a0ed35927650-kube-api-access-r5tjc\") pod \"managed-serviceaccount-addon-agent-8668bf47d8-rhrcs\" (UID: \"a1ee2400-c0fe-429d-a439-a0ed35927650\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8668bf47d8-rhrcs" Aug 04 15:07:31.587011 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:31.586886 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x774w\" (UniqueName: \"kubernetes.io/projected/efd49e4e-739d-4f61-a452-5894a3403cfd-kube-api-access-x774w\") pod \"klusterlet-addon-workmgr-5cddcb86f9-s655m\" (UID: \"efd49e4e-739d-4f61-a452-5894a3403cfd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cddcb86f9-s655m" Aug 04 15:07:31.587088 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:31.587019 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a1ee2400-c0fe-429d-a439-a0ed35927650-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-8668bf47d8-rhrcs\" (UID: \"a1ee2400-c0fe-429d-a439-a0ed35927650\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8668bf47d8-rhrcs" Aug 04 15:07:31.587088 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:31.587077 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/efd49e4e-739d-4f61-a452-5894a3403cfd-tmp\") pod \"klusterlet-addon-workmgr-5cddcb86f9-s655m\" (UID: \"efd49e4e-739d-4f61-a452-5894a3403cfd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cddcb86f9-s655m" Aug 04 15:07:31.587204 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:31.587132 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/efd49e4e-739d-4f61-a452-5894a3403cfd-klusterlet-config\") pod \"klusterlet-addon-workmgr-5cddcb86f9-s655m\" (UID: \"efd49e4e-739d-4f61-a452-5894a3403cfd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cddcb86f9-s655m" Aug 04 15:07:31.688019 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:31.687936 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x774w\" (UniqueName: \"kubernetes.io/projected/efd49e4e-739d-4f61-a452-5894a3403cfd-kube-api-access-x774w\") pod \"klusterlet-addon-workmgr-5cddcb86f9-s655m\" (UID: \"efd49e4e-739d-4f61-a452-5894a3403cfd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cddcb86f9-s655m" Aug 04 15:07:31.688019 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:31.687978 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a1ee2400-c0fe-429d-a439-a0ed35927650-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-8668bf47d8-rhrcs\" (UID: \"a1ee2400-c0fe-429d-a439-a0ed35927650\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8668bf47d8-rhrcs" Aug 04 15:07:31.688230 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:31.688169 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/efd49e4e-739d-4f61-a452-5894a3403cfd-tmp\") pod \"klusterlet-addon-workmgr-5cddcb86f9-s655m\" (UID: \"efd49e4e-739d-4f61-a452-5894a3403cfd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cddcb86f9-s655m" Aug 04 15:07:31.688288 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:31.688244 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/efd49e4e-739d-4f61-a452-5894a3403cfd-klusterlet-config\") pod \"klusterlet-addon-workmgr-5cddcb86f9-s655m\" (UID: \"efd49e4e-739d-4f61-a452-5894a3403cfd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cddcb86f9-s655m" Aug 04 15:07:31.688343 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:31.688292 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5tjc\" (UniqueName: \"kubernetes.io/projected/a1ee2400-c0fe-429d-a439-a0ed35927650-kube-api-access-r5tjc\") pod \"managed-serviceaccount-addon-agent-8668bf47d8-rhrcs\" (UID: \"a1ee2400-c0fe-429d-a439-a0ed35927650\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8668bf47d8-rhrcs" Aug 04 15:07:31.688577 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:31.688545 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/efd49e4e-739d-4f61-a452-5894a3403cfd-tmp\") pod \"klusterlet-addon-workmgr-5cddcb86f9-s655m\" (UID: \"efd49e4e-739d-4f61-a452-5894a3403cfd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cddcb86f9-s655m" Aug 04 15:07:31.690786 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:31.690754 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a1ee2400-c0fe-429d-a439-a0ed35927650-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-8668bf47d8-rhrcs\" (UID: \"a1ee2400-c0fe-429d-a439-a0ed35927650\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8668bf47d8-rhrcs" Aug 04 15:07:31.690786 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:31.690784 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/efd49e4e-739d-4f61-a452-5894a3403cfd-klusterlet-config\") pod \"klusterlet-addon-workmgr-5cddcb86f9-s655m\" (UID: \"efd49e4e-739d-4f61-a452-5894a3403cfd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cddcb86f9-s655m" Aug 04 15:07:31.721964 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:31.721934 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5tjc\" (UniqueName: \"kubernetes.io/projected/a1ee2400-c0fe-429d-a439-a0ed35927650-kube-api-access-r5tjc\") pod \"managed-serviceaccount-addon-agent-8668bf47d8-rhrcs\" (UID: \"a1ee2400-c0fe-429d-a439-a0ed35927650\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8668bf47d8-rhrcs" Aug 04 15:07:31.728331 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:31.728298 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x774w\" (UniqueName: \"kubernetes.io/projected/efd49e4e-739d-4f61-a452-5894a3403cfd-kube-api-access-x774w\") pod \"klusterlet-addon-workmgr-5cddcb86f9-s655m\" (UID: \"efd49e4e-739d-4f61-a452-5894a3403cfd\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cddcb86f9-s655m" Aug 04 15:07:31.829715 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:31.829678 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8668bf47d8-rhrcs" Aug 04 15:07:31.837474 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:31.837446 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cddcb86f9-s655m" Aug 04 15:07:32.022447 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:32.022417 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8668bf47d8-rhrcs"] Aug 04 15:07:32.025402 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:07:32.025367 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda1ee2400_c0fe_429d_a439_a0ed35927650.slice/crio-4150248252f7cc0daeb69698f825d9d9566785afe0291a58307ebf30e0cd4d00 WatchSource:0}: Error finding container 4150248252f7cc0daeb69698f825d9d9566785afe0291a58307ebf30e0cd4d00: Status 404 returned error can't find the container with id 4150248252f7cc0daeb69698f825d9d9566785afe0291a58307ebf30e0cd4d00 Aug 04 15:07:32.045978 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:32.045954 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cddcb86f9-s655m"] Aug 04 15:07:32.048649 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:07:32.048629 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefd49e4e_739d_4f61_a452_5894a3403cfd.slice/crio-1bdbb3a05c8524ddb64526d51b6e1bc162d6d212830a411553cb474493e02db1 WatchSource:0}: Error finding container 1bdbb3a05c8524ddb64526d51b6e1bc162d6d212830a411553cb474493e02db1: Status 404 returned error can't find the container with id 1bdbb3a05c8524ddb64526d51b6e1bc162d6d212830a411553cb474493e02db1 Aug 04 15:07:32.487355 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:32.487317 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8668bf47d8-rhrcs" event={"ID":"a1ee2400-c0fe-429d-a439-a0ed35927650","Type":"ContainerStarted","Data":"4150248252f7cc0daeb69698f825d9d9566785afe0291a58307ebf30e0cd4d00"} Aug 04 15:07:32.488377 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:32.488352 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cddcb86f9-s655m" event={"ID":"efd49e4e-739d-4f61-a452-5894a3403cfd","Type":"ContainerStarted","Data":"1bdbb3a05c8524ddb64526d51b6e1bc162d6d212830a411553cb474493e02db1"} Aug 04 15:07:33.850714 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:33.850073 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-57675b5c55-dtbz2"] Aug 04 15:07:33.864077 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:33.864044 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57675b5c55-dtbz2" Aug 04 15:07:33.869830 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:33.869802 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Aug 04 15:07:33.870127 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:33.870016 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-txzf8\"" Aug 04 15:07:33.872008 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:33.870209 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Aug 04 15:07:33.872008 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:33.870419 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Aug 04 15:07:33.872008 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:33.870624 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Aug 04 15:07:33.872008 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:33.870715 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Aug 04 15:07:33.872008 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:33.870943 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Aug 04 15:07:33.872008 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:33.871282 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Aug 04 15:07:33.873492 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:33.873115 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57675b5c55-dtbz2"] Aug 04 15:07:34.008589 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:34.008289 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cef83f55-1138-47ab-9700-4e0879c39b71-service-ca\") pod \"console-57675b5c55-dtbz2\" (UID: \"cef83f55-1138-47ab-9700-4e0879c39b71\") " pod="openshift-console/console-57675b5c55-dtbz2" Aug 04 15:07:34.008589 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:34.008339 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cef83f55-1138-47ab-9700-4e0879c39b71-oauth-serving-cert\") pod \"console-57675b5c55-dtbz2\" (UID: \"cef83f55-1138-47ab-9700-4e0879c39b71\") " pod="openshift-console/console-57675b5c55-dtbz2" Aug 04 15:07:34.008589 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:34.008441 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j855p\" (UniqueName: \"kubernetes.io/projected/cef83f55-1138-47ab-9700-4e0879c39b71-kube-api-access-j855p\") pod \"console-57675b5c55-dtbz2\" (UID: \"cef83f55-1138-47ab-9700-4e0879c39b71\") " pod="openshift-console/console-57675b5c55-dtbz2" Aug 04 15:07:34.008589 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:34.008465 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cef83f55-1138-47ab-9700-4e0879c39b71-console-serving-cert\") pod \"console-57675b5c55-dtbz2\" (UID: \"cef83f55-1138-47ab-9700-4e0879c39b71\") " pod="openshift-console/console-57675b5c55-dtbz2" Aug 04 15:07:34.008589 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:34.008501 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cef83f55-1138-47ab-9700-4e0879c39b71-console-config\") pod \"console-57675b5c55-dtbz2\" (UID: \"cef83f55-1138-47ab-9700-4e0879c39b71\") " pod="openshift-console/console-57675b5c55-dtbz2" Aug 04 15:07:34.008589 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:34.008537 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cef83f55-1138-47ab-9700-4e0879c39b71-console-oauth-config\") pod \"console-57675b5c55-dtbz2\" (UID: \"cef83f55-1138-47ab-9700-4e0879c39b71\") " pod="openshift-console/console-57675b5c55-dtbz2" Aug 04 15:07:34.109088 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:34.108995 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cef83f55-1138-47ab-9700-4e0879c39b71-console-config\") pod \"console-57675b5c55-dtbz2\" (UID: \"cef83f55-1138-47ab-9700-4e0879c39b71\") " pod="openshift-console/console-57675b5c55-dtbz2" Aug 04 15:07:34.109088 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:34.109050 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cef83f55-1138-47ab-9700-4e0879c39b71-console-oauth-config\") pod \"console-57675b5c55-dtbz2\" (UID: \"cef83f55-1138-47ab-9700-4e0879c39b71\") " pod="openshift-console/console-57675b5c55-dtbz2" Aug 04 15:07:34.109308 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:34.109093 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cef83f55-1138-47ab-9700-4e0879c39b71-service-ca\") pod \"console-57675b5c55-dtbz2\" (UID: \"cef83f55-1138-47ab-9700-4e0879c39b71\") " pod="openshift-console/console-57675b5c55-dtbz2" Aug 04 15:07:34.109308 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:34.109133 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cef83f55-1138-47ab-9700-4e0879c39b71-oauth-serving-cert\") pod \"console-57675b5c55-dtbz2\" (UID: \"cef83f55-1138-47ab-9700-4e0879c39b71\") " pod="openshift-console/console-57675b5c55-dtbz2" Aug 04 15:07:34.109308 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:34.109195 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j855p\" (UniqueName: \"kubernetes.io/projected/cef83f55-1138-47ab-9700-4e0879c39b71-kube-api-access-j855p\") pod \"console-57675b5c55-dtbz2\" (UID: \"cef83f55-1138-47ab-9700-4e0879c39b71\") " pod="openshift-console/console-57675b5c55-dtbz2" Aug 04 15:07:34.109308 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:34.109214 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cef83f55-1138-47ab-9700-4e0879c39b71-console-serving-cert\") pod \"console-57675b5c55-dtbz2\" (UID: \"cef83f55-1138-47ab-9700-4e0879c39b71\") " pod="openshift-console/console-57675b5c55-dtbz2" Aug 04 15:07:34.110329 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:34.110266 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cef83f55-1138-47ab-9700-4e0879c39b71-oauth-serving-cert\") pod \"console-57675b5c55-dtbz2\" (UID: \"cef83f55-1138-47ab-9700-4e0879c39b71\") " pod="openshift-console/console-57675b5c55-dtbz2" Aug 04 15:07:34.110487 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:34.110373 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cef83f55-1138-47ab-9700-4e0879c39b71-service-ca\") pod \"console-57675b5c55-dtbz2\" (UID: \"cef83f55-1138-47ab-9700-4e0879c39b71\") " pod="openshift-console/console-57675b5c55-dtbz2" Aug 04 15:07:34.110633 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:34.110588 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cef83f55-1138-47ab-9700-4e0879c39b71-console-config\") pod \"console-57675b5c55-dtbz2\" (UID: \"cef83f55-1138-47ab-9700-4e0879c39b71\") " pod="openshift-console/console-57675b5c55-dtbz2" Aug 04 15:07:34.112266 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:34.112231 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cef83f55-1138-47ab-9700-4e0879c39b71-console-serving-cert\") pod \"console-57675b5c55-dtbz2\" (UID: \"cef83f55-1138-47ab-9700-4e0879c39b71\") " pod="openshift-console/console-57675b5c55-dtbz2" Aug 04 15:07:34.112674 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:34.112649 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cef83f55-1138-47ab-9700-4e0879c39b71-console-oauth-config\") pod \"console-57675b5c55-dtbz2\" (UID: \"cef83f55-1138-47ab-9700-4e0879c39b71\") " pod="openshift-console/console-57675b5c55-dtbz2" Aug 04 15:07:34.121844 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:34.121794 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j855p\" (UniqueName: \"kubernetes.io/projected/cef83f55-1138-47ab-9700-4e0879c39b71-kube-api-access-j855p\") pod \"console-57675b5c55-dtbz2\" (UID: \"cef83f55-1138-47ab-9700-4e0879c39b71\") " pod="openshift-console/console-57675b5c55-dtbz2" Aug 04 15:07:34.179608 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:34.179573 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57675b5c55-dtbz2" Aug 04 15:07:37.121940 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:37.121814 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57675b5c55-dtbz2"] Aug 04 15:07:37.126052 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:07:37.126021 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcef83f55_1138_47ab_9700_4e0879c39b71.slice/crio-8105de7b33cc54383cea06286d5f0b32597a2cbcc836b27cf613247d6eeea6df WatchSource:0}: Error finding container 8105de7b33cc54383cea06286d5f0b32597a2cbcc836b27cf613247d6eeea6df: Status 404 returned error can't find the container with id 8105de7b33cc54383cea06286d5f0b32597a2cbcc836b27cf613247d6eeea6df Aug 04 15:07:37.510729 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:37.510687 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8668bf47d8-rhrcs" event={"ID":"a1ee2400-c0fe-429d-a439-a0ed35927650","Type":"ContainerStarted","Data":"81762f600a1613fc45c1b16b9cdf174c18e301ca624938c70ab6deac8e8c6f7c"} Aug 04 15:07:37.512142 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:37.512112 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cddcb86f9-s655m" event={"ID":"efd49e4e-739d-4f61-a452-5894a3403cfd","Type":"ContainerStarted","Data":"aa60034146ad2eed16f385439950b8228b657d4dc903def5f4626dcb3eb4d24b"} Aug 04 15:07:37.512291 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:37.512163 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cddcb86f9-s655m" Aug 04 15:07:37.513337 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:37.513314 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57675b5c55-dtbz2" event={"ID":"cef83f55-1138-47ab-9700-4e0879c39b71","Type":"ContainerStarted","Data":"8105de7b33cc54383cea06286d5f0b32597a2cbcc836b27cf613247d6eeea6df"} Aug 04 15:07:37.514339 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:37.514322 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cddcb86f9-s655m" Aug 04 15:07:37.534967 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:37.534924 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-8668bf47d8-rhrcs" podStartSLOduration=1.587453296 podStartE2EDuration="6.534909975s" totalImagesPullingTime="4.947456679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:07:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:07:37.534456212 +0000 UTC m=+80.994328561" watchObservedRunningTime="2026-08-04 15:07:37.534909975 +0000 UTC m=+80.994782323" Aug 04 15:07:40.485867 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:40.485836 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-hqwr6" Aug 04 15:07:40.521512 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:40.521467 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5cddcb86f9-s655m" podStartSLOduration=4.581427435 podStartE2EDuration="9.521453085s" totalImagesPullingTime="4.94002565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:07:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:07:37.565084444 +0000 UTC m=+81.024956792" watchObservedRunningTime="2026-08-04 15:07:40.521453085 +0000 UTC m=+83.981325435" Aug 04 15:07:40.526761 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:40.526738 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57675b5c55-dtbz2" event={"ID":"cef83f55-1138-47ab-9700-4e0879c39b71","Type":"ContainerStarted","Data":"77168c1cd0a280e1cc14b817626ad9ed0b0fb8079f84976a11a8b99d816eea75"} Aug 04 15:07:40.565439 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:40.565367 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-57675b5c55-dtbz2" podStartSLOduration=4.333164129 podStartE2EDuration="7.565347821s" totalImagesPullingTime="3.232183692s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:07:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:07:40.563690594 +0000 UTC m=+84.023562943" watchObservedRunningTime="2026-08-04 15:07:40.565347821 +0000 UTC m=+84.025220172" Aug 04 15:07:44.180332 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:44.180300 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-57675b5c55-dtbz2" Aug 04 15:07:44.180726 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:44.180351 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-57675b5c55-dtbz2" Aug 04 15:07:44.185707 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:44.185682 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-57675b5c55-dtbz2" Aug 04 15:07:44.545019 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:44.544994 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-57675b5c55-dtbz2" Aug 04 15:07:47.452062 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:47.452027 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" Aug 04 15:07:48.731214 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:48.731180 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mgmxj"] Aug 04 15:07:48.733965 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:48.733948 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mgmxj" Aug 04 15:07:48.736732 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:48.736705 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Aug 04 15:07:48.736870 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:48.736849 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Aug 04 15:07:48.736939 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:48.736857 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Aug 04 15:07:48.738003 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:48.737953 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Aug 04 15:07:48.738296 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:48.738033 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-c87cf\"" Aug 04 15:07:48.828010 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:48.827979 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c1225731-e66d-44a6-82db-3f00d98225fe-root\") pod \"node-exporter-mgmxj\" (UID: \"c1225731-e66d-44a6-82db-3f00d98225fe\") " pod="openshift-monitoring/node-exporter-mgmxj" Aug 04 15:07:48.828149 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:48.828027 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c1225731-e66d-44a6-82db-3f00d98225fe-node-exporter-tls\") pod \"node-exporter-mgmxj\" (UID: \"c1225731-e66d-44a6-82db-3f00d98225fe\") " pod="openshift-monitoring/node-exporter-mgmxj" Aug 04 15:07:48.828149 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:48.828058 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c1225731-e66d-44a6-82db-3f00d98225fe-node-exporter-accelerators-collector-config\") pod \"node-exporter-mgmxj\" (UID: \"c1225731-e66d-44a6-82db-3f00d98225fe\") " pod="openshift-monitoring/node-exporter-mgmxj" Aug 04 15:07:48.828149 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:48.828091 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c1225731-e66d-44a6-82db-3f00d98225fe-node-exporter-textfile\") pod \"node-exporter-mgmxj\" (UID: \"c1225731-e66d-44a6-82db-3f00d98225fe\") " pod="openshift-monitoring/node-exporter-mgmxj" Aug 04 15:07:48.828149 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:48.828111 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkwlj\" (UniqueName: \"kubernetes.io/projected/c1225731-e66d-44a6-82db-3f00d98225fe-kube-api-access-rkwlj\") pod \"node-exporter-mgmxj\" (UID: \"c1225731-e66d-44a6-82db-3f00d98225fe\") " pod="openshift-monitoring/node-exporter-mgmxj" Aug 04 15:07:48.828272 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:48.828166 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1225731-e66d-44a6-82db-3f00d98225fe-metrics-client-ca\") pod \"node-exporter-mgmxj\" (UID: \"c1225731-e66d-44a6-82db-3f00d98225fe\") " pod="openshift-monitoring/node-exporter-mgmxj" Aug 04 15:07:48.828272 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:48.828203 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c1225731-e66d-44a6-82db-3f00d98225fe-node-exporter-wtmp\") pod \"node-exporter-mgmxj\" (UID: \"c1225731-e66d-44a6-82db-3f00d98225fe\") " pod="openshift-monitoring/node-exporter-mgmxj" Aug 04 15:07:48.828272 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:48.828221 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c1225731-e66d-44a6-82db-3f00d98225fe-sys\") pod \"node-exporter-mgmxj\" (UID: \"c1225731-e66d-44a6-82db-3f00d98225fe\") " pod="openshift-monitoring/node-exporter-mgmxj" Aug 04 15:07:48.828361 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:48.828269 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c1225731-e66d-44a6-82db-3f00d98225fe-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mgmxj\" (UID: \"c1225731-e66d-44a6-82db-3f00d98225fe\") " pod="openshift-monitoring/node-exporter-mgmxj" Aug 04 15:07:48.929574 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:48.929543 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c1225731-e66d-44a6-82db-3f00d98225fe-sys\") pod \"node-exporter-mgmxj\" (UID: \"c1225731-e66d-44a6-82db-3f00d98225fe\") " pod="openshift-monitoring/node-exporter-mgmxj" Aug 04 15:07:48.929574 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:48.929576 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c1225731-e66d-44a6-82db-3f00d98225fe-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mgmxj\" (UID: \"c1225731-e66d-44a6-82db-3f00d98225fe\") " pod="openshift-monitoring/node-exporter-mgmxj" Aug 04 15:07:48.929754 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:48.929599 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c1225731-e66d-44a6-82db-3f00d98225fe-root\") pod \"node-exporter-mgmxj\" (UID: \"c1225731-e66d-44a6-82db-3f00d98225fe\") " pod="openshift-monitoring/node-exporter-mgmxj" Aug 04 15:07:48.929754 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:48.929740 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c1225731-e66d-44a6-82db-3f00d98225fe-sys\") pod \"node-exporter-mgmxj\" (UID: \"c1225731-e66d-44a6-82db-3f00d98225fe\") " pod="openshift-monitoring/node-exporter-mgmxj" Aug 04 15:07:48.929827 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:48.929804 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c1225731-e66d-44a6-82db-3f00d98225fe-root\") pod \"node-exporter-mgmxj\" (UID: \"c1225731-e66d-44a6-82db-3f00d98225fe\") " pod="openshift-monitoring/node-exporter-mgmxj" Aug 04 15:07:48.929863 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:48.929842 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c1225731-e66d-44a6-82db-3f00d98225fe-node-exporter-tls\") pod \"node-exporter-mgmxj\" (UID: \"c1225731-e66d-44a6-82db-3f00d98225fe\") " pod="openshift-monitoring/node-exporter-mgmxj" Aug 04 15:07:48.929902 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:48.929869 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c1225731-e66d-44a6-82db-3f00d98225fe-node-exporter-accelerators-collector-config\") pod \"node-exporter-mgmxj\" (UID: \"c1225731-e66d-44a6-82db-3f00d98225fe\") " pod="openshift-monitoring/node-exporter-mgmxj" Aug 04 15:07:48.929902 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:48.929900 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c1225731-e66d-44a6-82db-3f00d98225fe-node-exporter-textfile\") pod \"node-exporter-mgmxj\" (UID: \"c1225731-e66d-44a6-82db-3f00d98225fe\") " pod="openshift-monitoring/node-exporter-mgmxj" Aug 04 15:07:48.929992 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:48.929918 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkwlj\" (UniqueName: \"kubernetes.io/projected/c1225731-e66d-44a6-82db-3f00d98225fe-kube-api-access-rkwlj\") pod \"node-exporter-mgmxj\" (UID: \"c1225731-e66d-44a6-82db-3f00d98225fe\") " pod="openshift-monitoring/node-exporter-mgmxj" Aug 04 15:07:48.929992 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:48.929951 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1225731-e66d-44a6-82db-3f00d98225fe-metrics-client-ca\") pod \"node-exporter-mgmxj\" (UID: \"c1225731-e66d-44a6-82db-3f00d98225fe\") " pod="openshift-monitoring/node-exporter-mgmxj" Aug 04 15:07:48.930078 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:48.930016 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c1225731-e66d-44a6-82db-3f00d98225fe-node-exporter-wtmp\") pod \"node-exporter-mgmxj\" (UID: \"c1225731-e66d-44a6-82db-3f00d98225fe\") " pod="openshift-monitoring/node-exporter-mgmxj" Aug 04 15:07:48.930180 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:48.930162 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c1225731-e66d-44a6-82db-3f00d98225fe-node-exporter-wtmp\") pod \"node-exporter-mgmxj\" (UID: \"c1225731-e66d-44a6-82db-3f00d98225fe\") " pod="openshift-monitoring/node-exporter-mgmxj" Aug 04 15:07:48.930243 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:48.930229 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c1225731-e66d-44a6-82db-3f00d98225fe-node-exporter-textfile\") pod \"node-exporter-mgmxj\" (UID: \"c1225731-e66d-44a6-82db-3f00d98225fe\") " pod="openshift-monitoring/node-exporter-mgmxj" Aug 04 15:07:48.930513 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:48.930489 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c1225731-e66d-44a6-82db-3f00d98225fe-node-exporter-accelerators-collector-config\") pod \"node-exporter-mgmxj\" (UID: \"c1225731-e66d-44a6-82db-3f00d98225fe\") " pod="openshift-monitoring/node-exporter-mgmxj" Aug 04 15:07:48.930588 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:48.930543 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1225731-e66d-44a6-82db-3f00d98225fe-metrics-client-ca\") pod \"node-exporter-mgmxj\" (UID: \"c1225731-e66d-44a6-82db-3f00d98225fe\") " pod="openshift-monitoring/node-exporter-mgmxj" Aug 04 15:07:48.932060 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:48.932034 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c1225731-e66d-44a6-82db-3f00d98225fe-node-exporter-tls\") pod \"node-exporter-mgmxj\" (UID: \"c1225731-e66d-44a6-82db-3f00d98225fe\") " pod="openshift-monitoring/node-exporter-mgmxj" Aug 04 15:07:48.932227 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:48.932208 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c1225731-e66d-44a6-82db-3f00d98225fe-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mgmxj\" (UID: \"c1225731-e66d-44a6-82db-3f00d98225fe\") " pod="openshift-monitoring/node-exporter-mgmxj" Aug 04 15:07:48.940971 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:48.940944 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkwlj\" (UniqueName: \"kubernetes.io/projected/c1225731-e66d-44a6-82db-3f00d98225fe-kube-api-access-rkwlj\") pod \"node-exporter-mgmxj\" (UID: \"c1225731-e66d-44a6-82db-3f00d98225fe\") " pod="openshift-monitoring/node-exporter-mgmxj" Aug 04 15:07:49.043133 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:49.043105 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mgmxj" Aug 04 15:07:49.054102 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:07:49.054075 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1225731_e66d_44a6_82db_3f00d98225fe.slice/crio-9923eff9b5270cef0d3d0f1a81a3057b3b24274e069826d7e5d5d226aeaeb5af WatchSource:0}: Error finding container 9923eff9b5270cef0d3d0f1a81a3057b3b24274e069826d7e5d5d226aeaeb5af: Status 404 returned error can't find the container with id 9923eff9b5270cef0d3d0f1a81a3057b3b24274e069826d7e5d5d226aeaeb5af Aug 04 15:07:49.560680 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:49.560646 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mgmxj" event={"ID":"c1225731-e66d-44a6-82db-3f00d98225fe","Type":"ContainerStarted","Data":"9923eff9b5270cef0d3d0f1a81a3057b3b24274e069826d7e5d5d226aeaeb5af"} Aug 04 15:07:50.566080 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:50.565998 2574 generic.go:356] "Generic (PLEG): container finished" podID="c1225731-e66d-44a6-82db-3f00d98225fe" containerID="31e8d9674384781de712138105fff3a5f1ec7a4b908ee60b9a247f0aecde9e18" exitCode=0 Aug 04 15:07:50.566080 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:50.566049 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mgmxj" event={"ID":"c1225731-e66d-44a6-82db-3f00d98225fe","Type":"ContainerDied","Data":"31e8d9674384781de712138105fff3a5f1ec7a4b908ee60b9a247f0aecde9e18"} Aug 04 15:07:51.571633 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:51.571595 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mgmxj" event={"ID":"c1225731-e66d-44a6-82db-3f00d98225fe","Type":"ContainerStarted","Data":"b1df5a3a5f54583205be70edeed702c7c351d370d06b781b92a22bed18b97d43"} Aug 04 15:07:51.572002 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:51.571659 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mgmxj" event={"ID":"c1225731-e66d-44a6-82db-3f00d98225fe","Type":"ContainerStarted","Data":"e5943f4017571718aa37344c640dcac37e98d24ddd19f7d5b515a37be0353f3c"} Aug 04 15:07:51.599897 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:51.599846 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mgmxj" podStartSLOduration=2.320669397 podStartE2EDuration="3.599832035s" totalImagesPullingTime="1.214033595s" totalInitContainerRuntime="65.129043ms" isStatefulPod=true podCreationTimestamp="2026-08-04 15:07:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:07:51.598158806 +0000 UTC m=+95.058031156" watchObservedRunningTime="2026-08-04 15:07:51.599832035 +0000 UTC m=+95.059704383" Aug 04 15:07:53.318280 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:53.318243 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57675b5c55-dtbz2"] Aug 04 15:07:54.410703 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:54.410674 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2vsg4" Aug 04 15:07:54.683558 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:07:54.683484 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-bddc5f76d-42zvh"] Aug 04 15:08:02.491315 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:02.491280 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5cbcc8d8c9-7xwxn"] Aug 04 15:08:02.525241 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:02.525210 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cbcc8d8c9-7xwxn"] Aug 04 15:08:02.525420 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:02.525307 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cbcc8d8c9-7xwxn" Aug 04 15:08:02.534770 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:02.534747 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Aug 04 15:08:02.546620 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:02.546597 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9621c718-c201-42e2-abbd-62076fefb25e-console-serving-cert\") pod \"console-5cbcc8d8c9-7xwxn\" (UID: \"9621c718-c201-42e2-abbd-62076fefb25e\") " pod="openshift-console/console-5cbcc8d8c9-7xwxn" Aug 04 15:08:02.546711 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:02.546633 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9621c718-c201-42e2-abbd-62076fefb25e-oauth-serving-cert\") pod \"console-5cbcc8d8c9-7xwxn\" (UID: \"9621c718-c201-42e2-abbd-62076fefb25e\") " pod="openshift-console/console-5cbcc8d8c9-7xwxn" Aug 04 15:08:02.546711 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:02.546651 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9621c718-c201-42e2-abbd-62076fefb25e-service-ca\") pod \"console-5cbcc8d8c9-7xwxn\" (UID: \"9621c718-c201-42e2-abbd-62076fefb25e\") " pod="openshift-console/console-5cbcc8d8c9-7xwxn" Aug 04 15:08:02.546783 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:02.546724 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9621c718-c201-42e2-abbd-62076fefb25e-console-config\") pod \"console-5cbcc8d8c9-7xwxn\" (UID: \"9621c718-c201-42e2-abbd-62076fefb25e\") " pod="openshift-console/console-5cbcc8d8c9-7xwxn" Aug 04 15:08:02.546783 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:02.546758 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9621c718-c201-42e2-abbd-62076fefb25e-trusted-ca-bundle\") pod \"console-5cbcc8d8c9-7xwxn\" (UID: \"9621c718-c201-42e2-abbd-62076fefb25e\") " pod="openshift-console/console-5cbcc8d8c9-7xwxn" Aug 04 15:08:02.546844 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:02.546799 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9621c718-c201-42e2-abbd-62076fefb25e-console-oauth-config\") pod \"console-5cbcc8d8c9-7xwxn\" (UID: \"9621c718-c201-42e2-abbd-62076fefb25e\") " pod="openshift-console/console-5cbcc8d8c9-7xwxn" Aug 04 15:08:02.546844 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:02.546833 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4pk8\" (UniqueName: \"kubernetes.io/projected/9621c718-c201-42e2-abbd-62076fefb25e-kube-api-access-h4pk8\") pod \"console-5cbcc8d8c9-7xwxn\" (UID: \"9621c718-c201-42e2-abbd-62076fefb25e\") " pod="openshift-console/console-5cbcc8d8c9-7xwxn" Aug 04 15:08:02.648175 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:02.648143 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9621c718-c201-42e2-abbd-62076fefb25e-console-oauth-config\") pod \"console-5cbcc8d8c9-7xwxn\" (UID: \"9621c718-c201-42e2-abbd-62076fefb25e\") " pod="openshift-console/console-5cbcc8d8c9-7xwxn" Aug 04 15:08:02.648175 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:02.648179 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4pk8\" (UniqueName: \"kubernetes.io/projected/9621c718-c201-42e2-abbd-62076fefb25e-kube-api-access-h4pk8\") pod \"console-5cbcc8d8c9-7xwxn\" (UID: \"9621c718-c201-42e2-abbd-62076fefb25e\") " pod="openshift-console/console-5cbcc8d8c9-7xwxn" Aug 04 15:08:02.648425 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:02.648211 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9621c718-c201-42e2-abbd-62076fefb25e-console-serving-cert\") pod \"console-5cbcc8d8c9-7xwxn\" (UID: \"9621c718-c201-42e2-abbd-62076fefb25e\") " pod="openshift-console/console-5cbcc8d8c9-7xwxn" Aug 04 15:08:02.648425 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:02.648243 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9621c718-c201-42e2-abbd-62076fefb25e-oauth-serving-cert\") pod \"console-5cbcc8d8c9-7xwxn\" (UID: \"9621c718-c201-42e2-abbd-62076fefb25e\") " pod="openshift-console/console-5cbcc8d8c9-7xwxn" Aug 04 15:08:02.648425 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:02.648263 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9621c718-c201-42e2-abbd-62076fefb25e-service-ca\") pod \"console-5cbcc8d8c9-7xwxn\" (UID: \"9621c718-c201-42e2-abbd-62076fefb25e\") " pod="openshift-console/console-5cbcc8d8c9-7xwxn" Aug 04 15:08:02.648425 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:02.648306 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9621c718-c201-42e2-abbd-62076fefb25e-console-config\") pod \"console-5cbcc8d8c9-7xwxn\" (UID: \"9621c718-c201-42e2-abbd-62076fefb25e\") " pod="openshift-console/console-5cbcc8d8c9-7xwxn" Aug 04 15:08:02.648425 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:02.648334 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9621c718-c201-42e2-abbd-62076fefb25e-trusted-ca-bundle\") pod \"console-5cbcc8d8c9-7xwxn\" (UID: \"9621c718-c201-42e2-abbd-62076fefb25e\") " pod="openshift-console/console-5cbcc8d8c9-7xwxn" Aug 04 15:08:02.649036 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:02.649011 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9621c718-c201-42e2-abbd-62076fefb25e-service-ca\") pod \"console-5cbcc8d8c9-7xwxn\" (UID: \"9621c718-c201-42e2-abbd-62076fefb25e\") " pod="openshift-console/console-5cbcc8d8c9-7xwxn" Aug 04 15:08:02.649137 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:02.649084 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9621c718-c201-42e2-abbd-62076fefb25e-console-config\") pod \"console-5cbcc8d8c9-7xwxn\" (UID: \"9621c718-c201-42e2-abbd-62076fefb25e\") " pod="openshift-console/console-5cbcc8d8c9-7xwxn" Aug 04 15:08:02.649137 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:02.649106 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9621c718-c201-42e2-abbd-62076fefb25e-oauth-serving-cert\") pod \"console-5cbcc8d8c9-7xwxn\" (UID: \"9621c718-c201-42e2-abbd-62076fefb25e\") " pod="openshift-console/console-5cbcc8d8c9-7xwxn" Aug 04 15:08:02.649593 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:02.649571 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9621c718-c201-42e2-abbd-62076fefb25e-trusted-ca-bundle\") pod \"console-5cbcc8d8c9-7xwxn\" (UID: \"9621c718-c201-42e2-abbd-62076fefb25e\") " pod="openshift-console/console-5cbcc8d8c9-7xwxn" Aug 04 15:08:02.650744 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:02.650723 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9621c718-c201-42e2-abbd-62076fefb25e-console-serving-cert\") pod \"console-5cbcc8d8c9-7xwxn\" (UID: \"9621c718-c201-42e2-abbd-62076fefb25e\") " pod="openshift-console/console-5cbcc8d8c9-7xwxn" Aug 04 15:08:02.650845 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:02.650835 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9621c718-c201-42e2-abbd-62076fefb25e-console-oauth-config\") pod \"console-5cbcc8d8c9-7xwxn\" (UID: \"9621c718-c201-42e2-abbd-62076fefb25e\") " pod="openshift-console/console-5cbcc8d8c9-7xwxn" Aug 04 15:08:02.656602 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:02.656577 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4pk8\" (UniqueName: \"kubernetes.io/projected/9621c718-c201-42e2-abbd-62076fefb25e-kube-api-access-h4pk8\") pod \"console-5cbcc8d8c9-7xwxn\" (UID: \"9621c718-c201-42e2-abbd-62076fefb25e\") " pod="openshift-console/console-5cbcc8d8c9-7xwxn" Aug 04 15:08:02.835774 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:02.835694 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cbcc8d8c9-7xwxn" Aug 04 15:08:02.982295 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:02.982261 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cbcc8d8c9-7xwxn"] Aug 04 15:08:02.986552 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:08:02.986526 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9621c718_c201_42e2_abbd_62076fefb25e.slice/crio-f8a3dde9d5ee5e315aacd0bc3ef3c228006574d553d1605e99d0abcd42933d5e WatchSource:0}: Error finding container f8a3dde9d5ee5e315aacd0bc3ef3c228006574d553d1605e99d0abcd42933d5e: Status 404 returned error can't find the container with id f8a3dde9d5ee5e315aacd0bc3ef3c228006574d553d1605e99d0abcd42933d5e Aug 04 15:08:03.616979 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:03.616946 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cbcc8d8c9-7xwxn" event={"ID":"9621c718-c201-42e2-abbd-62076fefb25e","Type":"ContainerStarted","Data":"db13d3ae6a10a19d47622163401af95e7231736c9719a0ae3cd2a00732be9831"} Aug 04 15:08:03.617446 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:03.616995 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cbcc8d8c9-7xwxn" event={"ID":"9621c718-c201-42e2-abbd-62076fefb25e","Type":"ContainerStarted","Data":"f8a3dde9d5ee5e315aacd0bc3ef3c228006574d553d1605e99d0abcd42933d5e"} Aug 04 15:08:03.651290 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:03.651243 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5cbcc8d8c9-7xwxn" podStartSLOduration=1.651226078 podStartE2EDuration="1.651226078s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:08:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:08:03.648662345 +0000 UTC m=+107.108534692" watchObservedRunningTime="2026-08-04 15:08:03.651226078 +0000 UTC m=+107.111098477" Aug 04 15:08:12.836659 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:12.836621 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5cbcc8d8c9-7xwxn" Aug 04 15:08:12.837144 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:12.836739 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5cbcc8d8c9-7xwxn" Aug 04 15:08:12.841587 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:12.841564 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5cbcc8d8c9-7xwxn" Aug 04 15:08:13.659103 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:13.659071 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5cbcc8d8c9-7xwxn" Aug 04 15:08:18.343182 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:18.343140 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-57675b5c55-dtbz2" podUID="cef83f55-1138-47ab-9700-4e0879c39b71" containerName="console" containerID="cri-o://77168c1cd0a280e1cc14b817626ad9ed0b0fb8079f84976a11a8b99d816eea75" gracePeriod=15 Aug 04 15:08:18.577063 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:18.577040 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57675b5c55-dtbz2" Aug 04 15:08:18.674853 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:18.674828 2574 generic.go:356] "Generic (PLEG): container finished" podID="91c9aec0-0301-40bd-8bbd-ebd389fb4faf" containerID="462d2e13500f5049f7053e8325bc78d4f654ea5cd7d1469b5a011a8a216e4029" exitCode=0 Aug 04 15:08:18.674981 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:18.674911 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-kpmh2" event={"ID":"91c9aec0-0301-40bd-8bbd-ebd389fb4faf","Type":"ContainerDied","Data":"462d2e13500f5049f7053e8325bc78d4f654ea5cd7d1469b5a011a8a216e4029"} Aug 04 15:08:18.675333 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:18.675314 2574 scope.go:117] "RemoveContainer" containerID="462d2e13500f5049f7053e8325bc78d4f654ea5cd7d1469b5a011a8a216e4029" Aug 04 15:08:18.676338 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:18.676284 2574 generic.go:356] "Generic (PLEG): container finished" podID="cef83f55-1138-47ab-9700-4e0879c39b71" containerID="77168c1cd0a280e1cc14b817626ad9ed0b0fb8079f84976a11a8b99d816eea75" exitCode=0 Aug 04 15:08:18.676338 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:18.676326 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57675b5c55-dtbz2" Aug 04 15:08:18.676499 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:18.676318 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57675b5c55-dtbz2" event={"ID":"cef83f55-1138-47ab-9700-4e0879c39b71","Type":"ContainerDied","Data":"77168c1cd0a280e1cc14b817626ad9ed0b0fb8079f84976a11a8b99d816eea75"} Aug 04 15:08:18.676499 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:18.676382 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57675b5c55-dtbz2" event={"ID":"cef83f55-1138-47ab-9700-4e0879c39b71","Type":"ContainerDied","Data":"8105de7b33cc54383cea06286d5f0b32597a2cbcc836b27cf613247d6eeea6df"} Aug 04 15:08:18.676499 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:18.676424 2574 scope.go:117] "RemoveContainer" containerID="77168c1cd0a280e1cc14b817626ad9ed0b0fb8079f84976a11a8b99d816eea75" Aug 04 15:08:18.688582 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:18.688564 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cef83f55-1138-47ab-9700-4e0879c39b71-console-serving-cert\") pod \"cef83f55-1138-47ab-9700-4e0879c39b71\" (UID: \"cef83f55-1138-47ab-9700-4e0879c39b71\") " Aug 04 15:08:18.688663 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:18.688600 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cef83f55-1138-47ab-9700-4e0879c39b71-oauth-serving-cert\") pod \"cef83f55-1138-47ab-9700-4e0879c39b71\" (UID: \"cef83f55-1138-47ab-9700-4e0879c39b71\") " Aug 04 15:08:18.688663 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:18.688649 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cef83f55-1138-47ab-9700-4e0879c39b71-service-ca\") pod \"cef83f55-1138-47ab-9700-4e0879c39b71\" (UID: \"cef83f55-1138-47ab-9700-4e0879c39b71\") " Aug 04 15:08:18.688736 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:18.688677 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cef83f55-1138-47ab-9700-4e0879c39b71-console-config\") pod \"cef83f55-1138-47ab-9700-4e0879c39b71\" (UID: \"cef83f55-1138-47ab-9700-4e0879c39b71\") " Aug 04 15:08:18.688736 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:18.688727 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cef83f55-1138-47ab-9700-4e0879c39b71-console-oauth-config\") pod \"cef83f55-1138-47ab-9700-4e0879c39b71\" (UID: \"cef83f55-1138-47ab-9700-4e0879c39b71\") " Aug 04 15:08:18.688809 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:18.688769 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j855p\" (UniqueName: \"kubernetes.io/projected/cef83f55-1138-47ab-9700-4e0879c39b71-kube-api-access-j855p\") pod \"cef83f55-1138-47ab-9700-4e0879c39b71\" (UID: \"cef83f55-1138-47ab-9700-4e0879c39b71\") " Aug 04 15:08:18.689164 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:18.689053 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cef83f55-1138-47ab-9700-4e0879c39b71-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "cef83f55-1138-47ab-9700-4e0879c39b71" (UID: "cef83f55-1138-47ab-9700-4e0879c39b71"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 15:08:18.689164 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:18.689066 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cef83f55-1138-47ab-9700-4e0879c39b71-service-ca" (OuterVolumeSpecName: "service-ca") pod "cef83f55-1138-47ab-9700-4e0879c39b71" (UID: "cef83f55-1138-47ab-9700-4e0879c39b71"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 15:08:18.689164 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:18.689113 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cef83f55-1138-47ab-9700-4e0879c39b71-console-config" (OuterVolumeSpecName: "console-config") pod "cef83f55-1138-47ab-9700-4e0879c39b71" (UID: "cef83f55-1138-47ab-9700-4e0879c39b71"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 15:08:18.690788 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:18.690759 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cef83f55-1138-47ab-9700-4e0879c39b71-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "cef83f55-1138-47ab-9700-4e0879c39b71" (UID: "cef83f55-1138-47ab-9700-4e0879c39b71"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 15:08:18.690788 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:18.690778 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cef83f55-1138-47ab-9700-4e0879c39b71-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "cef83f55-1138-47ab-9700-4e0879c39b71" (UID: "cef83f55-1138-47ab-9700-4e0879c39b71"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 15:08:18.690900 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:18.690825 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cef83f55-1138-47ab-9700-4e0879c39b71-kube-api-access-j855p" (OuterVolumeSpecName: "kube-api-access-j855p") pod "cef83f55-1138-47ab-9700-4e0879c39b71" (UID: "cef83f55-1138-47ab-9700-4e0879c39b71"). InnerVolumeSpecName "kube-api-access-j855p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 15:08:18.705742 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:18.705723 2574 scope.go:117] "RemoveContainer" containerID="77168c1cd0a280e1cc14b817626ad9ed0b0fb8079f84976a11a8b99d816eea75" Aug 04 15:08:18.706101 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:08:18.706067 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"77168c1cd0a280e1cc14b817626ad9ed0b0fb8079f84976a11a8b99d816eea75\": container with ID starting with 77168c1cd0a280e1cc14b817626ad9ed0b0fb8079f84976a11a8b99d816eea75 not found: ID does not exist" containerID="77168c1cd0a280e1cc14b817626ad9ed0b0fb8079f84976a11a8b99d816eea75" Aug 04 15:08:18.706203 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:18.706099 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"77168c1cd0a280e1cc14b817626ad9ed0b0fb8079f84976a11a8b99d816eea75"} err="failed to get container status \"77168c1cd0a280e1cc14b817626ad9ed0b0fb8079f84976a11a8b99d816eea75\": rpc error: code = NotFound desc = could not find container \"77168c1cd0a280e1cc14b817626ad9ed0b0fb8079f84976a11a8b99d816eea75\": container with ID starting with 77168c1cd0a280e1cc14b817626ad9ed0b0fb8079f84976a11a8b99d816eea75 not found: ID does not exist" Aug 04 15:08:18.790444 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:18.790408 2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cef83f55-1138-47ab-9700-4e0879c39b71-console-oauth-config\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:08:18.790638 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:18.790560 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j855p\" (UniqueName: \"kubernetes.io/projected/cef83f55-1138-47ab-9700-4e0879c39b71-kube-api-access-j855p\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:08:18.790638 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:18.790593 2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cef83f55-1138-47ab-9700-4e0879c39b71-console-serving-cert\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:08:18.790638 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:18.790608 2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cef83f55-1138-47ab-9700-4e0879c39b71-oauth-serving-cert\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:08:18.790638 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:18.790626 2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cef83f55-1138-47ab-9700-4e0879c39b71-service-ca\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:08:18.790779 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:18.790646 2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cef83f55-1138-47ab-9700-4e0879c39b71-console-config\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:08:18.997820 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:18.997790 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57675b5c55-dtbz2"] Aug 04 15:08:19.002356 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:19.002334 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-57675b5c55-dtbz2"] Aug 04 15:08:19.052573 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:19.052548 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cef83f55-1138-47ab-9700-4e0879c39b71" path="/var/lib/kubelet/pods/cef83f55-1138-47ab-9700-4e0879c39b71/volumes" Aug 04 15:08:19.555119 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:19.555089 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7fc97bc64b-45b4c_d201490f-504d-4d2a-8b4f-c4eea2023469/router/0.log" Aug 04 15:08:19.584417 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:19.584371 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rcqrj_43e78d75-11c3-457d-aec1-5d4379ab2470/serve-healthcheck-canary/0.log" Aug 04 15:08:19.682101 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:19.682063 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-77696685b-kpmh2" event={"ID":"91c9aec0-0301-40bd-8bbd-ebd389fb4faf","Type":"ContainerStarted","Data":"b2120022f81f174e8b08a33e0413cdf4efd922530c3fc1696edfbda2df1f2cc4"} Aug 04 15:08:19.702327 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:19.702291 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" podUID="aab2968d-11b7-4174-b0c4-b3ce5a46b90e" containerName="registry" containerID="cri-o://0b7c8629f548380ddc854dedc1218029128ceb86a0345708ad4b638242bc5a62" gracePeriod=30 Aug 04 15:08:19.949858 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:19.949836 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" Aug 04 15:08:20.000116 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:20.000084 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-bound-sa-token\") pod \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\" (UID: \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\") " Aug 04 15:08:20.000269 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:20.000121 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-installation-pull-secrets\") pod \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\" (UID: \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\") " Aug 04 15:08:20.000269 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:20.000166 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-registry-tls\") pod \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\" (UID: \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\") " Aug 04 15:08:20.000269 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:20.000207 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-ca-trust-extracted\") pod \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\" (UID: \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\") " Aug 04 15:08:20.000269 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:20.000244 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v66fh\" (UniqueName: \"kubernetes.io/projected/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-kube-api-access-v66fh\") pod \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\" (UID: \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\") " Aug 04 15:08:20.000509 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:20.000362 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-registry-certificates\") pod \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\" (UID: \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\") " Aug 04 15:08:20.000509 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:20.000438 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-image-registry-private-configuration\") pod \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\" (UID: \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\") " Aug 04 15:08:20.000509 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:20.000497 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-trusted-ca\") pod \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\" (UID: \"aab2968d-11b7-4174-b0c4-b3ce5a46b90e\") " Aug 04 15:08:20.000816 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:20.000783 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "aab2968d-11b7-4174-b0c4-b3ce5a46b90e" (UID: "aab2968d-11b7-4174-b0c4-b3ce5a46b90e"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 15:08:20.001175 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:20.001147 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "aab2968d-11b7-4174-b0c4-b3ce5a46b90e" (UID: "aab2968d-11b7-4174-b0c4-b3ce5a46b90e"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 15:08:20.002841 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:20.002810 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "aab2968d-11b7-4174-b0c4-b3ce5a46b90e" (UID: "aab2968d-11b7-4174-b0c4-b3ce5a46b90e"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 15:08:20.003005 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:20.002985 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "aab2968d-11b7-4174-b0c4-b3ce5a46b90e" (UID: "aab2968d-11b7-4174-b0c4-b3ce5a46b90e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 15:08:20.003082 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:20.003050 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-kube-api-access-v66fh" (OuterVolumeSpecName: "kube-api-access-v66fh") pod "aab2968d-11b7-4174-b0c4-b3ce5a46b90e" (UID: "aab2968d-11b7-4174-b0c4-b3ce5a46b90e"). InnerVolumeSpecName "kube-api-access-v66fh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 15:08:20.003082 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:20.003048 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "aab2968d-11b7-4174-b0c4-b3ce5a46b90e" (UID: "aab2968d-11b7-4174-b0c4-b3ce5a46b90e"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 15:08:20.003315 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:20.003296 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "aab2968d-11b7-4174-b0c4-b3ce5a46b90e" (UID: "aab2968d-11b7-4174-b0c4-b3ce5a46b90e"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 15:08:20.010013 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:20.009987 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "aab2968d-11b7-4174-b0c4-b3ce5a46b90e" (UID: "aab2968d-11b7-4174-b0c4-b3ce5a46b90e"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:08:20.102030 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:20.101939 2574 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-registry-certificates\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:08:20.102030 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:20.101983 2574 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-image-registry-private-configuration\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:08:20.102030 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:20.101995 2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-trusted-ca\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:08:20.102030 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:20.102003 2574 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-bound-sa-token\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:08:20.102030 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:20.102012 2574 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-installation-pull-secrets\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:08:20.102030 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:20.102020 2574 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-registry-tls\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:08:20.102030 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:20.102028 2574 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-ca-trust-extracted\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:08:20.102030 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:20.102036 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v66fh\" (UniqueName: \"kubernetes.io/projected/aab2968d-11b7-4174-b0c4-b3ce5a46b90e-kube-api-access-v66fh\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:08:20.688088 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:20.688046 2574 generic.go:356] "Generic (PLEG): container finished" podID="aab2968d-11b7-4174-b0c4-b3ce5a46b90e" containerID="0b7c8629f548380ddc854dedc1218029128ceb86a0345708ad4b638242bc5a62" exitCode=0 Aug 04 15:08:20.688556 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:20.688099 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" Aug 04 15:08:20.688556 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:20.688132 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" event={"ID":"aab2968d-11b7-4174-b0c4-b3ce5a46b90e","Type":"ContainerDied","Data":"0b7c8629f548380ddc854dedc1218029128ceb86a0345708ad4b638242bc5a62"} Aug 04 15:08:20.688556 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:20.688185 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-bddc5f76d-42zvh" event={"ID":"aab2968d-11b7-4174-b0c4-b3ce5a46b90e","Type":"ContainerDied","Data":"40ade403054082ffb8fa7ef3cdcec04b8915df81f62b5b5d1f7deb80f9337120"} Aug 04 15:08:20.688556 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:20.688205 2574 scope.go:117] "RemoveContainer" containerID="0b7c8629f548380ddc854dedc1218029128ceb86a0345708ad4b638242bc5a62" Aug 04 15:08:20.699093 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:20.698948 2574 scope.go:117] "RemoveContainer" containerID="0b7c8629f548380ddc854dedc1218029128ceb86a0345708ad4b638242bc5a62" Aug 04 15:08:20.699576 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:08:20.699550 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b7c8629f548380ddc854dedc1218029128ceb86a0345708ad4b638242bc5a62\": container with ID starting with 0b7c8629f548380ddc854dedc1218029128ceb86a0345708ad4b638242bc5a62 not found: ID does not exist" containerID="0b7c8629f548380ddc854dedc1218029128ceb86a0345708ad4b638242bc5a62" Aug 04 15:08:20.699666 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:20.699583 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b7c8629f548380ddc854dedc1218029128ceb86a0345708ad4b638242bc5a62"} err="failed to get container status \"0b7c8629f548380ddc854dedc1218029128ceb86a0345708ad4b638242bc5a62\": rpc error: code = NotFound desc = could not find container \"0b7c8629f548380ddc854dedc1218029128ceb86a0345708ad4b638242bc5a62\": container with ID starting with 0b7c8629f548380ddc854dedc1218029128ceb86a0345708ad4b638242bc5a62 not found: ID does not exist" Aug 04 15:08:20.716624 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:20.716599 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-bddc5f76d-42zvh"] Aug 04 15:08:20.724637 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:20.724616 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-bddc5f76d-42zvh"] Aug 04 15:08:21.052249 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:21.052214 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aab2968d-11b7-4174-b0c4-b3ce5a46b90e" path="/var/lib/kubelet/pods/aab2968d-11b7-4174-b0c4-b3ce5a46b90e/volumes" Aug 04 15:08:33.738980 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:33.738951 2574 generic.go:356] "Generic (PLEG): container finished" podID="ad87b28c-e7a3-4354-af24-4e0d4aa3f464" containerID="5bce6b8bae28745bdae6024b35671e687159c673f7bd0af50b016d78b58b8eb4" exitCode=0 Aug 04 15:08:33.739345 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:33.738992 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-7jlpj" event={"ID":"ad87b28c-e7a3-4354-af24-4e0d4aa3f464","Type":"ContainerDied","Data":"5bce6b8bae28745bdae6024b35671e687159c673f7bd0af50b016d78b58b8eb4"} Aug 04 15:08:33.739423 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:33.739370 2574 scope.go:117] "RemoveContainer" containerID="5bce6b8bae28745bdae6024b35671e687159c673f7bd0af50b016d78b58b8eb4" Aug 04 15:08:34.745759 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:34.745724 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-784f97bcf6-7jlpj" event={"ID":"ad87b28c-e7a3-4354-af24-4e0d4aa3f464","Type":"ContainerStarted","Data":"29174ddf53a41ad2303fff1a74268bb6465e1199ce1ead05566f5266a1d3a3cf"} Aug 04 15:08:34.747261 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:34.747228 2574 generic.go:356] "Generic (PLEG): container finished" podID="1a2285d4-f4ce-47be-80d4-7b4be7f930b1" containerID="cbda7288e1252f8b1f64163a1c4213cd41539f7bf878294fd99f80201e57ee78" exitCode=0 Aug 04 15:08:34.747363 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:34.747295 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-tqzqn" event={"ID":"1a2285d4-f4ce-47be-80d4-7b4be7f930b1","Type":"ContainerDied","Data":"cbda7288e1252f8b1f64163a1c4213cd41539f7bf878294fd99f80201e57ee78"} Aug 04 15:08:34.747582 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:34.747567 2574 scope.go:117] "RemoveContainer" containerID="cbda7288e1252f8b1f64163a1c4213cd41539f7bf878294fd99f80201e57ee78" Aug 04 15:08:35.752728 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:08:35.752699 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7878f4b88c-tqzqn" event={"ID":"1a2285d4-f4ce-47be-80d4-7b4be7f930b1","Type":"ContainerStarted","Data":"fc40aa5069092db0fc9eecba9983407b610e2444d4b190f127cbecad09e02c9b"} Aug 04 15:09:23.912330 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:09:23.912298 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c5555494f-7jz9k"] Aug 04 15:09:23.912733 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:09:23.912630 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aab2968d-11b7-4174-b0c4-b3ce5a46b90e" containerName="registry" Aug 04 15:09:23.912733 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:09:23.912647 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="aab2968d-11b7-4174-b0c4-b3ce5a46b90e" containerName="registry" Aug 04 15:09:23.912733 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:09:23.912665 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cef83f55-1138-47ab-9700-4e0879c39b71" containerName="console" Aug 04 15:09:23.912733 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:09:23.912671 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="cef83f55-1138-47ab-9700-4e0879c39b71" containerName="console" Aug 04 15:09:23.912733 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:09:23.912724 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="aab2968d-11b7-4174-b0c4-b3ce5a46b90e" containerName="registry" Aug 04 15:09:23.912733 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:09:23.912734 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="cef83f55-1138-47ab-9700-4e0879c39b71" containerName="console" Aug 04 15:09:23.915629 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:09:23.915607 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c5555494f-7jz9k" Aug 04 15:09:23.933647 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:09:23.933623 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c5555494f-7jz9k"] Aug 04 15:09:24.004609 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:09:24.004574 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/161e7bb8-9e46-4b79-89c5-6e02060e2830-trusted-ca-bundle\") pod \"console-5c5555494f-7jz9k\" (UID: \"161e7bb8-9e46-4b79-89c5-6e02060e2830\") " pod="openshift-console/console-5c5555494f-7jz9k" Aug 04 15:09:24.004609 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:09:24.004614 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/161e7bb8-9e46-4b79-89c5-6e02060e2830-console-oauth-config\") pod \"console-5c5555494f-7jz9k\" (UID: \"161e7bb8-9e46-4b79-89c5-6e02060e2830\") " pod="openshift-console/console-5c5555494f-7jz9k" Aug 04 15:09:24.004790 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:09:24.004634 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/161e7bb8-9e46-4b79-89c5-6e02060e2830-console-config\") pod \"console-5c5555494f-7jz9k\" (UID: \"161e7bb8-9e46-4b79-89c5-6e02060e2830\") " pod="openshift-console/console-5c5555494f-7jz9k" Aug 04 15:09:24.004790 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:09:24.004720 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/161e7bb8-9e46-4b79-89c5-6e02060e2830-console-serving-cert\") pod \"console-5c5555494f-7jz9k\" (UID: \"161e7bb8-9e46-4b79-89c5-6e02060e2830\") " pod="openshift-console/console-5c5555494f-7jz9k" Aug 04 15:09:24.004790 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:09:24.004772 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/161e7bb8-9e46-4b79-89c5-6e02060e2830-oauth-serving-cert\") pod \"console-5c5555494f-7jz9k\" (UID: \"161e7bb8-9e46-4b79-89c5-6e02060e2830\") " pod="openshift-console/console-5c5555494f-7jz9k" Aug 04 15:09:24.004885 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:09:24.004828 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ss56l\" (UniqueName: \"kubernetes.io/projected/161e7bb8-9e46-4b79-89c5-6e02060e2830-kube-api-access-ss56l\") pod \"console-5c5555494f-7jz9k\" (UID: \"161e7bb8-9e46-4b79-89c5-6e02060e2830\") " pod="openshift-console/console-5c5555494f-7jz9k" Aug 04 15:09:24.004885 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:09:24.004848 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/161e7bb8-9e46-4b79-89c5-6e02060e2830-service-ca\") pod \"console-5c5555494f-7jz9k\" (UID: \"161e7bb8-9e46-4b79-89c5-6e02060e2830\") " pod="openshift-console/console-5c5555494f-7jz9k" Aug 04 15:09:24.105878 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:09:24.105846 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/161e7bb8-9e46-4b79-89c5-6e02060e2830-console-config\") pod \"console-5c5555494f-7jz9k\" (UID: \"161e7bb8-9e46-4b79-89c5-6e02060e2830\") " pod="openshift-console/console-5c5555494f-7jz9k" Aug 04 15:09:24.106027 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:09:24.105888 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/161e7bb8-9e46-4b79-89c5-6e02060e2830-console-serving-cert\") pod \"console-5c5555494f-7jz9k\" (UID: \"161e7bb8-9e46-4b79-89c5-6e02060e2830\") " pod="openshift-console/console-5c5555494f-7jz9k" Aug 04 15:09:24.106027 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:09:24.105915 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/161e7bb8-9e46-4b79-89c5-6e02060e2830-oauth-serving-cert\") pod \"console-5c5555494f-7jz9k\" (UID: \"161e7bb8-9e46-4b79-89c5-6e02060e2830\") " pod="openshift-console/console-5c5555494f-7jz9k" Aug 04 15:09:24.106027 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:09:24.105952 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ss56l\" (UniqueName: \"kubernetes.io/projected/161e7bb8-9e46-4b79-89c5-6e02060e2830-kube-api-access-ss56l\") pod \"console-5c5555494f-7jz9k\" (UID: \"161e7bb8-9e46-4b79-89c5-6e02060e2830\") " pod="openshift-console/console-5c5555494f-7jz9k" Aug 04 15:09:24.106027 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:09:24.105970 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/161e7bb8-9e46-4b79-89c5-6e02060e2830-service-ca\") pod \"console-5c5555494f-7jz9k\" (UID: \"161e7bb8-9e46-4b79-89c5-6e02060e2830\") " pod="openshift-console/console-5c5555494f-7jz9k" Aug 04 15:09:24.106027 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:09:24.105987 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/161e7bb8-9e46-4b79-89c5-6e02060e2830-trusted-ca-bundle\") pod \"console-5c5555494f-7jz9k\" (UID: \"161e7bb8-9e46-4b79-89c5-6e02060e2830\") " pod="openshift-console/console-5c5555494f-7jz9k" Aug 04 15:09:24.106027 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:09:24.106014 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/161e7bb8-9e46-4b79-89c5-6e02060e2830-console-oauth-config\") pod \"console-5c5555494f-7jz9k\" (UID: \"161e7bb8-9e46-4b79-89c5-6e02060e2830\") " pod="openshift-console/console-5c5555494f-7jz9k" Aug 04 15:09:24.106742 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:09:24.106713 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/161e7bb8-9e46-4b79-89c5-6e02060e2830-service-ca\") pod \"console-5c5555494f-7jz9k\" (UID: \"161e7bb8-9e46-4b79-89c5-6e02060e2830\") " pod="openshift-console/console-5c5555494f-7jz9k" Aug 04 15:09:24.106862 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:09:24.106722 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/161e7bb8-9e46-4b79-89c5-6e02060e2830-console-config\") pod \"console-5c5555494f-7jz9k\" (UID: \"161e7bb8-9e46-4b79-89c5-6e02060e2830\") " pod="openshift-console/console-5c5555494f-7jz9k" Aug 04 15:09:24.106862 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:09:24.106827 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/161e7bb8-9e46-4b79-89c5-6e02060e2830-oauth-serving-cert\") pod \"console-5c5555494f-7jz9k\" (UID: \"161e7bb8-9e46-4b79-89c5-6e02060e2830\") " pod="openshift-console/console-5c5555494f-7jz9k" Aug 04 15:09:24.106862 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:09:24.106841 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/161e7bb8-9e46-4b79-89c5-6e02060e2830-trusted-ca-bundle\") pod \"console-5c5555494f-7jz9k\" (UID: \"161e7bb8-9e46-4b79-89c5-6e02060e2830\") " pod="openshift-console/console-5c5555494f-7jz9k" Aug 04 15:09:24.108355 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:09:24.108339 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/161e7bb8-9e46-4b79-89c5-6e02060e2830-console-serving-cert\") pod \"console-5c5555494f-7jz9k\" (UID: \"161e7bb8-9e46-4b79-89c5-6e02060e2830\") " pod="openshift-console/console-5c5555494f-7jz9k" Aug 04 15:09:24.108503 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:09:24.108486 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/161e7bb8-9e46-4b79-89c5-6e02060e2830-console-oauth-config\") pod \"console-5c5555494f-7jz9k\" (UID: \"161e7bb8-9e46-4b79-89c5-6e02060e2830\") " pod="openshift-console/console-5c5555494f-7jz9k" Aug 04 15:09:24.116936 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:09:24.116914 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss56l\" (UniqueName: \"kubernetes.io/projected/161e7bb8-9e46-4b79-89c5-6e02060e2830-kube-api-access-ss56l\") pod \"console-5c5555494f-7jz9k\" (UID: \"161e7bb8-9e46-4b79-89c5-6e02060e2830\") " pod="openshift-console/console-5c5555494f-7jz9k" Aug 04 15:09:24.224142 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:09:24.224070 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c5555494f-7jz9k" Aug 04 15:09:24.362934 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:09:24.362898 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c5555494f-7jz9k"] Aug 04 15:09:24.365625 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:09:24.365595 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod161e7bb8_9e46_4b79_89c5_6e02060e2830.slice/crio-289ef1f76590f6cfcc7dc846bf6b08eaf11a5d71d3a6ff8bc3a9500d1077b463 WatchSource:0}: Error finding container 289ef1f76590f6cfcc7dc846bf6b08eaf11a5d71d3a6ff8bc3a9500d1077b463: Status 404 returned error can't find the container with id 289ef1f76590f6cfcc7dc846bf6b08eaf11a5d71d3a6ff8bc3a9500d1077b463 Aug 04 15:09:24.931102 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:09:24.931065 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c5555494f-7jz9k" event={"ID":"161e7bb8-9e46-4b79-89c5-6e02060e2830","Type":"ContainerStarted","Data":"bdb17e7900372b1c3a59aa8f7d3575108f21f3961cd211ff4a284de740e1d01b"} Aug 04 15:09:24.931508 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:09:24.931112 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c5555494f-7jz9k" event={"ID":"161e7bb8-9e46-4b79-89c5-6e02060e2830","Type":"ContainerStarted","Data":"289ef1f76590f6cfcc7dc846bf6b08eaf11a5d71d3a6ff8bc3a9500d1077b463"} Aug 04 15:09:24.969602 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:09:24.969555 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5c5555494f-7jz9k" podStartSLOduration=1.969540045 podStartE2EDuration="1.969540045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:09:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:09:24.968796246 +0000 UTC m=+188.428668594" watchObservedRunningTime="2026-08-04 15:09:24.969540045 +0000 UTC m=+188.429412394" Aug 04 15:09:34.225102 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:09:34.225059 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5c5555494f-7jz9k" Aug 04 15:09:34.225591 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:09:34.225130 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c5555494f-7jz9k" Aug 04 15:09:34.230608 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:09:34.230580 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5c5555494f-7jz9k" Aug 04 15:09:34.970631 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:09:34.970600 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5c5555494f-7jz9k" Aug 04 15:09:35.022365 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:09:35.022334 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cbcc8d8c9-7xwxn"] Aug 04 15:10:00.043646 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:00.043599 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5cbcc8d8c9-7xwxn" podUID="9621c718-c201-42e2-abbd-62076fefb25e" containerName="console" containerID="cri-o://db13d3ae6a10a19d47622163401af95e7231736c9719a0ae3cd2a00732be9831" gracePeriod=15 Aug 04 15:10:00.285560 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:00.285534 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cbcc8d8c9-7xwxn" Aug 04 15:10:00.394968 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:00.394883 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9621c718-c201-42e2-abbd-62076fefb25e-console-serving-cert\") pod \"9621c718-c201-42e2-abbd-62076fefb25e\" (UID: \"9621c718-c201-42e2-abbd-62076fefb25e\") " Aug 04 15:10:00.394968 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:00.394943 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h4pk8\" (UniqueName: \"kubernetes.io/projected/9621c718-c201-42e2-abbd-62076fefb25e-kube-api-access-h4pk8\") pod \"9621c718-c201-42e2-abbd-62076fefb25e\" (UID: \"9621c718-c201-42e2-abbd-62076fefb25e\") " Aug 04 15:10:00.395166 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:00.394975 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9621c718-c201-42e2-abbd-62076fefb25e-oauth-serving-cert\") pod \"9621c718-c201-42e2-abbd-62076fefb25e\" (UID: \"9621c718-c201-42e2-abbd-62076fefb25e\") " Aug 04 15:10:00.395166 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:00.395018 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9621c718-c201-42e2-abbd-62076fefb25e-console-oauth-config\") pod \"9621c718-c201-42e2-abbd-62076fefb25e\" (UID: \"9621c718-c201-42e2-abbd-62076fefb25e\") " Aug 04 15:10:00.395166 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:00.395042 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9621c718-c201-42e2-abbd-62076fefb25e-trusted-ca-bundle\") pod \"9621c718-c201-42e2-abbd-62076fefb25e\" (UID: \"9621c718-c201-42e2-abbd-62076fefb25e\") " Aug 04 15:10:00.395166 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:00.395089 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9621c718-c201-42e2-abbd-62076fefb25e-console-config\") pod \"9621c718-c201-42e2-abbd-62076fefb25e\" (UID: \"9621c718-c201-42e2-abbd-62076fefb25e\") " Aug 04 15:10:00.395365 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:00.395219 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9621c718-c201-42e2-abbd-62076fefb25e-service-ca\") pod \"9621c718-c201-42e2-abbd-62076fefb25e\" (UID: \"9621c718-c201-42e2-abbd-62076fefb25e\") " Aug 04 15:10:00.395467 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:00.395436 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9621c718-c201-42e2-abbd-62076fefb25e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9621c718-c201-42e2-abbd-62076fefb25e" (UID: "9621c718-c201-42e2-abbd-62076fefb25e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 15:10:00.395544 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:00.395500 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9621c718-c201-42e2-abbd-62076fefb25e-console-config" (OuterVolumeSpecName: "console-config") pod "9621c718-c201-42e2-abbd-62076fefb25e" (UID: "9621c718-c201-42e2-abbd-62076fefb25e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 15:10:00.395544 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:00.395509 2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9621c718-c201-42e2-abbd-62076fefb25e-oauth-serving-cert\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:10:00.395544 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:00.395515 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9621c718-c201-42e2-abbd-62076fefb25e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "9621c718-c201-42e2-abbd-62076fefb25e" (UID: "9621c718-c201-42e2-abbd-62076fefb25e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 15:10:00.395765 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:00.395748 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9621c718-c201-42e2-abbd-62076fefb25e-service-ca" (OuterVolumeSpecName: "service-ca") pod "9621c718-c201-42e2-abbd-62076fefb25e" (UID: "9621c718-c201-42e2-abbd-62076fefb25e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 15:10:00.397116 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:00.397094 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9621c718-c201-42e2-abbd-62076fefb25e-kube-api-access-h4pk8" (OuterVolumeSpecName: "kube-api-access-h4pk8") pod "9621c718-c201-42e2-abbd-62076fefb25e" (UID: "9621c718-c201-42e2-abbd-62076fefb25e"). InnerVolumeSpecName "kube-api-access-h4pk8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 15:10:00.397544 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:00.397519 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9621c718-c201-42e2-abbd-62076fefb25e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9621c718-c201-42e2-abbd-62076fefb25e" (UID: "9621c718-c201-42e2-abbd-62076fefb25e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 15:10:00.397544 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:00.397535 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9621c718-c201-42e2-abbd-62076fefb25e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9621c718-c201-42e2-abbd-62076fefb25e" (UID: "9621c718-c201-42e2-abbd-62076fefb25e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 15:10:00.496181 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:00.496136 2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9621c718-c201-42e2-abbd-62076fefb25e-console-oauth-config\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:10:00.496181 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:00.496175 2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9621c718-c201-42e2-abbd-62076fefb25e-trusted-ca-bundle\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:10:00.496181 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:00.496186 2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9621c718-c201-42e2-abbd-62076fefb25e-console-config\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:10:00.496181 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:00.496196 2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9621c718-c201-42e2-abbd-62076fefb25e-service-ca\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:10:00.496417 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:00.496205 2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9621c718-c201-42e2-abbd-62076fefb25e-console-serving-cert\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:10:00.496417 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:00.496213 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h4pk8\" (UniqueName: \"kubernetes.io/projected/9621c718-c201-42e2-abbd-62076fefb25e-kube-api-access-h4pk8\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:10:01.061768 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:01.061741 2574 generic.go:356] "Generic (PLEG): container finished" podID="9621c718-c201-42e2-abbd-62076fefb25e" containerID="db13d3ae6a10a19d47622163401af95e7231736c9719a0ae3cd2a00732be9831" exitCode=0 Aug 04 15:10:01.062048 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:01.061786 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cbcc8d8c9-7xwxn" Aug 04 15:10:01.062048 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:01.061787 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cbcc8d8c9-7xwxn" event={"ID":"9621c718-c201-42e2-abbd-62076fefb25e","Type":"ContainerDied","Data":"db13d3ae6a10a19d47622163401af95e7231736c9719a0ae3cd2a00732be9831"} Aug 04 15:10:01.062048 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:01.061915 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cbcc8d8c9-7xwxn" event={"ID":"9621c718-c201-42e2-abbd-62076fefb25e","Type":"ContainerDied","Data":"f8a3dde9d5ee5e315aacd0bc3ef3c228006574d553d1605e99d0abcd42933d5e"} Aug 04 15:10:01.062048 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:01.061944 2574 scope.go:117] "RemoveContainer" containerID="db13d3ae6a10a19d47622163401af95e7231736c9719a0ae3cd2a00732be9831" Aug 04 15:10:01.070419 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:01.070398 2574 scope.go:117] "RemoveContainer" containerID="db13d3ae6a10a19d47622163401af95e7231736c9719a0ae3cd2a00732be9831" Aug 04 15:10:01.070668 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:10:01.070653 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db13d3ae6a10a19d47622163401af95e7231736c9719a0ae3cd2a00732be9831\": container with ID starting with db13d3ae6a10a19d47622163401af95e7231736c9719a0ae3cd2a00732be9831 not found: ID does not exist" containerID="db13d3ae6a10a19d47622163401af95e7231736c9719a0ae3cd2a00732be9831" Aug 04 15:10:01.070716 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:01.070675 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db13d3ae6a10a19d47622163401af95e7231736c9719a0ae3cd2a00732be9831"} err="failed to get container status \"db13d3ae6a10a19d47622163401af95e7231736c9719a0ae3cd2a00732be9831\": rpc error: code = NotFound desc = could not find container \"db13d3ae6a10a19d47622163401af95e7231736c9719a0ae3cd2a00732be9831\": container with ID starting with db13d3ae6a10a19d47622163401af95e7231736c9719a0ae3cd2a00732be9831 not found: ID does not exist" Aug 04 15:10:01.079681 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:01.079656 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cbcc8d8c9-7xwxn"] Aug 04 15:10:01.083697 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:01.083676 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5cbcc8d8c9-7xwxn"] Aug 04 15:10:03.052126 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:03.052080 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9621c718-c201-42e2-abbd-62076fefb25e" path="/var/lib/kubelet/pods/9621c718-c201-42e2-abbd-62076fefb25e/volumes" Aug 04 15:10:53.450601 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:53.450518 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7zds"] Aug 04 15:10:53.451040 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:53.450796 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9621c718-c201-42e2-abbd-62076fefb25e" containerName="console" Aug 04 15:10:53.451040 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:53.450807 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9621c718-c201-42e2-abbd-62076fefb25e" containerName="console" Aug 04 15:10:53.451040 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:53.450857 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="9621c718-c201-42e2-abbd-62076fefb25e" containerName="console" Aug 04 15:10:53.453770 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:53.453754 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7zds" Aug 04 15:10:53.457052 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:53.457028 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Aug 04 15:10:53.457052 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:53.457039 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Aug 04 15:10:53.457638 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:53.457621 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ghcjn\"" Aug 04 15:10:53.465816 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:53.465789 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7zds"] Aug 04 15:10:53.503949 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:53.503913 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d99c337-ba7a-4fc0-815d-4869d637dbbd-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7zds\" (UID: \"9d99c337-ba7a-4fc0-815d-4869d637dbbd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7zds" Aug 04 15:10:53.503949 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:53.503951 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ssxn\" (UniqueName: \"kubernetes.io/projected/9d99c337-ba7a-4fc0-815d-4869d637dbbd-kube-api-access-8ssxn\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7zds\" (UID: \"9d99c337-ba7a-4fc0-815d-4869d637dbbd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7zds" Aug 04 15:10:53.504146 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:53.504030 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d99c337-ba7a-4fc0-815d-4869d637dbbd-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7zds\" (UID: \"9d99c337-ba7a-4fc0-815d-4869d637dbbd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7zds" Aug 04 15:10:53.604931 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:53.604897 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d99c337-ba7a-4fc0-815d-4869d637dbbd-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7zds\" (UID: \"9d99c337-ba7a-4fc0-815d-4869d637dbbd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7zds" Aug 04 15:10:53.605062 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:53.604939 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ssxn\" (UniqueName: \"kubernetes.io/projected/9d99c337-ba7a-4fc0-815d-4869d637dbbd-kube-api-access-8ssxn\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7zds\" (UID: \"9d99c337-ba7a-4fc0-815d-4869d637dbbd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7zds" Aug 04 15:10:53.605062 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:53.604992 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d99c337-ba7a-4fc0-815d-4869d637dbbd-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7zds\" (UID: \"9d99c337-ba7a-4fc0-815d-4869d637dbbd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7zds" Aug 04 15:10:53.605267 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:53.605249 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d99c337-ba7a-4fc0-815d-4869d637dbbd-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7zds\" (UID: \"9d99c337-ba7a-4fc0-815d-4869d637dbbd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7zds" Aug 04 15:10:53.605332 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:53.605313 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d99c337-ba7a-4fc0-815d-4869d637dbbd-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7zds\" (UID: \"9d99c337-ba7a-4fc0-815d-4869d637dbbd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7zds" Aug 04 15:10:53.614804 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:53.614774 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ssxn\" (UniqueName: \"kubernetes.io/projected/9d99c337-ba7a-4fc0-815d-4869d637dbbd-kube-api-access-8ssxn\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7zds\" (UID: \"9d99c337-ba7a-4fc0-815d-4869d637dbbd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7zds" Aug 04 15:10:53.762581 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:53.762549 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7zds" Aug 04 15:10:53.897462 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:53.897415 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7zds"] Aug 04 15:10:53.899817 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:10:53.899792 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d99c337_ba7a_4fc0_815d_4869d637dbbd.slice/crio-45e094c60b785d015e034441d10050b95e16a8b131baa99a979a76f0fe36a5e8 WatchSource:0}: Error finding container 45e094c60b785d015e034441d10050b95e16a8b131baa99a979a76f0fe36a5e8: Status 404 returned error can't find the container with id 45e094c60b785d015e034441d10050b95e16a8b131baa99a979a76f0fe36a5e8 Aug 04 15:10:54.250704 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:10:54.250667 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7zds" event={"ID":"9d99c337-ba7a-4fc0-815d-4869d637dbbd","Type":"ContainerStarted","Data":"45e094c60b785d015e034441d10050b95e16a8b131baa99a979a76f0fe36a5e8"} Aug 04 15:11:01.280736 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:01.280700 2574 generic.go:356] "Generic (PLEG): container finished" podID="9d99c337-ba7a-4fc0-815d-4869d637dbbd" containerID="23b6735eecae0c0729c850344c34d754aafb5333bda30c4aa33dfaafd945bf75" exitCode=0 Aug 04 15:11:01.281119 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:01.280787 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7zds" event={"ID":"9d99c337-ba7a-4fc0-815d-4869d637dbbd","Type":"ContainerDied","Data":"23b6735eecae0c0729c850344c34d754aafb5333bda30c4aa33dfaafd945bf75"} Aug 04 15:11:04.294134 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:04.294098 2574 generic.go:356] "Generic (PLEG): container finished" podID="9d99c337-ba7a-4fc0-815d-4869d637dbbd" containerID="05a03ac148119c43594f3100e5957b0a0d9542cbfc45751f6ff70629434f6ec6" exitCode=0 Aug 04 15:11:04.294524 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:04.294181 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7zds" event={"ID":"9d99c337-ba7a-4fc0-815d-4869d637dbbd","Type":"ContainerDied","Data":"05a03ac148119c43594f3100e5957b0a0d9542cbfc45751f6ff70629434f6ec6"} Aug 04 15:11:11.329458 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:11.329420 2574 generic.go:356] "Generic (PLEG): container finished" podID="9d99c337-ba7a-4fc0-815d-4869d637dbbd" containerID="2c30c06237e8a0ec7bbe451dd0914e5c0e26c240a744d4ddcef95d00c424927b" exitCode=0 Aug 04 15:11:11.329828 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:11.329503 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7zds" event={"ID":"9d99c337-ba7a-4fc0-815d-4869d637dbbd","Type":"ContainerDied","Data":"2c30c06237e8a0ec7bbe451dd0914e5c0e26c240a744d4ddcef95d00c424927b"} Aug 04 15:11:12.463161 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:12.463139 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7zds" Aug 04 15:11:12.563317 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:12.563287 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8ssxn\" (UniqueName: \"kubernetes.io/projected/9d99c337-ba7a-4fc0-815d-4869d637dbbd-kube-api-access-8ssxn\") pod \"9d99c337-ba7a-4fc0-815d-4869d637dbbd\" (UID: \"9d99c337-ba7a-4fc0-815d-4869d637dbbd\") " Aug 04 15:11:12.563495 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:12.563338 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d99c337-ba7a-4fc0-815d-4869d637dbbd-util\") pod \"9d99c337-ba7a-4fc0-815d-4869d637dbbd\" (UID: \"9d99c337-ba7a-4fc0-815d-4869d637dbbd\") " Aug 04 15:11:12.563495 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:12.563359 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d99c337-ba7a-4fc0-815d-4869d637dbbd-bundle\") pod \"9d99c337-ba7a-4fc0-815d-4869d637dbbd\" (UID: \"9d99c337-ba7a-4fc0-815d-4869d637dbbd\") " Aug 04 15:11:12.564055 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:12.564027 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d99c337-ba7a-4fc0-815d-4869d637dbbd-bundle" (OuterVolumeSpecName: "bundle") pod "9d99c337-ba7a-4fc0-815d-4869d637dbbd" (UID: "9d99c337-ba7a-4fc0-815d-4869d637dbbd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:11:12.565365 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:12.565344 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d99c337-ba7a-4fc0-815d-4869d637dbbd-kube-api-access-8ssxn" (OuterVolumeSpecName: "kube-api-access-8ssxn") pod "9d99c337-ba7a-4fc0-815d-4869d637dbbd" (UID: "9d99c337-ba7a-4fc0-815d-4869d637dbbd"). InnerVolumeSpecName "kube-api-access-8ssxn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 15:11:12.567666 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:12.567643 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d99c337-ba7a-4fc0-815d-4869d637dbbd-util" (OuterVolumeSpecName: "util") pod "9d99c337-ba7a-4fc0-815d-4869d637dbbd" (UID: "9d99c337-ba7a-4fc0-815d-4869d637dbbd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:11:12.664430 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:12.664332 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d99c337-ba7a-4fc0-815d-4869d637dbbd-util\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:11:12.664430 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:12.664360 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d99c337-ba7a-4fc0-815d-4869d637dbbd-bundle\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:11:12.664430 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:12.664370 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ssxn\" (UniqueName: \"kubernetes.io/projected/9d99c337-ba7a-4fc0-815d-4869d637dbbd-kube-api-access-8ssxn\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:11:13.339338 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:13.339258 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7zds" Aug 04 15:11:13.339338 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:13.339275 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540c7zds" event={"ID":"9d99c337-ba7a-4fc0-815d-4869d637dbbd","Type":"ContainerDied","Data":"45e094c60b785d015e034441d10050b95e16a8b131baa99a979a76f0fe36a5e8"} Aug 04 15:11:13.339338 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:13.339315 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="45e094c60b785d015e034441d10050b95e16a8b131baa99a979a76f0fe36a5e8" Aug 04 15:11:16.951661 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:16.951631 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-r8wkp_0580fdab-0381-4d8e-be5f-bde5eb44e0e6/console-operator/1.log" Aug 04 15:11:16.953166 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:16.953141 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-r8wkp_0580fdab-0381-4d8e-be5f-bde5eb44e0e6/console-operator/1.log" Aug 04 15:11:16.956011 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:16.955973 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8h6k_9c62e3b1-c2dc-4b2a-a61d-7d80049f355f/ovn-acl-logging/0.log" Aug 04 15:11:16.957733 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:16.957714 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8h6k_9c62e3b1-c2dc-4b2a-a61d-7d80049f355f/ovn-acl-logging/0.log" Aug 04 15:11:16.962586 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:16.962569 2574 kubelet.go:1648] "Image garbage collection succeeded" Aug 04 15:11:21.096261 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:21.096225 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rg8t"] Aug 04 15:11:21.118356 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:21.096536 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d99c337-ba7a-4fc0-815d-4869d637dbbd" containerName="util" Aug 04 15:11:21.118356 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:21.096548 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d99c337-ba7a-4fc0-815d-4869d637dbbd" containerName="util" Aug 04 15:11:21.118356 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:21.096559 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d99c337-ba7a-4fc0-815d-4869d637dbbd" containerName="extract" Aug 04 15:11:21.118356 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:21.096564 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d99c337-ba7a-4fc0-815d-4869d637dbbd" containerName="extract" Aug 04 15:11:21.118356 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:21.096577 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d99c337-ba7a-4fc0-815d-4869d637dbbd" containerName="pull" Aug 04 15:11:21.118356 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:21.096583 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d99c337-ba7a-4fc0-815d-4869d637dbbd" containerName="pull" Aug 04 15:11:21.118356 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:21.096634 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d99c337-ba7a-4fc0-815d-4869d637dbbd" containerName="extract" Aug 04 15:11:21.118673 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:21.118371 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rg8t" Aug 04 15:11:21.121569 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:21.121548 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ghcjn\"" Aug 04 15:11:21.121672 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:21.121613 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Aug 04 15:11:21.121729 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:21.121697 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Aug 04 15:11:21.129235 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:21.129214 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rg8t"] Aug 04 15:11:21.230158 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:21.230125 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/af4c9ccd-4a78-4c73-bb09-9ecfc80583c8-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rg8t\" (UID: \"af4c9ccd-4a78-4c73-bb09-9ecfc80583c8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rg8t" Aug 04 15:11:21.230316 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:21.230203 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdz7n\" (UniqueName: \"kubernetes.io/projected/af4c9ccd-4a78-4c73-bb09-9ecfc80583c8-kube-api-access-qdz7n\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rg8t\" (UID: \"af4c9ccd-4a78-4c73-bb09-9ecfc80583c8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rg8t" Aug 04 15:11:21.230316 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:21.230234 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/af4c9ccd-4a78-4c73-bb09-9ecfc80583c8-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rg8t\" (UID: \"af4c9ccd-4a78-4c73-bb09-9ecfc80583c8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rg8t" Aug 04 15:11:21.330804 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:21.330772 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdz7n\" (UniqueName: \"kubernetes.io/projected/af4c9ccd-4a78-4c73-bb09-9ecfc80583c8-kube-api-access-qdz7n\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rg8t\" (UID: \"af4c9ccd-4a78-4c73-bb09-9ecfc80583c8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rg8t" Aug 04 15:11:21.330939 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:21.330811 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/af4c9ccd-4a78-4c73-bb09-9ecfc80583c8-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rg8t\" (UID: \"af4c9ccd-4a78-4c73-bb09-9ecfc80583c8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rg8t" Aug 04 15:11:21.330939 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:21.330846 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/af4c9ccd-4a78-4c73-bb09-9ecfc80583c8-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rg8t\" (UID: \"af4c9ccd-4a78-4c73-bb09-9ecfc80583c8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rg8t" Aug 04 15:11:21.331237 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:21.331219 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/af4c9ccd-4a78-4c73-bb09-9ecfc80583c8-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rg8t\" (UID: \"af4c9ccd-4a78-4c73-bb09-9ecfc80583c8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rg8t" Aug 04 15:11:21.331272 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:21.331248 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/af4c9ccd-4a78-4c73-bb09-9ecfc80583c8-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rg8t\" (UID: \"af4c9ccd-4a78-4c73-bb09-9ecfc80583c8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rg8t" Aug 04 15:11:21.347049 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:21.347003 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdz7n\" (UniqueName: \"kubernetes.io/projected/af4c9ccd-4a78-4c73-bb09-9ecfc80583c8-kube-api-access-qdz7n\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rg8t\" (UID: \"af4c9ccd-4a78-4c73-bb09-9ecfc80583c8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rg8t" Aug 04 15:11:21.427676 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:21.427650 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rg8t" Aug 04 15:11:21.609160 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:21.609134 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rg8t"] Aug 04 15:11:21.611670 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:11:21.611642 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf4c9ccd_4a78_4c73_bb09_9ecfc80583c8.slice/crio-7fa08cd9ec2656e801328337de123c77881933aaba5023f396cd40fa4866d593 WatchSource:0}: Error finding container 7fa08cd9ec2656e801328337de123c77881933aaba5023f396cd40fa4866d593: Status 404 returned error can't find the container with id 7fa08cd9ec2656e801328337de123c77881933aaba5023f396cd40fa4866d593 Aug 04 15:11:22.374010 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:22.373973 2574 generic.go:356] "Generic (PLEG): container finished" podID="af4c9ccd-4a78-4c73-bb09-9ecfc80583c8" containerID="b573c7aa488f71ddb5696415a689eedd6320358c0c2f651228337b75cbdf39bc" exitCode=0 Aug 04 15:11:22.374385 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:22.374018 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rg8t" event={"ID":"af4c9ccd-4a78-4c73-bb09-9ecfc80583c8","Type":"ContainerDied","Data":"b573c7aa488f71ddb5696415a689eedd6320358c0c2f651228337b75cbdf39bc"} Aug 04 15:11:22.374385 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:22.374069 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rg8t" event={"ID":"af4c9ccd-4a78-4c73-bb09-9ecfc80583c8","Type":"ContainerStarted","Data":"7fa08cd9ec2656e801328337de123c77881933aaba5023f396cd40fa4866d593"} Aug 04 15:11:22.375076 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:22.375058 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 15:11:25.391350 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:25.391313 2574 generic.go:356] "Generic (PLEG): container finished" podID="af4c9ccd-4a78-4c73-bb09-9ecfc80583c8" containerID="1e9925202ae27f2a4ed82d3e25ea9725e316ff0cf8b49fb1c68d48558cfa4ef9" exitCode=0 Aug 04 15:11:25.391807 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:25.391411 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rg8t" event={"ID":"af4c9ccd-4a78-4c73-bb09-9ecfc80583c8","Type":"ContainerDied","Data":"1e9925202ae27f2a4ed82d3e25ea9725e316ff0cf8b49fb1c68d48558cfa4ef9"} Aug 04 15:11:26.402086 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:26.402051 2574 generic.go:356] "Generic (PLEG): container finished" podID="af4c9ccd-4a78-4c73-bb09-9ecfc80583c8" containerID="04dfa9c6f8298f84ea5f3bf0aff2aae7c28263470ddee432bcc0e8156cccdd28" exitCode=0 Aug 04 15:11:26.402445 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:26.402132 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rg8t" event={"ID":"af4c9ccd-4a78-4c73-bb09-9ecfc80583c8","Type":"ContainerDied","Data":"04dfa9c6f8298f84ea5f3bf0aff2aae7c28263470ddee432bcc0e8156cccdd28"} Aug 04 15:11:27.526032 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:27.526002 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rg8t" Aug 04 15:11:27.688034 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:27.687945 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qdz7n\" (UniqueName: \"kubernetes.io/projected/af4c9ccd-4a78-4c73-bb09-9ecfc80583c8-kube-api-access-qdz7n\") pod \"af4c9ccd-4a78-4c73-bb09-9ecfc80583c8\" (UID: \"af4c9ccd-4a78-4c73-bb09-9ecfc80583c8\") " Aug 04 15:11:27.688034 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:27.688019 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/af4c9ccd-4a78-4c73-bb09-9ecfc80583c8-util\") pod \"af4c9ccd-4a78-4c73-bb09-9ecfc80583c8\" (UID: \"af4c9ccd-4a78-4c73-bb09-9ecfc80583c8\") " Aug 04 15:11:27.688247 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:27.688120 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/af4c9ccd-4a78-4c73-bb09-9ecfc80583c8-bundle\") pod \"af4c9ccd-4a78-4c73-bb09-9ecfc80583c8\" (UID: \"af4c9ccd-4a78-4c73-bb09-9ecfc80583c8\") " Aug 04 15:11:27.688447 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:27.688427 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af4c9ccd-4a78-4c73-bb09-9ecfc80583c8-bundle" (OuterVolumeSpecName: "bundle") pod "af4c9ccd-4a78-4c73-bb09-9ecfc80583c8" (UID: "af4c9ccd-4a78-4c73-bb09-9ecfc80583c8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:11:27.690035 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:27.690002 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af4c9ccd-4a78-4c73-bb09-9ecfc80583c8-kube-api-access-qdz7n" (OuterVolumeSpecName: "kube-api-access-qdz7n") pod "af4c9ccd-4a78-4c73-bb09-9ecfc80583c8" (UID: "af4c9ccd-4a78-4c73-bb09-9ecfc80583c8"). InnerVolumeSpecName "kube-api-access-qdz7n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 15:11:27.692840 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:27.692800 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af4c9ccd-4a78-4c73-bb09-9ecfc80583c8-util" (OuterVolumeSpecName: "util") pod "af4c9ccd-4a78-4c73-bb09-9ecfc80583c8" (UID: "af4c9ccd-4a78-4c73-bb09-9ecfc80583c8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:11:27.789129 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:27.789103 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdz7n\" (UniqueName: \"kubernetes.io/projected/af4c9ccd-4a78-4c73-bb09-9ecfc80583c8-kube-api-access-qdz7n\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:11:27.789129 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:27.789130 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/af4c9ccd-4a78-4c73-bb09-9ecfc80583c8-util\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:11:27.789129 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:27.789141 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/af4c9ccd-4a78-4c73-bb09-9ecfc80583c8-bundle\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:11:28.220359 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:28.220324 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-qw8ds"] Aug 04 15:11:28.220609 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:28.220596 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af4c9ccd-4a78-4c73-bb09-9ecfc80583c8" containerName="pull" Aug 04 15:11:28.220652 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:28.220610 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="af4c9ccd-4a78-4c73-bb09-9ecfc80583c8" containerName="pull" Aug 04 15:11:28.220652 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:28.220625 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af4c9ccd-4a78-4c73-bb09-9ecfc80583c8" containerName="util" Aug 04 15:11:28.220652 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:28.220631 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="af4c9ccd-4a78-4c73-bb09-9ecfc80583c8" containerName="util" Aug 04 15:11:28.220652 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:28.220643 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af4c9ccd-4a78-4c73-bb09-9ecfc80583c8" containerName="extract" Aug 04 15:11:28.220652 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:28.220648 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="af4c9ccd-4a78-4c73-bb09-9ecfc80583c8" containerName="extract" Aug 04 15:11:28.220810 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:28.220697 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="af4c9ccd-4a78-4c73-bb09-9ecfc80583c8" containerName="extract" Aug 04 15:11:28.222481 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:28.222465 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-qw8ds" Aug 04 15:11:28.225059 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:28.225037 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Aug 04 15:11:28.226286 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:28.226268 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-r4rfv\"" Aug 04 15:11:28.226349 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:28.226278 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Aug 04 15:11:28.236000 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:28.235972 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-qw8ds"] Aug 04 15:11:28.293431 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:28.293401 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3acf82b6-7019-4c81-a0e9-5be305990d16-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-qw8ds\" (UID: \"3acf82b6-7019-4c81-a0e9-5be305990d16\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qw8ds" Aug 04 15:11:28.293559 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:28.293451 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q79tb\" (UniqueName: \"kubernetes.io/projected/3acf82b6-7019-4c81-a0e9-5be305990d16-kube-api-access-q79tb\") pod \"cert-manager-webhook-6486ffd997-qw8ds\" (UID: \"3acf82b6-7019-4c81-a0e9-5be305990d16\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qw8ds" Aug 04 15:11:28.394384 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:28.394340 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3acf82b6-7019-4c81-a0e9-5be305990d16-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-qw8ds\" (UID: \"3acf82b6-7019-4c81-a0e9-5be305990d16\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qw8ds" Aug 04 15:11:28.394521 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:28.394409 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q79tb\" (UniqueName: \"kubernetes.io/projected/3acf82b6-7019-4c81-a0e9-5be305990d16-kube-api-access-q79tb\") pod \"cert-manager-webhook-6486ffd997-qw8ds\" (UID: \"3acf82b6-7019-4c81-a0e9-5be305990d16\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qw8ds" Aug 04 15:11:28.403809 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:28.403782 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3acf82b6-7019-4c81-a0e9-5be305990d16-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-qw8ds\" (UID: \"3acf82b6-7019-4c81-a0e9-5be305990d16\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qw8ds" Aug 04 15:11:28.404047 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:28.404022 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q79tb\" (UniqueName: \"kubernetes.io/projected/3acf82b6-7019-4c81-a0e9-5be305990d16-kube-api-access-q79tb\") pod \"cert-manager-webhook-6486ffd997-qw8ds\" (UID: \"3acf82b6-7019-4c81-a0e9-5be305990d16\") " pod="cert-manager/cert-manager-webhook-6486ffd997-qw8ds" Aug 04 15:11:28.412314 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:28.412296 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rg8t" Aug 04 15:11:28.412412 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:28.412321 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6rg8t" event={"ID":"af4c9ccd-4a78-4c73-bb09-9ecfc80583c8","Type":"ContainerDied","Data":"7fa08cd9ec2656e801328337de123c77881933aaba5023f396cd40fa4866d593"} Aug 04 15:11:28.412412 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:28.412363 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7fa08cd9ec2656e801328337de123c77881933aaba5023f396cd40fa4866d593" Aug 04 15:11:28.530667 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:28.530632 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-qw8ds" Aug 04 15:11:28.656245 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:28.656200 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-qw8ds"] Aug 04 15:11:28.658325 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:11:28.658299 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3acf82b6_7019_4c81_a0e9_5be305990d16.slice/crio-a65a04c6e54ab150eacf4e6bcdff9e6cf6f43b7895124b37b30526c00ece6ea7 WatchSource:0}: Error finding container a65a04c6e54ab150eacf4e6bcdff9e6cf6f43b7895124b37b30526c00ece6ea7: Status 404 returned error can't find the container with id a65a04c6e54ab150eacf4e6bcdff9e6cf6f43b7895124b37b30526c00ece6ea7 Aug 04 15:11:29.416691 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:29.416656 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-qw8ds" event={"ID":"3acf82b6-7019-4c81-a0e9-5be305990d16","Type":"ContainerStarted","Data":"a65a04c6e54ab150eacf4e6bcdff9e6cf6f43b7895124b37b30526c00ece6ea7"} Aug 04 15:11:32.430703 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:32.430666 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-qw8ds" event={"ID":"3acf82b6-7019-4c81-a0e9-5be305990d16","Type":"ContainerStarted","Data":"16fcaa71a5f6f313ca157cca5b3ee41507de75daf4e694a485307c792439b13f"} Aug 04 15:11:32.431076 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:32.430716 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-qw8ds" Aug 04 15:11:32.451724 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:32.451682 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-qw8ds" podStartSLOduration=1.66806799 podStartE2EDuration="4.451668653s" totalImagesPullingTime="2.783600663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:11:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:11:32.451156901 +0000 UTC m=+315.911029251" watchObservedRunningTime="2026-08-04 15:11:32.451668653 +0000 UTC m=+315.911541001" Aug 04 15:11:38.436733 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:38.436700 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-qw8ds" Aug 04 15:11:40.834742 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:40.834704 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-pphd8"] Aug 04 15:11:40.836806 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:40.836783 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-pphd8" Aug 04 15:11:40.840725 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:40.840706 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-84btd\"" Aug 04 15:11:40.860743 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:40.860719 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-pphd8"] Aug 04 15:11:40.995448 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:40.995416 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlm2j\" (UniqueName: \"kubernetes.io/projected/d3c07162-36d3-4e26-8e1c-e94a142e8e6a-kube-api-access-xlm2j\") pod \"cert-manager-55b4d954d5-pphd8\" (UID: \"d3c07162-36d3-4e26-8e1c-e94a142e8e6a\") " pod="cert-manager/cert-manager-55b4d954d5-pphd8" Aug 04 15:11:40.995621 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:40.995463 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d3c07162-36d3-4e26-8e1c-e94a142e8e6a-bound-sa-token\") pod \"cert-manager-55b4d954d5-pphd8\" (UID: \"d3c07162-36d3-4e26-8e1c-e94a142e8e6a\") " pod="cert-manager/cert-manager-55b4d954d5-pphd8" Aug 04 15:11:41.096697 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:41.096615 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlm2j\" (UniqueName: \"kubernetes.io/projected/d3c07162-36d3-4e26-8e1c-e94a142e8e6a-kube-api-access-xlm2j\") pod \"cert-manager-55b4d954d5-pphd8\" (UID: \"d3c07162-36d3-4e26-8e1c-e94a142e8e6a\") " pod="cert-manager/cert-manager-55b4d954d5-pphd8" Aug 04 15:11:41.096697 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:41.096677 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d3c07162-36d3-4e26-8e1c-e94a142e8e6a-bound-sa-token\") pod \"cert-manager-55b4d954d5-pphd8\" (UID: \"d3c07162-36d3-4e26-8e1c-e94a142e8e6a\") " pod="cert-manager/cert-manager-55b4d954d5-pphd8" Aug 04 15:11:41.109491 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:41.109465 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlm2j\" (UniqueName: \"kubernetes.io/projected/d3c07162-36d3-4e26-8e1c-e94a142e8e6a-kube-api-access-xlm2j\") pod \"cert-manager-55b4d954d5-pphd8\" (UID: \"d3c07162-36d3-4e26-8e1c-e94a142e8e6a\") " pod="cert-manager/cert-manager-55b4d954d5-pphd8" Aug 04 15:11:41.109611 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:41.109561 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d3c07162-36d3-4e26-8e1c-e94a142e8e6a-bound-sa-token\") pod \"cert-manager-55b4d954d5-pphd8\" (UID: \"d3c07162-36d3-4e26-8e1c-e94a142e8e6a\") " pod="cert-manager/cert-manager-55b4d954d5-pphd8" Aug 04 15:11:41.145628 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:41.145606 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-pphd8" Aug 04 15:11:41.272819 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:41.272792 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-pphd8"] Aug 04 15:11:41.274377 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:11:41.274351 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd3c07162_36d3_4e26_8e1c_e94a142e8e6a.slice/crio-f1baaa165bfa76d640c8a888d6d4cde14ecd4cc92e90849ef312f955207e965b WatchSource:0}: Error finding container f1baaa165bfa76d640c8a888d6d4cde14ecd4cc92e90849ef312f955207e965b: Status 404 returned error can't find the container with id f1baaa165bfa76d640c8a888d6d4cde14ecd4cc92e90849ef312f955207e965b Aug 04 15:11:41.467110 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:41.467032 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-pphd8" event={"ID":"d3c07162-36d3-4e26-8e1c-e94a142e8e6a","Type":"ContainerStarted","Data":"800291344c1cd547aa9d4090b078bd8fad5222643a417d69cbccf4ccd6582b50"} Aug 04 15:11:41.467110 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:11:41.467073 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-pphd8" event={"ID":"d3c07162-36d3-4e26-8e1c-e94a142e8e6a","Type":"ContainerStarted","Data":"f1baaa165bfa76d640c8a888d6d4cde14ecd4cc92e90849ef312f955207e965b"} Aug 04 15:12:10.022344 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:10.022231 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-pphd8" podStartSLOduration=30.022212816 podStartE2EDuration="30.022212816s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:11:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:11:41.498315084 +0000 UTC m=+324.958187432" watchObservedRunningTime="2026-08-04 15:12:10.022212816 +0000 UTC m=+353.482085165" Aug 04 15:12:10.023617 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:10.023589 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6wtwv"] Aug 04 15:12:10.026041 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:10.026020 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6wtwv" Aug 04 15:12:10.028682 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:10.028653 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ghcjn\"" Aug 04 15:12:10.028968 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:10.028950 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Aug 04 15:12:10.029849 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:10.029834 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Aug 04 15:12:10.038282 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:10.038260 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6wtwv"] Aug 04 15:12:10.106824 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:10.106794 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3231e7e5-672e-4e04-9511-f81cff1ccecf-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6wtwv\" (UID: \"3231e7e5-672e-4e04-9511-f81cff1ccecf\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6wtwv" Aug 04 15:12:10.106979 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:10.106855 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3231e7e5-672e-4e04-9511-f81cff1ccecf-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6wtwv\" (UID: \"3231e7e5-672e-4e04-9511-f81cff1ccecf\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6wtwv" Aug 04 15:12:10.106979 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:10.106895 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p84bm\" (UniqueName: \"kubernetes.io/projected/3231e7e5-672e-4e04-9511-f81cff1ccecf-kube-api-access-p84bm\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6wtwv\" (UID: \"3231e7e5-672e-4e04-9511-f81cff1ccecf\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6wtwv" Aug 04 15:12:10.208235 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:10.208178 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3231e7e5-672e-4e04-9511-f81cff1ccecf-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6wtwv\" (UID: \"3231e7e5-672e-4e04-9511-f81cff1ccecf\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6wtwv" Aug 04 15:12:10.208497 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:10.208411 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3231e7e5-672e-4e04-9511-f81cff1ccecf-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6wtwv\" (UID: \"3231e7e5-672e-4e04-9511-f81cff1ccecf\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6wtwv" Aug 04 15:12:10.208949 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:10.208669 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3231e7e5-672e-4e04-9511-f81cff1ccecf-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6wtwv\" (UID: \"3231e7e5-672e-4e04-9511-f81cff1ccecf\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6wtwv" Aug 04 15:12:10.208949 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:10.208721 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3231e7e5-672e-4e04-9511-f81cff1ccecf-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6wtwv\" (UID: \"3231e7e5-672e-4e04-9511-f81cff1ccecf\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6wtwv" Aug 04 15:12:10.209101 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:10.209019 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p84bm\" (UniqueName: \"kubernetes.io/projected/3231e7e5-672e-4e04-9511-f81cff1ccecf-kube-api-access-p84bm\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6wtwv\" (UID: \"3231e7e5-672e-4e04-9511-f81cff1ccecf\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6wtwv" Aug 04 15:12:10.219851 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:10.219822 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p84bm\" (UniqueName: \"kubernetes.io/projected/3231e7e5-672e-4e04-9511-f81cff1ccecf-kube-api-access-p84bm\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6wtwv\" (UID: \"3231e7e5-672e-4e04-9511-f81cff1ccecf\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6wtwv" Aug 04 15:12:10.335274 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:10.335185 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6wtwv" Aug 04 15:12:10.463924 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:10.463861 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6wtwv"] Aug 04 15:12:10.466645 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:12:10.466612 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3231e7e5_672e_4e04_9511_f81cff1ccecf.slice/crio-a91cff2b9537821d32ec1d76cdd034b5551ff3ff3e62308545f81a4f75e1a1ff WatchSource:0}: Error finding container a91cff2b9537821d32ec1d76cdd034b5551ff3ff3e62308545f81a4f75e1a1ff: Status 404 returned error can't find the container with id a91cff2b9537821d32ec1d76cdd034b5551ff3ff3e62308545f81a4f75e1a1ff Aug 04 15:12:10.589840 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:10.589756 2574 generic.go:356] "Generic (PLEG): container finished" podID="3231e7e5-672e-4e04-9511-f81cff1ccecf" containerID="aeff15aa2509d5025181898d6e1e8d500d7ccf586f87447a6c06db4ffcd5d16e" exitCode=0 Aug 04 15:12:10.589973 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:10.589843 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6wtwv" event={"ID":"3231e7e5-672e-4e04-9511-f81cff1ccecf","Type":"ContainerDied","Data":"aeff15aa2509d5025181898d6e1e8d500d7ccf586f87447a6c06db4ffcd5d16e"} Aug 04 15:12:10.589973 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:10.589906 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6wtwv" event={"ID":"3231e7e5-672e-4e04-9511-f81cff1ccecf","Type":"ContainerStarted","Data":"a91cff2b9537821d32ec1d76cdd034b5551ff3ff3e62308545f81a4f75e1a1ff"} Aug 04 15:12:12.600997 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:12.600960 2574 generic.go:356] "Generic (PLEG): container finished" podID="3231e7e5-672e-4e04-9511-f81cff1ccecf" containerID="679b409dd7e374913bc8d69feb15a9fc3b16a6b2127a947d590ad96a3021d610" exitCode=0 Aug 04 15:12:12.601442 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:12.601038 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6wtwv" event={"ID":"3231e7e5-672e-4e04-9511-f81cff1ccecf","Type":"ContainerDied","Data":"679b409dd7e374913bc8d69feb15a9fc3b16a6b2127a947d590ad96a3021d610"} Aug 04 15:12:13.607157 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:13.607118 2574 generic.go:356] "Generic (PLEG): container finished" podID="3231e7e5-672e-4e04-9511-f81cff1ccecf" containerID="f780e65712bfa1c94b3f584d4ae2074082f9e82267fb1117438f94b5ad657605" exitCode=0 Aug 04 15:12:13.607617 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:13.607210 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6wtwv" event={"ID":"3231e7e5-672e-4e04-9511-f81cff1ccecf","Type":"ContainerDied","Data":"f780e65712bfa1c94b3f584d4ae2074082f9e82267fb1117438f94b5ad657605"} Aug 04 15:12:14.734274 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:14.734251 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6wtwv" Aug 04 15:12:14.846104 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:14.846066 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p84bm\" (UniqueName: \"kubernetes.io/projected/3231e7e5-672e-4e04-9511-f81cff1ccecf-kube-api-access-p84bm\") pod \"3231e7e5-672e-4e04-9511-f81cff1ccecf\" (UID: \"3231e7e5-672e-4e04-9511-f81cff1ccecf\") " Aug 04 15:12:14.846288 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:14.846114 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3231e7e5-672e-4e04-9511-f81cff1ccecf-bundle\") pod \"3231e7e5-672e-4e04-9511-f81cff1ccecf\" (UID: \"3231e7e5-672e-4e04-9511-f81cff1ccecf\") " Aug 04 15:12:14.846288 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:14.846168 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3231e7e5-672e-4e04-9511-f81cff1ccecf-util\") pod \"3231e7e5-672e-4e04-9511-f81cff1ccecf\" (UID: \"3231e7e5-672e-4e04-9511-f81cff1ccecf\") " Aug 04 15:12:14.846871 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:14.846846 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3231e7e5-672e-4e04-9511-f81cff1ccecf-bundle" (OuterVolumeSpecName: "bundle") pod "3231e7e5-672e-4e04-9511-f81cff1ccecf" (UID: "3231e7e5-672e-4e04-9511-f81cff1ccecf"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:12:14.848215 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:14.848183 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3231e7e5-672e-4e04-9511-f81cff1ccecf-kube-api-access-p84bm" (OuterVolumeSpecName: "kube-api-access-p84bm") pod "3231e7e5-672e-4e04-9511-f81cff1ccecf" (UID: "3231e7e5-672e-4e04-9511-f81cff1ccecf"). InnerVolumeSpecName "kube-api-access-p84bm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 15:12:14.851802 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:14.851769 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3231e7e5-672e-4e04-9511-f81cff1ccecf-util" (OuterVolumeSpecName: "util") pod "3231e7e5-672e-4e04-9511-f81cff1ccecf" (UID: "3231e7e5-672e-4e04-9511-f81cff1ccecf"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:12:14.946744 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:14.946664 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3231e7e5-672e-4e04-9511-f81cff1ccecf-util\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:12:14.946744 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:14.946691 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p84bm\" (UniqueName: \"kubernetes.io/projected/3231e7e5-672e-4e04-9511-f81cff1ccecf-kube-api-access-p84bm\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:12:14.946744 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:14.946702 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3231e7e5-672e-4e04-9511-f81cff1ccecf-bundle\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:12:15.617217 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:15.617187 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6wtwv" event={"ID":"3231e7e5-672e-4e04-9511-f81cff1ccecf","Type":"ContainerDied","Data":"a91cff2b9537821d32ec1d76cdd034b5551ff3ff3e62308545f81a4f75e1a1ff"} Aug 04 15:12:15.617377 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:15.617227 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af6wtwv" Aug 04 15:12:15.617377 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:15.617231 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a91cff2b9537821d32ec1d76cdd034b5551ff3ff3e62308545f81a4f75e1a1ff" Aug 04 15:12:27.132712 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:27.132681 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-764fcd8dc-mtplw"] Aug 04 15:12:27.133111 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:27.132948 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3231e7e5-672e-4e04-9511-f81cff1ccecf" containerName="extract" Aug 04 15:12:27.133111 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:27.132958 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3231e7e5-672e-4e04-9511-f81cff1ccecf" containerName="extract" Aug 04 15:12:27.133111 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:27.132968 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3231e7e5-672e-4e04-9511-f81cff1ccecf" containerName="pull" Aug 04 15:12:27.133111 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:27.132974 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3231e7e5-672e-4e04-9511-f81cff1ccecf" containerName="pull" Aug 04 15:12:27.133111 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:27.132981 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3231e7e5-672e-4e04-9511-f81cff1ccecf" containerName="util" Aug 04 15:12:27.133111 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:27.132993 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3231e7e5-672e-4e04-9511-f81cff1ccecf" containerName="util" Aug 04 15:12:27.133111 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:27.133050 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3231e7e5-672e-4e04-9511-f81cff1ccecf" containerName="extract" Aug 04 15:12:27.137285 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:27.137268 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-764fcd8dc-mtplw" Aug 04 15:12:27.158056 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:27.158032 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Aug 04 15:12:27.158185 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:27.158053 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-2c586\"" Aug 04 15:12:27.158185 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:27.158126 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Aug 04 15:12:27.158990 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:27.158971 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Aug 04 15:12:27.160918 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:27.160905 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Aug 04 15:12:27.226001 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:27.225974 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-764fcd8dc-mtplw"] Aug 04 15:12:27.245552 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:27.245523 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6d4760e1-d114-4df8-b496-5c91481b1c24-manifests\") pod \"opendatahub-operator-controller-manager-764fcd8dc-mtplw\" (UID: \"6d4760e1-d114-4df8-b496-5c91481b1c24\") " pod="opendatahub/opendatahub-operator-controller-manager-764fcd8dc-mtplw" Aug 04 15:12:27.245683 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:27.245582 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjrch\" (UniqueName: \"kubernetes.io/projected/6d4760e1-d114-4df8-b496-5c91481b1c24-kube-api-access-tjrch\") pod \"opendatahub-operator-controller-manager-764fcd8dc-mtplw\" (UID: \"6d4760e1-d114-4df8-b496-5c91481b1c24\") " pod="opendatahub/opendatahub-operator-controller-manager-764fcd8dc-mtplw" Aug 04 15:12:27.245683 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:27.245628 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6d4760e1-d114-4df8-b496-5c91481b1c24-webhook-cert\") pod \"opendatahub-operator-controller-manager-764fcd8dc-mtplw\" (UID: \"6d4760e1-d114-4df8-b496-5c91481b1c24\") " pod="opendatahub/opendatahub-operator-controller-manager-764fcd8dc-mtplw" Aug 04 15:12:27.245797 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:27.245691 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6d4760e1-d114-4df8-b496-5c91481b1c24-apiservice-cert\") pod \"opendatahub-operator-controller-manager-764fcd8dc-mtplw\" (UID: \"6d4760e1-d114-4df8-b496-5c91481b1c24\") " pod="opendatahub/opendatahub-operator-controller-manager-764fcd8dc-mtplw" Aug 04 15:12:27.245797 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:27.245758 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6d4760e1-d114-4df8-b496-5c91481b1c24-tmp\") pod \"opendatahub-operator-controller-manager-764fcd8dc-mtplw\" (UID: \"6d4760e1-d114-4df8-b496-5c91481b1c24\") " pod="opendatahub/opendatahub-operator-controller-manager-764fcd8dc-mtplw" Aug 04 15:12:27.347111 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:27.347078 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6d4760e1-d114-4df8-b496-5c91481b1c24-tmp\") pod \"opendatahub-operator-controller-manager-764fcd8dc-mtplw\" (UID: \"6d4760e1-d114-4df8-b496-5c91481b1c24\") " pod="opendatahub/opendatahub-operator-controller-manager-764fcd8dc-mtplw" Aug 04 15:12:27.347238 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:27.347127 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6d4760e1-d114-4df8-b496-5c91481b1c24-manifests\") pod \"opendatahub-operator-controller-manager-764fcd8dc-mtplw\" (UID: \"6d4760e1-d114-4df8-b496-5c91481b1c24\") " pod="opendatahub/opendatahub-operator-controller-manager-764fcd8dc-mtplw" Aug 04 15:12:27.347238 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:27.347182 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjrch\" (UniqueName: \"kubernetes.io/projected/6d4760e1-d114-4df8-b496-5c91481b1c24-kube-api-access-tjrch\") pod \"opendatahub-operator-controller-manager-764fcd8dc-mtplw\" (UID: \"6d4760e1-d114-4df8-b496-5c91481b1c24\") " pod="opendatahub/opendatahub-operator-controller-manager-764fcd8dc-mtplw" Aug 04 15:12:27.347238 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:27.347212 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6d4760e1-d114-4df8-b496-5c91481b1c24-webhook-cert\") pod \"opendatahub-operator-controller-manager-764fcd8dc-mtplw\" (UID: \"6d4760e1-d114-4df8-b496-5c91481b1c24\") " pod="opendatahub/opendatahub-operator-controller-manager-764fcd8dc-mtplw" Aug 04 15:12:27.347416 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:27.347255 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6d4760e1-d114-4df8-b496-5c91481b1c24-apiservice-cert\") pod \"opendatahub-operator-controller-manager-764fcd8dc-mtplw\" (UID: \"6d4760e1-d114-4df8-b496-5c91481b1c24\") " pod="opendatahub/opendatahub-operator-controller-manager-764fcd8dc-mtplw" Aug 04 15:12:27.347597 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:27.347577 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6d4760e1-d114-4df8-b496-5c91481b1c24-manifests\") pod \"opendatahub-operator-controller-manager-764fcd8dc-mtplw\" (UID: \"6d4760e1-d114-4df8-b496-5c91481b1c24\") " pod="opendatahub/opendatahub-operator-controller-manager-764fcd8dc-mtplw" Aug 04 15:12:27.349369 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:27.349345 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6d4760e1-d114-4df8-b496-5c91481b1c24-tmp\") pod \"opendatahub-operator-controller-manager-764fcd8dc-mtplw\" (UID: \"6d4760e1-d114-4df8-b496-5c91481b1c24\") " pod="opendatahub/opendatahub-operator-controller-manager-764fcd8dc-mtplw" Aug 04 15:12:27.349615 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:27.349598 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6d4760e1-d114-4df8-b496-5c91481b1c24-apiservice-cert\") pod \"opendatahub-operator-controller-manager-764fcd8dc-mtplw\" (UID: \"6d4760e1-d114-4df8-b496-5c91481b1c24\") " pod="opendatahub/opendatahub-operator-controller-manager-764fcd8dc-mtplw" Aug 04 15:12:27.349686 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:27.349666 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6d4760e1-d114-4df8-b496-5c91481b1c24-webhook-cert\") pod \"opendatahub-operator-controller-manager-764fcd8dc-mtplw\" (UID: \"6d4760e1-d114-4df8-b496-5c91481b1c24\") " pod="opendatahub/opendatahub-operator-controller-manager-764fcd8dc-mtplw" Aug 04 15:12:27.374322 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:27.374299 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjrch\" (UniqueName: \"kubernetes.io/projected/6d4760e1-d114-4df8-b496-5c91481b1c24-kube-api-access-tjrch\") pod \"opendatahub-operator-controller-manager-764fcd8dc-mtplw\" (UID: \"6d4760e1-d114-4df8-b496-5c91481b1c24\") " pod="opendatahub/opendatahub-operator-controller-manager-764fcd8dc-mtplw" Aug 04 15:12:27.446890 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:27.446824 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-764fcd8dc-mtplw" Aug 04 15:12:27.587067 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:27.587042 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-764fcd8dc-mtplw"] Aug 04 15:12:27.588017 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:12:27.587993 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d4760e1_d114_4df8_b496_5c91481b1c24.slice/crio-edd14be9994c7c5fff42e56edfe085b465a7fdac6378b80ed9536193778bcc15 WatchSource:0}: Error finding container edd14be9994c7c5fff42e56edfe085b465a7fdac6378b80ed9536193778bcc15: Status 404 returned error can't find the container with id edd14be9994c7c5fff42e56edfe085b465a7fdac6378b80ed9536193778bcc15 Aug 04 15:12:27.668384 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:27.668352 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-764fcd8dc-mtplw" event={"ID":"6d4760e1-d114-4df8-b496-5c91481b1c24","Type":"ContainerStarted","Data":"edd14be9994c7c5fff42e56edfe085b465a7fdac6378b80ed9536193778bcc15"} Aug 04 15:12:30.687197 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:30.687163 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-764fcd8dc-mtplw" event={"ID":"6d4760e1-d114-4df8-b496-5c91481b1c24","Type":"ContainerStarted","Data":"916cc892308188f35e382f73079389653a6099a8e05dc59d4434b3adf03cc7d7"} Aug 04 15:12:32.697205 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:32.697169 2574 generic.go:356] "Generic (PLEG): container finished" podID="6d4760e1-d114-4df8-b496-5c91481b1c24" containerID="916cc892308188f35e382f73079389653a6099a8e05dc59d4434b3adf03cc7d7" exitCode=0 Aug 04 15:12:32.697608 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:32.697218 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-764fcd8dc-mtplw" event={"ID":"6d4760e1-d114-4df8-b496-5c91481b1c24","Type":"ContainerDied","Data":"916cc892308188f35e382f73079389653a6099a8e05dc59d4434b3adf03cc7d7"} Aug 04 15:12:33.704240 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:33.704208 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-764fcd8dc-mtplw" event={"ID":"6d4760e1-d114-4df8-b496-5c91481b1c24","Type":"ContainerStarted","Data":"ddd64afc34ae904e670df9a0f8c78816b7f72043b26ef5c8939583e901457d3d"} Aug 04 15:12:33.704663 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:33.704450 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-764fcd8dc-mtplw" Aug 04 15:12:33.745299 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:33.745247 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-764fcd8dc-mtplw" podStartSLOduration=2.3475987959999998 podStartE2EDuration="6.745230549s" totalImagesPullingTime="3.335832532s" totalInitContainerRuntime="1.061799221s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:12:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:12:33.744899515 +0000 UTC m=+377.204771867" watchObservedRunningTime="2026-08-04 15:12:33.745230549 +0000 UTC m=+377.205102959" Aug 04 15:12:44.711384 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:44.711353 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-764fcd8dc-mtplw" Aug 04 15:12:48.327804 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:48.327767 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7c7cf99455-6rvm7"] Aug 04 15:12:48.333446 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:48.333427 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7c7cf99455-6rvm7" Aug 04 15:12:48.340631 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:48.340613 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Aug 04 15:12:48.341503 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:48.341487 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Aug 04 15:12:48.341580 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:48.341515 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-6nb8c\"" Aug 04 15:12:48.350331 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:48.350308 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-q4t7p"] Aug 04 15:12:48.352269 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:48.352253 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-q4t7p" Aug 04 15:12:48.372383 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:48.372361 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7c7cf99455-6rvm7"] Aug 04 15:12:48.386300 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:48.386274 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Aug 04 15:12:48.388425 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:48.388384 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-dchbb\"" Aug 04 15:12:48.424087 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:48.424063 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f18365b-4250-4cd6-8217-9f5efdc88129-tls-certs\") pod \"kube-auth-proxy-7c7cf99455-6rvm7\" (UID: \"6f18365b-4250-4cd6-8217-9f5efdc88129\") " pod="openshift-ingress/kube-auth-proxy-7c7cf99455-6rvm7" Aug 04 15:12:48.424205 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:48.424103 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n46dk\" (UniqueName: \"kubernetes.io/projected/6f18365b-4250-4cd6-8217-9f5efdc88129-kube-api-access-n46dk\") pod \"kube-auth-proxy-7c7cf99455-6rvm7\" (UID: \"6f18365b-4250-4cd6-8217-9f5efdc88129\") " pod="openshift-ingress/kube-auth-proxy-7c7cf99455-6rvm7" Aug 04 15:12:48.424205 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:48.424143 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f18365b-4250-4cd6-8217-9f5efdc88129-tmp\") pod \"kube-auth-proxy-7c7cf99455-6rvm7\" (UID: \"6f18365b-4250-4cd6-8217-9f5efdc88129\") " pod="openshift-ingress/kube-auth-proxy-7c7cf99455-6rvm7" Aug 04 15:12:48.449830 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:48.449803 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-q4t7p"] Aug 04 15:12:48.524808 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:48.524775 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7aaa449d-38a3-4003-af61-e36cbab919c5-cert\") pod \"odh-model-controller-7998c67dc4-q4t7p\" (UID: \"7aaa449d-38a3-4003-af61-e36cbab919c5\") " pod="opendatahub/odh-model-controller-7998c67dc4-q4t7p" Aug 04 15:12:48.524959 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:48.524820 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f18365b-4250-4cd6-8217-9f5efdc88129-tls-certs\") pod \"kube-auth-proxy-7c7cf99455-6rvm7\" (UID: \"6f18365b-4250-4cd6-8217-9f5efdc88129\") " pod="openshift-ingress/kube-auth-proxy-7c7cf99455-6rvm7" Aug 04 15:12:48.524959 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:48.524844 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sl9rt\" (UniqueName: \"kubernetes.io/projected/7aaa449d-38a3-4003-af61-e36cbab919c5-kube-api-access-sl9rt\") pod \"odh-model-controller-7998c67dc4-q4t7p\" (UID: \"7aaa449d-38a3-4003-af61-e36cbab919c5\") " pod="opendatahub/odh-model-controller-7998c67dc4-q4t7p" Aug 04 15:12:48.525037 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:48.524961 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n46dk\" (UniqueName: \"kubernetes.io/projected/6f18365b-4250-4cd6-8217-9f5efdc88129-kube-api-access-n46dk\") pod \"kube-auth-proxy-7c7cf99455-6rvm7\" (UID: \"6f18365b-4250-4cd6-8217-9f5efdc88129\") " pod="openshift-ingress/kube-auth-proxy-7c7cf99455-6rvm7" Aug 04 15:12:48.525037 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:48.525018 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f18365b-4250-4cd6-8217-9f5efdc88129-tmp\") pod \"kube-auth-proxy-7c7cf99455-6rvm7\" (UID: \"6f18365b-4250-4cd6-8217-9f5efdc88129\") " pod="openshift-ingress/kube-auth-proxy-7c7cf99455-6rvm7" Aug 04 15:12:48.527166 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:48.527135 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f18365b-4250-4cd6-8217-9f5efdc88129-tmp\") pod \"kube-auth-proxy-7c7cf99455-6rvm7\" (UID: \"6f18365b-4250-4cd6-8217-9f5efdc88129\") " pod="openshift-ingress/kube-auth-proxy-7c7cf99455-6rvm7" Aug 04 15:12:48.527343 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:48.527324 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6f18365b-4250-4cd6-8217-9f5efdc88129-tls-certs\") pod \"kube-auth-proxy-7c7cf99455-6rvm7\" (UID: \"6f18365b-4250-4cd6-8217-9f5efdc88129\") " pod="openshift-ingress/kube-auth-proxy-7c7cf99455-6rvm7" Aug 04 15:12:48.548108 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:48.548077 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n46dk\" (UniqueName: \"kubernetes.io/projected/6f18365b-4250-4cd6-8217-9f5efdc88129-kube-api-access-n46dk\") pod \"kube-auth-proxy-7c7cf99455-6rvm7\" (UID: \"6f18365b-4250-4cd6-8217-9f5efdc88129\") " pod="openshift-ingress/kube-auth-proxy-7c7cf99455-6rvm7" Aug 04 15:12:48.625634 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:48.625563 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7aaa449d-38a3-4003-af61-e36cbab919c5-cert\") pod \"odh-model-controller-7998c67dc4-q4t7p\" (UID: \"7aaa449d-38a3-4003-af61-e36cbab919c5\") " pod="opendatahub/odh-model-controller-7998c67dc4-q4t7p" Aug 04 15:12:48.625634 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:48.625602 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl9rt\" (UniqueName: \"kubernetes.io/projected/7aaa449d-38a3-4003-af61-e36cbab919c5-kube-api-access-sl9rt\") pod \"odh-model-controller-7998c67dc4-q4t7p\" (UID: \"7aaa449d-38a3-4003-af61-e36cbab919c5\") " pod="opendatahub/odh-model-controller-7998c67dc4-q4t7p" Aug 04 15:12:48.627764 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:48.627738 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7aaa449d-38a3-4003-af61-e36cbab919c5-cert\") pod \"odh-model-controller-7998c67dc4-q4t7p\" (UID: \"7aaa449d-38a3-4003-af61-e36cbab919c5\") " pod="opendatahub/odh-model-controller-7998c67dc4-q4t7p" Aug 04 15:12:48.642246 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:48.642221 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sl9rt\" (UniqueName: \"kubernetes.io/projected/7aaa449d-38a3-4003-af61-e36cbab919c5-kube-api-access-sl9rt\") pod \"odh-model-controller-7998c67dc4-q4t7p\" (UID: \"7aaa449d-38a3-4003-af61-e36cbab919c5\") " pod="opendatahub/odh-model-controller-7998c67dc4-q4t7p" Aug 04 15:12:48.643018 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:48.642992 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7c7cf99455-6rvm7" Aug 04 15:12:48.661175 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:48.661152 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-q4t7p" Aug 04 15:12:48.817314 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:48.817291 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7c7cf99455-6rvm7"] Aug 04 15:12:48.819611 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:12:48.819586 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f18365b_4250_4cd6_8217_9f5efdc88129.slice/crio-1b05858fc4b39725ced2e31c291fc53f0613ab5b12f644e38f67ad11cb57f080 WatchSource:0}: Error finding container 1b05858fc4b39725ced2e31c291fc53f0613ab5b12f644e38f67ad11cb57f080: Status 404 returned error can't find the container with id 1b05858fc4b39725ced2e31c291fc53f0613ab5b12f644e38f67ad11cb57f080 Aug 04 15:12:48.843279 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:48.843259 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-q4t7p"] Aug 04 15:12:48.843962 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:12:48.843937 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7aaa449d_38a3_4003_af61_e36cbab919c5.slice/crio-147ae4f98db3f4aed8d179f99b40d8210c88a1e9d5401be9496da0a24fdec334 WatchSource:0}: Error finding container 147ae4f98db3f4aed8d179f99b40d8210c88a1e9d5401be9496da0a24fdec334: Status 404 returned error can't find the container with id 147ae4f98db3f4aed8d179f99b40d8210c88a1e9d5401be9496da0a24fdec334 Aug 04 15:12:49.778196 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:49.778157 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7c7cf99455-6rvm7" event={"ID":"6f18365b-4250-4cd6-8217-9f5efdc88129","Type":"ContainerStarted","Data":"1b05858fc4b39725ced2e31c291fc53f0613ab5b12f644e38f67ad11cb57f080"} Aug 04 15:12:49.779606 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:49.779578 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-q4t7p" event={"ID":"7aaa449d-38a3-4003-af61-e36cbab919c5","Type":"ContainerStarted","Data":"147ae4f98db3f4aed8d179f99b40d8210c88a1e9d5401be9496da0a24fdec334"} Aug 04 15:12:49.821085 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:49.821019 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-dnbrf"] Aug 04 15:12:49.824094 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:49.823738 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-dnbrf" Aug 04 15:12:49.826856 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:49.826833 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Aug 04 15:12:49.845520 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:49.845454 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-dnbrf"] Aug 04 15:12:49.938410 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:49.938365 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbb8d592-c4e9-4263-bb54-7656658e165b-tls-certs\") pod \"model-serving-api-54d89bd5fc-dnbrf\" (UID: \"dbb8d592-c4e9-4263-bb54-7656658e165b\") " pod="opendatahub/model-serving-api-54d89bd5fc-dnbrf" Aug 04 15:12:49.938567 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:49.938430 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7kqh\" (UniqueName: \"kubernetes.io/projected/dbb8d592-c4e9-4263-bb54-7656658e165b-kube-api-access-b7kqh\") pod \"model-serving-api-54d89bd5fc-dnbrf\" (UID: \"dbb8d592-c4e9-4263-bb54-7656658e165b\") " pod="opendatahub/model-serving-api-54d89bd5fc-dnbrf" Aug 04 15:12:50.039616 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:50.039418 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbb8d592-c4e9-4263-bb54-7656658e165b-tls-certs\") pod \"model-serving-api-54d89bd5fc-dnbrf\" (UID: \"dbb8d592-c4e9-4263-bb54-7656658e165b\") " pod="opendatahub/model-serving-api-54d89bd5fc-dnbrf" Aug 04 15:12:50.039616 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:50.039492 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7kqh\" (UniqueName: \"kubernetes.io/projected/dbb8d592-c4e9-4263-bb54-7656658e165b-kube-api-access-b7kqh\") pod \"model-serving-api-54d89bd5fc-dnbrf\" (UID: \"dbb8d592-c4e9-4263-bb54-7656658e165b\") " pod="opendatahub/model-serving-api-54d89bd5fc-dnbrf" Aug 04 15:12:50.042814 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:50.042759 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbb8d592-c4e9-4263-bb54-7656658e165b-tls-certs\") pod \"model-serving-api-54d89bd5fc-dnbrf\" (UID: \"dbb8d592-c4e9-4263-bb54-7656658e165b\") " pod="opendatahub/model-serving-api-54d89bd5fc-dnbrf" Aug 04 15:12:50.054967 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:50.054908 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7kqh\" (UniqueName: \"kubernetes.io/projected/dbb8d592-c4e9-4263-bb54-7656658e165b-kube-api-access-b7kqh\") pod \"model-serving-api-54d89bd5fc-dnbrf\" (UID: \"dbb8d592-c4e9-4263-bb54-7656658e165b\") " pod="opendatahub/model-serving-api-54d89bd5fc-dnbrf" Aug 04 15:12:50.138453 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:50.138411 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-dnbrf" Aug 04 15:12:50.302086 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:50.301936 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-dnbrf"] Aug 04 15:12:50.304957 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:12:50.304918 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbb8d592_c4e9_4263_bb54_7656658e165b.slice/crio-dd8cc31785a126ca2ba3fafc2534270e72a8bab1324828ae039842514dad88af WatchSource:0}: Error finding container dd8cc31785a126ca2ba3fafc2534270e72a8bab1324828ae039842514dad88af: Status 404 returned error can't find the container with id dd8cc31785a126ca2ba3fafc2534270e72a8bab1324828ae039842514dad88af Aug 04 15:12:50.783908 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:50.783843 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xrm6z"] Aug 04 15:12:50.791260 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:50.791225 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-dnbrf" event={"ID":"dbb8d592-c4e9-4263-bb54-7656658e165b","Type":"ContainerStarted","Data":"dd8cc31785a126ca2ba3fafc2534270e72a8bab1324828ae039842514dad88af"} Aug 04 15:12:50.791425 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:50.791343 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xrm6z" Aug 04 15:12:50.795649 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:50.795623 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Aug 04 15:12:50.795777 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:50.795623 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Aug 04 15:12:50.796118 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:50.796095 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Aug 04 15:12:50.796118 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:50.796101 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Aug 04 15:12:50.796433 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:50.796414 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-z4f8l\"" Aug 04 15:12:50.806998 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:50.806976 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xrm6z"] Aug 04 15:12:50.946824 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:50.946787 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mf9n\" (UniqueName: \"kubernetes.io/projected/6e301f43-c965-491c-895b-78e5f2dfdfed-kube-api-access-7mf9n\") pod \"istiod-openshift-gateway-7b7995d5cd-xrm6z\" (UID: \"6e301f43-c965-491c-895b-78e5f2dfdfed\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xrm6z" Aug 04 15:12:50.947004 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:50.946829 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6e301f43-c965-491c-895b-78e5f2dfdfed-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-xrm6z\" (UID: \"6e301f43-c965-491c-895b-78e5f2dfdfed\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xrm6z" Aug 04 15:12:50.947073 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:50.946995 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e301f43-c965-491c-895b-78e5f2dfdfed-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-xrm6z\" (UID: \"6e301f43-c965-491c-895b-78e5f2dfdfed\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xrm6z" Aug 04 15:12:50.947073 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:50.947036 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6e301f43-c965-491c-895b-78e5f2dfdfed-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-xrm6z\" (UID: \"6e301f43-c965-491c-895b-78e5f2dfdfed\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xrm6z" Aug 04 15:12:50.947168 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:50.947075 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e301f43-c965-491c-895b-78e5f2dfdfed-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-xrm6z\" (UID: \"6e301f43-c965-491c-895b-78e5f2dfdfed\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xrm6z" Aug 04 15:12:50.947168 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:50.947105 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6e301f43-c965-491c-895b-78e5f2dfdfed-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-xrm6z\" (UID: \"6e301f43-c965-491c-895b-78e5f2dfdfed\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xrm6z" Aug 04 15:12:50.947264 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:50.947163 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6e301f43-c965-491c-895b-78e5f2dfdfed-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-xrm6z\" (UID: \"6e301f43-c965-491c-895b-78e5f2dfdfed\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xrm6z" Aug 04 15:12:51.048234 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:51.048147 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e301f43-c965-491c-895b-78e5f2dfdfed-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-xrm6z\" (UID: \"6e301f43-c965-491c-895b-78e5f2dfdfed\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xrm6z" Aug 04 15:12:51.048234 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:51.048190 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6e301f43-c965-491c-895b-78e5f2dfdfed-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-xrm6z\" (UID: \"6e301f43-c965-491c-895b-78e5f2dfdfed\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xrm6z" Aug 04 15:12:51.048234 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:51.048220 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e301f43-c965-491c-895b-78e5f2dfdfed-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-xrm6z\" (UID: \"6e301f43-c965-491c-895b-78e5f2dfdfed\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xrm6z" Aug 04 15:12:51.048632 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:51.048245 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6e301f43-c965-491c-895b-78e5f2dfdfed-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-xrm6z\" (UID: \"6e301f43-c965-491c-895b-78e5f2dfdfed\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xrm6z" Aug 04 15:12:51.048632 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:51.048280 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6e301f43-c965-491c-895b-78e5f2dfdfed-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-xrm6z\" (UID: \"6e301f43-c965-491c-895b-78e5f2dfdfed\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xrm6z" Aug 04 15:12:51.048632 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:51.048309 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mf9n\" (UniqueName: \"kubernetes.io/projected/6e301f43-c965-491c-895b-78e5f2dfdfed-kube-api-access-7mf9n\") pod \"istiod-openshift-gateway-7b7995d5cd-xrm6z\" (UID: \"6e301f43-c965-491c-895b-78e5f2dfdfed\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xrm6z" Aug 04 15:12:51.048632 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:51.048334 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6e301f43-c965-491c-895b-78e5f2dfdfed-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-xrm6z\" (UID: \"6e301f43-c965-491c-895b-78e5f2dfdfed\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xrm6z" Aug 04 15:12:51.049276 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:51.049225 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6e301f43-c965-491c-895b-78e5f2dfdfed-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-xrm6z\" (UID: \"6e301f43-c965-491c-895b-78e5f2dfdfed\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xrm6z" Aug 04 15:12:51.051157 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:51.051129 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6e301f43-c965-491c-895b-78e5f2dfdfed-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-xrm6z\" (UID: \"6e301f43-c965-491c-895b-78e5f2dfdfed\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xrm6z" Aug 04 15:12:51.051256 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:51.051226 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6e301f43-c965-491c-895b-78e5f2dfdfed-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-xrm6z\" (UID: \"6e301f43-c965-491c-895b-78e5f2dfdfed\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xrm6z" Aug 04 15:12:51.051256 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:51.051236 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e301f43-c965-491c-895b-78e5f2dfdfed-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-xrm6z\" (UID: \"6e301f43-c965-491c-895b-78e5f2dfdfed\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xrm6z" Aug 04 15:12:51.051401 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:51.051347 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6e301f43-c965-491c-895b-78e5f2dfdfed-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-xrm6z\" (UID: \"6e301f43-c965-491c-895b-78e5f2dfdfed\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xrm6z" Aug 04 15:12:51.059187 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:51.059161 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e301f43-c965-491c-895b-78e5f2dfdfed-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-xrm6z\" (UID: \"6e301f43-c965-491c-895b-78e5f2dfdfed\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xrm6z" Aug 04 15:12:51.060986 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:51.060964 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mf9n\" (UniqueName: \"kubernetes.io/projected/6e301f43-c965-491c-895b-78e5f2dfdfed-kube-api-access-7mf9n\") pod \"istiod-openshift-gateway-7b7995d5cd-xrm6z\" (UID: \"6e301f43-c965-491c-895b-78e5f2dfdfed\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xrm6z" Aug 04 15:12:51.102544 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:51.102515 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xrm6z" Aug 04 15:12:51.704482 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:51.704452 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xrm6z"] Aug 04 15:12:51.708629 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:12:51.708597 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e301f43_c965_491c_895b_78e5f2dfdfed.slice/crio-3d8699dba25bd10de84ece1fa4f43235c6425515d7a4d80ee6c3335f46fd504e WatchSource:0}: Error finding container 3d8699dba25bd10de84ece1fa4f43235c6425515d7a4d80ee6c3335f46fd504e: Status 404 returned error can't find the container with id 3d8699dba25bd10de84ece1fa4f43235c6425515d7a4d80ee6c3335f46fd504e Aug 04 15:12:51.797831 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:51.797737 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xrm6z" event={"ID":"6e301f43-c965-491c-895b-78e5f2dfdfed","Type":"ContainerStarted","Data":"3d8699dba25bd10de84ece1fa4f43235c6425515d7a4d80ee6c3335f46fd504e"} Aug 04 15:12:51.801109 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:51.801075 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-q4t7p" event={"ID":"7aaa449d-38a3-4003-af61-e36cbab919c5","Type":"ContainerStarted","Data":"4e7194afa2ae9cd86cd44a2ff60c20395e3b4557220808c37a556e2e5f8740a3"} Aug 04 15:12:51.801845 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:51.801826 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-q4t7p" Aug 04 15:12:51.822244 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:51.822151 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-q4t7p" podStartSLOduration=1.079289988 podStartE2EDuration="3.822130901s" totalImagesPullingTime="2.742840913s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:12:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:12:51.82132775 +0000 UTC m=+395.281200101" watchObservedRunningTime="2026-08-04 15:12:51.822130901 +0000 UTC m=+395.282003250" Aug 04 15:12:54.824989 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:54.824939 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7c7cf99455-6rvm7" event={"ID":"6f18365b-4250-4cd6-8217-9f5efdc88129","Type":"ContainerStarted","Data":"17c38746ced2a3528926dbe90160eb7ea7595fd4460ffaf7a4c37657ac3d3102"} Aug 04 15:12:54.826661 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:54.826633 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-dnbrf" event={"ID":"dbb8d592-c4e9-4263-bb54-7656658e165b","Type":"ContainerStarted","Data":"daf5cdb339d8da18da7aab1a9db8d73ed22683c834143110884db01ce04c9a92"} Aug 04 15:12:54.826870 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:54.826837 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-dnbrf" Aug 04 15:12:54.876509 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:54.876449 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7c7cf99455-6rvm7" podStartSLOduration=1.309073638 podStartE2EDuration="6.876429904s" totalImagesPullingTime="5.567356266s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:12:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:12:54.875583041 +0000 UTC m=+398.335455393" watchObservedRunningTime="2026-08-04 15:12:54.876429904 +0000 UTC m=+398.336302257" Aug 04 15:12:54.925331 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:54.925275 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-dnbrf" podStartSLOduration=1.8448186519999998 podStartE2EDuration="5.925258929s" totalImagesPullingTime="4.080440277s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:12:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:12:54.924946422 +0000 UTC m=+398.384818774" watchObservedRunningTime="2026-08-04 15:12:54.925258929 +0000 UTC m=+398.385131278" Aug 04 15:12:56.279686 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:56.279645 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:12:56.280045 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:56.279729 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:12:56.838539 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:56.838509 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xrm6z" event={"ID":"6e301f43-c965-491c-895b-78e5f2dfdfed","Type":"ContainerStarted","Data":"ba7eb7906a946477bdb525f8d152fbcb598cc7298b83ae686e5dd78d4a77e20b"} Aug 04 15:12:56.838698 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:56.838616 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xrm6z" Aug 04 15:12:56.876250 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:56.876203 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xrm6z" podStartSLOduration=2.308179242 podStartE2EDuration="6.876188292s" totalImagesPullingTime="4.56800905s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:12:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:12:56.87331524 +0000 UTC m=+400.333187589" watchObservedRunningTime="2026-08-04 15:12:56.876188292 +0000 UTC m=+400.336060640" Aug 04 15:12:57.849477 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:57.849437 2574 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-xrm6z container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 04 15:12:57.849869 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:57.849510 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xrm6z" podUID="6e301f43-c965-491c-895b-78e5f2dfdfed" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 15:12:58.733215 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:58.733180 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9"] Aug 04 15:12:58.736094 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:58.736078 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9" Aug 04 15:12:58.747358 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:58.747339 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-ftx67\"" Aug 04 15:12:58.789716 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:58.789688 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9"] Aug 04 15:12:58.922409 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:58.922356 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c6a911a-c21d-4ac0-beeb-c491710bfb72-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9\" (UID: \"5c6a911a-c21d-4ac0-beeb-c491710bfb72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9" Aug 04 15:12:58.922797 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:58.922423 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c6a911a-c21d-4ac0-beeb-c491710bfb72-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9\" (UID: \"5c6a911a-c21d-4ac0-beeb-c491710bfb72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9" Aug 04 15:12:58.922797 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:58.922442 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c6a911a-c21d-4ac0-beeb-c491710bfb72-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9\" (UID: \"5c6a911a-c21d-4ac0-beeb-c491710bfb72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9" Aug 04 15:12:58.922797 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:58.922467 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c6a911a-c21d-4ac0-beeb-c491710bfb72-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9\" (UID: \"5c6a911a-c21d-4ac0-beeb-c491710bfb72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9" Aug 04 15:12:58.922797 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:58.922482 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c6a911a-c21d-4ac0-beeb-c491710bfb72-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9\" (UID: \"5c6a911a-c21d-4ac0-beeb-c491710bfb72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9" Aug 04 15:12:58.922797 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:58.922533 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jh9g8\" (UniqueName: \"kubernetes.io/projected/5c6a911a-c21d-4ac0-beeb-c491710bfb72-kube-api-access-jh9g8\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9\" (UID: \"5c6a911a-c21d-4ac0-beeb-c491710bfb72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9" Aug 04 15:12:58.922797 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:58.922593 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c6a911a-c21d-4ac0-beeb-c491710bfb72-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9\" (UID: \"5c6a911a-c21d-4ac0-beeb-c491710bfb72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9" Aug 04 15:12:58.922797 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:58.922611 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c6a911a-c21d-4ac0-beeb-c491710bfb72-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9\" (UID: \"5c6a911a-c21d-4ac0-beeb-c491710bfb72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9" Aug 04 15:12:58.922797 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:58.922640 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c6a911a-c21d-4ac0-beeb-c491710bfb72-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9\" (UID: \"5c6a911a-c21d-4ac0-beeb-c491710bfb72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9" Aug 04 15:12:59.023567 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:59.023487 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c6a911a-c21d-4ac0-beeb-c491710bfb72-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9\" (UID: \"5c6a911a-c21d-4ac0-beeb-c491710bfb72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9" Aug 04 15:12:59.023567 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:59.023536 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c6a911a-c21d-4ac0-beeb-c491710bfb72-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9\" (UID: \"5c6a911a-c21d-4ac0-beeb-c491710bfb72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9" Aug 04 15:12:59.023785 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:59.023571 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c6a911a-c21d-4ac0-beeb-c491710bfb72-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9\" (UID: \"5c6a911a-c21d-4ac0-beeb-c491710bfb72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9" Aug 04 15:12:59.023785 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:59.023596 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c6a911a-c21d-4ac0-beeb-c491710bfb72-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9\" (UID: \"5c6a911a-c21d-4ac0-beeb-c491710bfb72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9" Aug 04 15:12:59.023785 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:59.023618 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c6a911a-c21d-4ac0-beeb-c491710bfb72-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9\" (UID: \"5c6a911a-c21d-4ac0-beeb-c491710bfb72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9" Aug 04 15:12:59.023785 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:59.023641 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c6a911a-c21d-4ac0-beeb-c491710bfb72-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9\" (UID: \"5c6a911a-c21d-4ac0-beeb-c491710bfb72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9" Aug 04 15:12:59.023785 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:59.023670 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jh9g8\" (UniqueName: \"kubernetes.io/projected/5c6a911a-c21d-4ac0-beeb-c491710bfb72-kube-api-access-jh9g8\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9\" (UID: \"5c6a911a-c21d-4ac0-beeb-c491710bfb72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9" Aug 04 15:12:59.023785 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:59.023734 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c6a911a-c21d-4ac0-beeb-c491710bfb72-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9\" (UID: \"5c6a911a-c21d-4ac0-beeb-c491710bfb72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9" Aug 04 15:12:59.023785 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:59.023760 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c6a911a-c21d-4ac0-beeb-c491710bfb72-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9\" (UID: \"5c6a911a-c21d-4ac0-beeb-c491710bfb72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9" Aug 04 15:12:59.024116 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:59.024055 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c6a911a-c21d-4ac0-beeb-c491710bfb72-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9\" (UID: \"5c6a911a-c21d-4ac0-beeb-c491710bfb72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9" Aug 04 15:12:59.024288 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:59.024263 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c6a911a-c21d-4ac0-beeb-c491710bfb72-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9\" (UID: \"5c6a911a-c21d-4ac0-beeb-c491710bfb72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9" Aug 04 15:12:59.024438 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:59.024285 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c6a911a-c21d-4ac0-beeb-c491710bfb72-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9\" (UID: \"5c6a911a-c21d-4ac0-beeb-c491710bfb72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9" Aug 04 15:12:59.024438 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:59.024429 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c6a911a-c21d-4ac0-beeb-c491710bfb72-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9\" (UID: \"5c6a911a-c21d-4ac0-beeb-c491710bfb72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9" Aug 04 15:12:59.024736 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:59.024712 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c6a911a-c21d-4ac0-beeb-c491710bfb72-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9\" (UID: \"5c6a911a-c21d-4ac0-beeb-c491710bfb72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9" Aug 04 15:12:59.026039 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:59.026019 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c6a911a-c21d-4ac0-beeb-c491710bfb72-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9\" (UID: \"5c6a911a-c21d-4ac0-beeb-c491710bfb72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9" Aug 04 15:12:59.026195 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:59.026177 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c6a911a-c21d-4ac0-beeb-c491710bfb72-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9\" (UID: \"5c6a911a-c21d-4ac0-beeb-c491710bfb72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9" Aug 04 15:12:59.040091 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:59.040071 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c6a911a-c21d-4ac0-beeb-c491710bfb72-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9\" (UID: \"5c6a911a-c21d-4ac0-beeb-c491710bfb72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9" Aug 04 15:12:59.043242 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:59.043213 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jh9g8\" (UniqueName: \"kubernetes.io/projected/5c6a911a-c21d-4ac0-beeb-c491710bfb72-kube-api-access-jh9g8\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9\" (UID: \"5c6a911a-c21d-4ac0-beeb-c491710bfb72\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9" Aug 04 15:12:59.046029 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:59.046004 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9" Aug 04 15:12:59.199377 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:59.199352 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9"] Aug 04 15:12:59.200940 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:12:59.200912 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c6a911a_c21d_4ac0_beeb_c491710bfb72.slice/crio-f6202b103f60d258db001a0adf0419cccaf54abaef472ac1cc43fa526d98986f WatchSource:0}: Error finding container f6202b103f60d258db001a0adf0419cccaf54abaef472ac1cc43fa526d98986f: Status 404 returned error can't find the container with id f6202b103f60d258db001a0adf0419cccaf54abaef472ac1cc43fa526d98986f Aug 04 15:12:59.754510 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:59.754464 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-w9wtz"] Aug 04 15:12:59.758953 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:59.758935 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w9wtz" Aug 04 15:12:59.763117 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:59.763095 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-zccp7\"" Aug 04 15:12:59.763340 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:59.763328 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Aug 04 15:12:59.782371 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:59.782348 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-tkrnj"] Aug 04 15:12:59.785356 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:59.785335 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tkrnj" Aug 04 15:12:59.788579 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:59.788560 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Aug 04 15:12:59.796244 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:59.796224 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-glpdb\"" Aug 04 15:12:59.796608 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:59.796591 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-w9wtz"] Aug 04 15:12:59.830729 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:59.830704 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-tkrnj"] Aug 04 15:12:59.854731 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:59.854705 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9" event={"ID":"5c6a911a-c21d-4ac0-beeb-c491710bfb72","Type":"ContainerStarted","Data":"f6202b103f60d258db001a0adf0419cccaf54abaef472ac1cc43fa526d98986f"} Aug 04 15:12:59.931502 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:59.931477 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/632ef889-6c6f-437f-9af4-ba9e3b00bf2a-cert\") pod \"kserve-controller-manager-d6f4b65b8-w9wtz\" (UID: \"632ef889-6c6f-437f-9af4-ba9e3b00bf2a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w9wtz" Aug 04 15:12:59.931847 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:59.931527 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbfbf\" (UniqueName: \"kubernetes.io/projected/b7992fa0-a315-4350-ba4c-0704718db979-kube-api-access-qbfbf\") pod \"llmisvc-controller-manager-777c6bd7d5-tkrnj\" (UID: \"b7992fa0-a315-4350-ba4c-0704718db979\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tkrnj" Aug 04 15:12:59.931847 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:59.931552 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b7992fa0-a315-4350-ba4c-0704718db979-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-tkrnj\" (UID: \"b7992fa0-a315-4350-ba4c-0704718db979\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tkrnj" Aug 04 15:12:59.931847 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:12:59.931576 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nnm5\" (UniqueName: \"kubernetes.io/projected/632ef889-6c6f-437f-9af4-ba9e3b00bf2a-kube-api-access-9nnm5\") pod \"kserve-controller-manager-d6f4b65b8-w9wtz\" (UID: \"632ef889-6c6f-437f-9af4-ba9e3b00bf2a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w9wtz" Aug 04 15:13:00.033033 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:00.032955 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbfbf\" (UniqueName: \"kubernetes.io/projected/b7992fa0-a315-4350-ba4c-0704718db979-kube-api-access-qbfbf\") pod \"llmisvc-controller-manager-777c6bd7d5-tkrnj\" (UID: \"b7992fa0-a315-4350-ba4c-0704718db979\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tkrnj" Aug 04 15:13:00.033033 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:00.033006 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b7992fa0-a315-4350-ba4c-0704718db979-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-tkrnj\" (UID: \"b7992fa0-a315-4350-ba4c-0704718db979\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tkrnj" Aug 04 15:13:00.033224 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:00.033041 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nnm5\" (UniqueName: \"kubernetes.io/projected/632ef889-6c6f-437f-9af4-ba9e3b00bf2a-kube-api-access-9nnm5\") pod \"kserve-controller-manager-d6f4b65b8-w9wtz\" (UID: \"632ef889-6c6f-437f-9af4-ba9e3b00bf2a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w9wtz" Aug 04 15:13:00.033224 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:00.033118 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/632ef889-6c6f-437f-9af4-ba9e3b00bf2a-cert\") pod \"kserve-controller-manager-d6f4b65b8-w9wtz\" (UID: \"632ef889-6c6f-437f-9af4-ba9e3b00bf2a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w9wtz" Aug 04 15:13:00.033224 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:13:00.033200 2574 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Aug 04 15:13:00.033371 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:13:00.033298 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7992fa0-a315-4350-ba4c-0704718db979-cert podName:b7992fa0-a315-4350-ba4c-0704718db979 nodeName:}" failed. No retries permitted until 2026-08-04 15:13:00.533275502 +0000 UTC m=+403.993147832 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b7992fa0-a315-4350-ba4c-0704718db979-cert") pod "llmisvc-controller-manager-777c6bd7d5-tkrnj" (UID: "b7992fa0-a315-4350-ba4c-0704718db979") : secret "llmisvc-webhook-server-cert" not found Aug 04 15:13:00.035554 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:00.035535 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/632ef889-6c6f-437f-9af4-ba9e3b00bf2a-cert\") pod \"kserve-controller-manager-d6f4b65b8-w9wtz\" (UID: \"632ef889-6c6f-437f-9af4-ba9e3b00bf2a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w9wtz" Aug 04 15:13:00.061868 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:00.061848 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbfbf\" (UniqueName: \"kubernetes.io/projected/b7992fa0-a315-4350-ba4c-0704718db979-kube-api-access-qbfbf\") pod \"llmisvc-controller-manager-777c6bd7d5-tkrnj\" (UID: \"b7992fa0-a315-4350-ba4c-0704718db979\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tkrnj" Aug 04 15:13:00.063781 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:00.063757 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nnm5\" (UniqueName: \"kubernetes.io/projected/632ef889-6c6f-437f-9af4-ba9e3b00bf2a-kube-api-access-9nnm5\") pod \"kserve-controller-manager-d6f4b65b8-w9wtz\" (UID: \"632ef889-6c6f-437f-9af4-ba9e3b00bf2a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w9wtz" Aug 04 15:13:00.069536 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:00.069515 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w9wtz" Aug 04 15:13:00.287585 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:13:00.287557 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod632ef889_6c6f_437f_9af4_ba9e3b00bf2a.slice/crio-ab33b0ed87f7dd3d3bf3cae1efb6b5db8f4beee03bc37c18d110a757cb6d120b WatchSource:0}: Error finding container ab33b0ed87f7dd3d3bf3cae1efb6b5db8f4beee03bc37c18d110a757cb6d120b: Status 404 returned error can't find the container with id ab33b0ed87f7dd3d3bf3cae1efb6b5db8f4beee03bc37c18d110a757cb6d120b Aug 04 15:13:00.313513 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:00.313486 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-w9wtz"] Aug 04 15:13:00.537941 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:00.537866 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b7992fa0-a315-4350-ba4c-0704718db979-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-tkrnj\" (UID: \"b7992fa0-a315-4350-ba4c-0704718db979\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tkrnj" Aug 04 15:13:00.540160 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:00.540133 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b7992fa0-a315-4350-ba4c-0704718db979-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-tkrnj\" (UID: \"b7992fa0-a315-4350-ba4c-0704718db979\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tkrnj" Aug 04 15:13:00.695515 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:00.695482 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tkrnj" Aug 04 15:13:00.839353 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:00.839327 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-tkrnj"] Aug 04 15:13:00.839872 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:13:00.839848 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb7992fa0_a315_4350_ba4c_0704718db979.slice/crio-46e8a7c3ea45369543a709c195c47c2f2b2c97e710c7c5c55e487262bf79e27e WatchSource:0}: Error finding container 46e8a7c3ea45369543a709c195c47c2f2b2c97e710c7c5c55e487262bf79e27e: Status 404 returned error can't find the container with id 46e8a7c3ea45369543a709c195c47c2f2b2c97e710c7c5c55e487262bf79e27e Aug 04 15:13:00.844777 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:00.844758 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xrm6z" Aug 04 15:13:00.860131 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:00.860103 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tkrnj" event={"ID":"b7992fa0-a315-4350-ba4c-0704718db979","Type":"ContainerStarted","Data":"46e8a7c3ea45369543a709c195c47c2f2b2c97e710c7c5c55e487262bf79e27e"} Aug 04 15:13:00.861231 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:00.861209 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w9wtz" event={"ID":"632ef889-6c6f-437f-9af4-ba9e3b00bf2a","Type":"ContainerStarted","Data":"ab33b0ed87f7dd3d3bf3cae1efb6b5db8f4beee03bc37c18d110a757cb6d120b"} Aug 04 15:13:00.935703 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:00.935662 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xrm6z"] Aug 04 15:13:00.936157 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:00.935847 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xrm6z" podUID="6e301f43-c965-491c-895b-78e5f2dfdfed" containerName="discovery" containerID="cri-o://ba7eb7906a946477bdb525f8d152fbcb598cc7298b83ae686e5dd78d4a77e20b" gracePeriod=30 Aug 04 15:13:01.200351 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:01.200322 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xrm6z" Aug 04 15:13:01.242405 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:01.242356 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6e301f43-c965-491c-895b-78e5f2dfdfed-istio-csr-ca-configmap\") pod \"6e301f43-c965-491c-895b-78e5f2dfdfed\" (UID: \"6e301f43-c965-491c-895b-78e5f2dfdfed\") " Aug 04 15:13:01.242548 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:01.242439 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6e301f43-c965-491c-895b-78e5f2dfdfed-istio-kubeconfig\") pod \"6e301f43-c965-491c-895b-78e5f2dfdfed\" (UID: \"6e301f43-c965-491c-895b-78e5f2dfdfed\") " Aug 04 15:13:01.242548 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:01.242464 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6e301f43-c965-491c-895b-78e5f2dfdfed-istio-csr-dns-cert\") pod \"6e301f43-c965-491c-895b-78e5f2dfdfed\" (UID: \"6e301f43-c965-491c-895b-78e5f2dfdfed\") " Aug 04 15:13:01.242548 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:01.242511 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e301f43-c965-491c-895b-78e5f2dfdfed-istio-token\") pod \"6e301f43-c965-491c-895b-78e5f2dfdfed\" (UID: \"6e301f43-c965-491c-895b-78e5f2dfdfed\") " Aug 04 15:13:01.242725 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:01.242555 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e301f43-c965-491c-895b-78e5f2dfdfed-local-certs\") pod \"6e301f43-c965-491c-895b-78e5f2dfdfed\" (UID: \"6e301f43-c965-491c-895b-78e5f2dfdfed\") " Aug 04 15:13:01.242776 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:01.242736 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e301f43-c965-491c-895b-78e5f2dfdfed-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "6e301f43-c965-491c-895b-78e5f2dfdfed" (UID: "6e301f43-c965-491c-895b-78e5f2dfdfed"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 15:13:01.242878 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:01.242836 2574 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6e301f43-c965-491c-895b-78e5f2dfdfed-istio-csr-ca-configmap\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:13:01.245198 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:01.245151 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e301f43-c965-491c-895b-78e5f2dfdfed-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "6e301f43-c965-491c-895b-78e5f2dfdfed" (UID: "6e301f43-c965-491c-895b-78e5f2dfdfed"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 15:13:01.245198 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:01.245166 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e301f43-c965-491c-895b-78e5f2dfdfed-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "6e301f43-c965-491c-895b-78e5f2dfdfed" (UID: "6e301f43-c965-491c-895b-78e5f2dfdfed"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 15:13:01.245351 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:01.245232 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e301f43-c965-491c-895b-78e5f2dfdfed-local-certs" (OuterVolumeSpecName: "local-certs") pod "6e301f43-c965-491c-895b-78e5f2dfdfed" (UID: "6e301f43-c965-491c-895b-78e5f2dfdfed"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:13:01.246222 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:01.246201 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e301f43-c965-491c-895b-78e5f2dfdfed-istio-token" (OuterVolumeSpecName: "istio-token") pod "6e301f43-c965-491c-895b-78e5f2dfdfed" (UID: "6e301f43-c965-491c-895b-78e5f2dfdfed"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 15:13:01.343235 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:01.343204 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7mf9n\" (UniqueName: \"kubernetes.io/projected/6e301f43-c965-491c-895b-78e5f2dfdfed-kube-api-access-7mf9n\") pod \"6e301f43-c965-491c-895b-78e5f2dfdfed\" (UID: \"6e301f43-c965-491c-895b-78e5f2dfdfed\") " Aug 04 15:13:01.343531 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:01.343323 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6e301f43-c965-491c-895b-78e5f2dfdfed-cacerts\") pod \"6e301f43-c965-491c-895b-78e5f2dfdfed\" (UID: \"6e301f43-c965-491c-895b-78e5f2dfdfed\") " Aug 04 15:13:01.343613 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:01.343556 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e301f43-c965-491c-895b-78e5f2dfdfed-istio-token\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:13:01.343613 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:01.343575 2574 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e301f43-c965-491c-895b-78e5f2dfdfed-local-certs\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:13:01.343613 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:01.343589 2574 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6e301f43-c965-491c-895b-78e5f2dfdfed-istio-kubeconfig\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:13:01.343613 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:01.343602 2574 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6e301f43-c965-491c-895b-78e5f2dfdfed-istio-csr-dns-cert\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:13:01.345845 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:01.345813 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e301f43-c965-491c-895b-78e5f2dfdfed-cacerts" (OuterVolumeSpecName: "cacerts") pod "6e301f43-c965-491c-895b-78e5f2dfdfed" (UID: "6e301f43-c965-491c-895b-78e5f2dfdfed"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 15:13:01.345993 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:01.345860 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e301f43-c965-491c-895b-78e5f2dfdfed-kube-api-access-7mf9n" (OuterVolumeSpecName: "kube-api-access-7mf9n") pod "6e301f43-c965-491c-895b-78e5f2dfdfed" (UID: "6e301f43-c965-491c-895b-78e5f2dfdfed"). InnerVolumeSpecName "kube-api-access-7mf9n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 15:13:01.444620 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:01.444529 2574 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6e301f43-c965-491c-895b-78e5f2dfdfed-cacerts\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:13:01.444620 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:01.444569 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mf9n\" (UniqueName: \"kubernetes.io/projected/6e301f43-c965-491c-895b-78e5f2dfdfed-kube-api-access-7mf9n\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:13:01.868339 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:01.868289 2574 generic.go:356] "Generic (PLEG): container finished" podID="6e301f43-c965-491c-895b-78e5f2dfdfed" containerID="ba7eb7906a946477bdb525f8d152fbcb598cc7298b83ae686e5dd78d4a77e20b" exitCode=0 Aug 04 15:13:01.868534 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:01.868356 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xrm6z" Aug 04 15:13:01.868534 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:01.868477 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xrm6z" event={"ID":"6e301f43-c965-491c-895b-78e5f2dfdfed","Type":"ContainerDied","Data":"ba7eb7906a946477bdb525f8d152fbcb598cc7298b83ae686e5dd78d4a77e20b"} Aug 04 15:13:01.868534 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:01.868521 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xrm6z" event={"ID":"6e301f43-c965-491c-895b-78e5f2dfdfed","Type":"ContainerDied","Data":"3d8699dba25bd10de84ece1fa4f43235c6425515d7a4d80ee6c3335f46fd504e"} Aug 04 15:13:01.868704 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:01.868541 2574 scope.go:117] "RemoveContainer" containerID="ba7eb7906a946477bdb525f8d152fbcb598cc7298b83ae686e5dd78d4a77e20b" Aug 04 15:13:01.891267 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:01.891242 2574 scope.go:117] "RemoveContainer" containerID="ba7eb7906a946477bdb525f8d152fbcb598cc7298b83ae686e5dd78d4a77e20b" Aug 04 15:13:01.891655 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:13:01.891593 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba7eb7906a946477bdb525f8d152fbcb598cc7298b83ae686e5dd78d4a77e20b\": container with ID starting with ba7eb7906a946477bdb525f8d152fbcb598cc7298b83ae686e5dd78d4a77e20b not found: ID does not exist" containerID="ba7eb7906a946477bdb525f8d152fbcb598cc7298b83ae686e5dd78d4a77e20b" Aug 04 15:13:01.891655 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:01.891630 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba7eb7906a946477bdb525f8d152fbcb598cc7298b83ae686e5dd78d4a77e20b"} err="failed to get container status \"ba7eb7906a946477bdb525f8d152fbcb598cc7298b83ae686e5dd78d4a77e20b\": rpc error: code = NotFound desc = could not find container \"ba7eb7906a946477bdb525f8d152fbcb598cc7298b83ae686e5dd78d4a77e20b\": container with ID starting with ba7eb7906a946477bdb525f8d152fbcb598cc7298b83ae686e5dd78d4a77e20b not found: ID does not exist" Aug 04 15:13:01.910771 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:01.910720 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xrm6z"] Aug 04 15:13:01.914984 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:01.914961 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-xrm6z"] Aug 04 15:13:03.054723 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:03.054688 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e301f43-c965-491c-895b-78e5f2dfdfed" path="/var/lib/kubelet/pods/6e301f43-c965-491c-895b-78e5f2dfdfed/volumes" Aug 04 15:13:03.476729 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:03.476687 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:13:03.476894 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:03.476777 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:13:03.476894 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:03.476820 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:13:03.817427 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:03.817379 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-q4t7p" Aug 04 15:13:03.884921 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:03.884797 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9" event={"ID":"5c6a911a-c21d-4ac0-beeb-c491710bfb72","Type":"ContainerStarted","Data":"cc5ce7418ef90d34928d57c82792ce9d67cab776803516cc1428bb4007dffe7e"} Aug 04 15:13:03.927816 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:03.927764 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9" podStartSLOduration=1.6541728660000001 podStartE2EDuration="5.927748204s" totalImagesPullingTime="4.273575338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:12:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:13:03.923830979 +0000 UTC m=+407.383703358" watchObservedRunningTime="2026-08-04 15:13:03.927748204 +0000 UTC m=+407.387620552" Aug 04 15:13:04.047034 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:04.046980 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9" Aug 04 15:13:04.048664 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:04.048627 2574 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Aug 04 15:13:04.048787 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:04.048680 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9" podUID="5c6a911a-c21d-4ac0-beeb-c491710bfb72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Aug 04 15:13:04.893013 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:04.892971 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tkrnj" event={"ID":"b7992fa0-a315-4350-ba4c-0704718db979","Type":"ContainerStarted","Data":"2183e3b1cca23bb8025586dba0c16ef0f187dd5d7abd8917f835bd776f3f1f34"} Aug 04 15:13:04.893515 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:04.893052 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tkrnj" Aug 04 15:13:04.921208 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:04.921160 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tkrnj" podStartSLOduration=2.388387701 podStartE2EDuration="5.921144371s" totalImagesPullingTime="3.53275667s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:12:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:13:04.920678037 +0000 UTC m=+408.380550387" watchObservedRunningTime="2026-08-04 15:13:04.921144371 +0000 UTC m=+408.381016774" Aug 04 15:13:05.053631 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:05.053603 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9" Aug 04 15:13:05.836927 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:05.836895 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-dnbrf" Aug 04 15:13:05.897829 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:05.897799 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9" Aug 04 15:13:05.898807 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:05.898782 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9" Aug 04 15:13:06.905129 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:06.905008 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w9wtz" event={"ID":"632ef889-6c6f-437f-9af4-ba9e3b00bf2a","Type":"ContainerStarted","Data":"a78312360f3125eedc21334ff7b3c5b51d50a01e272074753f27fb4f0b872d5b"} Aug 04 15:13:06.905595 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:06.905242 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w9wtz" Aug 04 15:13:06.946281 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:06.946227 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w9wtz" podStartSLOduration=1.684926945 podStartE2EDuration="7.946209083s" totalImagesPullingTime="6.261282138s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:12:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:13:06.946012765 +0000 UTC m=+410.405885113" watchObservedRunningTime="2026-08-04 15:13:06.946209083 +0000 UTC m=+410.406081432" Aug 04 15:13:35.899384 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:35.899353 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-tkrnj" Aug 04 15:13:37.915203 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:37.915170 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w9wtz" Aug 04 15:13:48.754633 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:48.754553 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qfrhm"] Aug 04 15:13:48.755099 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:48.754905 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e301f43-c965-491c-895b-78e5f2dfdfed" containerName="discovery" Aug 04 15:13:48.755099 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:48.754917 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e301f43-c965-491c-895b-78e5f2dfdfed" containerName="discovery" Aug 04 15:13:48.755099 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:48.754979 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e301f43-c965-491c-895b-78e5f2dfdfed" containerName="discovery" Aug 04 15:13:48.758141 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:48.758111 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qfrhm" Aug 04 15:13:48.761494 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:48.761470 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Aug 04 15:13:48.762687 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:48.762665 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Aug 04 15:13:48.762687 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:48.762675 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-lw5gj\"" Aug 04 15:13:48.773197 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:48.773172 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qfrhm"] Aug 04 15:13:48.841319 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:48.841283 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gj4c\" (UniqueName: \"kubernetes.io/projected/f4c3e5ef-d0ad-4315-aae1-2ddffb73d0d0-kube-api-access-5gj4c\") pod \"kuadrant-operator-catalog-qfrhm\" (UID: \"f4c3e5ef-d0ad-4315-aae1-2ddffb73d0d0\") " pod="kuadrant-system/kuadrant-operator-catalog-qfrhm" Aug 04 15:13:48.942670 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:48.942626 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gj4c\" (UniqueName: \"kubernetes.io/projected/f4c3e5ef-d0ad-4315-aae1-2ddffb73d0d0-kube-api-access-5gj4c\") pod \"kuadrant-operator-catalog-qfrhm\" (UID: \"f4c3e5ef-d0ad-4315-aae1-2ddffb73d0d0\") " pod="kuadrant-system/kuadrant-operator-catalog-qfrhm" Aug 04 15:13:48.957074 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:48.957049 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gj4c\" (UniqueName: \"kubernetes.io/projected/f4c3e5ef-d0ad-4315-aae1-2ddffb73d0d0-kube-api-access-5gj4c\") pod \"kuadrant-operator-catalog-qfrhm\" (UID: \"f4c3e5ef-d0ad-4315-aae1-2ddffb73d0d0\") " pod="kuadrant-system/kuadrant-operator-catalog-qfrhm" Aug 04 15:13:49.068678 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:49.068585 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qfrhm" Aug 04 15:13:49.130310 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:49.130207 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qfrhm"] Aug 04 15:13:49.247349 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:49.247324 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qfrhm"] Aug 04 15:13:49.249533 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:13:49.249512 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4c3e5ef_d0ad_4315_aae1_2ddffb73d0d0.slice/crio-7f3ae1048e117f20b82242745b326bc971d1a59e5d909afc4dbdd18ea22b9769 WatchSource:0}: Error finding container 7f3ae1048e117f20b82242745b326bc971d1a59e5d909afc4dbdd18ea22b9769: Status 404 returned error can't find the container with id 7f3ae1048e117f20b82242745b326bc971d1a59e5d909afc4dbdd18ea22b9769 Aug 04 15:13:49.337617 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:49.337537 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2vwn8"] Aug 04 15:13:49.342057 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:49.342040 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2vwn8" Aug 04 15:13:49.354434 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:49.354406 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2vwn8"] Aug 04 15:13:49.446769 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:49.446736 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gscmb\" (UniqueName: \"kubernetes.io/projected/07a74b29-3084-41c0-b79f-5848579687a8-kube-api-access-gscmb\") pod \"kuadrant-operator-catalog-2vwn8\" (UID: \"07a74b29-3084-41c0-b79f-5848579687a8\") " pod="kuadrant-system/kuadrant-operator-catalog-2vwn8" Aug 04 15:13:49.547735 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:49.547701 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gscmb\" (UniqueName: \"kubernetes.io/projected/07a74b29-3084-41c0-b79f-5848579687a8-kube-api-access-gscmb\") pod \"kuadrant-operator-catalog-2vwn8\" (UID: \"07a74b29-3084-41c0-b79f-5848579687a8\") " pod="kuadrant-system/kuadrant-operator-catalog-2vwn8" Aug 04 15:13:49.556317 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:49.556295 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gscmb\" (UniqueName: \"kubernetes.io/projected/07a74b29-3084-41c0-b79f-5848579687a8-kube-api-access-gscmb\") pod \"kuadrant-operator-catalog-2vwn8\" (UID: \"07a74b29-3084-41c0-b79f-5848579687a8\") " pod="kuadrant-system/kuadrant-operator-catalog-2vwn8" Aug 04 15:13:49.651540 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:49.651464 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2vwn8" Aug 04 15:13:49.777198 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:49.777175 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2vwn8"] Aug 04 15:13:49.779264 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:13:49.779239 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07a74b29_3084_41c0_b79f_5848579687a8.slice/crio-38ee5b2d958b64f435cd2e68fd6d4c823f8dce1e151d9d9c51c413686e8b2bf4 WatchSource:0}: Error finding container 38ee5b2d958b64f435cd2e68fd6d4c823f8dce1e151d9d9c51c413686e8b2bf4: Status 404 returned error can't find the container with id 38ee5b2d958b64f435cd2e68fd6d4c823f8dce1e151d9d9c51c413686e8b2bf4 Aug 04 15:13:50.102140 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:50.102101 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qfrhm" event={"ID":"f4c3e5ef-d0ad-4315-aae1-2ddffb73d0d0","Type":"ContainerStarted","Data":"7f3ae1048e117f20b82242745b326bc971d1a59e5d909afc4dbdd18ea22b9769"} Aug 04 15:13:50.103289 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:50.103256 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2vwn8" event={"ID":"07a74b29-3084-41c0-b79f-5848579687a8","Type":"ContainerStarted","Data":"38ee5b2d958b64f435cd2e68fd6d4c823f8dce1e151d9d9c51c413686e8b2bf4"} Aug 04 15:13:52.115686 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:52.115586 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2vwn8" event={"ID":"07a74b29-3084-41c0-b79f-5848579687a8","Type":"ContainerStarted","Data":"2b7d266407b9bf358794c349be516e835823c85912d5b4aec4d935afd1d6d49e"} Aug 04 15:13:52.116925 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:52.116903 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qfrhm" event={"ID":"f4c3e5ef-d0ad-4315-aae1-2ddffb73d0d0","Type":"ContainerStarted","Data":"dca67a9af5356829490957a200b818a086d2ab4643cf16f424ae2f4077b2c10e"} Aug 04 15:13:52.117021 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:52.116960 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-qfrhm" podUID="f4c3e5ef-d0ad-4315-aae1-2ddffb73d0d0" containerName="registry-server" containerID="cri-o://dca67a9af5356829490957a200b818a086d2ab4643cf16f424ae2f4077b2c10e" gracePeriod=2 Aug 04 15:13:52.136142 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:52.136098 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-2vwn8" podStartSLOduration=1.154594162 podStartE2EDuration="3.136083836s" totalImagesPullingTime="1.981489674s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:13:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:13:52.134157505 +0000 UTC m=+455.594029846" watchObservedRunningTime="2026-08-04 15:13:52.136083836 +0000 UTC m=+455.595956185" Aug 04 15:13:52.152605 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:52.152560 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-qfrhm" podStartSLOduration=1.637443591 podStartE2EDuration="4.152547397s" totalImagesPullingTime="2.515103806s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:13:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:13:52.150567439 +0000 UTC m=+455.610439809" watchObservedRunningTime="2026-08-04 15:13:52.152547397 +0000 UTC m=+455.612419785" Aug 04 15:13:52.359886 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:52.359863 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qfrhm" Aug 04 15:13:52.472961 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:52.472875 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5gj4c\" (UniqueName: \"kubernetes.io/projected/f4c3e5ef-d0ad-4315-aae1-2ddffb73d0d0-kube-api-access-5gj4c\") pod \"f4c3e5ef-d0ad-4315-aae1-2ddffb73d0d0\" (UID: \"f4c3e5ef-d0ad-4315-aae1-2ddffb73d0d0\") " Aug 04 15:13:52.474964 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:52.474941 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4c3e5ef-d0ad-4315-aae1-2ddffb73d0d0-kube-api-access-5gj4c" (OuterVolumeSpecName: "kube-api-access-5gj4c") pod "f4c3e5ef-d0ad-4315-aae1-2ddffb73d0d0" (UID: "f4c3e5ef-d0ad-4315-aae1-2ddffb73d0d0"). InnerVolumeSpecName "kube-api-access-5gj4c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 15:13:52.574119 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:52.574075 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5gj4c\" (UniqueName: \"kubernetes.io/projected/f4c3e5ef-d0ad-4315-aae1-2ddffb73d0d0-kube-api-access-5gj4c\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:13:53.122212 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:53.122178 2574 generic.go:356] "Generic (PLEG): container finished" podID="f4c3e5ef-d0ad-4315-aae1-2ddffb73d0d0" containerID="dca67a9af5356829490957a200b818a086d2ab4643cf16f424ae2f4077b2c10e" exitCode=0 Aug 04 15:13:53.122672 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:53.122221 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qfrhm" Aug 04 15:13:53.122672 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:53.122269 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qfrhm" event={"ID":"f4c3e5ef-d0ad-4315-aae1-2ddffb73d0d0","Type":"ContainerDied","Data":"dca67a9af5356829490957a200b818a086d2ab4643cf16f424ae2f4077b2c10e"} Aug 04 15:13:53.122672 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:53.122328 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qfrhm" event={"ID":"f4c3e5ef-d0ad-4315-aae1-2ddffb73d0d0","Type":"ContainerDied","Data":"7f3ae1048e117f20b82242745b326bc971d1a59e5d909afc4dbdd18ea22b9769"} Aug 04 15:13:53.122672 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:53.122361 2574 scope.go:117] "RemoveContainer" containerID="dca67a9af5356829490957a200b818a086d2ab4643cf16f424ae2f4077b2c10e" Aug 04 15:13:53.131646 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:53.131630 2574 scope.go:117] "RemoveContainer" containerID="dca67a9af5356829490957a200b818a086d2ab4643cf16f424ae2f4077b2c10e" Aug 04 15:13:53.131910 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:13:53.131891 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dca67a9af5356829490957a200b818a086d2ab4643cf16f424ae2f4077b2c10e\": container with ID starting with dca67a9af5356829490957a200b818a086d2ab4643cf16f424ae2f4077b2c10e not found: ID does not exist" containerID="dca67a9af5356829490957a200b818a086d2ab4643cf16f424ae2f4077b2c10e" Aug 04 15:13:53.131964 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:53.131917 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dca67a9af5356829490957a200b818a086d2ab4643cf16f424ae2f4077b2c10e"} err="failed to get container status \"dca67a9af5356829490957a200b818a086d2ab4643cf16f424ae2f4077b2c10e\": rpc error: code = NotFound desc = could not find container \"dca67a9af5356829490957a200b818a086d2ab4643cf16f424ae2f4077b2c10e\": container with ID starting with dca67a9af5356829490957a200b818a086d2ab4643cf16f424ae2f4077b2c10e not found: ID does not exist" Aug 04 15:13:53.142930 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:53.142908 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qfrhm"] Aug 04 15:13:53.150468 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:53.150449 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qfrhm"] Aug 04 15:13:55.053227 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:55.053196 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4c3e5ef-d0ad-4315-aae1-2ddffb73d0d0" path="/var/lib/kubelet/pods/f4c3e5ef-d0ad-4315-aae1-2ddffb73d0d0/volumes" Aug 04 15:13:59.652081 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:59.652043 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-2vwn8" Aug 04 15:13:59.652532 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:59.652105 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-2vwn8" Aug 04 15:13:59.674372 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:13:59.674342 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-2vwn8" Aug 04 15:14:00.181820 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:00.181792 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-2vwn8" Aug 04 15:14:04.157356 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:04.157318 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06lzhh"] Aug 04 15:14:04.157774 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:04.157700 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4c3e5ef-d0ad-4315-aae1-2ddffb73d0d0" containerName="registry-server" Aug 04 15:14:04.157774 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:04.157713 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4c3e5ef-d0ad-4315-aae1-2ddffb73d0d0" containerName="registry-server" Aug 04 15:14:04.157860 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:04.157785 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4c3e5ef-d0ad-4315-aae1-2ddffb73d0d0" containerName="registry-server" Aug 04 15:14:04.162331 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:04.162311 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06lzhh" Aug 04 15:14:04.165058 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:04.165037 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-g9sm8\"" Aug 04 15:14:04.170070 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:04.170048 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06lzhh"] Aug 04 15:14:04.261977 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:04.261947 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/04d06a27-7273-48eb-bc1b-fbe7ac3ff51e-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06lzhh\" (UID: \"04d06a27-7273-48eb-bc1b-fbe7ac3ff51e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06lzhh" Aug 04 15:14:04.262151 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:04.261987 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6fgm\" (UniqueName: \"kubernetes.io/projected/04d06a27-7273-48eb-bc1b-fbe7ac3ff51e-kube-api-access-t6fgm\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06lzhh\" (UID: \"04d06a27-7273-48eb-bc1b-fbe7ac3ff51e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06lzhh" Aug 04 15:14:04.262151 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:04.262021 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/04d06a27-7273-48eb-bc1b-fbe7ac3ff51e-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06lzhh\" (UID: \"04d06a27-7273-48eb-bc1b-fbe7ac3ff51e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06lzhh" Aug 04 15:14:04.362571 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:04.362523 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/04d06a27-7273-48eb-bc1b-fbe7ac3ff51e-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06lzhh\" (UID: \"04d06a27-7273-48eb-bc1b-fbe7ac3ff51e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06lzhh" Aug 04 15:14:04.362571 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:04.362578 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6fgm\" (UniqueName: \"kubernetes.io/projected/04d06a27-7273-48eb-bc1b-fbe7ac3ff51e-kube-api-access-t6fgm\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06lzhh\" (UID: \"04d06a27-7273-48eb-bc1b-fbe7ac3ff51e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06lzhh" Aug 04 15:14:04.362789 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:04.362627 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/04d06a27-7273-48eb-bc1b-fbe7ac3ff51e-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06lzhh\" (UID: \"04d06a27-7273-48eb-bc1b-fbe7ac3ff51e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06lzhh" Aug 04 15:14:04.362939 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:04.362918 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/04d06a27-7273-48eb-bc1b-fbe7ac3ff51e-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06lzhh\" (UID: \"04d06a27-7273-48eb-bc1b-fbe7ac3ff51e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06lzhh" Aug 04 15:14:04.362997 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:04.362977 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/04d06a27-7273-48eb-bc1b-fbe7ac3ff51e-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06lzhh\" (UID: \"04d06a27-7273-48eb-bc1b-fbe7ac3ff51e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06lzhh" Aug 04 15:14:04.372246 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:04.372221 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6fgm\" (UniqueName: \"kubernetes.io/projected/04d06a27-7273-48eb-bc1b-fbe7ac3ff51e-kube-api-access-t6fgm\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06lzhh\" (UID: \"04d06a27-7273-48eb-bc1b-fbe7ac3ff51e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06lzhh" Aug 04 15:14:04.472649 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:04.472560 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06lzhh" Aug 04 15:14:04.610712 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:04.610686 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06lzhh"] Aug 04 15:14:04.612159 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:14:04.612133 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04d06a27_7273_48eb_bc1b_fbe7ac3ff51e.slice/crio-175e11b955bd5d4ef770ff4834f382aaa2cc8994fbc889e378e19971ca194473 WatchSource:0}: Error finding container 175e11b955bd5d4ef770ff4834f382aaa2cc8994fbc889e378e19971ca194473: Status 404 returned error can't find the container with id 175e11b955bd5d4ef770ff4834f382aaa2cc8994fbc889e378e19971ca194473 Aug 04 15:14:04.758224 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:04.758189 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738s8t6"] Aug 04 15:14:04.761633 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:04.761617 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738s8t6" Aug 04 15:14:04.765578 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:04.765556 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jx7zm\" (UniqueName: \"kubernetes.io/projected/a6e10417-aaa7-426d-9d7a-8f946e766d35-kube-api-access-jx7zm\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738s8t6\" (UID: \"a6e10417-aaa7-426d-9d7a-8f946e766d35\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738s8t6" Aug 04 15:14:04.765669 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:04.765599 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a6e10417-aaa7-426d-9d7a-8f946e766d35-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738s8t6\" (UID: \"a6e10417-aaa7-426d-9d7a-8f946e766d35\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738s8t6" Aug 04 15:14:04.765717 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:04.765666 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a6e10417-aaa7-426d-9d7a-8f946e766d35-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738s8t6\" (UID: \"a6e10417-aaa7-426d-9d7a-8f946e766d35\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738s8t6" Aug 04 15:14:04.770448 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:04.770420 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738s8t6"] Aug 04 15:14:04.866964 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:04.866933 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jx7zm\" (UniqueName: \"kubernetes.io/projected/a6e10417-aaa7-426d-9d7a-8f946e766d35-kube-api-access-jx7zm\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738s8t6\" (UID: \"a6e10417-aaa7-426d-9d7a-8f946e766d35\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738s8t6" Aug 04 15:14:04.867135 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:04.866971 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a6e10417-aaa7-426d-9d7a-8f946e766d35-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738s8t6\" (UID: \"a6e10417-aaa7-426d-9d7a-8f946e766d35\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738s8t6" Aug 04 15:14:04.867135 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:04.867013 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a6e10417-aaa7-426d-9d7a-8f946e766d35-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738s8t6\" (UID: \"a6e10417-aaa7-426d-9d7a-8f946e766d35\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738s8t6" Aug 04 15:14:04.867435 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:04.867415 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a6e10417-aaa7-426d-9d7a-8f946e766d35-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738s8t6\" (UID: \"a6e10417-aaa7-426d-9d7a-8f946e766d35\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738s8t6" Aug 04 15:14:04.867435 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:04.867428 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a6e10417-aaa7-426d-9d7a-8f946e766d35-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738s8t6\" (UID: \"a6e10417-aaa7-426d-9d7a-8f946e766d35\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738s8t6" Aug 04 15:14:04.876085 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:04.876064 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jx7zm\" (UniqueName: \"kubernetes.io/projected/a6e10417-aaa7-426d-9d7a-8f946e766d35-kube-api-access-jx7zm\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738s8t6\" (UID: \"a6e10417-aaa7-426d-9d7a-8f946e766d35\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738s8t6" Aug 04 15:14:05.071620 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:05.071540 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738s8t6" Aug 04 15:14:05.166732 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:05.166695 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hsrn7"] Aug 04 15:14:05.172745 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:05.172642 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hsrn7" Aug 04 15:14:05.179531 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:05.179505 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hsrn7"] Aug 04 15:14:05.188054 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:05.188029 2574 generic.go:356] "Generic (PLEG): container finished" podID="04d06a27-7273-48eb-bc1b-fbe7ac3ff51e" containerID="654d9aa87c8eb22d5801470153927a35679ebdcb180e6e0d41a4d959a3fc753f" exitCode=0 Aug 04 15:14:05.188160 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:05.188107 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06lzhh" event={"ID":"04d06a27-7273-48eb-bc1b-fbe7ac3ff51e","Type":"ContainerDied","Data":"654d9aa87c8eb22d5801470153927a35679ebdcb180e6e0d41a4d959a3fc753f"} Aug 04 15:14:05.188160 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:05.188145 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06lzhh" event={"ID":"04d06a27-7273-48eb-bc1b-fbe7ac3ff51e","Type":"ContainerStarted","Data":"175e11b955bd5d4ef770ff4834f382aaa2cc8994fbc889e378e19971ca194473"} Aug 04 15:14:05.206283 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:05.206260 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738s8t6"] Aug 04 15:14:05.207954 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:14:05.207925 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6e10417_aaa7_426d_9d7a_8f946e766d35.slice/crio-f37650fbc54e4fe4db666136e45c350ef6eb6b43ec49ed0fb710484455797c87 WatchSource:0}: Error finding container f37650fbc54e4fe4db666136e45c350ef6eb6b43ec49ed0fb710484455797c87: Status 404 returned error can't find the container with id f37650fbc54e4fe4db666136e45c350ef6eb6b43ec49ed0fb710484455797c87 Aug 04 15:14:05.270126 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:05.270100 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjn5l\" (UniqueName: \"kubernetes.io/projected/034b04b9-45c8-4ee1-b3cf-98754b6c93d0-kube-api-access-jjn5l\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hsrn7\" (UID: \"034b04b9-45c8-4ee1-b3cf-98754b6c93d0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hsrn7" Aug 04 15:14:05.270238 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:05.270193 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/034b04b9-45c8-4ee1-b3cf-98754b6c93d0-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hsrn7\" (UID: \"034b04b9-45c8-4ee1-b3cf-98754b6c93d0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hsrn7" Aug 04 15:14:05.270304 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:05.270281 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/034b04b9-45c8-4ee1-b3cf-98754b6c93d0-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hsrn7\" (UID: \"034b04b9-45c8-4ee1-b3cf-98754b6c93d0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hsrn7" Aug 04 15:14:05.371479 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:05.371407 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/034b04b9-45c8-4ee1-b3cf-98754b6c93d0-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hsrn7\" (UID: \"034b04b9-45c8-4ee1-b3cf-98754b6c93d0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hsrn7" Aug 04 15:14:05.371610 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:05.371499 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjn5l\" (UniqueName: \"kubernetes.io/projected/034b04b9-45c8-4ee1-b3cf-98754b6c93d0-kube-api-access-jjn5l\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hsrn7\" (UID: \"034b04b9-45c8-4ee1-b3cf-98754b6c93d0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hsrn7" Aug 04 15:14:05.371610 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:05.371539 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/034b04b9-45c8-4ee1-b3cf-98754b6c93d0-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hsrn7\" (UID: \"034b04b9-45c8-4ee1-b3cf-98754b6c93d0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hsrn7" Aug 04 15:14:05.371745 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:05.371723 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/034b04b9-45c8-4ee1-b3cf-98754b6c93d0-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hsrn7\" (UID: \"034b04b9-45c8-4ee1-b3cf-98754b6c93d0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hsrn7" Aug 04 15:14:05.371821 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:05.371787 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/034b04b9-45c8-4ee1-b3cf-98754b6c93d0-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hsrn7\" (UID: \"034b04b9-45c8-4ee1-b3cf-98754b6c93d0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hsrn7" Aug 04 15:14:05.380929 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:05.380901 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjn5l\" (UniqueName: \"kubernetes.io/projected/034b04b9-45c8-4ee1-b3cf-98754b6c93d0-kube-api-access-jjn5l\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hsrn7\" (UID: \"034b04b9-45c8-4ee1-b3cf-98754b6c93d0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hsrn7" Aug 04 15:14:05.485057 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:05.485021 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hsrn7" Aug 04 15:14:05.558901 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:05.558870 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lxrwj"] Aug 04 15:14:05.563485 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:05.563464 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lxrwj" Aug 04 15:14:05.573111 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:05.572847 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/53d8340a-04de-4a94-9865-a6259ed37066-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lxrwj\" (UID: \"53d8340a-04de-4a94-9865-a6259ed37066\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lxrwj" Aug 04 15:14:05.573111 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:05.572955 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jdkb\" (UniqueName: \"kubernetes.io/projected/53d8340a-04de-4a94-9865-a6259ed37066-kube-api-access-6jdkb\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lxrwj\" (UID: \"53d8340a-04de-4a94-9865-a6259ed37066\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lxrwj" Aug 04 15:14:05.573111 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:05.572986 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/53d8340a-04de-4a94-9865-a6259ed37066-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lxrwj\" (UID: \"53d8340a-04de-4a94-9865-a6259ed37066\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lxrwj" Aug 04 15:14:05.574823 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:05.574796 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lxrwj"] Aug 04 15:14:05.621888 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:05.621826 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hsrn7"] Aug 04 15:14:05.623130 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:14:05.623107 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod034b04b9_45c8_4ee1_b3cf_98754b6c93d0.slice/crio-d0af850b1db2bd7dedb0df4c22a8cab376315ccfc32e6bfd28018959fa783bce WatchSource:0}: Error finding container d0af850b1db2bd7dedb0df4c22a8cab376315ccfc32e6bfd28018959fa783bce: Status 404 returned error can't find the container with id d0af850b1db2bd7dedb0df4c22a8cab376315ccfc32e6bfd28018959fa783bce Aug 04 15:14:05.674174 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:05.674144 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/53d8340a-04de-4a94-9865-a6259ed37066-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lxrwj\" (UID: \"53d8340a-04de-4a94-9865-a6259ed37066\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lxrwj" Aug 04 15:14:05.674279 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:05.674202 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/53d8340a-04de-4a94-9865-a6259ed37066-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lxrwj\" (UID: \"53d8340a-04de-4a94-9865-a6259ed37066\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lxrwj" Aug 04 15:14:05.674279 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:05.674257 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jdkb\" (UniqueName: \"kubernetes.io/projected/53d8340a-04de-4a94-9865-a6259ed37066-kube-api-access-6jdkb\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lxrwj\" (UID: \"53d8340a-04de-4a94-9865-a6259ed37066\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lxrwj" Aug 04 15:14:05.674588 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:05.674566 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/53d8340a-04de-4a94-9865-a6259ed37066-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lxrwj\" (UID: \"53d8340a-04de-4a94-9865-a6259ed37066\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lxrwj" Aug 04 15:14:05.674658 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:05.674619 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/53d8340a-04de-4a94-9865-a6259ed37066-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lxrwj\" (UID: \"53d8340a-04de-4a94-9865-a6259ed37066\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lxrwj" Aug 04 15:14:05.682838 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:05.682795 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jdkb\" (UniqueName: \"kubernetes.io/projected/53d8340a-04de-4a94-9865-a6259ed37066-kube-api-access-6jdkb\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lxrwj\" (UID: \"53d8340a-04de-4a94-9865-a6259ed37066\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lxrwj" Aug 04 15:14:05.875374 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:05.875291 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lxrwj" Aug 04 15:14:06.003627 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:06.003601 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lxrwj"] Aug 04 15:14:06.005235 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:14:06.005203 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53d8340a_04de_4a94_9865_a6259ed37066.slice/crio-44ccfbd762751df4b4253acd896ec7629ac3d459a5e23fd3caabb24a971f1a75 WatchSource:0}: Error finding container 44ccfbd762751df4b4253acd896ec7629ac3d459a5e23fd3caabb24a971f1a75: Status 404 returned error can't find the container with id 44ccfbd762751df4b4253acd896ec7629ac3d459a5e23fd3caabb24a971f1a75 Aug 04 15:14:06.195042 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:06.194971 2574 generic.go:356] "Generic (PLEG): container finished" podID="034b04b9-45c8-4ee1-b3cf-98754b6c93d0" containerID="dc281e3ab182531c3b21178a8f4e373d5cc06f5a8625156195ca604b221a571c" exitCode=0 Aug 04 15:14:06.195462 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:06.195068 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hsrn7" event={"ID":"034b04b9-45c8-4ee1-b3cf-98754b6c93d0","Type":"ContainerDied","Data":"dc281e3ab182531c3b21178a8f4e373d5cc06f5a8625156195ca604b221a571c"} Aug 04 15:14:06.195462 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:06.195126 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hsrn7" event={"ID":"034b04b9-45c8-4ee1-b3cf-98754b6c93d0","Type":"ContainerStarted","Data":"d0af850b1db2bd7dedb0df4c22a8cab376315ccfc32e6bfd28018959fa783bce"} Aug 04 15:14:06.196723 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:06.196699 2574 generic.go:356] "Generic (PLEG): container finished" podID="a6e10417-aaa7-426d-9d7a-8f946e766d35" containerID="34b30f09e15eb5cb458f4143e39a851e38ebf6bcd2e1c9d375c12f35fe03c364" exitCode=0 Aug 04 15:14:06.196831 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:06.196786 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738s8t6" event={"ID":"a6e10417-aaa7-426d-9d7a-8f946e766d35","Type":"ContainerDied","Data":"34b30f09e15eb5cb458f4143e39a851e38ebf6bcd2e1c9d375c12f35fe03c364"} Aug 04 15:14:06.196889 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:06.196836 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738s8t6" event={"ID":"a6e10417-aaa7-426d-9d7a-8f946e766d35","Type":"ContainerStarted","Data":"f37650fbc54e4fe4db666136e45c350ef6eb6b43ec49ed0fb710484455797c87"} Aug 04 15:14:06.201875 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:06.201772 2574 generic.go:356] "Generic (PLEG): container finished" podID="53d8340a-04de-4a94-9865-a6259ed37066" containerID="da5064fbcf974c2062841e9cc16d979846e0c8c36f8c1d1ba9a1f1353499c26c" exitCode=0 Aug 04 15:14:06.201875 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:06.201811 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lxrwj" event={"ID":"53d8340a-04de-4a94-9865-a6259ed37066","Type":"ContainerDied","Data":"da5064fbcf974c2062841e9cc16d979846e0c8c36f8c1d1ba9a1f1353499c26c"} Aug 04 15:14:06.201875 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:06.201849 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lxrwj" event={"ID":"53d8340a-04de-4a94-9865-a6259ed37066","Type":"ContainerStarted","Data":"44ccfbd762751df4b4253acd896ec7629ac3d459a5e23fd3caabb24a971f1a75"} Aug 04 15:14:07.208762 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:07.208731 2574 generic.go:356] "Generic (PLEG): container finished" podID="04d06a27-7273-48eb-bc1b-fbe7ac3ff51e" containerID="f4e3881cd9fb0b31f6542c690810eb248b4c4aa66286b7fe43d3061b3aa6f751" exitCode=0 Aug 04 15:14:07.209110 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:07.208819 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06lzhh" event={"ID":"04d06a27-7273-48eb-bc1b-fbe7ac3ff51e","Type":"ContainerDied","Data":"f4e3881cd9fb0b31f6542c690810eb248b4c4aa66286b7fe43d3061b3aa6f751"} Aug 04 15:14:08.221082 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:08.221044 2574 generic.go:356] "Generic (PLEG): container finished" podID="53d8340a-04de-4a94-9865-a6259ed37066" containerID="2838595fa52881353189a41d8a1967de484d02ac485b199e17059564b53dad5e" exitCode=0 Aug 04 15:14:08.221557 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:08.221130 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lxrwj" event={"ID":"53d8340a-04de-4a94-9865-a6259ed37066","Type":"ContainerDied","Data":"2838595fa52881353189a41d8a1967de484d02ac485b199e17059564b53dad5e"} Aug 04 15:14:08.223592 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:08.223567 2574 generic.go:356] "Generic (PLEG): container finished" podID="04d06a27-7273-48eb-bc1b-fbe7ac3ff51e" containerID="7b7a3d1c96613301ee29ffa4adfac4dd689bfb62f0f601f7b4ce692f0c92efe0" exitCode=0 Aug 04 15:14:08.223683 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:08.223651 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06lzhh" event={"ID":"04d06a27-7273-48eb-bc1b-fbe7ac3ff51e","Type":"ContainerDied","Data":"7b7a3d1c96613301ee29ffa4adfac4dd689bfb62f0f601f7b4ce692f0c92efe0"} Aug 04 15:14:08.225491 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:08.225471 2574 generic.go:356] "Generic (PLEG): container finished" podID="034b04b9-45c8-4ee1-b3cf-98754b6c93d0" containerID="980a376f4c399f0136fcb1fc7cc1e3c5174dcbd2d7abaf74c628a8971530c223" exitCode=0 Aug 04 15:14:08.225578 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:08.225547 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hsrn7" event={"ID":"034b04b9-45c8-4ee1-b3cf-98754b6c93d0","Type":"ContainerDied","Data":"980a376f4c399f0136fcb1fc7cc1e3c5174dcbd2d7abaf74c628a8971530c223"} Aug 04 15:14:08.227435 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:08.227415 2574 generic.go:356] "Generic (PLEG): container finished" podID="a6e10417-aaa7-426d-9d7a-8f946e766d35" containerID="e55611ad3bc3944fbbb0d05c8123ca14c83a853bac3645e35641f6326e45af6c" exitCode=0 Aug 04 15:14:08.227577 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:08.227446 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738s8t6" event={"ID":"a6e10417-aaa7-426d-9d7a-8f946e766d35","Type":"ContainerDied","Data":"e55611ad3bc3944fbbb0d05c8123ca14c83a853bac3645e35641f6326e45af6c"} Aug 04 15:14:09.234870 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:09.234831 2574 generic.go:356] "Generic (PLEG): container finished" podID="53d8340a-04de-4a94-9865-a6259ed37066" containerID="bb15d7b23de04ee1ead4a0d351dcf0c6b5f416d94493b4340f302a60949ba0dc" exitCode=0 Aug 04 15:14:09.235321 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:09.234920 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lxrwj" event={"ID":"53d8340a-04de-4a94-9865-a6259ed37066","Type":"ContainerDied","Data":"bb15d7b23de04ee1ead4a0d351dcf0c6b5f416d94493b4340f302a60949ba0dc"} Aug 04 15:14:09.237044 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:09.237020 2574 generic.go:356] "Generic (PLEG): container finished" podID="034b04b9-45c8-4ee1-b3cf-98754b6c93d0" containerID="3244d3bbab731226cafe920068bf0e95b645a2864b099295474752c3d9d663bd" exitCode=0 Aug 04 15:14:09.237183 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:09.237108 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hsrn7" event={"ID":"034b04b9-45c8-4ee1-b3cf-98754b6c93d0","Type":"ContainerDied","Data":"3244d3bbab731226cafe920068bf0e95b645a2864b099295474752c3d9d663bd"} Aug 04 15:14:09.239139 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:09.239119 2574 generic.go:356] "Generic (PLEG): container finished" podID="a6e10417-aaa7-426d-9d7a-8f946e766d35" containerID="619519ff3a4a9cc7e22a9e0d7f20fced04b1dbd8c7ce696575689a7bf4a137a4" exitCode=0 Aug 04 15:14:09.239226 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:09.239186 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738s8t6" event={"ID":"a6e10417-aaa7-426d-9d7a-8f946e766d35","Type":"ContainerDied","Data":"619519ff3a4a9cc7e22a9e0d7f20fced04b1dbd8c7ce696575689a7bf4a137a4"} Aug 04 15:14:09.361520 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:09.361496 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06lzhh" Aug 04 15:14:09.407250 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:09.407220 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/04d06a27-7273-48eb-bc1b-fbe7ac3ff51e-util\") pod \"04d06a27-7273-48eb-bc1b-fbe7ac3ff51e\" (UID: \"04d06a27-7273-48eb-bc1b-fbe7ac3ff51e\") " Aug 04 15:14:09.407380 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:09.407284 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/04d06a27-7273-48eb-bc1b-fbe7ac3ff51e-bundle\") pod \"04d06a27-7273-48eb-bc1b-fbe7ac3ff51e\" (UID: \"04d06a27-7273-48eb-bc1b-fbe7ac3ff51e\") " Aug 04 15:14:09.407380 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:09.407319 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6fgm\" (UniqueName: \"kubernetes.io/projected/04d06a27-7273-48eb-bc1b-fbe7ac3ff51e-kube-api-access-t6fgm\") pod \"04d06a27-7273-48eb-bc1b-fbe7ac3ff51e\" (UID: \"04d06a27-7273-48eb-bc1b-fbe7ac3ff51e\") " Aug 04 15:14:09.407752 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:09.407726 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04d06a27-7273-48eb-bc1b-fbe7ac3ff51e-bundle" (OuterVolumeSpecName: "bundle") pod "04d06a27-7273-48eb-bc1b-fbe7ac3ff51e" (UID: "04d06a27-7273-48eb-bc1b-fbe7ac3ff51e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:14:09.409422 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:09.409371 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04d06a27-7273-48eb-bc1b-fbe7ac3ff51e-kube-api-access-t6fgm" (OuterVolumeSpecName: "kube-api-access-t6fgm") pod "04d06a27-7273-48eb-bc1b-fbe7ac3ff51e" (UID: "04d06a27-7273-48eb-bc1b-fbe7ac3ff51e"). InnerVolumeSpecName "kube-api-access-t6fgm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 15:14:09.413330 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:09.413311 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04d06a27-7273-48eb-bc1b-fbe7ac3ff51e-util" (OuterVolumeSpecName: "util") pod "04d06a27-7273-48eb-bc1b-fbe7ac3ff51e" (UID: "04d06a27-7273-48eb-bc1b-fbe7ac3ff51e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:14:09.508182 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:09.508113 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/04d06a27-7273-48eb-bc1b-fbe7ac3ff51e-util\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:14:09.508182 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:09.508138 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/04d06a27-7273-48eb-bc1b-fbe7ac3ff51e-bundle\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:14:09.508182 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:09.508148 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6fgm\" (UniqueName: \"kubernetes.io/projected/04d06a27-7273-48eb-bc1b-fbe7ac3ff51e-kube-api-access-t6fgm\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:14:10.246124 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:10.246096 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06lzhh" Aug 04 15:14:10.246590 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:10.246129 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06lzhh" event={"ID":"04d06a27-7273-48eb-bc1b-fbe7ac3ff51e","Type":"ContainerDied","Data":"175e11b955bd5d4ef770ff4834f382aaa2cc8994fbc889e378e19971ca194473"} Aug 04 15:14:10.246590 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:10.246174 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="175e11b955bd5d4ef770ff4834f382aaa2cc8994fbc889e378e19971ca194473" Aug 04 15:14:10.416351 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:10.416328 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hsrn7" Aug 04 15:14:10.442948 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:10.442924 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lxrwj" Aug 04 15:14:10.445290 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:10.445270 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738s8t6" Aug 04 15:14:10.517221 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:10.517153 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6jdkb\" (UniqueName: \"kubernetes.io/projected/53d8340a-04de-4a94-9865-a6259ed37066-kube-api-access-6jdkb\") pod \"53d8340a-04de-4a94-9865-a6259ed37066\" (UID: \"53d8340a-04de-4a94-9865-a6259ed37066\") " Aug 04 15:14:10.517221 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:10.517188 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/034b04b9-45c8-4ee1-b3cf-98754b6c93d0-bundle\") pod \"034b04b9-45c8-4ee1-b3cf-98754b6c93d0\" (UID: \"034b04b9-45c8-4ee1-b3cf-98754b6c93d0\") " Aug 04 15:14:10.517221 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:10.517211 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/034b04b9-45c8-4ee1-b3cf-98754b6c93d0-util\") pod \"034b04b9-45c8-4ee1-b3cf-98754b6c93d0\" (UID: \"034b04b9-45c8-4ee1-b3cf-98754b6c93d0\") " Aug 04 15:14:10.517485 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:10.517244 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a6e10417-aaa7-426d-9d7a-8f946e766d35-bundle\") pod \"a6e10417-aaa7-426d-9d7a-8f946e766d35\" (UID: \"a6e10417-aaa7-426d-9d7a-8f946e766d35\") " Aug 04 15:14:10.517485 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:10.517267 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/53d8340a-04de-4a94-9865-a6259ed37066-util\") pod \"53d8340a-04de-4a94-9865-a6259ed37066\" (UID: \"53d8340a-04de-4a94-9865-a6259ed37066\") " Aug 04 15:14:10.517485 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:10.517305 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a6e10417-aaa7-426d-9d7a-8f946e766d35-util\") pod \"a6e10417-aaa7-426d-9d7a-8f946e766d35\" (UID: \"a6e10417-aaa7-426d-9d7a-8f946e766d35\") " Aug 04 15:14:10.517485 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:10.517338 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/53d8340a-04de-4a94-9865-a6259ed37066-bundle\") pod \"53d8340a-04de-4a94-9865-a6259ed37066\" (UID: \"53d8340a-04de-4a94-9865-a6259ed37066\") " Aug 04 15:14:10.517485 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:10.517371 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jjn5l\" (UniqueName: \"kubernetes.io/projected/034b04b9-45c8-4ee1-b3cf-98754b6c93d0-kube-api-access-jjn5l\") pod \"034b04b9-45c8-4ee1-b3cf-98754b6c93d0\" (UID: \"034b04b9-45c8-4ee1-b3cf-98754b6c93d0\") " Aug 04 15:14:10.517485 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:10.517463 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jx7zm\" (UniqueName: \"kubernetes.io/projected/a6e10417-aaa7-426d-9d7a-8f946e766d35-kube-api-access-jx7zm\") pod \"a6e10417-aaa7-426d-9d7a-8f946e766d35\" (UID: \"a6e10417-aaa7-426d-9d7a-8f946e766d35\") " Aug 04 15:14:10.517967 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:10.517897 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/034b04b9-45c8-4ee1-b3cf-98754b6c93d0-bundle" (OuterVolumeSpecName: "bundle") pod "034b04b9-45c8-4ee1-b3cf-98754b6c93d0" (UID: "034b04b9-45c8-4ee1-b3cf-98754b6c93d0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:14:10.517967 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:10.517918 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53d8340a-04de-4a94-9865-a6259ed37066-bundle" (OuterVolumeSpecName: "bundle") pod "53d8340a-04de-4a94-9865-a6259ed37066" (UID: "53d8340a-04de-4a94-9865-a6259ed37066"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:14:10.518287 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:10.518261 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6e10417-aaa7-426d-9d7a-8f946e766d35-bundle" (OuterVolumeSpecName: "bundle") pod "a6e10417-aaa7-426d-9d7a-8f946e766d35" (UID: "a6e10417-aaa7-426d-9d7a-8f946e766d35"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:14:10.519453 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:10.519426 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53d8340a-04de-4a94-9865-a6259ed37066-kube-api-access-6jdkb" (OuterVolumeSpecName: "kube-api-access-6jdkb") pod "53d8340a-04de-4a94-9865-a6259ed37066" (UID: "53d8340a-04de-4a94-9865-a6259ed37066"). InnerVolumeSpecName "kube-api-access-6jdkb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 15:14:10.519987 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:10.519956 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6e10417-aaa7-426d-9d7a-8f946e766d35-kube-api-access-jx7zm" (OuterVolumeSpecName: "kube-api-access-jx7zm") pod "a6e10417-aaa7-426d-9d7a-8f946e766d35" (UID: "a6e10417-aaa7-426d-9d7a-8f946e766d35"). InnerVolumeSpecName "kube-api-access-jx7zm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 15:14:10.520075 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:10.519985 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/034b04b9-45c8-4ee1-b3cf-98754b6c93d0-kube-api-access-jjn5l" (OuterVolumeSpecName: "kube-api-access-jjn5l") pod "034b04b9-45c8-4ee1-b3cf-98754b6c93d0" (UID: "034b04b9-45c8-4ee1-b3cf-98754b6c93d0"). InnerVolumeSpecName "kube-api-access-jjn5l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 15:14:10.523558 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:10.523535 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53d8340a-04de-4a94-9865-a6259ed37066-util" (OuterVolumeSpecName: "util") pod "53d8340a-04de-4a94-9865-a6259ed37066" (UID: "53d8340a-04de-4a94-9865-a6259ed37066"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:14:10.524211 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:10.524192 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/034b04b9-45c8-4ee1-b3cf-98754b6c93d0-util" (OuterVolumeSpecName: "util") pod "034b04b9-45c8-4ee1-b3cf-98754b6c93d0" (UID: "034b04b9-45c8-4ee1-b3cf-98754b6c93d0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:14:10.535720 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:10.535697 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6e10417-aaa7-426d-9d7a-8f946e766d35-util" (OuterVolumeSpecName: "util") pod "a6e10417-aaa7-426d-9d7a-8f946e766d35" (UID: "a6e10417-aaa7-426d-9d7a-8f946e766d35"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:14:10.618983 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:10.618956 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6jdkb\" (UniqueName: \"kubernetes.io/projected/53d8340a-04de-4a94-9865-a6259ed37066-kube-api-access-6jdkb\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:14:10.618983 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:10.618982 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/034b04b9-45c8-4ee1-b3cf-98754b6c93d0-bundle\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:14:10.619132 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:10.618992 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/034b04b9-45c8-4ee1-b3cf-98754b6c93d0-util\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:14:10.619132 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:10.619000 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a6e10417-aaa7-426d-9d7a-8f946e766d35-bundle\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:14:10.619132 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:10.619007 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/53d8340a-04de-4a94-9865-a6259ed37066-util\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:14:10.619132 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:10.619015 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a6e10417-aaa7-426d-9d7a-8f946e766d35-util\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:14:10.619132 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:10.619022 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/53d8340a-04de-4a94-9865-a6259ed37066-bundle\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:14:10.619132 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:10.619029 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjn5l\" (UniqueName: \"kubernetes.io/projected/034b04b9-45c8-4ee1-b3cf-98754b6c93d0-kube-api-access-jjn5l\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:14:10.619132 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:10.619038 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jx7zm\" (UniqueName: \"kubernetes.io/projected/a6e10417-aaa7-426d-9d7a-8f946e766d35-kube-api-access-jx7zm\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:14:11.252788 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:11.252756 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738s8t6" Aug 04 15:14:11.253203 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:11.252788 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738s8t6" event={"ID":"a6e10417-aaa7-426d-9d7a-8f946e766d35","Type":"ContainerDied","Data":"f37650fbc54e4fe4db666136e45c350ef6eb6b43ec49ed0fb710484455797c87"} Aug 04 15:14:11.253203 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:11.252845 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f37650fbc54e4fe4db666136e45c350ef6eb6b43ec49ed0fb710484455797c87" Aug 04 15:14:11.254739 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:11.254713 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lxrwj" Aug 04 15:14:11.254739 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:11.254726 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lxrwj" event={"ID":"53d8340a-04de-4a94-9865-a6259ed37066","Type":"ContainerDied","Data":"44ccfbd762751df4b4253acd896ec7629ac3d459a5e23fd3caabb24a971f1a75"} Aug 04 15:14:11.254923 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:11.254768 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="44ccfbd762751df4b4253acd896ec7629ac3d459a5e23fd3caabb24a971f1a75" Aug 04 15:14:11.256910 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:11.256879 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hsrn7" event={"ID":"034b04b9-45c8-4ee1-b3cf-98754b6c93d0","Type":"ContainerDied","Data":"d0af850b1db2bd7dedb0df4c22a8cab376315ccfc32e6bfd28018959fa783bce"} Aug 04 15:14:11.257015 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:11.256915 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d0af850b1db2bd7dedb0df4c22a8cab376315ccfc32e6bfd28018959fa783bce" Aug 04 15:14:11.257084 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:11.257058 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hsrn7" Aug 04 15:14:21.258863 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:21.258819 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-qvxnw"] Aug 04 15:14:21.259358 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:21.259300 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53d8340a-04de-4a94-9865-a6259ed37066" containerName="extract" Aug 04 15:14:21.259358 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:21.259320 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="53d8340a-04de-4a94-9865-a6259ed37066" containerName="extract" Aug 04 15:14:21.259358 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:21.259336 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53d8340a-04de-4a94-9865-a6259ed37066" containerName="util" Aug 04 15:14:21.259358 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:21.259345 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="53d8340a-04de-4a94-9865-a6259ed37066" containerName="util" Aug 04 15:14:21.259358 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:21.259358 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6e10417-aaa7-426d-9d7a-8f946e766d35" containerName="util" Aug 04 15:14:21.259663 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:21.259367 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6e10417-aaa7-426d-9d7a-8f946e766d35" containerName="util" Aug 04 15:14:21.259663 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:21.259379 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6e10417-aaa7-426d-9d7a-8f946e766d35" containerName="extract" Aug 04 15:14:21.259663 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:21.259405 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6e10417-aaa7-426d-9d7a-8f946e766d35" containerName="extract" Aug 04 15:14:21.259663 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:21.259426 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04d06a27-7273-48eb-bc1b-fbe7ac3ff51e" containerName="pull" Aug 04 15:14:21.259663 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:21.259434 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="04d06a27-7273-48eb-bc1b-fbe7ac3ff51e" containerName="pull" Aug 04 15:14:21.259663 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:21.259449 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="034b04b9-45c8-4ee1-b3cf-98754b6c93d0" containerName="util" Aug 04 15:14:21.259663 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:21.259456 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="034b04b9-45c8-4ee1-b3cf-98754b6c93d0" containerName="util" Aug 04 15:14:21.259663 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:21.259466 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6e10417-aaa7-426d-9d7a-8f946e766d35" containerName="pull" Aug 04 15:14:21.259663 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:21.259474 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6e10417-aaa7-426d-9d7a-8f946e766d35" containerName="pull" Aug 04 15:14:21.259663 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:21.259483 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="034b04b9-45c8-4ee1-b3cf-98754b6c93d0" containerName="pull" Aug 04 15:14:21.259663 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:21.259492 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="034b04b9-45c8-4ee1-b3cf-98754b6c93d0" containerName="pull" Aug 04 15:14:21.259663 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:21.259503 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53d8340a-04de-4a94-9865-a6259ed37066" containerName="pull" Aug 04 15:14:21.259663 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:21.259511 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="53d8340a-04de-4a94-9865-a6259ed37066" containerName="pull" Aug 04 15:14:21.259663 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:21.259522 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04d06a27-7273-48eb-bc1b-fbe7ac3ff51e" containerName="extract" Aug 04 15:14:21.259663 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:21.259532 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="04d06a27-7273-48eb-bc1b-fbe7ac3ff51e" containerName="extract" Aug 04 15:14:21.259663 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:21.259546 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="034b04b9-45c8-4ee1-b3cf-98754b6c93d0" containerName="extract" Aug 04 15:14:21.259663 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:21.259554 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="034b04b9-45c8-4ee1-b3cf-98754b6c93d0" containerName="extract" Aug 04 15:14:21.259663 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:21.259573 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04d06a27-7273-48eb-bc1b-fbe7ac3ff51e" containerName="util" Aug 04 15:14:21.259663 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:21.259581 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="04d06a27-7273-48eb-bc1b-fbe7ac3ff51e" containerName="util" Aug 04 15:14:21.259663 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:21.259668 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="53d8340a-04de-4a94-9865-a6259ed37066" containerName="extract" Aug 04 15:14:21.260577 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:21.259684 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6e10417-aaa7-426d-9d7a-8f946e766d35" containerName="extract" Aug 04 15:14:21.260577 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:21.259698 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="034b04b9-45c8-4ee1-b3cf-98754b6c93d0" containerName="extract" Aug 04 15:14:21.260577 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:21.259707 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="04d06a27-7273-48eb-bc1b-fbe7ac3ff51e" containerName="extract" Aug 04 15:14:21.264478 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:21.264457 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qvxnw" Aug 04 15:14:21.271495 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:21.271470 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-ffpjg\"" Aug 04 15:14:21.282890 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:21.282866 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-qvxnw"] Aug 04 15:14:21.309532 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:21.309507 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8k4q\" (UniqueName: \"kubernetes.io/projected/00bbf14a-0e93-4b87-b93f-a894364f6ede-kube-api-access-m8k4q\") pod \"authorino-operator-75cc8d5cb7-qvxnw\" (UID: \"00bbf14a-0e93-4b87-b93f-a894364f6ede\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-qvxnw" Aug 04 15:14:21.410948 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:21.410915 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8k4q\" (UniqueName: \"kubernetes.io/projected/00bbf14a-0e93-4b87-b93f-a894364f6ede-kube-api-access-m8k4q\") pod \"authorino-operator-75cc8d5cb7-qvxnw\" (UID: \"00bbf14a-0e93-4b87-b93f-a894364f6ede\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-qvxnw" Aug 04 15:14:21.422343 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:21.422319 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8k4q\" (UniqueName: \"kubernetes.io/projected/00bbf14a-0e93-4b87-b93f-a894364f6ede-kube-api-access-m8k4q\") pod \"authorino-operator-75cc8d5cb7-qvxnw\" (UID: \"00bbf14a-0e93-4b87-b93f-a894364f6ede\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-qvxnw" Aug 04 15:14:21.574299 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:21.574206 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qvxnw" Aug 04 15:14:21.711329 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:21.711295 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-qvxnw"] Aug 04 15:14:21.712907 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:14:21.712877 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00bbf14a_0e93_4b87_b93f_a894364f6ede.slice/crio-ac8bcdae96517f46ee756ee61f9baa2718102101b54d3af0087ba43d04331143 WatchSource:0}: Error finding container ac8bcdae96517f46ee756ee61f9baa2718102101b54d3af0087ba43d04331143: Status 404 returned error can't find the container with id ac8bcdae96517f46ee756ee61f9baa2718102101b54d3af0087ba43d04331143 Aug 04 15:14:22.313422 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:22.313373 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qvxnw" event={"ID":"00bbf14a-0e93-4b87-b93f-a894364f6ede","Type":"ContainerStarted","Data":"ac8bcdae96517f46ee756ee61f9baa2718102101b54d3af0087ba43d04331143"} Aug 04 15:14:23.945243 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:23.945223 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5gwhg"] Aug 04 15:14:23.952117 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:23.948698 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5gwhg" Aug 04 15:14:23.952117 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:23.951667 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-jjhcb\"" Aug 04 15:14:23.964499 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:23.964476 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5gwhg"] Aug 04 15:14:24.034656 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:24.034623 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9krt\" (UniqueName: \"kubernetes.io/projected/1c196aa9-0c6c-4935-a05d-1ecb5a84cdc8-kube-api-access-l9krt\") pod \"limitador-operator-controller-manager-859d7b78bd-5gwhg\" (UID: \"1c196aa9-0c6c-4935-a05d-1ecb5a84cdc8\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5gwhg" Aug 04 15:14:24.135804 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:24.135774 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9krt\" (UniqueName: \"kubernetes.io/projected/1c196aa9-0c6c-4935-a05d-1ecb5a84cdc8-kube-api-access-l9krt\") pod \"limitador-operator-controller-manager-859d7b78bd-5gwhg\" (UID: \"1c196aa9-0c6c-4935-a05d-1ecb5a84cdc8\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5gwhg" Aug 04 15:14:24.152288 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:24.152262 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9krt\" (UniqueName: \"kubernetes.io/projected/1c196aa9-0c6c-4935-a05d-1ecb5a84cdc8-kube-api-access-l9krt\") pod \"limitador-operator-controller-manager-859d7b78bd-5gwhg\" (UID: \"1c196aa9-0c6c-4935-a05d-1ecb5a84cdc8\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5gwhg" Aug 04 15:14:24.269448 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:24.269418 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5gwhg" Aug 04 15:14:24.327676 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:24.327600 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qvxnw" event={"ID":"00bbf14a-0e93-4b87-b93f-a894364f6ede","Type":"ContainerStarted","Data":"2dc5fb28017ae27137adecde97f0176932457a0af6e504b617a0d99b528b7d3d"} Aug 04 15:14:24.327676 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:24.327677 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qvxnw" Aug 04 15:14:24.357319 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:24.357265 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qvxnw" podStartSLOduration=1.133711435 podStartE2EDuration="3.357249866s" totalImagesPullingTime="2.223538431s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:14:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:14:24.355311543 +0000 UTC m=+487.815183893" watchObservedRunningTime="2026-08-04 15:14:24.357249866 +0000 UTC m=+487.817122214" Aug 04 15:14:24.410454 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:24.410431 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5gwhg"] Aug 04 15:14:24.411821 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:14:24.411796 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c196aa9_0c6c_4935_a05d_1ecb5a84cdc8.slice/crio-411f89f66d5be9331af1b301c6b188148b9de7e844266a0120df01c3d19bfef5 WatchSource:0}: Error finding container 411f89f66d5be9331af1b301c6b188148b9de7e844266a0120df01c3d19bfef5: Status 404 returned error can't find the container with id 411f89f66d5be9331af1b301c6b188148b9de7e844266a0120df01c3d19bfef5 Aug 04 15:14:25.336589 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:25.336476 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5gwhg" event={"ID":"1c196aa9-0c6c-4935-a05d-1ecb5a84cdc8","Type":"ContainerStarted","Data":"411f89f66d5be9331af1b301c6b188148b9de7e844266a0120df01c3d19bfef5"} Aug 04 15:14:26.343467 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:26.343427 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5gwhg" event={"ID":"1c196aa9-0c6c-4935-a05d-1ecb5a84cdc8","Type":"ContainerStarted","Data":"ec6c4f89f9e0e9b2b9c2ba6833ecf7ef9cbbb4677a42320a116f5291ff4a83cf"} Aug 04 15:14:26.343854 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:26.343491 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5gwhg" Aug 04 15:14:26.371132 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:26.371086 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5gwhg" podStartSLOduration=1.660524241 podStartE2EDuration="3.37107157s" totalImagesPullingTime="1.710547329s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:14:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:14:26.369762837 +0000 UTC m=+489.829635187" watchObservedRunningTime="2026-08-04 15:14:26.37107157 +0000 UTC m=+489.830943919" Aug 04 15:14:29.913191 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:29.913162 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-vczk2"] Aug 04 15:14:29.916647 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:29.916627 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-vczk2" Aug 04 15:14:29.919336 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:29.919315 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Aug 04 15:14:29.919452 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:29.919360 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-bvx2f\"" Aug 04 15:14:29.931972 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:29.931942 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-vczk2"] Aug 04 15:14:29.986288 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:29.986260 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nk759\" (UniqueName: \"kubernetes.io/projected/5d0b639a-d4fe-4409-aa4a-5a4a8db4e889-kube-api-access-nk759\") pod \"dns-operator-controller-manager-864dd78c4c-vczk2\" (UID: \"5d0b639a-d4fe-4409-aa4a-5a4a8db4e889\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-vczk2" Aug 04 15:14:30.087540 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:30.087512 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nk759\" (UniqueName: \"kubernetes.io/projected/5d0b639a-d4fe-4409-aa4a-5a4a8db4e889-kube-api-access-nk759\") pod \"dns-operator-controller-manager-864dd78c4c-vczk2\" (UID: \"5d0b639a-d4fe-4409-aa4a-5a4a8db4e889\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-vczk2" Aug 04 15:14:30.098886 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:30.098858 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nk759\" (UniqueName: \"kubernetes.io/projected/5d0b639a-d4fe-4409-aa4a-5a4a8db4e889-kube-api-access-nk759\") pod \"dns-operator-controller-manager-864dd78c4c-vczk2\" (UID: \"5d0b639a-d4fe-4409-aa4a-5a4a8db4e889\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-vczk2" Aug 04 15:14:30.226213 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:30.226128 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-vczk2" Aug 04 15:14:30.360738 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:30.360598 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-vczk2"] Aug 04 15:14:31.374219 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:31.374181 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-vczk2" event={"ID":"5d0b639a-d4fe-4409-aa4a-5a4a8db4e889","Type":"ContainerStarted","Data":"7ff7df086c9d21323e9401d626060f95815d2e7d93588ad9e97afc650b7571bb"} Aug 04 15:14:33.387147 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:33.387109 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-vczk2" event={"ID":"5d0b639a-d4fe-4409-aa4a-5a4a8db4e889","Type":"ContainerStarted","Data":"f1c287b1210fcd1b7149d1851d9c307ff07c57835dfe09c37a69a9b9d8f36b00"} Aug 04 15:14:33.387538 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:33.387172 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-vczk2" Aug 04 15:14:33.407680 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:33.407624 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-vczk2" podStartSLOduration=2.07500467 podStartE2EDuration="4.407609578s" totalImagesPullingTime="2.332604908s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:14:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:14:33.407485417 +0000 UTC m=+496.867357767" watchObservedRunningTime="2026-08-04 15:14:33.407609578 +0000 UTC m=+496.867481927" Aug 04 15:14:35.339309 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:35.339281 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-qvxnw" Aug 04 15:14:37.349934 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:37.349905 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5gwhg" Aug 04 15:14:37.478676 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:37.478646 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-sp4x9"] Aug 04 15:14:37.482323 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:37.482301 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sp4x9" Aug 04 15:14:37.485931 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:37.485904 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Aug 04 15:14:37.486067 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:37.485985 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-g9sm8\"" Aug 04 15:14:37.486067 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:37.486050 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Aug 04 15:14:37.495974 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:37.495938 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-sp4x9"] Aug 04 15:14:37.556225 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:37.556195 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bdacdb71-39a3-4cb7-9c61-8c5844764a09-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-sp4x9\" (UID: \"bdacdb71-39a3-4cb7-9c61-8c5844764a09\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sp4x9" Aug 04 15:14:37.556375 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:37.556244 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5657\" (UniqueName: \"kubernetes.io/projected/bdacdb71-39a3-4cb7-9c61-8c5844764a09-kube-api-access-w5657\") pod \"kuadrant-console-plugin-66b5774985-sp4x9\" (UID: \"bdacdb71-39a3-4cb7-9c61-8c5844764a09\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sp4x9" Aug 04 15:14:37.556375 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:37.556324 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bdacdb71-39a3-4cb7-9c61-8c5844764a09-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-sp4x9\" (UID: \"bdacdb71-39a3-4cb7-9c61-8c5844764a09\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sp4x9" Aug 04 15:14:37.657522 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:37.657444 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5657\" (UniqueName: \"kubernetes.io/projected/bdacdb71-39a3-4cb7-9c61-8c5844764a09-kube-api-access-w5657\") pod \"kuadrant-console-plugin-66b5774985-sp4x9\" (UID: \"bdacdb71-39a3-4cb7-9c61-8c5844764a09\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sp4x9" Aug 04 15:14:37.657522 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:37.657508 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bdacdb71-39a3-4cb7-9c61-8c5844764a09-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-sp4x9\" (UID: \"bdacdb71-39a3-4cb7-9c61-8c5844764a09\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sp4x9" Aug 04 15:14:37.657725 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:37.657573 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bdacdb71-39a3-4cb7-9c61-8c5844764a09-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-sp4x9\" (UID: \"bdacdb71-39a3-4cb7-9c61-8c5844764a09\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sp4x9" Aug 04 15:14:37.657725 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:14:37.657590 2574 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Aug 04 15:14:37.657725 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:14:37.657646 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdacdb71-39a3-4cb7-9c61-8c5844764a09-plugin-serving-cert podName:bdacdb71-39a3-4cb7-9c61-8c5844764a09 nodeName:}" failed. No retries permitted until 2026-08-04 15:14:38.157630708 +0000 UTC m=+501.617503035 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/bdacdb71-39a3-4cb7-9c61-8c5844764a09-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-sp4x9" (UID: "bdacdb71-39a3-4cb7-9c61-8c5844764a09") : secret "plugin-serving-cert" not found Aug 04 15:14:37.658322 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:37.658301 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bdacdb71-39a3-4cb7-9c61-8c5844764a09-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-sp4x9\" (UID: \"bdacdb71-39a3-4cb7-9c61-8c5844764a09\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sp4x9" Aug 04 15:14:37.677536 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:37.677512 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5657\" (UniqueName: \"kubernetes.io/projected/bdacdb71-39a3-4cb7-9c61-8c5844764a09-kube-api-access-w5657\") pod \"kuadrant-console-plugin-66b5774985-sp4x9\" (UID: \"bdacdb71-39a3-4cb7-9c61-8c5844764a09\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sp4x9" Aug 04 15:14:38.160853 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:38.160817 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bdacdb71-39a3-4cb7-9c61-8c5844764a09-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-sp4x9\" (UID: \"bdacdb71-39a3-4cb7-9c61-8c5844764a09\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sp4x9" Aug 04 15:14:38.163192 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:38.163170 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bdacdb71-39a3-4cb7-9c61-8c5844764a09-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-sp4x9\" (UID: \"bdacdb71-39a3-4cb7-9c61-8c5844764a09\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sp4x9" Aug 04 15:14:38.391472 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:38.391439 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sp4x9" Aug 04 15:14:38.541782 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:38.541754 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-sp4x9"] Aug 04 15:14:38.543102 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:14:38.543069 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbdacdb71_39a3_4cb7_9c61_8c5844764a09.slice/crio-5d6df519f11bdfe43449c990a7a7d9c5c4c7a9f0b909ed69afb164d0107a2394 WatchSource:0}: Error finding container 5d6df519f11bdfe43449c990a7a7d9c5c4c7a9f0b909ed69afb164d0107a2394: Status 404 returned error can't find the container with id 5d6df519f11bdfe43449c990a7a7d9c5c4c7a9f0b909ed69afb164d0107a2394 Aug 04 15:14:39.419622 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:39.419588 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sp4x9" event={"ID":"bdacdb71-39a3-4cb7-9c61-8c5844764a09","Type":"ContainerStarted","Data":"5d6df519f11bdfe43449c990a7a7d9c5c4c7a9f0b909ed69afb164d0107a2394"} Aug 04 15:14:44.394844 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:44.394812 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-vczk2" Aug 04 15:14:48.159166 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:48.159133 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5gwhg"] Aug 04 15:14:48.159740 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:48.159324 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5gwhg" podUID="1c196aa9-0c6c-4935-a05d-1ecb5a84cdc8" containerName="manager" containerID="cri-o://ec6c4f89f9e0e9b2b9c2ba6833ecf7ef9cbbb4677a42320a116f5291ff4a83cf" gracePeriod=2 Aug 04 15:14:48.166670 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:48.166637 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5gwhg"] Aug 04 15:14:48.203268 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:48.203236 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p7g8l"] Aug 04 15:14:48.203629 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:48.203615 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c196aa9-0c6c-4935-a05d-1ecb5a84cdc8" containerName="manager" Aug 04 15:14:48.203629 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:48.203629 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c196aa9-0c6c-4935-a05d-1ecb5a84cdc8" containerName="manager" Aug 04 15:14:48.203755 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:48.203687 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c196aa9-0c6c-4935-a05d-1ecb5a84cdc8" containerName="manager" Aug 04 15:14:48.206691 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:48.206671 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p7g8l" Aug 04 15:14:48.209800 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:14:48.209772 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-5gwhg\" is forbidden: User \"system:node:ip-10-0-132-155.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-155.ec2.internal' and this object" podUID="1c196aa9-0c6c-4935-a05d-1ecb5a84cdc8" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5gwhg" Aug 04 15:14:48.223009 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:48.222986 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p7g8l"] Aug 04 15:14:48.357627 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:48.356349 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lz25q\" (UniqueName: \"kubernetes.io/projected/f9fc1877-be08-44d4-b5fc-5b5349718121-kube-api-access-lz25q\") pod \"limitador-operator-controller-manager-859d7b78bd-p7g8l\" (UID: \"f9fc1877-be08-44d4-b5fc-5b5349718121\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p7g8l" Aug 04 15:14:48.410163 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:48.410110 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5gwhg" Aug 04 15:14:48.412613 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:14:48.412587 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-5gwhg\" is forbidden: User \"system:node:ip-10-0-132-155.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-155.ec2.internal' and this object" podUID="1c196aa9-0c6c-4935-a05d-1ecb5a84cdc8" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5gwhg" Aug 04 15:14:48.457031 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:48.456997 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz25q\" (UniqueName: \"kubernetes.io/projected/f9fc1877-be08-44d4-b5fc-5b5349718121-kube-api-access-lz25q\") pod \"limitador-operator-controller-manager-859d7b78bd-p7g8l\" (UID: \"f9fc1877-be08-44d4-b5fc-5b5349718121\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p7g8l" Aug 04 15:14:48.470763 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:48.470735 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lz25q\" (UniqueName: \"kubernetes.io/projected/f9fc1877-be08-44d4-b5fc-5b5349718121-kube-api-access-lz25q\") pod \"limitador-operator-controller-manager-859d7b78bd-p7g8l\" (UID: \"f9fc1877-be08-44d4-b5fc-5b5349718121\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p7g8l" Aug 04 15:14:48.480663 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:48.480641 2574 generic.go:356] "Generic (PLEG): container finished" podID="1c196aa9-0c6c-4935-a05d-1ecb5a84cdc8" containerID="ec6c4f89f9e0e9b2b9c2ba6833ecf7ef9cbbb4677a42320a116f5291ff4a83cf" exitCode=0 Aug 04 15:14:48.480759 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:48.480678 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5gwhg" Aug 04 15:14:48.480759 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:48.480695 2574 scope.go:117] "RemoveContainer" containerID="ec6c4f89f9e0e9b2b9c2ba6833ecf7ef9cbbb4677a42320a116f5291ff4a83cf" Aug 04 15:14:48.483042 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:14:48.483012 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-5gwhg\" is forbidden: User \"system:node:ip-10-0-132-155.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-155.ec2.internal' and this object" podUID="1c196aa9-0c6c-4935-a05d-1ecb5a84cdc8" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5gwhg" Aug 04 15:14:48.489210 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:48.489195 2574 scope.go:117] "RemoveContainer" containerID="ec6c4f89f9e0e9b2b9c2ba6833ecf7ef9cbbb4677a42320a116f5291ff4a83cf" Aug 04 15:14:48.489479 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:14:48.489461 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec6c4f89f9e0e9b2b9c2ba6833ecf7ef9cbbb4677a42320a116f5291ff4a83cf\": container with ID starting with ec6c4f89f9e0e9b2b9c2ba6833ecf7ef9cbbb4677a42320a116f5291ff4a83cf not found: ID does not exist" containerID="ec6c4f89f9e0e9b2b9c2ba6833ecf7ef9cbbb4677a42320a116f5291ff4a83cf" Aug 04 15:14:48.489522 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:48.489487 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec6c4f89f9e0e9b2b9c2ba6833ecf7ef9cbbb4677a42320a116f5291ff4a83cf"} err="failed to get container status \"ec6c4f89f9e0e9b2b9c2ba6833ecf7ef9cbbb4677a42320a116f5291ff4a83cf\": rpc error: code = NotFound desc = could not find container \"ec6c4f89f9e0e9b2b9c2ba6833ecf7ef9cbbb4677a42320a116f5291ff4a83cf\": container with ID starting with ec6c4f89f9e0e9b2b9c2ba6833ecf7ef9cbbb4677a42320a116f5291ff4a83cf not found: ID does not exist" Aug 04 15:14:48.533311 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:48.533285 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p7g8l" Aug 04 15:14:48.557598 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:48.557575 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l9krt\" (UniqueName: \"kubernetes.io/projected/1c196aa9-0c6c-4935-a05d-1ecb5a84cdc8-kube-api-access-l9krt\") pod \"1c196aa9-0c6c-4935-a05d-1ecb5a84cdc8\" (UID: \"1c196aa9-0c6c-4935-a05d-1ecb5a84cdc8\") " Aug 04 15:14:48.559671 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:48.559646 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c196aa9-0c6c-4935-a05d-1ecb5a84cdc8-kube-api-access-l9krt" (OuterVolumeSpecName: "kube-api-access-l9krt") pod "1c196aa9-0c6c-4935-a05d-1ecb5a84cdc8" (UID: "1c196aa9-0c6c-4935-a05d-1ecb5a84cdc8"). InnerVolumeSpecName "kube-api-access-l9krt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 15:14:48.658792 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:48.658757 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l9krt\" (UniqueName: \"kubernetes.io/projected/1c196aa9-0c6c-4935-a05d-1ecb5a84cdc8-kube-api-access-l9krt\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:14:48.672585 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:48.672561 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p7g8l"] Aug 04 15:14:48.673635 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:14:48.673611 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9fc1877_be08_44d4_b5fc_5b5349718121.slice/crio-15dee0a7a144af73a62dc903ff5425979708d690f5c7b874232ef8cc8a5f6398 WatchSource:0}: Error finding container 15dee0a7a144af73a62dc903ff5425979708d690f5c7b874232ef8cc8a5f6398: Status 404 returned error can't find the container with id 15dee0a7a144af73a62dc903ff5425979708d690f5c7b874232ef8cc8a5f6398 Aug 04 15:14:48.793956 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:14:48.793799 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-5gwhg\" is forbidden: User \"system:node:ip-10-0-132-155.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-155.ec2.internal' and this object" podUID="1c196aa9-0c6c-4935-a05d-1ecb5a84cdc8" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5gwhg" Aug 04 15:14:49.053401 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:49.053360 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c196aa9-0c6c-4935-a05d-1ecb5a84cdc8" path="/var/lib/kubelet/pods/1c196aa9-0c6c-4935-a05d-1ecb5a84cdc8/volumes" Aug 04 15:14:49.490457 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:49.490421 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p7g8l" event={"ID":"f9fc1877-be08-44d4-b5fc-5b5349718121","Type":"ContainerStarted","Data":"617fcbedfe97c78119ac919d5bd2d5267503ca36d35301e4509ccd238f5445d1"} Aug 04 15:14:49.490903 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:49.490484 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p7g8l" event={"ID":"f9fc1877-be08-44d4-b5fc-5b5349718121","Type":"ContainerStarted","Data":"15dee0a7a144af73a62dc903ff5425979708d690f5c7b874232ef8cc8a5f6398"} Aug 04 15:14:49.490903 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:49.490643 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p7g8l" Aug 04 15:14:49.509188 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:14:49.509144 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p7g8l" podStartSLOduration=1.509130781 podStartE2EDuration="1.509130781s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:14:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:14:49.50767514 +0000 UTC m=+512.967547490" watchObservedRunningTime="2026-08-04 15:14:49.509130781 +0000 UTC m=+512.969003108" Aug 04 15:15:00.499987 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:00.499942 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-p7g8l" Aug 04 15:15:04.582245 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:04.582210 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sp4x9" event={"ID":"bdacdb71-39a3-4cb7-9c61-8c5844764a09","Type":"ContainerStarted","Data":"83c4c78534456d3fbbb197ed482f8ec95acf3439e80c1b869a2d9b2c37b7bc12"} Aug 04 15:15:04.600407 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:04.600348 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sp4x9" podStartSLOduration=2.090149643 podStartE2EDuration="27.600333065s" totalImagesPullingTime="25.510183422s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:14:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:15:04.599176617 +0000 UTC m=+528.059048967" watchObservedRunningTime="2026-08-04 15:15:04.600333065 +0000 UTC m=+528.060205414" Aug 04 15:15:21.849996 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:21.849955 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn"] Aug 04 15:15:21.868637 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:21.868600 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" Aug 04 15:15:21.872065 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:21.872043 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-c9dxh\"" Aug 04 15:15:21.880875 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:21.880846 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn"] Aug 04 15:15:21.946097 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:21.946066 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dd49fc0-1d34-4825-a841-249a773b23ae-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-27rnn\" (UID: \"6dd49fc0-1d34-4825-a841-249a773b23ae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" Aug 04 15:15:21.946252 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:21.946100 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltfqs\" (UniqueName: \"kubernetes.io/projected/6dd49fc0-1d34-4825-a841-249a773b23ae-kube-api-access-ltfqs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-27rnn\" (UID: \"6dd49fc0-1d34-4825-a841-249a773b23ae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" Aug 04 15:15:21.946252 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:21.946132 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6dd49fc0-1d34-4825-a841-249a773b23ae-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-27rnn\" (UID: \"6dd49fc0-1d34-4825-a841-249a773b23ae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" Aug 04 15:15:21.946252 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:21.946196 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dd49fc0-1d34-4825-a841-249a773b23ae-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-27rnn\" (UID: \"6dd49fc0-1d34-4825-a841-249a773b23ae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" Aug 04 15:15:21.946252 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:21.946240 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dd49fc0-1d34-4825-a841-249a773b23ae-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-27rnn\" (UID: \"6dd49fc0-1d34-4825-a841-249a773b23ae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" Aug 04 15:15:21.946522 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:21.946276 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6dd49fc0-1d34-4825-a841-249a773b23ae-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-27rnn\" (UID: \"6dd49fc0-1d34-4825-a841-249a773b23ae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" Aug 04 15:15:21.946522 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:21.946304 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dd49fc0-1d34-4825-a841-249a773b23ae-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-27rnn\" (UID: \"6dd49fc0-1d34-4825-a841-249a773b23ae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" Aug 04 15:15:21.946522 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:21.946357 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6dd49fc0-1d34-4825-a841-249a773b23ae-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-27rnn\" (UID: \"6dd49fc0-1d34-4825-a841-249a773b23ae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" Aug 04 15:15:21.946522 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:21.946456 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6dd49fc0-1d34-4825-a841-249a773b23ae-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-27rnn\" (UID: \"6dd49fc0-1d34-4825-a841-249a773b23ae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" Aug 04 15:15:22.047093 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:22.047059 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6dd49fc0-1d34-4825-a841-249a773b23ae-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-27rnn\" (UID: \"6dd49fc0-1d34-4825-a841-249a773b23ae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" Aug 04 15:15:22.047276 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:22.047108 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6dd49fc0-1d34-4825-a841-249a773b23ae-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-27rnn\" (UID: \"6dd49fc0-1d34-4825-a841-249a773b23ae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" Aug 04 15:15:22.047276 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:22.047147 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dd49fc0-1d34-4825-a841-249a773b23ae-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-27rnn\" (UID: \"6dd49fc0-1d34-4825-a841-249a773b23ae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" Aug 04 15:15:22.047276 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:22.047165 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltfqs\" (UniqueName: \"kubernetes.io/projected/6dd49fc0-1d34-4825-a841-249a773b23ae-kube-api-access-ltfqs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-27rnn\" (UID: \"6dd49fc0-1d34-4825-a841-249a773b23ae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" Aug 04 15:15:22.047507 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:22.047305 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6dd49fc0-1d34-4825-a841-249a773b23ae-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-27rnn\" (UID: \"6dd49fc0-1d34-4825-a841-249a773b23ae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" Aug 04 15:15:22.047507 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:22.047346 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dd49fc0-1d34-4825-a841-249a773b23ae-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-27rnn\" (UID: \"6dd49fc0-1d34-4825-a841-249a773b23ae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" Aug 04 15:15:22.047507 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:22.047369 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dd49fc0-1d34-4825-a841-249a773b23ae-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-27rnn\" (UID: \"6dd49fc0-1d34-4825-a841-249a773b23ae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" Aug 04 15:15:22.047507 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:22.047402 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6dd49fc0-1d34-4825-a841-249a773b23ae-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-27rnn\" (UID: \"6dd49fc0-1d34-4825-a841-249a773b23ae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" Aug 04 15:15:22.047507 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:22.047429 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dd49fc0-1d34-4825-a841-249a773b23ae-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-27rnn\" (UID: \"6dd49fc0-1d34-4825-a841-249a773b23ae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" Aug 04 15:15:22.047822 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:22.047793 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6dd49fc0-1d34-4825-a841-249a773b23ae-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-27rnn\" (UID: \"6dd49fc0-1d34-4825-a841-249a773b23ae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" Aug 04 15:15:22.047897 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:22.047864 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dd49fc0-1d34-4825-a841-249a773b23ae-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-27rnn\" (UID: \"6dd49fc0-1d34-4825-a841-249a773b23ae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" Aug 04 15:15:22.047897 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:22.047886 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dd49fc0-1d34-4825-a841-249a773b23ae-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-27rnn\" (UID: \"6dd49fc0-1d34-4825-a841-249a773b23ae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" Aug 04 15:15:22.048145 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:22.048129 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dd49fc0-1d34-4825-a841-249a773b23ae-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-27rnn\" (UID: \"6dd49fc0-1d34-4825-a841-249a773b23ae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" Aug 04 15:15:22.048193 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:22.048172 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6dd49fc0-1d34-4825-a841-249a773b23ae-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-27rnn\" (UID: \"6dd49fc0-1d34-4825-a841-249a773b23ae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" Aug 04 15:15:22.049609 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:22.049584 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6dd49fc0-1d34-4825-a841-249a773b23ae-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-27rnn\" (UID: \"6dd49fc0-1d34-4825-a841-249a773b23ae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" Aug 04 15:15:22.049713 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:22.049676 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6dd49fc0-1d34-4825-a841-249a773b23ae-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-27rnn\" (UID: \"6dd49fc0-1d34-4825-a841-249a773b23ae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" Aug 04 15:15:22.056031 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:22.056005 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltfqs\" (UniqueName: \"kubernetes.io/projected/6dd49fc0-1d34-4825-a841-249a773b23ae-kube-api-access-ltfqs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-27rnn\" (UID: \"6dd49fc0-1d34-4825-a841-249a773b23ae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" Aug 04 15:15:22.056258 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:22.056241 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dd49fc0-1d34-4825-a841-249a773b23ae-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-27rnn\" (UID: \"6dd49fc0-1d34-4825-a841-249a773b23ae\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" Aug 04 15:15:22.179149 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:22.179061 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" Aug 04 15:15:22.316218 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:22.316191 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn"] Aug 04 15:15:22.317640 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:15:22.317614 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dd49fc0_1d34_4825_a841_249a773b23ae.slice/crio-1b99a4da994c43db5d95b75dc9232de4565bd6e37b9af271e242d8230699ddbc WatchSource:0}: Error finding container 1b99a4da994c43db5d95b75dc9232de4565bd6e37b9af271e242d8230699ddbc: Status 404 returned error can't find the container with id 1b99a4da994c43db5d95b75dc9232de4565bd6e37b9af271e242d8230699ddbc Aug 04 15:15:22.319757 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:22.319729 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:15:22.319825 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:22.319790 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:15:22.319825 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:22.319817 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:15:22.679470 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:22.679423 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" event={"ID":"6dd49fc0-1d34-4825-a841-249a773b23ae","Type":"ContainerStarted","Data":"e7ab6cf1d189cafcdfb2a7371a86f10725695723bfa88a94421bcfa21108f640"} Aug 04 15:15:22.679669 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:22.679499 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" event={"ID":"6dd49fc0-1d34-4825-a841-249a773b23ae","Type":"ContainerStarted","Data":"1b99a4da994c43db5d95b75dc9232de4565bd6e37b9af271e242d8230699ddbc"} Aug 04 15:15:22.703280 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:22.703230 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" podStartSLOduration=1.703213858 podStartE2EDuration="1.703213858s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:15:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:15:22.70252379 +0000 UTC m=+546.162396138" watchObservedRunningTime="2026-08-04 15:15:22.703213858 +0000 UTC m=+546.163086206" Aug 04 15:15:23.179811 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:23.179778 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" Aug 04 15:15:23.184339 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:23.184319 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" Aug 04 15:15:23.689894 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:23.687582 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" Aug 04 15:15:23.690711 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:23.690685 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" Aug 04 15:15:24.176298 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:24.176265 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:15:24.180292 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:24.180276 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-gpqx7" Aug 04 15:15:24.183253 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:24.183233 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Aug 04 15:15:24.191602 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:24.191574 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:15:24.220488 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:24.220467 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:15:24.269879 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:24.269850 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lqq7\" (UniqueName: \"kubernetes.io/projected/2ce38caa-ed33-46bf-854c-ee78469016c1-kube-api-access-8lqq7\") pod \"limitador-limitador-54d6d988d5-gpqx7\" (UID: \"2ce38caa-ed33-46bf-854c-ee78469016c1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-gpqx7" Aug 04 15:15:24.270012 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:24.269934 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2ce38caa-ed33-46bf-854c-ee78469016c1-config-file\") pod \"limitador-limitador-54d6d988d5-gpqx7\" (UID: \"2ce38caa-ed33-46bf-854c-ee78469016c1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-gpqx7" Aug 04 15:15:24.371143 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:24.371106 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2ce38caa-ed33-46bf-854c-ee78469016c1-config-file\") pod \"limitador-limitador-54d6d988d5-gpqx7\" (UID: \"2ce38caa-ed33-46bf-854c-ee78469016c1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-gpqx7" Aug 04 15:15:24.371289 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:24.371269 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lqq7\" (UniqueName: \"kubernetes.io/projected/2ce38caa-ed33-46bf-854c-ee78469016c1-kube-api-access-8lqq7\") pod \"limitador-limitador-54d6d988d5-gpqx7\" (UID: \"2ce38caa-ed33-46bf-854c-ee78469016c1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-gpqx7" Aug 04 15:15:24.371782 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:24.371760 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2ce38caa-ed33-46bf-854c-ee78469016c1-config-file\") pod \"limitador-limitador-54d6d988d5-gpqx7\" (UID: \"2ce38caa-ed33-46bf-854c-ee78469016c1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-gpqx7" Aug 04 15:15:24.380542 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:24.380521 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lqq7\" (UniqueName: \"kubernetes.io/projected/2ce38caa-ed33-46bf-854c-ee78469016c1-kube-api-access-8lqq7\") pod \"limitador-limitador-54d6d988d5-gpqx7\" (UID: \"2ce38caa-ed33-46bf-854c-ee78469016c1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-gpqx7" Aug 04 15:15:24.492966 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:24.492931 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-gpqx7" Aug 04 15:15:24.644044 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:24.644019 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:15:24.645726 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:15:24.645699 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ce38caa_ed33_46bf_854c_ee78469016c1.slice/crio-49baa25ff600122897040c10cc25e10e526acd1e9562c44dae87cb9d436e5526 WatchSource:0}: Error finding container 49baa25ff600122897040c10cc25e10e526acd1e9562c44dae87cb9d436e5526: Status 404 returned error can't find the container with id 49baa25ff600122897040c10cc25e10e526acd1e9562c44dae87cb9d436e5526 Aug 04 15:15:24.692841 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:24.692806 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-gpqx7" event={"ID":"2ce38caa-ed33-46bf-854c-ee78469016c1","Type":"ContainerStarted","Data":"49baa25ff600122897040c10cc25e10e526acd1e9562c44dae87cb9d436e5526"} Aug 04 15:15:28.719401 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:28.719362 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-gpqx7" event={"ID":"2ce38caa-ed33-46bf-854c-ee78469016c1","Type":"ContainerStarted","Data":"d273d765ffc554911168996b4f85a535df9411e573fb622dfd09478e3747e23f"} Aug 04 15:15:28.719787 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:28.719536 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-gpqx7" Aug 04 15:15:28.739118 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:28.739073 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-gpqx7" podStartSLOduration=1.346465765 podStartE2EDuration="4.739059792s" totalImagesPullingTime="3.392594027s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:15:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:15:28.737455641 +0000 UTC m=+552.197327993" watchObservedRunningTime="2026-08-04 15:15:28.739059792 +0000 UTC m=+552.198932194" Aug 04 15:15:39.727293 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:15:39.727261 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-gpqx7" Aug 04 15:16:16.989584 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:16:16.989556 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-r8wkp_0580fdab-0381-4d8e-be5f-bde5eb44e0e6/console-operator/1.log" Aug 04 15:16:16.990135 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:16:16.990017 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-r8wkp_0580fdab-0381-4d8e-be5f-bde5eb44e0e6/console-operator/1.log" Aug 04 15:16:16.993544 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:16:16.993525 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8h6k_9c62e3b1-c2dc-4b2a-a61d-7d80049f355f/ovn-acl-logging/0.log" Aug 04 15:16:16.993805 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:16:16.993788 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8h6k_9c62e3b1-c2dc-4b2a-a61d-7d80049f355f/ovn-acl-logging/0.log" Aug 04 15:16:41.387128 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:16:41.387046 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:16:49.135529 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:16:49.135452 2574 generic.go:356] "Generic (PLEG): container finished" podID="6dd49fc0-1d34-4825-a841-249a773b23ae" containerID="e7ab6cf1d189cafcdfb2a7371a86f10725695723bfa88a94421bcfa21108f640" exitCode=0 Aug 04 15:16:49.135529 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:16:49.135496 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" event={"ID":"6dd49fc0-1d34-4825-a841-249a773b23ae","Type":"ContainerDied","Data":"e7ab6cf1d189cafcdfb2a7371a86f10725695723bfa88a94421bcfa21108f640"} Aug 04 15:16:49.135948 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:16:49.135792 2574 scope.go:117] "RemoveContainer" containerID="e7ab6cf1d189cafcdfb2a7371a86f10725695723bfa88a94421bcfa21108f640" Aug 04 15:16:49.136810 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:16:49.136774 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:16:49.136878 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:16:49.136850 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:16:49.136920 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:16:49.136892 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:16:50.148180 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:16:50.148133 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" event={"ID":"6dd49fc0-1d34-4825-a841-249a773b23ae","Type":"ContainerStarted","Data":"5f344ab6216b58b80d6f95284f41a5c979c88e17090e58e0386d4c83ae2bb198"} Aug 04 15:16:50.179352 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:16:50.179323 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" Aug 04 15:16:50.180624 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:16:50.180599 2574 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-27rnn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Aug 04 15:16:50.180764 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:16:50.180654 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" podUID="6dd49fc0-1d34-4825-a841-249a773b23ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Aug 04 15:16:51.180265 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:16:51.180232 2574 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-27rnn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Aug 04 15:16:51.180647 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:16:51.180284 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" podUID="6dd49fc0-1d34-4825-a841-249a773b23ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Aug 04 15:16:52.180210 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:16:52.180173 2574 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-27rnn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Aug 04 15:16:52.180418 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:16:52.180227 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" podUID="6dd49fc0-1d34-4825-a841-249a773b23ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Aug 04 15:16:53.180072 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:16:53.180036 2574 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-27rnn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Aug 04 15:16:53.180240 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:16:53.180086 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" podUID="6dd49fc0-1d34-4825-a841-249a773b23ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Aug 04 15:16:53.688194 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:16:53.688152 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" Aug 04 15:16:54.183464 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:16:54.183437 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" Aug 04 15:16:54.184362 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:16:54.184346 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-27rnn" Aug 04 15:17:03.814632 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:17:03.814593 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:17:22.116381 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:17:22.116345 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:17:31.415716 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:17:31.415673 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:17:40.518014 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:17:40.517977 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:17:54.502760 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:17:54.502729 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-ztjrs"] Aug 04 15:17:54.506225 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:17:54.506207 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ztjrs" Aug 04 15:17:54.509139 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:17:54.509116 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-l75xt\"" Aug 04 15:17:54.510494 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:17:54.510473 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Aug 04 15:17:54.510595 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:17:54.510506 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Aug 04 15:17:54.510782 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:17:54.510767 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Aug 04 15:17:54.517241 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:17:54.517217 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-ztjrs"] Aug 04 15:17:54.586164 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:17:54.586127 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/800feb8e-32f3-4d4d-bb6d-262d6a1bafe5-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ztjrs\" (UID: \"800feb8e-32f3-4d4d-bb6d-262d6a1bafe5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ztjrs" Aug 04 15:17:54.586316 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:17:54.586169 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/800feb8e-32f3-4d4d-bb6d-262d6a1bafe5-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ztjrs\" (UID: \"800feb8e-32f3-4d4d-bb6d-262d6a1bafe5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ztjrs" Aug 04 15:17:54.586316 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:17:54.586195 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/800feb8e-32f3-4d4d-bb6d-262d6a1bafe5-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ztjrs\" (UID: \"800feb8e-32f3-4d4d-bb6d-262d6a1bafe5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ztjrs" Aug 04 15:17:54.586316 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:17:54.586265 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/800feb8e-32f3-4d4d-bb6d-262d6a1bafe5-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ztjrs\" (UID: \"800feb8e-32f3-4d4d-bb6d-262d6a1bafe5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ztjrs" Aug 04 15:17:54.586464 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:17:54.586357 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/800feb8e-32f3-4d4d-bb6d-262d6a1bafe5-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ztjrs\" (UID: \"800feb8e-32f3-4d4d-bb6d-262d6a1bafe5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ztjrs" Aug 04 15:17:54.586464 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:17:54.586404 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l66lq\" (UniqueName: \"kubernetes.io/projected/800feb8e-32f3-4d4d-bb6d-262d6a1bafe5-kube-api-access-l66lq\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ztjrs\" (UID: \"800feb8e-32f3-4d4d-bb6d-262d6a1bafe5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ztjrs" Aug 04 15:17:54.687827 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:17:54.687791 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/800feb8e-32f3-4d4d-bb6d-262d6a1bafe5-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ztjrs\" (UID: \"800feb8e-32f3-4d4d-bb6d-262d6a1bafe5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ztjrs" Aug 04 15:17:54.688026 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:17:54.687850 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l66lq\" (UniqueName: \"kubernetes.io/projected/800feb8e-32f3-4d4d-bb6d-262d6a1bafe5-kube-api-access-l66lq\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ztjrs\" (UID: \"800feb8e-32f3-4d4d-bb6d-262d6a1bafe5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ztjrs" Aug 04 15:17:54.688026 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:17:54.687915 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/800feb8e-32f3-4d4d-bb6d-262d6a1bafe5-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ztjrs\" (UID: \"800feb8e-32f3-4d4d-bb6d-262d6a1bafe5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ztjrs" Aug 04 15:17:54.688026 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:17:54.687966 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/800feb8e-32f3-4d4d-bb6d-262d6a1bafe5-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ztjrs\" (UID: \"800feb8e-32f3-4d4d-bb6d-262d6a1bafe5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ztjrs" Aug 04 15:17:54.688026 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:17:54.687994 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/800feb8e-32f3-4d4d-bb6d-262d6a1bafe5-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ztjrs\" (UID: \"800feb8e-32f3-4d4d-bb6d-262d6a1bafe5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ztjrs" Aug 04 15:17:54.688026 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:17:54.688030 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/800feb8e-32f3-4d4d-bb6d-262d6a1bafe5-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ztjrs\" (UID: \"800feb8e-32f3-4d4d-bb6d-262d6a1bafe5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ztjrs" Aug 04 15:17:54.688514 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:17:54.688341 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/800feb8e-32f3-4d4d-bb6d-262d6a1bafe5-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ztjrs\" (UID: \"800feb8e-32f3-4d4d-bb6d-262d6a1bafe5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ztjrs" Aug 04 15:17:54.688514 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:17:54.688422 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/800feb8e-32f3-4d4d-bb6d-262d6a1bafe5-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ztjrs\" (UID: \"800feb8e-32f3-4d4d-bb6d-262d6a1bafe5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ztjrs" Aug 04 15:17:54.688514 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:17:54.688503 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/800feb8e-32f3-4d4d-bb6d-262d6a1bafe5-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ztjrs\" (UID: \"800feb8e-32f3-4d4d-bb6d-262d6a1bafe5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ztjrs" Aug 04 15:17:54.690479 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:17:54.690446 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/800feb8e-32f3-4d4d-bb6d-262d6a1bafe5-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ztjrs\" (UID: \"800feb8e-32f3-4d4d-bb6d-262d6a1bafe5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ztjrs" Aug 04 15:17:54.691029 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:17:54.691008 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/800feb8e-32f3-4d4d-bb6d-262d6a1bafe5-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ztjrs\" (UID: \"800feb8e-32f3-4d4d-bb6d-262d6a1bafe5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ztjrs" Aug 04 15:17:54.697428 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:17:54.697381 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l66lq\" (UniqueName: \"kubernetes.io/projected/800feb8e-32f3-4d4d-bb6d-262d6a1bafe5-kube-api-access-l66lq\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ztjrs\" (UID: \"800feb8e-32f3-4d4d-bb6d-262d6a1bafe5\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ztjrs" Aug 04 15:17:54.817460 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:17:54.817354 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ztjrs" Aug 04 15:17:54.964670 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:17:54.964644 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-ztjrs"] Aug 04 15:17:54.966331 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:17:54.966302 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod800feb8e_32f3_4d4d_bb6d_262d6a1bafe5.slice/crio-c1a26282e207436189b81b8f9192df131e02822bbfdb9c826783fda17216b7ef WatchSource:0}: Error finding container c1a26282e207436189b81b8f9192df131e02822bbfdb9c826783fda17216b7ef: Status 404 returned error can't find the container with id c1a26282e207436189b81b8f9192df131e02822bbfdb9c826783fda17216b7ef Aug 04 15:17:54.967727 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:17:54.967708 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 15:17:55.500001 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:17:55.499962 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ztjrs" event={"ID":"800feb8e-32f3-4d4d-bb6d-262d6a1bafe5","Type":"ContainerStarted","Data":"c1a26282e207436189b81b8f9192df131e02822bbfdb9c826783fda17216b7ef"} Aug 04 15:17:56.506974 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:17:56.506932 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ztjrs" event={"ID":"800feb8e-32f3-4d4d-bb6d-262d6a1bafe5","Type":"ContainerStarted","Data":"c2017e1f40bdedd9b5d9b5760a4f9dc56584d9adf924c75083cd8bcae63b84a8"} Aug 04 15:17:56.527462 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:17:56.527417 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ztjrs" podStartSLOduration=1.496680146 podStartE2EDuration="2.527401049s" totalImagesPullingTime="1.030720903s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:17:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:17:56.525756665 +0000 UTC m=+699.985629014" watchObservedRunningTime="2026-08-04 15:17:56.527401049 +0000 UTC m=+699.987273390" Aug 04 15:17:57.431045 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:17:57.430953 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:18:01.387444 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:18:01.387405 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-tscq2"] Aug 04 15:18:01.391171 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:18:01.391150 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tscq2" Aug 04 15:18:01.399542 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:18:01.399524 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Aug 04 15:18:01.409342 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:18:01.409314 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-tscq2"] Aug 04 15:18:01.550951 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:18:01.550916 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbrc6\" (UniqueName: \"kubernetes.io/projected/a6a6ac91-159b-46ae-8b44-c8b6d490a3c3-kube-api-access-sbrc6\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tscq2\" (UID: \"a6a6ac91-159b-46ae-8b44-c8b6d490a3c3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tscq2" Aug 04 15:18:01.550951 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:18:01.550950 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6a6ac91-159b-46ae-8b44-c8b6d490a3c3-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tscq2\" (UID: \"a6a6ac91-159b-46ae-8b44-c8b6d490a3c3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tscq2" Aug 04 15:18:01.551153 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:18:01.551064 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6a6ac91-159b-46ae-8b44-c8b6d490a3c3-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tscq2\" (UID: \"a6a6ac91-159b-46ae-8b44-c8b6d490a3c3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tscq2" Aug 04 15:18:01.551153 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:18:01.551092 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6a6ac91-159b-46ae-8b44-c8b6d490a3c3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tscq2\" (UID: \"a6a6ac91-159b-46ae-8b44-c8b6d490a3c3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tscq2" Aug 04 15:18:01.551153 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:18:01.551131 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6a6ac91-159b-46ae-8b44-c8b6d490a3c3-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tscq2\" (UID: \"a6a6ac91-159b-46ae-8b44-c8b6d490a3c3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tscq2" Aug 04 15:18:01.551260 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:18:01.551153 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6a6ac91-159b-46ae-8b44-c8b6d490a3c3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tscq2\" (UID: \"a6a6ac91-159b-46ae-8b44-c8b6d490a3c3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tscq2" Aug 04 15:18:01.652165 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:18:01.652073 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbrc6\" (UniqueName: \"kubernetes.io/projected/a6a6ac91-159b-46ae-8b44-c8b6d490a3c3-kube-api-access-sbrc6\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tscq2\" (UID: \"a6a6ac91-159b-46ae-8b44-c8b6d490a3c3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tscq2" Aug 04 15:18:01.652165 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:18:01.652127 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6a6ac91-159b-46ae-8b44-c8b6d490a3c3-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tscq2\" (UID: \"a6a6ac91-159b-46ae-8b44-c8b6d490a3c3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tscq2" Aug 04 15:18:01.652375 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:18:01.652231 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6a6ac91-159b-46ae-8b44-c8b6d490a3c3-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tscq2\" (UID: \"a6a6ac91-159b-46ae-8b44-c8b6d490a3c3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tscq2" Aug 04 15:18:01.652375 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:18:01.652259 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6a6ac91-159b-46ae-8b44-c8b6d490a3c3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tscq2\" (UID: \"a6a6ac91-159b-46ae-8b44-c8b6d490a3c3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tscq2" Aug 04 15:18:01.652375 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:18:01.652297 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6a6ac91-159b-46ae-8b44-c8b6d490a3c3-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tscq2\" (UID: \"a6a6ac91-159b-46ae-8b44-c8b6d490a3c3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tscq2" Aug 04 15:18:01.652375 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:18:01.652318 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6a6ac91-159b-46ae-8b44-c8b6d490a3c3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tscq2\" (UID: \"a6a6ac91-159b-46ae-8b44-c8b6d490a3c3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tscq2" Aug 04 15:18:01.652650 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:18:01.652623 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6a6ac91-159b-46ae-8b44-c8b6d490a3c3-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tscq2\" (UID: \"a6a6ac91-159b-46ae-8b44-c8b6d490a3c3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tscq2" Aug 04 15:18:01.652728 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:18:01.652636 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6a6ac91-159b-46ae-8b44-c8b6d490a3c3-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tscq2\" (UID: \"a6a6ac91-159b-46ae-8b44-c8b6d490a3c3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tscq2" Aug 04 15:18:01.652779 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:18:01.652718 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6a6ac91-159b-46ae-8b44-c8b6d490a3c3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tscq2\" (UID: \"a6a6ac91-159b-46ae-8b44-c8b6d490a3c3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tscq2" Aug 04 15:18:01.654596 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:18:01.654576 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6a6ac91-159b-46ae-8b44-c8b6d490a3c3-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tscq2\" (UID: \"a6a6ac91-159b-46ae-8b44-c8b6d490a3c3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tscq2" Aug 04 15:18:01.654764 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:18:01.654748 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6a6ac91-159b-46ae-8b44-c8b6d490a3c3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tscq2\" (UID: \"a6a6ac91-159b-46ae-8b44-c8b6d490a3c3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tscq2" Aug 04 15:18:01.660984 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:18:01.660961 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbrc6\" (UniqueName: \"kubernetes.io/projected/a6a6ac91-159b-46ae-8b44-c8b6d490a3c3-kube-api-access-sbrc6\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-tscq2\" (UID: \"a6a6ac91-159b-46ae-8b44-c8b6d490a3c3\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tscq2" Aug 04 15:18:01.701060 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:18:01.701029 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tscq2" Aug 04 15:18:01.840314 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:18:01.840290 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-tscq2"] Aug 04 15:18:01.841874 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:18:01.841846 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda6a6ac91_159b_46ae_8b44_c8b6d490a3c3.slice/crio-12bbe2a93cfc2d99414e5d804de8ccbc2a1b04f62a7894bd26b8d265a9e68a64 WatchSource:0}: Error finding container 12bbe2a93cfc2d99414e5d804de8ccbc2a1b04f62a7894bd26b8d265a9e68a64: Status 404 returned error can't find the container with id 12bbe2a93cfc2d99414e5d804de8ccbc2a1b04f62a7894bd26b8d265a9e68a64 Aug 04 15:18:02.546949 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:18:02.546918 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tscq2" event={"ID":"a6a6ac91-159b-46ae-8b44-c8b6d490a3c3","Type":"ContainerStarted","Data":"19372313b76017b3f0969e6e707ab16258d4ff14de229122a613c349dd5ab582"} Aug 04 15:18:02.547330 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:18:02.546958 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tscq2" event={"ID":"a6a6ac91-159b-46ae-8b44-c8b6d490a3c3","Type":"ContainerStarted","Data":"12bbe2a93cfc2d99414e5d804de8ccbc2a1b04f62a7894bd26b8d265a9e68a64"} Aug 04 15:18:02.567788 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:18:02.567741 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tscq2" podStartSLOduration=1.334714945 podStartE2EDuration="1.567728118s" totalImagesPullingTime="233.013173ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:18:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:18:02.566002267 +0000 UTC m=+706.025874616" watchObservedRunningTime="2026-08-04 15:18:02.567728118 +0000 UTC m=+706.027600467" Aug 04 15:18:04.818491 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:18:04.818458 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ztjrs" Aug 04 15:18:04.818931 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:18:04.818530 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ztjrs" Aug 04 15:18:04.831500 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:18:04.831477 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ztjrs" Aug 04 15:18:05.575284 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:18:05.575251 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ztjrs" Aug 04 15:18:11.615303 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:18:11.615223 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:18:11.701996 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:18:11.701959 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tscq2" Aug 04 15:18:11.702157 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:18:11.702042 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tscq2" Aug 04 15:18:11.714102 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:18:11.714081 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tscq2" Aug 04 15:18:12.612128 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:18:12.612094 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-tscq2" Aug 04 15:19:29.611937 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:19:29.611903 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:19:38.816458 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:19:38.816415 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:19:49.412273 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:19:49.412186 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:21:17.031326 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:21:17.031257 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-r8wkp_0580fdab-0381-4d8e-be5f-bde5eb44e0e6/console-operator/1.log" Aug 04 15:21:17.031326 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:21:17.031280 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-r8wkp_0580fdab-0381-4d8e-be5f-bde5eb44e0e6/console-operator/1.log" Aug 04 15:21:17.035500 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:21:17.035479 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8h6k_9c62e3b1-c2dc-4b2a-a61d-7d80049f355f/ovn-acl-logging/0.log" Aug 04 15:21:17.035622 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:21:17.035577 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8h6k_9c62e3b1-c2dc-4b2a-a61d-7d80049f355f/ovn-acl-logging/0.log" Aug 04 15:21:29.818087 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:21:29.818051 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:21:38.317899 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:21:38.317863 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:22:40.214513 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:22:40.214431 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:22:49.521711 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:22:49.521679 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:23:15.915421 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:23:15.915374 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:23:25.018443 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:23:25.018402 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:23:35.121302 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:23:35.121268 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:23:44.021378 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:23:44.021343 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:23:56.215724 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:23:56.215692 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:24:05.918452 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:24:05.918412 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:24:11.916482 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:24:11.916397 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:24:19.619365 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:24:19.619332 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:24:31.524290 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:24:31.524252 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:24:42.913409 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:24:42.913355 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:25:05.523062 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:25:05.523024 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:25:26.518687 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:25:26.518649 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:25:33.721315 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:25:33.721269 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:25:44.320672 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:25:44.320579 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:26:04.620963 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:26:04.620914 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:26:17.076019 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:26:17.075990 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-r8wkp_0580fdab-0381-4d8e-be5f-bde5eb44e0e6/console-operator/1.log" Aug 04 15:26:17.076605 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:26:17.076535 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-r8wkp_0580fdab-0381-4d8e-be5f-bde5eb44e0e6/console-operator/1.log" Aug 04 15:26:17.079789 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:26:17.079767 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8h6k_9c62e3b1-c2dc-4b2a-a61d-7d80049f355f/ovn-acl-logging/0.log" Aug 04 15:26:17.080189 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:26:17.080173 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8h6k_9c62e3b1-c2dc-4b2a-a61d-7d80049f355f/ovn-acl-logging/0.log" Aug 04 15:26:19.623850 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:26:19.623813 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:26:28.921298 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:26:28.921261 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:26:38.525455 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:26:38.525412 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:26:47.817779 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:26:47.817734 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:26:56.820071 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:26:56.820032 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:27:17.720709 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:27:17.720628 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:27:29.017278 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:27:29.017235 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:27:39.016035 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:27:39.015996 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:28:35.348261 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:28:35.348224 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c5555494f-7jz9k"] Aug 04 15:29:00.370314 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:29:00.370221 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5c5555494f-7jz9k" podUID="161e7bb8-9e46-4b79-89c5-6e02060e2830" containerName="console" containerID="cri-o://bdb17e7900372b1c3a59aa8f7d3575108f21f3961cd211ff4a284de740e1d01b" gracePeriod=15 Aug 04 15:29:00.631556 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:29:00.631496 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c5555494f-7jz9k" Aug 04 15:29:00.731465 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:29:00.731426 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/161e7bb8-9e46-4b79-89c5-6e02060e2830-console-oauth-config\") pod \"161e7bb8-9e46-4b79-89c5-6e02060e2830\" (UID: \"161e7bb8-9e46-4b79-89c5-6e02060e2830\") " Aug 04 15:29:00.731665 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:29:00.731486 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ss56l\" (UniqueName: \"kubernetes.io/projected/161e7bb8-9e46-4b79-89c5-6e02060e2830-kube-api-access-ss56l\") pod \"161e7bb8-9e46-4b79-89c5-6e02060e2830\" (UID: \"161e7bb8-9e46-4b79-89c5-6e02060e2830\") " Aug 04 15:29:00.731665 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:29:00.731589 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/161e7bb8-9e46-4b79-89c5-6e02060e2830-trusted-ca-bundle\") pod \"161e7bb8-9e46-4b79-89c5-6e02060e2830\" (UID: \"161e7bb8-9e46-4b79-89c5-6e02060e2830\") " Aug 04 15:29:00.731665 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:29:00.731635 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/161e7bb8-9e46-4b79-89c5-6e02060e2830-service-ca\") pod \"161e7bb8-9e46-4b79-89c5-6e02060e2830\" (UID: \"161e7bb8-9e46-4b79-89c5-6e02060e2830\") " Aug 04 15:29:00.731845 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:29:00.731668 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/161e7bb8-9e46-4b79-89c5-6e02060e2830-console-serving-cert\") pod \"161e7bb8-9e46-4b79-89c5-6e02060e2830\" (UID: \"161e7bb8-9e46-4b79-89c5-6e02060e2830\") " Aug 04 15:29:00.731845 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:29:00.731704 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/161e7bb8-9e46-4b79-89c5-6e02060e2830-oauth-serving-cert\") pod \"161e7bb8-9e46-4b79-89c5-6e02060e2830\" (UID: \"161e7bb8-9e46-4b79-89c5-6e02060e2830\") " Aug 04 15:29:00.731845 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:29:00.731733 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/161e7bb8-9e46-4b79-89c5-6e02060e2830-console-config\") pod \"161e7bb8-9e46-4b79-89c5-6e02060e2830\" (UID: \"161e7bb8-9e46-4b79-89c5-6e02060e2830\") " Aug 04 15:29:00.732122 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:29:00.732100 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/161e7bb8-9e46-4b79-89c5-6e02060e2830-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "161e7bb8-9e46-4b79-89c5-6e02060e2830" (UID: "161e7bb8-9e46-4b79-89c5-6e02060e2830"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 15:29:00.732196 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:29:00.732112 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/161e7bb8-9e46-4b79-89c5-6e02060e2830-service-ca" (OuterVolumeSpecName: "service-ca") pod "161e7bb8-9e46-4b79-89c5-6e02060e2830" (UID: "161e7bb8-9e46-4b79-89c5-6e02060e2830"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 15:29:00.732196 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:29:00.732096 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/161e7bb8-9e46-4b79-89c5-6e02060e2830-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "161e7bb8-9e46-4b79-89c5-6e02060e2830" (UID: "161e7bb8-9e46-4b79-89c5-6e02060e2830"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 15:29:00.732503 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:29:00.732479 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/161e7bb8-9e46-4b79-89c5-6e02060e2830-console-config" (OuterVolumeSpecName: "console-config") pod "161e7bb8-9e46-4b79-89c5-6e02060e2830" (UID: "161e7bb8-9e46-4b79-89c5-6e02060e2830"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 15:29:00.733781 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:29:00.733761 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/161e7bb8-9e46-4b79-89c5-6e02060e2830-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "161e7bb8-9e46-4b79-89c5-6e02060e2830" (UID: "161e7bb8-9e46-4b79-89c5-6e02060e2830"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 15:29:00.733781 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:29:00.733766 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/161e7bb8-9e46-4b79-89c5-6e02060e2830-kube-api-access-ss56l" (OuterVolumeSpecName: "kube-api-access-ss56l") pod "161e7bb8-9e46-4b79-89c5-6e02060e2830" (UID: "161e7bb8-9e46-4b79-89c5-6e02060e2830"). InnerVolumeSpecName "kube-api-access-ss56l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 15:29:00.733912 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:29:00.733838 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/161e7bb8-9e46-4b79-89c5-6e02060e2830-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "161e7bb8-9e46-4b79-89c5-6e02060e2830" (UID: "161e7bb8-9e46-4b79-89c5-6e02060e2830"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 15:29:00.832759 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:29:00.832720 2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/161e7bb8-9e46-4b79-89c5-6e02060e2830-trusted-ca-bundle\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:29:00.832759 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:29:00.832755 2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/161e7bb8-9e46-4b79-89c5-6e02060e2830-service-ca\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:29:00.832759 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:29:00.832764 2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/161e7bb8-9e46-4b79-89c5-6e02060e2830-console-serving-cert\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:29:00.833062 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:29:00.832773 2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/161e7bb8-9e46-4b79-89c5-6e02060e2830-oauth-serving-cert\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:29:00.833062 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:29:00.832781 2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/161e7bb8-9e46-4b79-89c5-6e02060e2830-console-config\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:29:00.833062 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:29:00.832790 2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/161e7bb8-9e46-4b79-89c5-6e02060e2830-console-oauth-config\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:29:00.833062 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:29:00.832798 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ss56l\" (UniqueName: \"kubernetes.io/projected/161e7bb8-9e46-4b79-89c5-6e02060e2830-kube-api-access-ss56l\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:29:01.128062 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:29:01.128029 2574 generic.go:356] "Generic (PLEG): container finished" podID="161e7bb8-9e46-4b79-89c5-6e02060e2830" containerID="bdb17e7900372b1c3a59aa8f7d3575108f21f3961cd211ff4a284de740e1d01b" exitCode=0 Aug 04 15:29:01.128232 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:29:01.128073 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c5555494f-7jz9k" Aug 04 15:29:01.128232 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:29:01.128094 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c5555494f-7jz9k" event={"ID":"161e7bb8-9e46-4b79-89c5-6e02060e2830","Type":"ContainerDied","Data":"bdb17e7900372b1c3a59aa8f7d3575108f21f3961cd211ff4a284de740e1d01b"} Aug 04 15:29:01.128232 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:29:01.128133 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c5555494f-7jz9k" event={"ID":"161e7bb8-9e46-4b79-89c5-6e02060e2830","Type":"ContainerDied","Data":"289ef1f76590f6cfcc7dc846bf6b08eaf11a5d71d3a6ff8bc3a9500d1077b463"} Aug 04 15:29:01.128232 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:29:01.128153 2574 scope.go:117] "RemoveContainer" containerID="bdb17e7900372b1c3a59aa8f7d3575108f21f3961cd211ff4a284de740e1d01b" Aug 04 15:29:01.137346 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:29:01.137331 2574 scope.go:117] "RemoveContainer" containerID="bdb17e7900372b1c3a59aa8f7d3575108f21f3961cd211ff4a284de740e1d01b" Aug 04 15:29:01.137631 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:29:01.137610 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bdb17e7900372b1c3a59aa8f7d3575108f21f3961cd211ff4a284de740e1d01b\": container with ID starting with bdb17e7900372b1c3a59aa8f7d3575108f21f3961cd211ff4a284de740e1d01b not found: ID does not exist" containerID="bdb17e7900372b1c3a59aa8f7d3575108f21f3961cd211ff4a284de740e1d01b" Aug 04 15:29:01.137739 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:29:01.137636 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdb17e7900372b1c3a59aa8f7d3575108f21f3961cd211ff4a284de740e1d01b"} err="failed to get container status \"bdb17e7900372b1c3a59aa8f7d3575108f21f3961cd211ff4a284de740e1d01b\": rpc error: code = NotFound desc = could not find container \"bdb17e7900372b1c3a59aa8f7d3575108f21f3961cd211ff4a284de740e1d01b\": container with ID starting with bdb17e7900372b1c3a59aa8f7d3575108f21f3961cd211ff4a284de740e1d01b not found: ID does not exist" Aug 04 15:29:01.148606 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:29:01.148584 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c5555494f-7jz9k"] Aug 04 15:29:01.152230 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:29:01.152206 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5c5555494f-7jz9k"] Aug 04 15:29:03.053689 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:29:03.053651 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="161e7bb8-9e46-4b79-89c5-6e02060e2830" path="/var/lib/kubelet/pods/161e7bb8-9e46-4b79-89c5-6e02060e2830/volumes" Aug 04 15:29:50.115223 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:29:50.115186 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:30:00.225495 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:30:00.225462 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:30:20.016238 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:30:20.016151 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:30:32.543066 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:30:32.543026 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:30:42.008909 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:30:42.008872 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:30:52.214456 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:30:52.214419 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:31:02.521037 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:31:02.520999 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:31:13.728041 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:31:13.728003 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:31:17.115225 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:31:17.115196 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-r8wkp_0580fdab-0381-4d8e-be5f-bde5eb44e0e6/console-operator/1.log" Aug 04 15:31:17.116561 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:31:17.116536 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-r8wkp_0580fdab-0381-4d8e-be5f-bde5eb44e0e6/console-operator/1.log" Aug 04 15:31:17.119462 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:31:17.119440 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8h6k_9c62e3b1-c2dc-4b2a-a61d-7d80049f355f/ovn-acl-logging/0.log" Aug 04 15:31:17.120589 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:31:17.120570 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8h6k_9c62e3b1-c2dc-4b2a-a61d-7d80049f355f/ovn-acl-logging/0.log" Aug 04 15:31:22.823131 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:31:22.823093 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:31:34.126793 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:31:34.126760 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:31:45.711055 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:31:45.710973 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:31:55.120618 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:31:55.120586 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:32:12.924747 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:32:12.924705 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:32:22.826668 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:32:22.826628 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:32:35.803625 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:32:35.803593 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw"] Aug 04 15:32:35.804015 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:32:35.804002 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="161e7bb8-9e46-4b79-89c5-6e02060e2830" containerName="console" Aug 04 15:32:35.804015 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:32:35.804016 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="161e7bb8-9e46-4b79-89c5-6e02060e2830" containerName="console" Aug 04 15:32:35.804091 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:32:35.804073 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="161e7bb8-9e46-4b79-89c5-6e02060e2830" containerName="console" Aug 04 15:32:35.807459 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:32:35.807436 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw" Aug 04 15:32:35.810550 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:32:35.810531 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-ffe4e6a1-kserve-self-signed-certs\"" Aug 04 15:32:35.824952 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:32:35.824929 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw"] Aug 04 15:32:35.921060 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:32:35.921024 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17704b56-b8ae-4bdf-839c-5c6077711f49-model-cache\") pod \"e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw\" (UID: \"17704b56-b8ae-4bdf-839c-5c6077711f49\") " pod="llm/e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw" Aug 04 15:32:35.921226 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:32:35.921066 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17704b56-b8ae-4bdf-839c-5c6077711f49-home\") pod \"e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw\" (UID: \"17704b56-b8ae-4bdf-839c-5c6077711f49\") " pod="llm/e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw" Aug 04 15:32:35.921226 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:32:35.921099 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17704b56-b8ae-4bdf-839c-5c6077711f49-tls-certs\") pod \"e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw\" (UID: \"17704b56-b8ae-4bdf-839c-5c6077711f49\") " pod="llm/e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw" Aug 04 15:32:35.921226 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:32:35.921143 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17704b56-b8ae-4bdf-839c-5c6077711f49-dshm\") pod \"e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw\" (UID: \"17704b56-b8ae-4bdf-839c-5c6077711f49\") " pod="llm/e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw" Aug 04 15:32:35.921226 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:32:35.921161 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17704b56-b8ae-4bdf-839c-5c6077711f49-tmp-dir\") pod \"e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw\" (UID: \"17704b56-b8ae-4bdf-839c-5c6077711f49\") " pod="llm/e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw" Aug 04 15:32:35.921226 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:32:35.921204 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xwbd\" (UniqueName: \"kubernetes.io/projected/17704b56-b8ae-4bdf-839c-5c6077711f49-kube-api-access-9xwbd\") pod \"e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw\" (UID: \"17704b56-b8ae-4bdf-839c-5c6077711f49\") " pod="llm/e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw" Aug 04 15:32:36.022269 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:32:36.022229 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17704b56-b8ae-4bdf-839c-5c6077711f49-home\") pod \"e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw\" (UID: \"17704b56-b8ae-4bdf-839c-5c6077711f49\") " pod="llm/e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw" Aug 04 15:32:36.022474 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:32:36.022285 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17704b56-b8ae-4bdf-839c-5c6077711f49-tls-certs\") pod \"e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw\" (UID: \"17704b56-b8ae-4bdf-839c-5c6077711f49\") " pod="llm/e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw" Aug 04 15:32:36.022474 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:32:36.022350 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17704b56-b8ae-4bdf-839c-5c6077711f49-dshm\") pod \"e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw\" (UID: \"17704b56-b8ae-4bdf-839c-5c6077711f49\") " pod="llm/e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw" Aug 04 15:32:36.022474 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:32:36.022379 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17704b56-b8ae-4bdf-839c-5c6077711f49-tmp-dir\") pod \"e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw\" (UID: \"17704b56-b8ae-4bdf-839c-5c6077711f49\") " pod="llm/e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw" Aug 04 15:32:36.022474 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:32:36.022442 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xwbd\" (UniqueName: \"kubernetes.io/projected/17704b56-b8ae-4bdf-839c-5c6077711f49-kube-api-access-9xwbd\") pod \"e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw\" (UID: \"17704b56-b8ae-4bdf-839c-5c6077711f49\") " pod="llm/e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw" Aug 04 15:32:36.022718 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:32:36.022483 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17704b56-b8ae-4bdf-839c-5c6077711f49-model-cache\") pod \"e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw\" (UID: \"17704b56-b8ae-4bdf-839c-5c6077711f49\") " pod="llm/e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw" Aug 04 15:32:36.022718 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:32:36.022663 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17704b56-b8ae-4bdf-839c-5c6077711f49-home\") pod \"e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw\" (UID: \"17704b56-b8ae-4bdf-839c-5c6077711f49\") " pod="llm/e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw" Aug 04 15:32:36.022829 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:32:36.022754 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17704b56-b8ae-4bdf-839c-5c6077711f49-tmp-dir\") pod \"e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw\" (UID: \"17704b56-b8ae-4bdf-839c-5c6077711f49\") " pod="llm/e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw" Aug 04 15:32:36.022829 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:32:36.022805 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17704b56-b8ae-4bdf-839c-5c6077711f49-model-cache\") pod \"e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw\" (UID: \"17704b56-b8ae-4bdf-839c-5c6077711f49\") " pod="llm/e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw" Aug 04 15:32:36.024654 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:32:36.024633 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17704b56-b8ae-4bdf-839c-5c6077711f49-dshm\") pod \"e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw\" (UID: \"17704b56-b8ae-4bdf-839c-5c6077711f49\") " pod="llm/e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw" Aug 04 15:32:36.024935 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:32:36.024913 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17704b56-b8ae-4bdf-839c-5c6077711f49-tls-certs\") pod \"e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw\" (UID: \"17704b56-b8ae-4bdf-839c-5c6077711f49\") " pod="llm/e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw" Aug 04 15:32:36.032042 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:32:36.032021 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xwbd\" (UniqueName: \"kubernetes.io/projected/17704b56-b8ae-4bdf-839c-5c6077711f49-kube-api-access-9xwbd\") pod \"e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw\" (UID: \"17704b56-b8ae-4bdf-839c-5c6077711f49\") " pod="llm/e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw" Aug 04 15:32:36.117041 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:32:36.116948 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw" Aug 04 15:32:36.255518 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:32:36.255494 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw"] Aug 04 15:32:36.257178 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:32:36.257156 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod17704b56_b8ae_4bdf_839c_5c6077711f49.slice/crio-5c0d30ff1526af4c6d00d91e3d2235ba98429c3586da263e67d6baa22947b7db WatchSource:0}: Error finding container 5c0d30ff1526af4c6d00d91e3d2235ba98429c3586da263e67d6baa22947b7db: Status 404 returned error can't find the container with id 5c0d30ff1526af4c6d00d91e3d2235ba98429c3586da263e67d6baa22947b7db Aug 04 15:32:36.283992 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:32:36.283965 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw" event={"ID":"17704b56-b8ae-4bdf-839c-5c6077711f49","Type":"ContainerStarted","Data":"5c0d30ff1526af4c6d00d91e3d2235ba98429c3586da263e67d6baa22947b7db"} Aug 04 15:32:37.290614 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:32:37.290580 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw" event={"ID":"17704b56-b8ae-4bdf-839c-5c6077711f49","Type":"ContainerStarted","Data":"6be6efae843300f9cfaae881fd1d329a908cad7f6b09c732bc6cb9a1ea46f88f"} Aug 04 15:32:37.310080 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:32:37.310036 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw" podStartSLOduration=2.310021315 podStartE2EDuration="2.310021315s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:32:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:32:37.309011586 +0000 UTC m=+1580.768883936" watchObservedRunningTime="2026-08-04 15:32:37.310021315 +0000 UTC m=+1580.769893642" Aug 04 15:32:39.637855 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:32:39.637820 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:32:46.117338 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:32:46.117259 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw" Aug 04 15:32:46.117752 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:32:46.117602 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw" Aug 04 15:32:46.129929 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:32:46.129903 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw" Aug 04 15:32:46.349694 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:32:46.349660 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw" Aug 04 15:32:55.422491 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:32:55.422453 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:33:08.590543 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:33:08.590503 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw"] Aug 04 15:33:12.622968 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:33:12.622895 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:33:23.609786 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:33:23.609718 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw" podUID="17704b56-b8ae-4bdf-839c-5c6077711f49" containerName="main" containerID="cri-o://6be6efae843300f9cfaae881fd1d329a908cad7f6b09c732bc6cb9a1ea46f88f" gracePeriod=45 Aug 04 15:33:23.864950 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:33:23.864892 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw" Aug 04 15:33:23.941286 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:33:23.941254 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17704b56-b8ae-4bdf-839c-5c6077711f49-home\") pod \"17704b56-b8ae-4bdf-839c-5c6077711f49\" (UID: \"17704b56-b8ae-4bdf-839c-5c6077711f49\") " Aug 04 15:33:23.941474 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:33:23.941318 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17704b56-b8ae-4bdf-839c-5c6077711f49-tls-certs\") pod \"17704b56-b8ae-4bdf-839c-5c6077711f49\" (UID: \"17704b56-b8ae-4bdf-839c-5c6077711f49\") " Aug 04 15:33:23.941474 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:33:23.941343 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xwbd\" (UniqueName: \"kubernetes.io/projected/17704b56-b8ae-4bdf-839c-5c6077711f49-kube-api-access-9xwbd\") pod \"17704b56-b8ae-4bdf-839c-5c6077711f49\" (UID: \"17704b56-b8ae-4bdf-839c-5c6077711f49\") " Aug 04 15:33:23.941474 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:33:23.941365 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17704b56-b8ae-4bdf-839c-5c6077711f49-tmp-dir\") pod \"17704b56-b8ae-4bdf-839c-5c6077711f49\" (UID: \"17704b56-b8ae-4bdf-839c-5c6077711f49\") " Aug 04 15:33:23.941474 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:33:23.941423 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17704b56-b8ae-4bdf-839c-5c6077711f49-dshm\") pod \"17704b56-b8ae-4bdf-839c-5c6077711f49\" (UID: \"17704b56-b8ae-4bdf-839c-5c6077711f49\") " Aug 04 15:33:23.941701 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:33:23.941477 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17704b56-b8ae-4bdf-839c-5c6077711f49-model-cache\") pod \"17704b56-b8ae-4bdf-839c-5c6077711f49\" (UID: \"17704b56-b8ae-4bdf-839c-5c6077711f49\") " Aug 04 15:33:23.941701 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:33:23.941570 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17704b56-b8ae-4bdf-839c-5c6077711f49-home" (OuterVolumeSpecName: "home") pod "17704b56-b8ae-4bdf-839c-5c6077711f49" (UID: "17704b56-b8ae-4bdf-839c-5c6077711f49"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:33:23.941701 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:33:23.941667 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17704b56-b8ae-4bdf-839c-5c6077711f49-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "17704b56-b8ae-4bdf-839c-5c6077711f49" (UID: "17704b56-b8ae-4bdf-839c-5c6077711f49"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:33:23.941857 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:33:23.941765 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17704b56-b8ae-4bdf-839c-5c6077711f49-model-cache" (OuterVolumeSpecName: "model-cache") pod "17704b56-b8ae-4bdf-839c-5c6077711f49" (UID: "17704b56-b8ae-4bdf-839c-5c6077711f49"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:33:23.941857 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:33:23.941819 2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17704b56-b8ae-4bdf-839c-5c6077711f49-home\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:33:23.941857 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:33:23.941833 2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17704b56-b8ae-4bdf-839c-5c6077711f49-tmp-dir\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:33:23.943750 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:33:23.943724 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17704b56-b8ae-4bdf-839c-5c6077711f49-kube-api-access-9xwbd" (OuterVolumeSpecName: "kube-api-access-9xwbd") pod "17704b56-b8ae-4bdf-839c-5c6077711f49" (UID: "17704b56-b8ae-4bdf-839c-5c6077711f49"). InnerVolumeSpecName "kube-api-access-9xwbd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 15:33:23.943885 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:33:23.943758 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17704b56-b8ae-4bdf-839c-5c6077711f49-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "17704b56-b8ae-4bdf-839c-5c6077711f49" (UID: "17704b56-b8ae-4bdf-839c-5c6077711f49"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 15:33:23.943885 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:33:23.943839 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17704b56-b8ae-4bdf-839c-5c6077711f49-dshm" (OuterVolumeSpecName: "dshm") pod "17704b56-b8ae-4bdf-839c-5c6077711f49" (UID: "17704b56-b8ae-4bdf-839c-5c6077711f49"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:33:24.042536 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:33:24.042505 2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17704b56-b8ae-4bdf-839c-5c6077711f49-dshm\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:33:24.042536 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:33:24.042530 2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17704b56-b8ae-4bdf-839c-5c6077711f49-model-cache\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:33:24.042536 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:33:24.042540 2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17704b56-b8ae-4bdf-839c-5c6077711f49-tls-certs\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:33:24.042749 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:33:24.042548 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xwbd\" (UniqueName: \"kubernetes.io/projected/17704b56-b8ae-4bdf-839c-5c6077711f49-kube-api-access-9xwbd\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:33:24.546832 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:33:24.546799 2574 generic.go:356] "Generic (PLEG): container finished" podID="17704b56-b8ae-4bdf-839c-5c6077711f49" containerID="6be6efae843300f9cfaae881fd1d329a908cad7f6b09c732bc6cb9a1ea46f88f" exitCode=0 Aug 04 15:33:24.546832 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:33:24.546844 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw" Aug 04 15:33:24.547048 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:33:24.546890 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw" event={"ID":"17704b56-b8ae-4bdf-839c-5c6077711f49","Type":"ContainerDied","Data":"6be6efae843300f9cfaae881fd1d329a908cad7f6b09c732bc6cb9a1ea46f88f"} Aug 04 15:33:24.547048 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:33:24.546947 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw" event={"ID":"17704b56-b8ae-4bdf-839c-5c6077711f49","Type":"ContainerDied","Data":"5c0d30ff1526af4c6d00d91e3d2235ba98429c3586da263e67d6baa22947b7db"} Aug 04 15:33:24.547048 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:33:24.546971 2574 scope.go:117] "RemoveContainer" containerID="6be6efae843300f9cfaae881fd1d329a908cad7f6b09c732bc6cb9a1ea46f88f" Aug 04 15:33:24.556579 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:33:24.556564 2574 scope.go:117] "RemoveContainer" containerID="6be6efae843300f9cfaae881fd1d329a908cad7f6b09c732bc6cb9a1ea46f88f" Aug 04 15:33:24.556827 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:33:24.556808 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6be6efae843300f9cfaae881fd1d329a908cad7f6b09c732bc6cb9a1ea46f88f\": container with ID starting with 6be6efae843300f9cfaae881fd1d329a908cad7f6b09c732bc6cb9a1ea46f88f not found: ID does not exist" containerID="6be6efae843300f9cfaae881fd1d329a908cad7f6b09c732bc6cb9a1ea46f88f" Aug 04 15:33:24.556918 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:33:24.556833 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6be6efae843300f9cfaae881fd1d329a908cad7f6b09c732bc6cb9a1ea46f88f"} err="failed to get container status \"6be6efae843300f9cfaae881fd1d329a908cad7f6b09c732bc6cb9a1ea46f88f\": rpc error: code = NotFound desc = could not find container \"6be6efae843300f9cfaae881fd1d329a908cad7f6b09c732bc6cb9a1ea46f88f\": container with ID starting with 6be6efae843300f9cfaae881fd1d329a908cad7f6b09c732bc6cb9a1ea46f88f not found: ID does not exist" Aug 04 15:33:24.577376 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:33:24.577332 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw"] Aug 04 15:33:24.577524 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:33:24.577401 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-ffe4e6a1-kserve-5565b6c4cf-79fhw"] Aug 04 15:33:25.053682 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:33:25.053649 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17704b56-b8ae-4bdf-839c-5c6077711f49" path="/var/lib/kubelet/pods/17704b56-b8ae-4bdf-839c-5c6077711f49/volumes" Aug 04 15:33:25.720821 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:33:25.720789 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:33:38.329485 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:33:38.329450 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:33:58.122375 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:33:58.122327 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:34:12.522300 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:34:12.522265 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:34:22.628241 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:34:22.628205 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:34:33.057739 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:34:33.057701 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:34:42.630072 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:34:42.629992 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:34:52.533341 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:34:52.533306 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:35:06.849184 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:35:06.849143 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:35:21.418788 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:35:21.418751 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:35:31.028430 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:35:31.028380 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:35:39.824117 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:35:39.824083 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:35:51.335397 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:35:51.335350 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:36:00.834964 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:36:00.834924 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:36:08.319579 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:36:08.319545 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:36:17.150593 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:36:17.150562 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-r8wkp_0580fdab-0381-4d8e-be5f-bde5eb44e0e6/console-operator/1.log" Aug 04 15:36:17.153904 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:36:17.153881 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-r8wkp_0580fdab-0381-4d8e-be5f-bde5eb44e0e6/console-operator/1.log" Aug 04 15:36:17.154503 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:36:17.154487 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8h6k_9c62e3b1-c2dc-4b2a-a61d-7d80049f355f/ovn-acl-logging/0.log" Aug 04 15:36:17.157748 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:36:17.157725 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8h6k_9c62e3b1-c2dc-4b2a-a61d-7d80049f355f/ovn-acl-logging/0.log" Aug 04 15:36:19.322801 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:36:19.322766 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:36:34.328270 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:36:34.328232 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:36:49.524367 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:36:49.524329 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:36:58.921769 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:36:58.921734 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:37:12.222066 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:12.222028 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:37:26.822622 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:26.822586 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:37:36.323975 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:36.323936 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:37:46.227292 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:46.227214 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:37:48.203494 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:48.203460 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p"] Aug 04 15:37:48.203857 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:48.203848 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17704b56-b8ae-4bdf-839c-5c6077711f49" containerName="main" Aug 04 15:37:48.203857 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:48.203858 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="17704b56-b8ae-4bdf-839c-5c6077711f49" containerName="main" Aug 04 15:37:48.203941 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:48.203919 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="17704b56-b8ae-4bdf-839c-5c6077711f49" containerName="main" Aug 04 15:37:48.207081 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:48.207060 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" Aug 04 15:37:48.210689 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:48.210664 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-dc6f2748-openshift-default-dockercfg-8wnkk\"" Aug 04 15:37:48.253208 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:48.253173 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p"] Aug 04 15:37:48.304917 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:48.304883 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f343916-0e44-4a8b-bf7f-8ba980997b5a-credential-socket\") pod \"e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p\" (UID: \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\") " pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" Aug 04 15:37:48.305091 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:48.304940 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f343916-0e44-4a8b-bf7f-8ba980997b5a-istio-data\") pod \"e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p\" (UID: \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\") " pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" Aug 04 15:37:48.305091 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:48.305021 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68d9v\" (UniqueName: \"kubernetes.io/projected/6f343916-0e44-4a8b-bf7f-8ba980997b5a-kube-api-access-68d9v\") pod \"e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p\" (UID: \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\") " pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" Aug 04 15:37:48.305220 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:48.305093 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f343916-0e44-4a8b-bf7f-8ba980997b5a-istio-token\") pod \"e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p\" (UID: \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\") " pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" Aug 04 15:37:48.305220 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:48.305154 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f343916-0e44-4a8b-bf7f-8ba980997b5a-workload-socket\") pod \"e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p\" (UID: \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\") " pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" Aug 04 15:37:48.305220 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:48.305188 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f343916-0e44-4a8b-bf7f-8ba980997b5a-istiod-ca-cert\") pod \"e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p\" (UID: \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\") " pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" Aug 04 15:37:48.305376 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:48.305255 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f343916-0e44-4a8b-bf7f-8ba980997b5a-workload-certs\") pod \"e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p\" (UID: \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\") " pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" Aug 04 15:37:48.305376 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:48.305280 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f343916-0e44-4a8b-bf7f-8ba980997b5a-istio-podinfo\") pod \"e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p\" (UID: \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\") " pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" Aug 04 15:37:48.305376 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:48.305312 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f343916-0e44-4a8b-bf7f-8ba980997b5a-istio-envoy\") pod \"e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p\" (UID: \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\") " pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" Aug 04 15:37:48.406361 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:48.406312 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68d9v\" (UniqueName: \"kubernetes.io/projected/6f343916-0e44-4a8b-bf7f-8ba980997b5a-kube-api-access-68d9v\") pod \"e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p\" (UID: \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\") " pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" Aug 04 15:37:48.406361 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:48.406367 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f343916-0e44-4a8b-bf7f-8ba980997b5a-istio-token\") pod \"e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p\" (UID: \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\") " pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" Aug 04 15:37:48.406643 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:48.406411 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f343916-0e44-4a8b-bf7f-8ba980997b5a-workload-socket\") pod \"e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p\" (UID: \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\") " pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" Aug 04 15:37:48.406643 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:48.406440 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f343916-0e44-4a8b-bf7f-8ba980997b5a-istiod-ca-cert\") pod \"e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p\" (UID: \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\") " pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" Aug 04 15:37:48.406643 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:48.406471 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f343916-0e44-4a8b-bf7f-8ba980997b5a-workload-certs\") pod \"e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p\" (UID: \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\") " pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" Aug 04 15:37:48.406643 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:48.406492 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f343916-0e44-4a8b-bf7f-8ba980997b5a-istio-podinfo\") pod \"e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p\" (UID: \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\") " pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" Aug 04 15:37:48.406643 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:48.406511 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f343916-0e44-4a8b-bf7f-8ba980997b5a-istio-envoy\") pod \"e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p\" (UID: \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\") " pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" Aug 04 15:37:48.406643 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:48.406582 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f343916-0e44-4a8b-bf7f-8ba980997b5a-credential-socket\") pod \"e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p\" (UID: \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\") " pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" Aug 04 15:37:48.406643 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:48.406619 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f343916-0e44-4a8b-bf7f-8ba980997b5a-istio-data\") pod \"e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p\" (UID: \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\") " pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" Aug 04 15:37:48.407011 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:48.406915 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f343916-0e44-4a8b-bf7f-8ba980997b5a-workload-certs\") pod \"e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p\" (UID: \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\") " pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" Aug 04 15:37:48.407011 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:48.406993 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f343916-0e44-4a8b-bf7f-8ba980997b5a-credential-socket\") pod \"e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p\" (UID: \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\") " pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" Aug 04 15:37:48.407112 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:48.407043 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f343916-0e44-4a8b-bf7f-8ba980997b5a-istio-data\") pod \"e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p\" (UID: \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\") " pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" Aug 04 15:37:48.407204 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:48.407184 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f343916-0e44-4a8b-bf7f-8ba980997b5a-workload-socket\") pod \"e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p\" (UID: \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\") " pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" Aug 04 15:37:48.407290 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:48.407268 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f343916-0e44-4a8b-bf7f-8ba980997b5a-istiod-ca-cert\") pod \"e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p\" (UID: \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\") " pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" Aug 04 15:37:48.408943 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:48.408920 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f343916-0e44-4a8b-bf7f-8ba980997b5a-istio-podinfo\") pod \"e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p\" (UID: \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\") " pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" Aug 04 15:37:48.409021 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:48.408989 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f343916-0e44-4a8b-bf7f-8ba980997b5a-istio-envoy\") pod \"e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p\" (UID: \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\") " pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" Aug 04 15:37:48.415292 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:48.415273 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f343916-0e44-4a8b-bf7f-8ba980997b5a-istio-token\") pod \"e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p\" (UID: \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\") " pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" Aug 04 15:37:48.417714 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:48.417682 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68d9v\" (UniqueName: \"kubernetes.io/projected/6f343916-0e44-4a8b-bf7f-8ba980997b5a-kube-api-access-68d9v\") pod \"e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p\" (UID: \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\") " pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" Aug 04 15:37:48.518020 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:48.517995 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" Aug 04 15:37:48.663694 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:48.663669 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p"] Aug 04 15:37:48.665948 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:37:48.665915 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f343916_0e44_4a8b_bf7f_8ba980997b5a.slice/crio-45c53d16a057b7b39490fed227ff55fa8ca4fa3589db58ab2fedc6e31d0343e7 WatchSource:0}: Error finding container 45c53d16a057b7b39490fed227ff55fa8ca4fa3589db58ab2fedc6e31d0343e7: Status 404 returned error can't find the container with id 45c53d16a057b7b39490fed227ff55fa8ca4fa3589db58ab2fedc6e31d0343e7 Aug 04 15:37:48.668113 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:48.668076 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:37:48.668227 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:48.668142 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:37:48.668227 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:48.668177 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:37:48.980951 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:48.980867 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" event={"ID":"6f343916-0e44-4a8b-bf7f-8ba980997b5a","Type":"ContainerStarted","Data":"e10e3bb626c2eb87dbb7b74bf374be11aea83802eb236eee6dad46d0c5f4a284"} Aug 04 15:37:48.980951 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:48.980930 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" event={"ID":"6f343916-0e44-4a8b-bf7f-8ba980997b5a","Type":"ContainerStarted","Data":"45c53d16a057b7b39490fed227ff55fa8ca4fa3589db58ab2fedc6e31d0343e7"} Aug 04 15:37:49.518725 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:49.518683 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" Aug 04 15:37:49.523879 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:49.523849 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" Aug 04 15:37:49.554718 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:49.554661 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" podStartSLOduration=1.554644439 podStartE2EDuration="1.554644439s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:37:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:37:49.010686252 +0000 UTC m=+1892.470558601" watchObservedRunningTime="2026-08-04 15:37:49.554644439 +0000 UTC m=+1893.014516788" Aug 04 15:37:49.989061 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:49.989032 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" Aug 04 15:37:49.990052 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:49.990030 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" Aug 04 15:37:57.258069 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:37:57.258034 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:38:06.031569 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:06.031520 2574 patch_prober.go:28] interesting pod/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Aug 04 15:38:06.031980 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:06.031599 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" podUID="6f343916-0e44-4a8b-bf7f-8ba980997b5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Aug 04 15:38:07.085533 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:07.085500 2574 generic.go:356] "Generic (PLEG): container finished" podID="6f343916-0e44-4a8b-bf7f-8ba980997b5a" containerID="e10e3bb626c2eb87dbb7b74bf374be11aea83802eb236eee6dad46d0c5f4a284" exitCode=0 Aug 04 15:38:07.085962 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:07.085575 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" event={"ID":"6f343916-0e44-4a8b-bf7f-8ba980997b5a","Type":"ContainerDied","Data":"e10e3bb626c2eb87dbb7b74bf374be11aea83802eb236eee6dad46d0c5f4a284"} Aug 04 15:38:07.085962 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:07.085959 2574 scope.go:117] "RemoveContainer" containerID="e10e3bb626c2eb87dbb7b74bf374be11aea83802eb236eee6dad46d0c5f4a284" Aug 04 15:38:07.086990 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:07.086958 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:38:07.087093 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:07.087043 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:38:07.087155 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:07.087090 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:38:08.092427 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:08.092376 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" event={"ID":"6f343916-0e44-4a8b-bf7f-8ba980997b5a","Type":"ContainerStarted","Data":"5424ec375359d00884ae0de7c8d12c3b9b6c3846dfdb696dad48f70906241462"} Aug 04 15:38:08.518904 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:08.518867 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" Aug 04 15:38:08.519228 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:08.519203 2574 patch_prober.go:28] interesting pod/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Aug 04 15:38:08.519281 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:08.519252 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" podUID="6f343916-0e44-4a8b-bf7f-8ba980997b5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Aug 04 15:38:09.518534 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:09.518496 2574 patch_prober.go:28] interesting pod/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Aug 04 15:38:09.518918 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:09.518552 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" podUID="6f343916-0e44-4a8b-bf7f-8ba980997b5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Aug 04 15:38:10.519022 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:10.518985 2574 patch_prober.go:28] interesting pod/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Aug 04 15:38:10.519429 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:10.519039 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" podUID="6f343916-0e44-4a8b-bf7f-8ba980997b5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Aug 04 15:38:11.519219 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:11.519181 2574 patch_prober.go:28] interesting pod/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Aug 04 15:38:11.519717 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:11.519271 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" podUID="6f343916-0e44-4a8b-bf7f-8ba980997b5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Aug 04 15:38:12.521882 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:12.521854 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" Aug 04 15:38:12.522299 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:12.522121 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" Aug 04 15:38:12.522866 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:12.522850 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" Aug 04 15:38:32.981704 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:32.981663 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p"] Aug 04 15:38:32.982211 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:32.981954 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" podUID="6f343916-0e44-4a8b-bf7f-8ba980997b5a" containerName="istio-proxy" containerID="cri-o://5424ec375359d00884ae0de7c8d12c3b9b6c3846dfdb696dad48f70906241462" gracePeriod=30 Aug 04 15:38:34.164425 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:34.164375 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph"] Aug 04 15:38:34.168115 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:34.168088 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" Aug 04 15:38:34.172691 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:34.172673 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-2a0c7713-gw-openshift-default-dockercfg-76z2b\"" Aug 04 15:38:34.191481 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:34.191462 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph"] Aug 04 15:38:34.313747 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:34.313712 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-istio-token\") pod \"e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph\" (UID: \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\") " pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" Aug 04 15:38:34.313912 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:34.313749 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wlrv\" (UniqueName: \"kubernetes.io/projected/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-kube-api-access-6wlrv\") pod \"e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph\" (UID: \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\") " pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" Aug 04 15:38:34.313912 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:34.313778 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-workload-socket\") pod \"e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph\" (UID: \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\") " pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" Aug 04 15:38:34.313912 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:34.313835 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-credential-socket\") pod \"e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph\" (UID: \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\") " pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" Aug 04 15:38:34.313912 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:34.313894 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-istio-podinfo\") pod \"e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph\" (UID: \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\") " pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" Aug 04 15:38:34.314085 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:34.313916 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-workload-certs\") pod \"e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph\" (UID: \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\") " pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" Aug 04 15:38:34.314085 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:34.313951 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-istio-data\") pod \"e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph\" (UID: \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\") " pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" Aug 04 15:38:34.314085 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:34.314028 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-istio-envoy\") pod \"e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph\" (UID: \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\") " pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" Aug 04 15:38:34.314085 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:34.314057 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-istiod-ca-cert\") pod \"e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph\" (UID: \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\") " pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" Aug 04 15:38:34.415191 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:34.415115 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-istiod-ca-cert\") pod \"e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph\" (UID: \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\") " pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" Aug 04 15:38:34.415191 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:34.415164 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-istio-token\") pod \"e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph\" (UID: \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\") " pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" Aug 04 15:38:34.415191 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:34.415183 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wlrv\" (UniqueName: \"kubernetes.io/projected/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-kube-api-access-6wlrv\") pod \"e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph\" (UID: \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\") " pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" Aug 04 15:38:34.415511 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:34.415203 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-workload-socket\") pod \"e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph\" (UID: \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\") " pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" Aug 04 15:38:34.415511 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:34.415225 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-credential-socket\") pod \"e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph\" (UID: \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\") " pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" Aug 04 15:38:34.415511 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:34.415251 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-istio-podinfo\") pod \"e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph\" (UID: \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\") " pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" Aug 04 15:38:34.415511 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:34.415266 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-workload-certs\") pod \"e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph\" (UID: \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\") " pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" Aug 04 15:38:34.415511 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:34.415352 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-istio-data\") pod \"e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph\" (UID: \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\") " pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" Aug 04 15:38:34.415511 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:34.415476 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-istio-envoy\") pod \"e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph\" (UID: \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\") " pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" Aug 04 15:38:34.415820 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:34.415646 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-workload-certs\") pod \"e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph\" (UID: \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\") " pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" Aug 04 15:38:34.415820 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:34.415678 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-credential-socket\") pod \"e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph\" (UID: \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\") " pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" Aug 04 15:38:34.415820 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:34.415748 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-istio-data\") pod \"e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph\" (UID: \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\") " pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" Aug 04 15:38:34.415963 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:34.415894 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-workload-socket\") pod \"e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph\" (UID: \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\") " pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" Aug 04 15:38:34.415963 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:34.415934 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-istiod-ca-cert\") pod \"e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph\" (UID: \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\") " pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" Aug 04 15:38:34.417687 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:34.417665 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-istio-envoy\") pod \"e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph\" (UID: \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\") " pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" Aug 04 15:38:34.417905 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:34.417886 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-istio-podinfo\") pod \"e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph\" (UID: \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\") " pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" Aug 04 15:38:34.431867 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:34.431844 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wlrv\" (UniqueName: \"kubernetes.io/projected/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-kube-api-access-6wlrv\") pod \"e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph\" (UID: \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\") " pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" Aug 04 15:38:34.432105 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:34.432081 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-istio-token\") pod \"e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph\" (UID: \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\") " pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" Aug 04 15:38:34.478105 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:34.478080 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" Aug 04 15:38:34.635696 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:34.635671 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph"] Aug 04 15:38:34.637502 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:38:34.637477 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod514f7176_0ac8_4adc_aa63_ef3382cd4ca1.slice/crio-19bc2b5205c0f3f2085b29abe531df70769e8c908911db86df1ee883d4d3a19b WatchSource:0}: Error finding container 19bc2b5205c0f3f2085b29abe531df70769e8c908911db86df1ee883d4d3a19b: Status 404 returned error can't find the container with id 19bc2b5205c0f3f2085b29abe531df70769e8c908911db86df1ee883d4d3a19b Aug 04 15:38:34.639695 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:34.639662 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:38:34.639764 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:34.639728 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:38:34.639764 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:34.639756 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:38:35.250435 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:35.250378 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" event={"ID":"514f7176-0ac8-4adc-aa63-ef3382cd4ca1","Type":"ContainerStarted","Data":"3852ff1d066aff5b2dff4e1cda1e23510f7eda275adc64797d77f0f84c1c79f9"} Aug 04 15:38:35.250862 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:35.250446 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" event={"ID":"514f7176-0ac8-4adc-aa63-ef3382cd4ca1","Type":"ContainerStarted","Data":"19bc2b5205c0f3f2085b29abe531df70769e8c908911db86df1ee883d4d3a19b"} Aug 04 15:38:35.275433 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:35.275369 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" podStartSLOduration=1.275353409 podStartE2EDuration="1.275353409s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:38:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:38:35.273620422 +0000 UTC m=+1938.733492771" watchObservedRunningTime="2026-08-04 15:38:35.275353409 +0000 UTC m=+1938.735225836" Aug 04 15:38:35.478865 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:35.478832 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" Aug 04 15:38:35.483562 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:35.483536 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" Aug 04 15:38:36.255842 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:36.255814 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" Aug 04 15:38:36.256751 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:36.256733 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" Aug 04 15:38:36.347463 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:36.343364 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:38:38.231056 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:38.231030 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" Aug 04 15:38:38.268131 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:38.268069 2574 generic.go:356] "Generic (PLEG): container finished" podID="6f343916-0e44-4a8b-bf7f-8ba980997b5a" containerID="5424ec375359d00884ae0de7c8d12c3b9b6c3846dfdb696dad48f70906241462" exitCode=0 Aug 04 15:38:38.268131 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:38.268116 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" Aug 04 15:38:38.268265 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:38.268160 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" event={"ID":"6f343916-0e44-4a8b-bf7f-8ba980997b5a","Type":"ContainerDied","Data":"5424ec375359d00884ae0de7c8d12c3b9b6c3846dfdb696dad48f70906241462"} Aug 04 15:38:38.268265 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:38.268216 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p" event={"ID":"6f343916-0e44-4a8b-bf7f-8ba980997b5a","Type":"ContainerDied","Data":"45c53d16a057b7b39490fed227ff55fa8ca4fa3589db58ab2fedc6e31d0343e7"} Aug 04 15:38:38.268265 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:38.268245 2574 scope.go:117] "RemoveContainer" containerID="5424ec375359d00884ae0de7c8d12c3b9b6c3846dfdb696dad48f70906241462" Aug 04 15:38:38.276991 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:38.276973 2574 scope.go:117] "RemoveContainer" containerID="e10e3bb626c2eb87dbb7b74bf374be11aea83802eb236eee6dad46d0c5f4a284" Aug 04 15:38:38.285335 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:38.285320 2574 scope.go:117] "RemoveContainer" containerID="5424ec375359d00884ae0de7c8d12c3b9b6c3846dfdb696dad48f70906241462" Aug 04 15:38:38.285624 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:38:38.285605 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5424ec375359d00884ae0de7c8d12c3b9b6c3846dfdb696dad48f70906241462\": container with ID starting with 5424ec375359d00884ae0de7c8d12c3b9b6c3846dfdb696dad48f70906241462 not found: ID does not exist" containerID="5424ec375359d00884ae0de7c8d12c3b9b6c3846dfdb696dad48f70906241462" Aug 04 15:38:38.285672 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:38.285631 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5424ec375359d00884ae0de7c8d12c3b9b6c3846dfdb696dad48f70906241462"} err="failed to get container status \"5424ec375359d00884ae0de7c8d12c3b9b6c3846dfdb696dad48f70906241462\": rpc error: code = NotFound desc = could not find container \"5424ec375359d00884ae0de7c8d12c3b9b6c3846dfdb696dad48f70906241462\": container with ID starting with 5424ec375359d00884ae0de7c8d12c3b9b6c3846dfdb696dad48f70906241462 not found: ID does not exist" Aug 04 15:38:38.285672 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:38.285647 2574 scope.go:117] "RemoveContainer" containerID="e10e3bb626c2eb87dbb7b74bf374be11aea83802eb236eee6dad46d0c5f4a284" Aug 04 15:38:38.285865 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:38:38.285842 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e10e3bb626c2eb87dbb7b74bf374be11aea83802eb236eee6dad46d0c5f4a284\": container with ID starting with e10e3bb626c2eb87dbb7b74bf374be11aea83802eb236eee6dad46d0c5f4a284 not found: ID does not exist" containerID="e10e3bb626c2eb87dbb7b74bf374be11aea83802eb236eee6dad46d0c5f4a284" Aug 04 15:38:38.285907 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:38.285872 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e10e3bb626c2eb87dbb7b74bf374be11aea83802eb236eee6dad46d0c5f4a284"} err="failed to get container status \"e10e3bb626c2eb87dbb7b74bf374be11aea83802eb236eee6dad46d0c5f4a284\": rpc error: code = NotFound desc = could not find container \"e10e3bb626c2eb87dbb7b74bf374be11aea83802eb236eee6dad46d0c5f4a284\": container with ID starting with e10e3bb626c2eb87dbb7b74bf374be11aea83802eb236eee6dad46d0c5f4a284 not found: ID does not exist" Aug 04 15:38:38.354114 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:38.354085 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-68d9v\" (UniqueName: \"kubernetes.io/projected/6f343916-0e44-4a8b-bf7f-8ba980997b5a-kube-api-access-68d9v\") pod \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\" (UID: \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\") " Aug 04 15:38:38.354287 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:38.354122 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f343916-0e44-4a8b-bf7f-8ba980997b5a-istio-token\") pod \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\" (UID: \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\") " Aug 04 15:38:38.354287 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:38.354157 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f343916-0e44-4a8b-bf7f-8ba980997b5a-istiod-ca-cert\") pod \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\" (UID: \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\") " Aug 04 15:38:38.354287 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:38.354202 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f343916-0e44-4a8b-bf7f-8ba980997b5a-workload-certs\") pod \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\" (UID: \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\") " Aug 04 15:38:38.354287 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:38.354266 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f343916-0e44-4a8b-bf7f-8ba980997b5a-istio-data\") pod \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\" (UID: \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\") " Aug 04 15:38:38.354534 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:38.354291 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f343916-0e44-4a8b-bf7f-8ba980997b5a-workload-socket\") pod \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\" (UID: \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\") " Aug 04 15:38:38.354534 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:38.354313 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f343916-0e44-4a8b-bf7f-8ba980997b5a-istio-podinfo\") pod \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\" (UID: \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\") " Aug 04 15:38:38.354534 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:38.354340 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f343916-0e44-4a8b-bf7f-8ba980997b5a-istio-envoy\") pod \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\" (UID: \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\") " Aug 04 15:38:38.354534 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:38.354441 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f343916-0e44-4a8b-bf7f-8ba980997b5a-credential-socket\") pod \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\" (UID: \"6f343916-0e44-4a8b-bf7f-8ba980997b5a\") " Aug 04 15:38:38.354719 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:38.354596 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f343916-0e44-4a8b-bf7f-8ba980997b5a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6f343916-0e44-4a8b-bf7f-8ba980997b5a" (UID: "6f343916-0e44-4a8b-bf7f-8ba980997b5a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:38:38.354719 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:38.354610 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f343916-0e44-4a8b-bf7f-8ba980997b5a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6f343916-0e44-4a8b-bf7f-8ba980997b5a" (UID: "6f343916-0e44-4a8b-bf7f-8ba980997b5a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 15:38:38.354719 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:38.354618 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f343916-0e44-4a8b-bf7f-8ba980997b5a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6f343916-0e44-4a8b-bf7f-8ba980997b5a" (UID: "6f343916-0e44-4a8b-bf7f-8ba980997b5a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:38:38.354887 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:38.354862 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f343916-0e44-4a8b-bf7f-8ba980997b5a-istiod-ca-cert\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:38:38.354887 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:38.354884 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f343916-0e44-4a8b-bf7f-8ba980997b5a-workload-certs\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:38:38.354993 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:38.354897 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f343916-0e44-4a8b-bf7f-8ba980997b5a-workload-socket\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:38:38.355295 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:38.355086 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f343916-0e44-4a8b-bf7f-8ba980997b5a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6f343916-0e44-4a8b-bf7f-8ba980997b5a" (UID: "6f343916-0e44-4a8b-bf7f-8ba980997b5a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:38:38.355295 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:38.355181 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f343916-0e44-4a8b-bf7f-8ba980997b5a-istio-data" (OuterVolumeSpecName: "istio-data") pod "6f343916-0e44-4a8b-bf7f-8ba980997b5a" (UID: "6f343916-0e44-4a8b-bf7f-8ba980997b5a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:38:38.356600 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:38.356567 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f343916-0e44-4a8b-bf7f-8ba980997b5a-istio-token" (OuterVolumeSpecName: "istio-token") pod "6f343916-0e44-4a8b-bf7f-8ba980997b5a" (UID: "6f343916-0e44-4a8b-bf7f-8ba980997b5a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 15:38:38.356721 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:38.356703 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f343916-0e44-4a8b-bf7f-8ba980997b5a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6f343916-0e44-4a8b-bf7f-8ba980997b5a" (UID: "6f343916-0e44-4a8b-bf7f-8ba980997b5a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:38:38.357011 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:38.356992 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f343916-0e44-4a8b-bf7f-8ba980997b5a-kube-api-access-68d9v" (OuterVolumeSpecName: "kube-api-access-68d9v") pod "6f343916-0e44-4a8b-bf7f-8ba980997b5a" (UID: "6f343916-0e44-4a8b-bf7f-8ba980997b5a"). InnerVolumeSpecName "kube-api-access-68d9v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 15:38:38.357096 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:38.357080 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6f343916-0e44-4a8b-bf7f-8ba980997b5a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6f343916-0e44-4a8b-bf7f-8ba980997b5a" (UID: "6f343916-0e44-4a8b-bf7f-8ba980997b5a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 15:38:38.455627 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:38.455595 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-68d9v\" (UniqueName: \"kubernetes.io/projected/6f343916-0e44-4a8b-bf7f-8ba980997b5a-kube-api-access-68d9v\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:38:38.455627 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:38.455622 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f343916-0e44-4a8b-bf7f-8ba980997b5a-istio-token\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:38:38.455627 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:38.455633 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f343916-0e44-4a8b-bf7f-8ba980997b5a-istio-data\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:38:38.455840 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:38.455640 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f343916-0e44-4a8b-bf7f-8ba980997b5a-istio-podinfo\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:38:38.455840 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:38.455649 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f343916-0e44-4a8b-bf7f-8ba980997b5a-istio-envoy\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:38:38.455840 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:38.455657 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f343916-0e44-4a8b-bf7f-8ba980997b5a-credential-socket\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:38:38.598219 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:38.598188 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p"] Aug 04 15:38:38.604510 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:38.604488 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-dc6f2748-openshift-default-7c454dbcfb-k789p"] Aug 04 15:38:39.053475 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:39.053434 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f343916-0e44-4a8b-bf7f-8ba980997b5a" path="/var/lib/kubelet/pods/6f343916-0e44-4a8b-bf7f-8ba980997b5a/volumes" Aug 04 15:38:46.215251 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:46.215221 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:38:50.340847 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:50.340764 2574 generic.go:356] "Generic (PLEG): container finished" podID="514f7176-0ac8-4adc-aa63-ef3382cd4ca1" containerID="3852ff1d066aff5b2dff4e1cda1e23510f7eda275adc64797d77f0f84c1c79f9" exitCode=0 Aug 04 15:38:50.341294 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:50.340841 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" event={"ID":"514f7176-0ac8-4adc-aa63-ef3382cd4ca1","Type":"ContainerDied","Data":"3852ff1d066aff5b2dff4e1cda1e23510f7eda275adc64797d77f0f84c1c79f9"} Aug 04 15:38:50.341294 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:50.341241 2574 scope.go:117] "RemoveContainer" containerID="3852ff1d066aff5b2dff4e1cda1e23510f7eda275adc64797d77f0f84c1c79f9" Aug 04 15:38:50.342247 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:50.342211 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:38:50.342340 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:50.342285 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:38:50.342340 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:50.342315 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:38:51.256656 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:51.256620 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" Aug 04 15:38:51.351343 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:51.351307 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" event={"ID":"514f7176-0ac8-4adc-aa63-ef3382cd4ca1","Type":"ContainerStarted","Data":"eee73087fde26b2798d32ad39d4c820e386583c655bbb3e8d49575a9d6c4ae6d"} Aug 04 15:38:51.479008 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:51.478968 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" Aug 04 15:38:51.480280 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:51.480247 2574 patch_prober.go:28] interesting pod/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Aug 04 15:38:51.480454 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:51.480301 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" podUID="514f7176-0ac8-4adc-aa63-ef3382cd4ca1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Aug 04 15:38:52.478572 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:52.478525 2574 patch_prober.go:28] interesting pod/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Aug 04 15:38:52.479075 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:52.478586 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" podUID="514f7176-0ac8-4adc-aa63-ef3382cd4ca1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Aug 04 15:38:53.478728 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:53.478693 2574 patch_prober.go:28] interesting pod/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Aug 04 15:38:53.479119 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:53.478745 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" podUID="514f7176-0ac8-4adc-aa63-ef3382cd4ca1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Aug 04 15:38:54.478658 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:54.478618 2574 patch_prober.go:28] interesting pod/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Aug 04 15:38:54.478830 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:54.478678 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" podUID="514f7176-0ac8-4adc-aa63-ef3382cd4ca1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Aug 04 15:38:55.482259 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:55.482225 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" Aug 04 15:38:55.482742 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:55.482537 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" Aug 04 15:38:55.483168 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:38:55.483149 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" Aug 04 15:39:09.045232 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:09.045192 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph"] Aug 04 15:39:09.045759 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:09.045448 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" podUID="514f7176-0ac8-4adc-aa63-ef3382cd4ca1" containerName="istio-proxy" containerID="cri-o://eee73087fde26b2798d32ad39d4c820e386583c655bbb3e8d49575a9d6c4ae6d" gracePeriod=30 Aug 04 15:39:09.928912 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:09.928874 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l"] Aug 04 15:39:09.929507 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:09.929490 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f343916-0e44-4a8b-bf7f-8ba980997b5a" containerName="istio-proxy" Aug 04 15:39:09.929570 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:09.929511 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f343916-0e44-4a8b-bf7f-8ba980997b5a" containerName="istio-proxy" Aug 04 15:39:09.929570 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:09.929529 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f343916-0e44-4a8b-bf7f-8ba980997b5a" containerName="istio-proxy" Aug 04 15:39:09.929570 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:09.929537 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f343916-0e44-4a8b-bf7f-8ba980997b5a" containerName="istio-proxy" Aug 04 15:39:09.929664 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:09.929652 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f343916-0e44-4a8b-bf7f-8ba980997b5a" containerName="istio-proxy" Aug 04 15:39:09.929698 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:09.929666 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f343916-0e44-4a8b-bf7f-8ba980997b5a" containerName="istio-proxy" Aug 04 15:39:09.933449 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:09.933430 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" Aug 04 15:39:09.937697 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:09.937678 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-8202ac71-openshift-default-dockercfg-wkn2z\"" Aug 04 15:39:09.946725 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:09.946701 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l"] Aug 04 15:39:10.043201 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:10.043166 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6c8d44c-0d49-493a-b995-f8eb8236d663-istio-data\") pod \"e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l\" (UID: \"d6c8d44c-0d49-493a-b995-f8eb8236d663\") " pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" Aug 04 15:39:10.043201 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:10.043202 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6c8d44c-0d49-493a-b995-f8eb8236d663-istiod-ca-cert\") pod \"e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l\" (UID: \"d6c8d44c-0d49-493a-b995-f8eb8236d663\") " pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" Aug 04 15:39:10.043460 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:10.043225 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thd86\" (UniqueName: \"kubernetes.io/projected/d6c8d44c-0d49-493a-b995-f8eb8236d663-kube-api-access-thd86\") pod \"e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l\" (UID: \"d6c8d44c-0d49-493a-b995-f8eb8236d663\") " pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" Aug 04 15:39:10.043460 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:10.043281 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6c8d44c-0d49-493a-b995-f8eb8236d663-istio-envoy\") pod \"e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l\" (UID: \"d6c8d44c-0d49-493a-b995-f8eb8236d663\") " pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" Aug 04 15:39:10.043460 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:10.043369 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6c8d44c-0d49-493a-b995-f8eb8236d663-istio-podinfo\") pod \"e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l\" (UID: \"d6c8d44c-0d49-493a-b995-f8eb8236d663\") " pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" Aug 04 15:39:10.043460 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:10.043423 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6c8d44c-0d49-493a-b995-f8eb8236d663-istio-token\") pod \"e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l\" (UID: \"d6c8d44c-0d49-493a-b995-f8eb8236d663\") " pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" Aug 04 15:39:10.043624 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:10.043507 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6c8d44c-0d49-493a-b995-f8eb8236d663-credential-socket\") pod \"e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l\" (UID: \"d6c8d44c-0d49-493a-b995-f8eb8236d663\") " pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" Aug 04 15:39:10.043624 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:10.043535 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6c8d44c-0d49-493a-b995-f8eb8236d663-workload-certs\") pod \"e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l\" (UID: \"d6c8d44c-0d49-493a-b995-f8eb8236d663\") " pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" Aug 04 15:39:10.043624 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:10.043592 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6c8d44c-0d49-493a-b995-f8eb8236d663-workload-socket\") pod \"e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l\" (UID: \"d6c8d44c-0d49-493a-b995-f8eb8236d663\") " pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" Aug 04 15:39:10.145037 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:10.145003 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6c8d44c-0d49-493a-b995-f8eb8236d663-credential-socket\") pod \"e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l\" (UID: \"d6c8d44c-0d49-493a-b995-f8eb8236d663\") " pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" Aug 04 15:39:10.145037 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:10.145038 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6c8d44c-0d49-493a-b995-f8eb8236d663-workload-certs\") pod \"e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l\" (UID: \"d6c8d44c-0d49-493a-b995-f8eb8236d663\") " pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" Aug 04 15:39:10.145595 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:10.145078 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6c8d44c-0d49-493a-b995-f8eb8236d663-workload-socket\") pod \"e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l\" (UID: \"d6c8d44c-0d49-493a-b995-f8eb8236d663\") " pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" Aug 04 15:39:10.145595 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:10.145128 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6c8d44c-0d49-493a-b995-f8eb8236d663-istio-data\") pod \"e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l\" (UID: \"d6c8d44c-0d49-493a-b995-f8eb8236d663\") " pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" Aug 04 15:39:10.145595 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:10.145153 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6c8d44c-0d49-493a-b995-f8eb8236d663-istiod-ca-cert\") pod \"e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l\" (UID: \"d6c8d44c-0d49-493a-b995-f8eb8236d663\") " pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" Aug 04 15:39:10.145595 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:10.145177 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thd86\" (UniqueName: \"kubernetes.io/projected/d6c8d44c-0d49-493a-b995-f8eb8236d663-kube-api-access-thd86\") pod \"e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l\" (UID: \"d6c8d44c-0d49-493a-b995-f8eb8236d663\") " pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" Aug 04 15:39:10.145595 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:10.145244 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6c8d44c-0d49-493a-b995-f8eb8236d663-istio-envoy\") pod \"e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l\" (UID: \"d6c8d44c-0d49-493a-b995-f8eb8236d663\") " pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" Aug 04 15:39:10.145595 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:10.145284 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6c8d44c-0d49-493a-b995-f8eb8236d663-istio-podinfo\") pod \"e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l\" (UID: \"d6c8d44c-0d49-493a-b995-f8eb8236d663\") " pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" Aug 04 15:39:10.145595 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:10.145465 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6c8d44c-0d49-493a-b995-f8eb8236d663-workload-socket\") pod \"e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l\" (UID: \"d6c8d44c-0d49-493a-b995-f8eb8236d663\") " pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" Aug 04 15:39:10.145595 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:10.145586 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6c8d44c-0d49-493a-b995-f8eb8236d663-workload-certs\") pod \"e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l\" (UID: \"d6c8d44c-0d49-493a-b995-f8eb8236d663\") " pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" Aug 04 15:39:10.146093 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:10.145641 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6c8d44c-0d49-493a-b995-f8eb8236d663-istio-token\") pod \"e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l\" (UID: \"d6c8d44c-0d49-493a-b995-f8eb8236d663\") " pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" Aug 04 15:39:10.146093 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:10.145723 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6c8d44c-0d49-493a-b995-f8eb8236d663-istio-data\") pod \"e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l\" (UID: \"d6c8d44c-0d49-493a-b995-f8eb8236d663\") " pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" Aug 04 15:39:10.146093 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:10.145944 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6c8d44c-0d49-493a-b995-f8eb8236d663-istiod-ca-cert\") pod \"e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l\" (UID: \"d6c8d44c-0d49-493a-b995-f8eb8236d663\") " pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" Aug 04 15:39:10.146093 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:10.146044 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6c8d44c-0d49-493a-b995-f8eb8236d663-credential-socket\") pod \"e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l\" (UID: \"d6c8d44c-0d49-493a-b995-f8eb8236d663\") " pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" Aug 04 15:39:10.147679 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:10.147654 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6c8d44c-0d49-493a-b995-f8eb8236d663-istio-envoy\") pod \"e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l\" (UID: \"d6c8d44c-0d49-493a-b995-f8eb8236d663\") " pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" Aug 04 15:39:10.147800 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:10.147706 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6c8d44c-0d49-493a-b995-f8eb8236d663-istio-podinfo\") pod \"e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l\" (UID: \"d6c8d44c-0d49-493a-b995-f8eb8236d663\") " pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" Aug 04 15:39:10.154614 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:10.154594 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6c8d44c-0d49-493a-b995-f8eb8236d663-istio-token\") pod \"e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l\" (UID: \"d6c8d44c-0d49-493a-b995-f8eb8236d663\") " pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" Aug 04 15:39:10.154786 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:10.154768 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thd86\" (UniqueName: \"kubernetes.io/projected/d6c8d44c-0d49-493a-b995-f8eb8236d663-kube-api-access-thd86\") pod \"e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l\" (UID: \"d6c8d44c-0d49-493a-b995-f8eb8236d663\") " pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" Aug 04 15:39:10.244371 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:10.244346 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" Aug 04 15:39:10.387365 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:10.387331 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l"] Aug 04 15:39:10.387918 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:39:10.387895 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6c8d44c_0d49_493a_b995_f8eb8236d663.slice/crio-e60de8268d340ea2930e1d15e299fe65c3820bbff654ef440f5bad9293d1e58c WatchSource:0}: Error finding container e60de8268d340ea2930e1d15e299fe65c3820bbff654ef440f5bad9293d1e58c: Status 404 returned error can't find the container with id e60de8268d340ea2930e1d15e299fe65c3820bbff654ef440f5bad9293d1e58c Aug 04 15:39:10.390034 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:10.389999 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:39:10.390130 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:10.390063 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:39:10.390130 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:10.390092 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:39:10.455448 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:10.455420 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" event={"ID":"d6c8d44c-0d49-493a-b995-f8eb8236d663","Type":"ContainerStarted","Data":"e60de8268d340ea2930e1d15e299fe65c3820bbff654ef440f5bad9293d1e58c"} Aug 04 15:39:10.484156 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:10.484127 2574 patch_prober.go:28] interesting pod/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 04 15:39:10.484277 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:10.484198 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" podUID="514f7176-0ac8-4adc-aa63-ef3382cd4ca1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 15:39:11.462404 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:11.462351 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" event={"ID":"d6c8d44c-0d49-493a-b995-f8eb8236d663","Type":"ContainerStarted","Data":"60c552261bea858945b7e56d941d05858c164f1a9f818e0c2cf670cffcbca93e"} Aug 04 15:39:11.491561 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:11.491512 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" podStartSLOduration=2.491496993 podStartE2EDuration="2.491496993s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:39:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:39:11.48965056 +0000 UTC m=+1974.949522948" watchObservedRunningTime="2026-08-04 15:39:11.491496993 +0000 UTC m=+1974.951369341" Aug 04 15:39:12.245429 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:12.245371 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" Aug 04 15:39:12.250498 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:12.250464 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" Aug 04 15:39:12.468562 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:12.468532 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" Aug 04 15:39:12.469589 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:12.469568 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" Aug 04 15:39:12.635010 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:12.634933 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:39:14.302178 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:14.302154 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" Aug 04 15:39:14.390560 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:14.390481 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-istio-token\") pod \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\" (UID: \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\") " Aug 04 15:39:14.390560 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:14.390525 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-workload-socket\") pod \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\" (UID: \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\") " Aug 04 15:39:14.390560 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:14.390547 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6wlrv\" (UniqueName: \"kubernetes.io/projected/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-kube-api-access-6wlrv\") pod \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\" (UID: \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\") " Aug 04 15:39:14.390824 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:14.390563 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-credential-socket\") pod \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\" (UID: \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\") " Aug 04 15:39:14.390824 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:14.390611 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-istiod-ca-cert\") pod \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\" (UID: \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\") " Aug 04 15:39:14.390824 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:14.390652 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-istio-envoy\") pod \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\" (UID: \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\") " Aug 04 15:39:14.390824 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:14.390686 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-istio-data\") pod \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\" (UID: \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\") " Aug 04 15:39:14.390824 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:14.390743 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-istio-podinfo\") pod \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\" (UID: \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\") " Aug 04 15:39:14.390824 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:14.390772 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-workload-certs\") pod \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\" (UID: \"514f7176-0ac8-4adc-aa63-ef3382cd4ca1\") " Aug 04 15:39:14.390824 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:14.390779 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "514f7176-0ac8-4adc-aa63-ef3382cd4ca1" (UID: "514f7176-0ac8-4adc-aa63-ef3382cd4ca1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:39:14.391198 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:14.390971 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "514f7176-0ac8-4adc-aa63-ef3382cd4ca1" (UID: "514f7176-0ac8-4adc-aa63-ef3382cd4ca1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:39:14.391198 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:14.391135 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-workload-socket\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:39:14.391198 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:14.391156 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-credential-socket\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:39:14.391198 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:14.391158 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "514f7176-0ac8-4adc-aa63-ef3382cd4ca1" (UID: "514f7176-0ac8-4adc-aa63-ef3382cd4ca1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:39:14.391434 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:14.391362 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "514f7176-0ac8-4adc-aa63-ef3382cd4ca1" (UID: "514f7176-0ac8-4adc-aa63-ef3382cd4ca1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 15:39:14.391749 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:14.391726 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-istio-data" (OuterVolumeSpecName: "istio-data") pod "514f7176-0ac8-4adc-aa63-ef3382cd4ca1" (UID: "514f7176-0ac8-4adc-aa63-ef3382cd4ca1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:39:14.392902 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:14.392877 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-kube-api-access-6wlrv" (OuterVolumeSpecName: "kube-api-access-6wlrv") pod "514f7176-0ac8-4adc-aa63-ef3382cd4ca1" (UID: "514f7176-0ac8-4adc-aa63-ef3382cd4ca1"). InnerVolumeSpecName "kube-api-access-6wlrv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 15:39:14.393028 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:14.392951 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "514f7176-0ac8-4adc-aa63-ef3382cd4ca1" (UID: "514f7176-0ac8-4adc-aa63-ef3382cd4ca1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:39:14.393152 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:14.393131 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-istio-token" (OuterVolumeSpecName: "istio-token") pod "514f7176-0ac8-4adc-aa63-ef3382cd4ca1" (UID: "514f7176-0ac8-4adc-aa63-ef3382cd4ca1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 15:39:14.393237 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:14.393223 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "514f7176-0ac8-4adc-aa63-ef3382cd4ca1" (UID: "514f7176-0ac8-4adc-aa63-ef3382cd4ca1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 15:39:14.480882 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:14.480848 2574 generic.go:356] "Generic (PLEG): container finished" podID="514f7176-0ac8-4adc-aa63-ef3382cd4ca1" containerID="eee73087fde26b2798d32ad39d4c820e386583c655bbb3e8d49575a9d6c4ae6d" exitCode=0 Aug 04 15:39:14.481077 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:14.480901 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" Aug 04 15:39:14.481077 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:14.480936 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" event={"ID":"514f7176-0ac8-4adc-aa63-ef3382cd4ca1","Type":"ContainerDied","Data":"eee73087fde26b2798d32ad39d4c820e386583c655bbb3e8d49575a9d6c4ae6d"} Aug 04 15:39:14.481077 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:14.481001 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph" event={"ID":"514f7176-0ac8-4adc-aa63-ef3382cd4ca1","Type":"ContainerDied","Data":"19bc2b5205c0f3f2085b29abe531df70769e8c908911db86df1ee883d4d3a19b"} Aug 04 15:39:14.481077 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:14.481032 2574 scope.go:117] "RemoveContainer" containerID="eee73087fde26b2798d32ad39d4c820e386583c655bbb3e8d49575a9d6c4ae6d" Aug 04 15:39:14.491358 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:14.491342 2574 scope.go:117] "RemoveContainer" containerID="3852ff1d066aff5b2dff4e1cda1e23510f7eda275adc64797d77f0f84c1c79f9" Aug 04 15:39:14.491659 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:14.491635 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-istio-token\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:39:14.491749 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:14.491661 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6wlrv\" (UniqueName: \"kubernetes.io/projected/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-kube-api-access-6wlrv\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:39:14.491749 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:14.491677 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-istiod-ca-cert\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:39:14.491749 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:14.491690 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-istio-envoy\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:39:14.491749 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:14.491702 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-istio-data\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:39:14.491749 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:14.491714 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-istio-podinfo\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:39:14.491749 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:14.491725 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/514f7176-0ac8-4adc-aa63-ef3382cd4ca1-workload-certs\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:39:14.500321 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:14.500303 2574 scope.go:117] "RemoveContainer" containerID="eee73087fde26b2798d32ad39d4c820e386583c655bbb3e8d49575a9d6c4ae6d" Aug 04 15:39:14.500578 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:39:14.500559 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eee73087fde26b2798d32ad39d4c820e386583c655bbb3e8d49575a9d6c4ae6d\": container with ID starting with eee73087fde26b2798d32ad39d4c820e386583c655bbb3e8d49575a9d6c4ae6d not found: ID does not exist" containerID="eee73087fde26b2798d32ad39d4c820e386583c655bbb3e8d49575a9d6c4ae6d" Aug 04 15:39:14.500641 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:14.500586 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eee73087fde26b2798d32ad39d4c820e386583c655bbb3e8d49575a9d6c4ae6d"} err="failed to get container status \"eee73087fde26b2798d32ad39d4c820e386583c655bbb3e8d49575a9d6c4ae6d\": rpc error: code = NotFound desc = could not find container \"eee73087fde26b2798d32ad39d4c820e386583c655bbb3e8d49575a9d6c4ae6d\": container with ID starting with eee73087fde26b2798d32ad39d4c820e386583c655bbb3e8d49575a9d6c4ae6d not found: ID does not exist" Aug 04 15:39:14.500641 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:14.500602 2574 scope.go:117] "RemoveContainer" containerID="3852ff1d066aff5b2dff4e1cda1e23510f7eda275adc64797d77f0f84c1c79f9" Aug 04 15:39:14.500848 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:39:14.500830 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3852ff1d066aff5b2dff4e1cda1e23510f7eda275adc64797d77f0f84c1c79f9\": container with ID starting with 3852ff1d066aff5b2dff4e1cda1e23510f7eda275adc64797d77f0f84c1c79f9 not found: ID does not exist" containerID="3852ff1d066aff5b2dff4e1cda1e23510f7eda275adc64797d77f0f84c1c79f9" Aug 04 15:39:14.500903 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:14.500861 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3852ff1d066aff5b2dff4e1cda1e23510f7eda275adc64797d77f0f84c1c79f9"} err="failed to get container status \"3852ff1d066aff5b2dff4e1cda1e23510f7eda275adc64797d77f0f84c1c79f9\": rpc error: code = NotFound desc = could not find container \"3852ff1d066aff5b2dff4e1cda1e23510f7eda275adc64797d77f0f84c1c79f9\": container with ID starting with 3852ff1d066aff5b2dff4e1cda1e23510f7eda275adc64797d77f0f84c1c79f9 not found: ID does not exist" Aug 04 15:39:14.509676 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:14.509653 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph"] Aug 04 15:39:14.521308 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:14.521288 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-2a0c7713-gw-openshift-default-744bc6596-tlzph"] Aug 04 15:39:15.053928 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:15.053897 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="514f7176-0ac8-4adc-aa63-ef3382cd4ca1" path="/var/lib/kubelet/pods/514f7176-0ac8-4adc-aa63-ef3382cd4ca1/volumes" Aug 04 15:39:22.736068 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:22.736027 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:39:26.555911 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:26.555876 2574 generic.go:356] "Generic (PLEG): container finished" podID="d6c8d44c-0d49-493a-b995-f8eb8236d663" containerID="60c552261bea858945b7e56d941d05858c164f1a9f818e0c2cf670cffcbca93e" exitCode=0 Aug 04 15:39:26.556330 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:26.555945 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" event={"ID":"d6c8d44c-0d49-493a-b995-f8eb8236d663","Type":"ContainerDied","Data":"60c552261bea858945b7e56d941d05858c164f1a9f818e0c2cf670cffcbca93e"} Aug 04 15:39:26.556330 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:26.556326 2574 scope.go:117] "RemoveContainer" containerID="60c552261bea858945b7e56d941d05858c164f1a9f818e0c2cf670cffcbca93e" Aug 04 15:39:26.557320 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:26.557289 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:39:26.557446 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:26.557376 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:39:26.557518 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:26.557443 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:39:27.469045 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:27.469009 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" Aug 04 15:39:27.564915 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:27.564880 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" event={"ID":"d6c8d44c-0d49-493a-b995-f8eb8236d663","Type":"ContainerStarted","Data":"a488fa3c809fee36935824f018ae105d6d5113f0451cec7104d271f4ae59faee"} Aug 04 15:39:28.246414 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:28.246246 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" Aug 04 15:39:28.246414 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:28.246371 2574 patch_prober.go:28] interesting pod/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Aug 04 15:39:28.246685 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:28.246449 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" podUID="d6c8d44c-0d49-493a-b995-f8eb8236d663" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Aug 04 15:39:29.244902 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:29.244851 2574 patch_prober.go:28] interesting pod/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Aug 04 15:39:29.245491 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:29.244939 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" podUID="d6c8d44c-0d49-493a-b995-f8eb8236d663" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Aug 04 15:39:30.244764 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:30.244717 2574 patch_prober.go:28] interesting pod/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Aug 04 15:39:30.244968 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:30.244803 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" podUID="d6c8d44c-0d49-493a-b995-f8eb8236d663" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Aug 04 15:39:31.245100 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:31.245055 2574 patch_prober.go:28] interesting pod/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Aug 04 15:39:31.245635 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:31.245123 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" podUID="d6c8d44c-0d49-493a-b995-f8eb8236d663" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Aug 04 15:39:32.250062 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:32.250024 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" Aug 04 15:39:32.250491 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:32.250317 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" Aug 04 15:39:32.251153 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:32.251137 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" Aug 04 15:39:46.329373 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:46.329337 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l"] Aug 04 15:39:46.329950 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:46.329600 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" podUID="d6c8d44c-0d49-493a-b995-f8eb8236d663" containerName="istio-proxy" containerID="cri-o://a488fa3c809fee36935824f018ae105d6d5113f0451cec7104d271f4ae59faee" gracePeriod=30 Aug 04 15:39:47.238806 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.238771 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj"] Aug 04 15:39:47.239227 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.239208 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="514f7176-0ac8-4adc-aa63-ef3382cd4ca1" containerName="istio-proxy" Aug 04 15:39:47.239312 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.239229 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="514f7176-0ac8-4adc-aa63-ef3382cd4ca1" containerName="istio-proxy" Aug 04 15:39:47.239369 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.239361 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="514f7176-0ac8-4adc-aa63-ef3382cd4ca1" containerName="istio-proxy" Aug 04 15:39:47.239487 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.239375 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="514f7176-0ac8-4adc-aa63-ef3382cd4ca1" containerName="istio-proxy" Aug 04 15:39:47.242727 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.242706 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" Aug 04 15:39:47.248189 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.248170 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-0f33af57-openshift-default-dockercfg-8zv44\"" Aug 04 15:39:47.252171 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.252143 2574 patch_prober.go:28] interesting pod/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 04 15:39:47.252285 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.252196 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" podUID="d6c8d44c-0d49-493a-b995-f8eb8236d663" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 15:39:47.276034 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.276011 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj"] Aug 04 15:39:47.305466 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.305426 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74570765-7077-44c1-8d7a-9a19b5e3d1a8-workload-certs\") pod \"e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj\" (UID: \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\") " pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" Aug 04 15:39:47.305778 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.305728 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74570765-7077-44c1-8d7a-9a19b5e3d1a8-credential-socket\") pod \"e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj\" (UID: \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\") " pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" Aug 04 15:39:47.305900 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.305795 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8hln\" (UniqueName: \"kubernetes.io/projected/74570765-7077-44c1-8d7a-9a19b5e3d1a8-kube-api-access-r8hln\") pod \"e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj\" (UID: \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\") " pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" Aug 04 15:39:47.305900 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.305863 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74570765-7077-44c1-8d7a-9a19b5e3d1a8-istiod-ca-cert\") pod \"e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj\" (UID: \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\") " pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" Aug 04 15:39:47.306075 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.305927 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74570765-7077-44c1-8d7a-9a19b5e3d1a8-istio-podinfo\") pod \"e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj\" (UID: \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\") " pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" Aug 04 15:39:47.306075 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.306009 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74570765-7077-44c1-8d7a-9a19b5e3d1a8-workload-socket\") pod \"e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj\" (UID: \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\") " pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" Aug 04 15:39:47.306075 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.306046 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74570765-7077-44c1-8d7a-9a19b5e3d1a8-istio-envoy\") pod \"e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj\" (UID: \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\") " pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" Aug 04 15:39:47.306075 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.306069 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74570765-7077-44c1-8d7a-9a19b5e3d1a8-istio-token\") pod \"e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj\" (UID: \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\") " pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" Aug 04 15:39:47.306260 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.306141 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74570765-7077-44c1-8d7a-9a19b5e3d1a8-istio-data\") pod \"e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj\" (UID: \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\") " pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" Aug 04 15:39:47.407520 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.407489 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74570765-7077-44c1-8d7a-9a19b5e3d1a8-workload-socket\") pod \"e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj\" (UID: \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\") " pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" Aug 04 15:39:47.407520 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.407526 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74570765-7077-44c1-8d7a-9a19b5e3d1a8-istio-envoy\") pod \"e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj\" (UID: \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\") " pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" Aug 04 15:39:47.408064 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.407542 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74570765-7077-44c1-8d7a-9a19b5e3d1a8-istio-token\") pod \"e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj\" (UID: \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\") " pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" Aug 04 15:39:47.408064 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.407567 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74570765-7077-44c1-8d7a-9a19b5e3d1a8-istio-data\") pod \"e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj\" (UID: \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\") " pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" Aug 04 15:39:47.408064 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.407589 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74570765-7077-44c1-8d7a-9a19b5e3d1a8-workload-certs\") pod \"e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj\" (UID: \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\") " pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" Aug 04 15:39:47.408064 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.407631 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74570765-7077-44c1-8d7a-9a19b5e3d1a8-credential-socket\") pod \"e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj\" (UID: \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\") " pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" Aug 04 15:39:47.408064 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.407648 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8hln\" (UniqueName: \"kubernetes.io/projected/74570765-7077-44c1-8d7a-9a19b5e3d1a8-kube-api-access-r8hln\") pod \"e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj\" (UID: \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\") " pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" Aug 04 15:39:47.408064 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.407692 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74570765-7077-44c1-8d7a-9a19b5e3d1a8-istiod-ca-cert\") pod \"e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj\" (UID: \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\") " pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" Aug 04 15:39:47.408064 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.407719 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74570765-7077-44c1-8d7a-9a19b5e3d1a8-istio-podinfo\") pod \"e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj\" (UID: \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\") " pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" Aug 04 15:39:47.408064 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.407970 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74570765-7077-44c1-8d7a-9a19b5e3d1a8-workload-socket\") pod \"e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj\" (UID: \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\") " pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" Aug 04 15:39:47.408519 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.408131 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74570765-7077-44c1-8d7a-9a19b5e3d1a8-workload-certs\") pod \"e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj\" (UID: \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\") " pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" Aug 04 15:39:47.408519 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.408170 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74570765-7077-44c1-8d7a-9a19b5e3d1a8-istio-data\") pod \"e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj\" (UID: \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\") " pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" Aug 04 15:39:47.408519 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.408209 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74570765-7077-44c1-8d7a-9a19b5e3d1a8-credential-socket\") pod \"e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj\" (UID: \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\") " pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" Aug 04 15:39:47.408689 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.408533 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74570765-7077-44c1-8d7a-9a19b5e3d1a8-istiod-ca-cert\") pod \"e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj\" (UID: \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\") " pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" Aug 04 15:39:47.409965 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.409940 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74570765-7077-44c1-8d7a-9a19b5e3d1a8-istio-envoy\") pod \"e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj\" (UID: \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\") " pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" Aug 04 15:39:47.410164 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.410147 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74570765-7077-44c1-8d7a-9a19b5e3d1a8-istio-podinfo\") pod \"e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj\" (UID: \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\") " pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" Aug 04 15:39:47.423718 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.423689 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74570765-7077-44c1-8d7a-9a19b5e3d1a8-istio-token\") pod \"e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj\" (UID: \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\") " pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" Aug 04 15:39:47.423819 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.423809 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8hln\" (UniqueName: \"kubernetes.io/projected/74570765-7077-44c1-8d7a-9a19b5e3d1a8-kube-api-access-r8hln\") pod \"e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj\" (UID: \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\") " pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" Aug 04 15:39:47.554318 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.554240 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" Aug 04 15:39:47.714576 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.714457 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj"] Aug 04 15:39:47.718762 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:39:47.718732 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74570765_7077_44c1_8d7a_9a19b5e3d1a8.slice/crio-62a7f795c8d1ed0d388326ee135059efd752fa33260b145c339e0ba2c48a5c97 WatchSource:0}: Error finding container 62a7f795c8d1ed0d388326ee135059efd752fa33260b145c339e0ba2c48a5c97: Status 404 returned error can't find the container with id 62a7f795c8d1ed0d388326ee135059efd752fa33260b145c339e0ba2c48a5c97 Aug 04 15:39:47.720869 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.720834 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:39:47.720958 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.720897 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:39:47.720958 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:47.720926 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:39:48.686669 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:48.686631 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" event={"ID":"74570765-7077-44c1-8d7a-9a19b5e3d1a8","Type":"ContainerStarted","Data":"700d589bb2e78811d802cc8ae968580c2c41ddb7a144d2ba251f2f0cac72a5f9"} Aug 04 15:39:48.687050 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:48.686690 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" event={"ID":"74570765-7077-44c1-8d7a-9a19b5e3d1a8","Type":"ContainerStarted","Data":"62a7f795c8d1ed0d388326ee135059efd752fa33260b145c339e0ba2c48a5c97"} Aug 04 15:39:48.726764 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:48.726715 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" podStartSLOduration=1.7267003619999999 podStartE2EDuration="1.726700362s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:39:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:39:48.725470619 +0000 UTC m=+2012.185342969" watchObservedRunningTime="2026-08-04 15:39:48.726700362 +0000 UTC m=+2012.186572710" Aug 04 15:39:49.066029 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:49.065996 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:39:49.554550 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:49.554515 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" Aug 04 15:39:49.559303 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:49.559280 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" Aug 04 15:39:49.692303 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:49.692272 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" Aug 04 15:39:49.693139 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:49.693120 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" Aug 04 15:39:51.594277 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:51.594221 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" Aug 04 15:39:51.652795 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:51.652770 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6c8d44c-0d49-493a-b995-f8eb8236d663-istiod-ca-cert\") pod \"d6c8d44c-0d49-493a-b995-f8eb8236d663\" (UID: \"d6c8d44c-0d49-493a-b995-f8eb8236d663\") " Aug 04 15:39:51.652795 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:51.652799 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6c8d44c-0d49-493a-b995-f8eb8236d663-workload-socket\") pod \"d6c8d44c-0d49-493a-b995-f8eb8236d663\" (UID: \"d6c8d44c-0d49-493a-b995-f8eb8236d663\") " Aug 04 15:39:51.653027 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:51.652825 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6c8d44c-0d49-493a-b995-f8eb8236d663-istio-envoy\") pod \"d6c8d44c-0d49-493a-b995-f8eb8236d663\" (UID: \"d6c8d44c-0d49-493a-b995-f8eb8236d663\") " Aug 04 15:39:51.653027 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:51.652843 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6c8d44c-0d49-493a-b995-f8eb8236d663-workload-certs\") pod \"d6c8d44c-0d49-493a-b995-f8eb8236d663\" (UID: \"d6c8d44c-0d49-493a-b995-f8eb8236d663\") " Aug 04 15:39:51.653027 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:51.652980 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6c8d44c-0d49-493a-b995-f8eb8236d663-istio-podinfo\") pod \"d6c8d44c-0d49-493a-b995-f8eb8236d663\" (UID: \"d6c8d44c-0d49-493a-b995-f8eb8236d663\") " Aug 04 15:39:51.653195 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:51.653055 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6c8d44c-0d49-493a-b995-f8eb8236d663-credential-socket\") pod \"d6c8d44c-0d49-493a-b995-f8eb8236d663\" (UID: \"d6c8d44c-0d49-493a-b995-f8eb8236d663\") " Aug 04 15:39:51.653195 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:51.653079 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6c8d44c-0d49-493a-b995-f8eb8236d663-istio-token\") pod \"d6c8d44c-0d49-493a-b995-f8eb8236d663\" (UID: \"d6c8d44c-0d49-493a-b995-f8eb8236d663\") " Aug 04 15:39:51.653195 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:51.653102 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6c8d44c-0d49-493a-b995-f8eb8236d663-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d6c8d44c-0d49-493a-b995-f8eb8236d663" (UID: "d6c8d44c-0d49-493a-b995-f8eb8236d663"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:39:51.653195 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:51.653133 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6c8d44c-0d49-493a-b995-f8eb8236d663-istio-data\") pod \"d6c8d44c-0d49-493a-b995-f8eb8236d663\" (UID: \"d6c8d44c-0d49-493a-b995-f8eb8236d663\") " Aug 04 15:39:51.653195 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:51.653168 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-thd86\" (UniqueName: \"kubernetes.io/projected/d6c8d44c-0d49-493a-b995-f8eb8236d663-kube-api-access-thd86\") pod \"d6c8d44c-0d49-493a-b995-f8eb8236d663\" (UID: \"d6c8d44c-0d49-493a-b995-f8eb8236d663\") " Aug 04 15:39:51.653499 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:51.653234 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6c8d44c-0d49-493a-b995-f8eb8236d663-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d6c8d44c-0d49-493a-b995-f8eb8236d663" (UID: "d6c8d44c-0d49-493a-b995-f8eb8236d663"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 15:39:51.653563 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:51.653519 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6c8d44c-0d49-493a-b995-f8eb8236d663-istiod-ca-cert\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:39:51.653563 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:51.653537 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6c8d44c-0d49-493a-b995-f8eb8236d663-workload-socket\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:39:51.653768 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:51.653729 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6c8d44c-0d49-493a-b995-f8eb8236d663-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d6c8d44c-0d49-493a-b995-f8eb8236d663" (UID: "d6c8d44c-0d49-493a-b995-f8eb8236d663"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:39:51.653857 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:51.653797 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6c8d44c-0d49-493a-b995-f8eb8236d663-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d6c8d44c-0d49-493a-b995-f8eb8236d663" (UID: "d6c8d44c-0d49-493a-b995-f8eb8236d663"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:39:51.654026 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:51.654005 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6c8d44c-0d49-493a-b995-f8eb8236d663-istio-data" (OuterVolumeSpecName: "istio-data") pod "d6c8d44c-0d49-493a-b995-f8eb8236d663" (UID: "d6c8d44c-0d49-493a-b995-f8eb8236d663"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:39:51.655554 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:51.655531 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6c8d44c-0d49-493a-b995-f8eb8236d663-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d6c8d44c-0d49-493a-b995-f8eb8236d663" (UID: "d6c8d44c-0d49-493a-b995-f8eb8236d663"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:39:51.655554 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:51.655543 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d6c8d44c-0d49-493a-b995-f8eb8236d663-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d6c8d44c-0d49-493a-b995-f8eb8236d663" (UID: "d6c8d44c-0d49-493a-b995-f8eb8236d663"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 15:39:51.655736 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:51.655567 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6c8d44c-0d49-493a-b995-f8eb8236d663-istio-token" (OuterVolumeSpecName: "istio-token") pod "d6c8d44c-0d49-493a-b995-f8eb8236d663" (UID: "d6c8d44c-0d49-493a-b995-f8eb8236d663"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 15:39:51.655736 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:51.655675 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6c8d44c-0d49-493a-b995-f8eb8236d663-kube-api-access-thd86" (OuterVolumeSpecName: "kube-api-access-thd86") pod "d6c8d44c-0d49-493a-b995-f8eb8236d663" (UID: "d6c8d44c-0d49-493a-b995-f8eb8236d663"). InnerVolumeSpecName "kube-api-access-thd86". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 15:39:51.705734 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:51.705701 2574 generic.go:356] "Generic (PLEG): container finished" podID="d6c8d44c-0d49-493a-b995-f8eb8236d663" containerID="a488fa3c809fee36935824f018ae105d6d5113f0451cec7104d271f4ae59faee" exitCode=0 Aug 04 15:39:51.705856 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:51.705750 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" Aug 04 15:39:51.705856 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:51.705784 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" event={"ID":"d6c8d44c-0d49-493a-b995-f8eb8236d663","Type":"ContainerDied","Data":"a488fa3c809fee36935824f018ae105d6d5113f0451cec7104d271f4ae59faee"} Aug 04 15:39:51.705856 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:51.705839 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l" event={"ID":"d6c8d44c-0d49-493a-b995-f8eb8236d663","Type":"ContainerDied","Data":"e60de8268d340ea2930e1d15e299fe65c3820bbff654ef440f5bad9293d1e58c"} Aug 04 15:39:51.705986 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:51.705872 2574 scope.go:117] "RemoveContainer" containerID="a488fa3c809fee36935824f018ae105d6d5113f0451cec7104d271f4ae59faee" Aug 04 15:39:51.716526 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:51.716505 2574 scope.go:117] "RemoveContainer" containerID="60c552261bea858945b7e56d941d05858c164f1a9f818e0c2cf670cffcbca93e" Aug 04 15:39:51.724926 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:51.724906 2574 scope.go:117] "RemoveContainer" containerID="a488fa3c809fee36935824f018ae105d6d5113f0451cec7104d271f4ae59faee" Aug 04 15:39:51.725172 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:39:51.725154 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a488fa3c809fee36935824f018ae105d6d5113f0451cec7104d271f4ae59faee\": container with ID starting with a488fa3c809fee36935824f018ae105d6d5113f0451cec7104d271f4ae59faee not found: ID does not exist" containerID="a488fa3c809fee36935824f018ae105d6d5113f0451cec7104d271f4ae59faee" Aug 04 15:39:51.725223 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:51.725180 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a488fa3c809fee36935824f018ae105d6d5113f0451cec7104d271f4ae59faee"} err="failed to get container status \"a488fa3c809fee36935824f018ae105d6d5113f0451cec7104d271f4ae59faee\": rpc error: code = NotFound desc = could not find container \"a488fa3c809fee36935824f018ae105d6d5113f0451cec7104d271f4ae59faee\": container with ID starting with a488fa3c809fee36935824f018ae105d6d5113f0451cec7104d271f4ae59faee not found: ID does not exist" Aug 04 15:39:51.725223 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:51.725196 2574 scope.go:117] "RemoveContainer" containerID="60c552261bea858945b7e56d941d05858c164f1a9f818e0c2cf670cffcbca93e" Aug 04 15:39:51.725459 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:39:51.725440 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60c552261bea858945b7e56d941d05858c164f1a9f818e0c2cf670cffcbca93e\": container with ID starting with 60c552261bea858945b7e56d941d05858c164f1a9f818e0c2cf670cffcbca93e not found: ID does not exist" containerID="60c552261bea858945b7e56d941d05858c164f1a9f818e0c2cf670cffcbca93e" Aug 04 15:39:51.725517 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:51.725465 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60c552261bea858945b7e56d941d05858c164f1a9f818e0c2cf670cffcbca93e"} err="failed to get container status \"60c552261bea858945b7e56d941d05858c164f1a9f818e0c2cf670cffcbca93e\": rpc error: code = NotFound desc = could not find container \"60c552261bea858945b7e56d941d05858c164f1a9f818e0c2cf670cffcbca93e\": container with ID starting with 60c552261bea858945b7e56d941d05858c164f1a9f818e0c2cf670cffcbca93e not found: ID does not exist" Aug 04 15:39:51.732348 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:51.732327 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l"] Aug 04 15:39:51.741741 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:51.741711 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-8202ac71-openshift-default-7c8fc569b4-jbl2l"] Aug 04 15:39:51.755224 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:51.755197 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6c8d44c-0d49-493a-b995-f8eb8236d663-istio-envoy\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:39:51.755224 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:51.755216 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6c8d44c-0d49-493a-b995-f8eb8236d663-workload-certs\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:39:51.755224 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:51.755224 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6c8d44c-0d49-493a-b995-f8eb8236d663-istio-podinfo\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:39:51.755431 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:51.755234 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6c8d44c-0d49-493a-b995-f8eb8236d663-credential-socket\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:39:51.755431 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:51.755245 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6c8d44c-0d49-493a-b995-f8eb8236d663-istio-token\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:39:51.755431 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:51.755253 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6c8d44c-0d49-493a-b995-f8eb8236d663-istio-data\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:39:51.755431 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:51.755261 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thd86\" (UniqueName: \"kubernetes.io/projected/d6c8d44c-0d49-493a-b995-f8eb8236d663-kube-api-access-thd86\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:39:53.053851 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:39:53.053811 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6c8d44c-0d49-493a-b995-f8eb8236d663" path="/var/lib/kubelet/pods/d6c8d44c-0d49-493a-b995-f8eb8236d663/volumes" Aug 04 15:40:00.622684 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:00.622645 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:40:02.777783 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:02.777744 2574 generic.go:356] "Generic (PLEG): container finished" podID="74570765-7077-44c1-8d7a-9a19b5e3d1a8" containerID="700d589bb2e78811d802cc8ae968580c2c41ddb7a144d2ba251f2f0cac72a5f9" exitCode=0 Aug 04 15:40:02.778285 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:02.777823 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" event={"ID":"74570765-7077-44c1-8d7a-9a19b5e3d1a8","Type":"ContainerDied","Data":"700d589bb2e78811d802cc8ae968580c2c41ddb7a144d2ba251f2f0cac72a5f9"} Aug 04 15:40:02.778374 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:02.778356 2574 scope.go:117] "RemoveContainer" containerID="700d589bb2e78811d802cc8ae968580c2c41ddb7a144d2ba251f2f0cac72a5f9" Aug 04 15:40:02.779408 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:02.779365 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:40:02.779476 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:02.779450 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:40:02.779515 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:02.779488 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:40:03.784562 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:03.784526 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" event={"ID":"74570765-7077-44c1-8d7a-9a19b5e3d1a8","Type":"ContainerStarted","Data":"9fed8931c24b577cac97f3ea7c71954fd86e2ecd9fe97c1d9f84ce577a011b9f"} Aug 04 15:40:04.554449 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:04.554382 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" Aug 04 15:40:04.555476 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:04.555449 2574 patch_prober.go:28] interesting pod/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Aug 04 15:40:04.555598 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:04.555499 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" podUID="74570765-7077-44c1-8d7a-9a19b5e3d1a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Aug 04 15:40:04.693178 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:04.693121 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" Aug 04 15:40:05.555020 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:05.554980 2574 patch_prober.go:28] interesting pod/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Aug 04 15:40:05.555439 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:05.555049 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" podUID="74570765-7077-44c1-8d7a-9a19b5e3d1a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Aug 04 15:40:06.555340 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:06.555305 2574 patch_prober.go:28] interesting pod/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Aug 04 15:40:06.555762 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:06.555358 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" podUID="74570765-7077-44c1-8d7a-9a19b5e3d1a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Aug 04 15:40:07.554793 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:07.554749 2574 patch_prober.go:28] interesting pod/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Aug 04 15:40:07.554972 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:07.554822 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" podUID="74570765-7077-44c1-8d7a-9a19b5e3d1a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Aug 04 15:40:08.558940 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:08.558911 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" Aug 04 15:40:08.560057 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:08.560034 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" Aug 04 15:40:22.555315 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:22.555277 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj"] Aug 04 15:40:22.555839 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:22.555552 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" podUID="74570765-7077-44c1-8d7a-9a19b5e3d1a8" containerName="istio-proxy" containerID="cri-o://9fed8931c24b577cac97f3ea7c71954fd86e2ecd9fe97c1d9f84ce577a011b9f" gracePeriod=30 Aug 04 15:40:23.562128 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:23.562095 2574 patch_prober.go:28] interesting pod/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 04 15:40:23.562549 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:23.562149 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" podUID="74570765-7077-44c1-8d7a-9a19b5e3d1a8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 15:40:26.083006 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:26.082966 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:40:27.809497 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:27.809472 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" Aug 04 15:40:27.891225 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:27.891142 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74570765-7077-44c1-8d7a-9a19b5e3d1a8-workload-certs\") pod \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\" (UID: \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\") " Aug 04 15:40:27.891225 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:27.891196 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74570765-7077-44c1-8d7a-9a19b5e3d1a8-istio-podinfo\") pod \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\" (UID: \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\") " Aug 04 15:40:27.891479 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:27.891234 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74570765-7077-44c1-8d7a-9a19b5e3d1a8-istio-envoy\") pod \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\" (UID: \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\") " Aug 04 15:40:27.891479 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:27.891261 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74570765-7077-44c1-8d7a-9a19b5e3d1a8-istiod-ca-cert\") pod \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\" (UID: \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\") " Aug 04 15:40:27.891479 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:27.891280 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74570765-7077-44c1-8d7a-9a19b5e3d1a8-istio-token\") pod \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\" (UID: \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\") " Aug 04 15:40:27.891479 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:27.891321 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74570765-7077-44c1-8d7a-9a19b5e3d1a8-credential-socket\") pod \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\" (UID: \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\") " Aug 04 15:40:27.891479 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:27.891454 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74570765-7077-44c1-8d7a-9a19b5e3d1a8-workload-socket\") pod \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\" (UID: \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\") " Aug 04 15:40:27.891755 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:27.891493 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74570765-7077-44c1-8d7a-9a19b5e3d1a8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "74570765-7077-44c1-8d7a-9a19b5e3d1a8" (UID: "74570765-7077-44c1-8d7a-9a19b5e3d1a8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:40:27.891755 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:27.891500 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8hln\" (UniqueName: \"kubernetes.io/projected/74570765-7077-44c1-8d7a-9a19b5e3d1a8-kube-api-access-r8hln\") pod \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\" (UID: \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\") " Aug 04 15:40:27.891755 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:27.891586 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74570765-7077-44c1-8d7a-9a19b5e3d1a8-istio-data\") pod \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\" (UID: \"74570765-7077-44c1-8d7a-9a19b5e3d1a8\") " Aug 04 15:40:27.891755 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:27.891608 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74570765-7077-44c1-8d7a-9a19b5e3d1a8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "74570765-7077-44c1-8d7a-9a19b5e3d1a8" (UID: "74570765-7077-44c1-8d7a-9a19b5e3d1a8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:40:27.891755 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:27.891689 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74570765-7077-44c1-8d7a-9a19b5e3d1a8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "74570765-7077-44c1-8d7a-9a19b5e3d1a8" (UID: "74570765-7077-44c1-8d7a-9a19b5e3d1a8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 15:40:27.892027 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:27.891803 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74570765-7077-44c1-8d7a-9a19b5e3d1a8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "74570765-7077-44c1-8d7a-9a19b5e3d1a8" (UID: "74570765-7077-44c1-8d7a-9a19b5e3d1a8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:40:27.892090 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:27.892055 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74570765-7077-44c1-8d7a-9a19b5e3d1a8-istiod-ca-cert\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:40:27.892090 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:27.892079 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74570765-7077-44c1-8d7a-9a19b5e3d1a8-credential-socket\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:40:27.892202 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:27.892092 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74570765-7077-44c1-8d7a-9a19b5e3d1a8-workload-socket\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:40:27.892202 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:27.892102 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74570765-7077-44c1-8d7a-9a19b5e3d1a8-workload-certs\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:40:27.892739 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:27.892712 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74570765-7077-44c1-8d7a-9a19b5e3d1a8-istio-data" (OuterVolumeSpecName: "istio-data") pod "74570765-7077-44c1-8d7a-9a19b5e3d1a8" (UID: "74570765-7077-44c1-8d7a-9a19b5e3d1a8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:40:27.893674 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:27.893641 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/74570765-7077-44c1-8d7a-9a19b5e3d1a8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "74570765-7077-44c1-8d7a-9a19b5e3d1a8" (UID: "74570765-7077-44c1-8d7a-9a19b5e3d1a8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 15:40:27.893755 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:27.893671 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74570765-7077-44c1-8d7a-9a19b5e3d1a8-istio-token" (OuterVolumeSpecName: "istio-token") pod "74570765-7077-44c1-8d7a-9a19b5e3d1a8" (UID: "74570765-7077-44c1-8d7a-9a19b5e3d1a8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 15:40:27.893935 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:27.893914 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74570765-7077-44c1-8d7a-9a19b5e3d1a8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "74570765-7077-44c1-8d7a-9a19b5e3d1a8" (UID: "74570765-7077-44c1-8d7a-9a19b5e3d1a8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:40:27.893994 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:27.893951 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74570765-7077-44c1-8d7a-9a19b5e3d1a8-kube-api-access-r8hln" (OuterVolumeSpecName: "kube-api-access-r8hln") pod "74570765-7077-44c1-8d7a-9a19b5e3d1a8" (UID: "74570765-7077-44c1-8d7a-9a19b5e3d1a8"). InnerVolumeSpecName "kube-api-access-r8hln". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 15:40:27.922195 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:27.922165 2574 generic.go:356] "Generic (PLEG): container finished" podID="74570765-7077-44c1-8d7a-9a19b5e3d1a8" containerID="9fed8931c24b577cac97f3ea7c71954fd86e2ecd9fe97c1d9f84ce577a011b9f" exitCode=0 Aug 04 15:40:27.922327 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:27.922214 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" Aug 04 15:40:27.922327 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:27.922248 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" event={"ID":"74570765-7077-44c1-8d7a-9a19b5e3d1a8","Type":"ContainerDied","Data":"9fed8931c24b577cac97f3ea7c71954fd86e2ecd9fe97c1d9f84ce577a011b9f"} Aug 04 15:40:27.922327 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:27.922303 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj" event={"ID":"74570765-7077-44c1-8d7a-9a19b5e3d1a8","Type":"ContainerDied","Data":"62a7f795c8d1ed0d388326ee135059efd752fa33260b145c339e0ba2c48a5c97"} Aug 04 15:40:27.922327 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:27.922326 2574 scope.go:117] "RemoveContainer" containerID="9fed8931c24b577cac97f3ea7c71954fd86e2ecd9fe97c1d9f84ce577a011b9f" Aug 04 15:40:27.932020 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:27.932001 2574 scope.go:117] "RemoveContainer" containerID="700d589bb2e78811d802cc8ae968580c2c41ddb7a144d2ba251f2f0cac72a5f9" Aug 04 15:40:27.939915 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:27.939894 2574 scope.go:117] "RemoveContainer" containerID="9fed8931c24b577cac97f3ea7c71954fd86e2ecd9fe97c1d9f84ce577a011b9f" Aug 04 15:40:27.940168 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:40:27.940149 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fed8931c24b577cac97f3ea7c71954fd86e2ecd9fe97c1d9f84ce577a011b9f\": container with ID starting with 9fed8931c24b577cac97f3ea7c71954fd86e2ecd9fe97c1d9f84ce577a011b9f not found: ID does not exist" containerID="9fed8931c24b577cac97f3ea7c71954fd86e2ecd9fe97c1d9f84ce577a011b9f" Aug 04 15:40:27.940213 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:27.940175 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fed8931c24b577cac97f3ea7c71954fd86e2ecd9fe97c1d9f84ce577a011b9f"} err="failed to get container status \"9fed8931c24b577cac97f3ea7c71954fd86e2ecd9fe97c1d9f84ce577a011b9f\": rpc error: code = NotFound desc = could not find container \"9fed8931c24b577cac97f3ea7c71954fd86e2ecd9fe97c1d9f84ce577a011b9f\": container with ID starting with 9fed8931c24b577cac97f3ea7c71954fd86e2ecd9fe97c1d9f84ce577a011b9f not found: ID does not exist" Aug 04 15:40:27.940213 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:27.940193 2574 scope.go:117] "RemoveContainer" containerID="700d589bb2e78811d802cc8ae968580c2c41ddb7a144d2ba251f2f0cac72a5f9" Aug 04 15:40:27.940473 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:40:27.940453 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"700d589bb2e78811d802cc8ae968580c2c41ddb7a144d2ba251f2f0cac72a5f9\": container with ID starting with 700d589bb2e78811d802cc8ae968580c2c41ddb7a144d2ba251f2f0cac72a5f9 not found: ID does not exist" containerID="700d589bb2e78811d802cc8ae968580c2c41ddb7a144d2ba251f2f0cac72a5f9" Aug 04 15:40:27.940534 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:27.940478 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"700d589bb2e78811d802cc8ae968580c2c41ddb7a144d2ba251f2f0cac72a5f9"} err="failed to get container status \"700d589bb2e78811d802cc8ae968580c2c41ddb7a144d2ba251f2f0cac72a5f9\": rpc error: code = NotFound desc = could not find container \"700d589bb2e78811d802cc8ae968580c2c41ddb7a144d2ba251f2f0cac72a5f9\": container with ID starting with 700d589bb2e78811d802cc8ae968580c2c41ddb7a144d2ba251f2f0cac72a5f9 not found: ID does not exist" Aug 04 15:40:27.948556 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:27.948537 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj"] Aug 04 15:40:27.957807 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:27.957787 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-0f33af57-openshift-default-5d9bd89f8d-rrmhj"] Aug 04 15:40:27.992773 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:27.992750 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74570765-7077-44c1-8d7a-9a19b5e3d1a8-istio-podinfo\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:40:27.992773 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:27.992771 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74570765-7077-44c1-8d7a-9a19b5e3d1a8-istio-envoy\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:40:27.992905 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:27.992782 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74570765-7077-44c1-8d7a-9a19b5e3d1a8-istio-token\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:40:27.992905 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:27.992792 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8hln\" (UniqueName: \"kubernetes.io/projected/74570765-7077-44c1-8d7a-9a19b5e3d1a8-kube-api-access-r8hln\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:40:27.992905 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:27.992805 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74570765-7077-44c1-8d7a-9a19b5e3d1a8-istio-data\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:40:29.054535 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:29.054504 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74570765-7077-44c1-8d7a-9a19b5e3d1a8" path="/var/lib/kubelet/pods/74570765-7077-44c1-8d7a-9a19b5e3d1a8/volumes" Aug 04 15:40:49.025679 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:40:49.025589 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:41:17.186427 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:41:17.186383 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-r8wkp_0580fdab-0381-4d8e-be5f-bde5eb44e0e6/console-operator/1.log" Aug 04 15:41:17.190062 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:41:17.190041 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8h6k_9c62e3b1-c2dc-4b2a-a61d-7d80049f355f/ovn-acl-logging/0.log" Aug 04 15:41:17.190683 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:41:17.190664 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-r8wkp_0580fdab-0381-4d8e-be5f-bde5eb44e0e6/console-operator/1.log" Aug 04 15:41:17.194733 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:41:17.194717 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8h6k_9c62e3b1-c2dc-4b2a-a61d-7d80049f355f/ovn-acl-logging/0.log" Aug 04 15:42:23.428724 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:23.428644 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:42:35.816465 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:35.816428 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:42:37.807905 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:37.807869 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk"] Aug 04 15:42:37.808301 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:37.808263 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="514f7176-0ac8-4adc-aa63-ef3382cd4ca1" containerName="istio-proxy" Aug 04 15:42:37.808301 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:37.808275 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="514f7176-0ac8-4adc-aa63-ef3382cd4ca1" containerName="istio-proxy" Aug 04 15:42:37.808301 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:37.808291 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74570765-7077-44c1-8d7a-9a19b5e3d1a8" containerName="istio-proxy" Aug 04 15:42:37.808301 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:37.808297 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="74570765-7077-44c1-8d7a-9a19b5e3d1a8" containerName="istio-proxy" Aug 04 15:42:37.808453 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:37.808312 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6c8d44c-0d49-493a-b995-f8eb8236d663" containerName="istio-proxy" Aug 04 15:42:37.808453 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:37.808318 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6c8d44c-0d49-493a-b995-f8eb8236d663" containerName="istio-proxy" Aug 04 15:42:37.808453 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:37.808329 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6c8d44c-0d49-493a-b995-f8eb8236d663" containerName="istio-proxy" Aug 04 15:42:37.808453 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:37.808335 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6c8d44c-0d49-493a-b995-f8eb8236d663" containerName="istio-proxy" Aug 04 15:42:37.808453 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:37.808341 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74570765-7077-44c1-8d7a-9a19b5e3d1a8" containerName="istio-proxy" Aug 04 15:42:37.808453 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:37.808346 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="74570765-7077-44c1-8d7a-9a19b5e3d1a8" containerName="istio-proxy" Aug 04 15:42:37.808453 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:37.808423 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6c8d44c-0d49-493a-b995-f8eb8236d663" containerName="istio-proxy" Aug 04 15:42:37.808453 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:37.808430 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="74570765-7077-44c1-8d7a-9a19b5e3d1a8" containerName="istio-proxy" Aug 04 15:42:37.811521 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:37.811496 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" Aug 04 15:42:37.814543 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:37.814515 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-96b2e1f8-openshift-default-dockercfg-xm2pj\"" Aug 04 15:42:37.853764 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:37.853728 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk"] Aug 04 15:42:37.952114 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:37.952076 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20e68244-c84d-4b64-95c2-bc50489c32ba-istio-data\") pod \"e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk\" (UID: \"20e68244-c84d-4b64-95c2-bc50489c32ba\") " pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" Aug 04 15:42:37.952320 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:37.952141 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20e68244-c84d-4b64-95c2-bc50489c32ba-workload-certs\") pod \"e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk\" (UID: \"20e68244-c84d-4b64-95c2-bc50489c32ba\") " pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" Aug 04 15:42:37.952320 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:37.952183 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20e68244-c84d-4b64-95c2-bc50489c32ba-workload-socket\") pod \"e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk\" (UID: \"20e68244-c84d-4b64-95c2-bc50489c32ba\") " pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" Aug 04 15:42:37.952320 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:37.952215 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20e68244-c84d-4b64-95c2-bc50489c32ba-istio-podinfo\") pod \"e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk\" (UID: \"20e68244-c84d-4b64-95c2-bc50489c32ba\") " pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" Aug 04 15:42:37.952320 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:37.952268 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20e68244-c84d-4b64-95c2-bc50489c32ba-credential-socket\") pod \"e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk\" (UID: \"20e68244-c84d-4b64-95c2-bc50489c32ba\") " pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" Aug 04 15:42:37.952560 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:37.952326 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8g4w\" (UniqueName: \"kubernetes.io/projected/20e68244-c84d-4b64-95c2-bc50489c32ba-kube-api-access-k8g4w\") pod \"e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk\" (UID: \"20e68244-c84d-4b64-95c2-bc50489c32ba\") " pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" Aug 04 15:42:37.952560 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:37.952364 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20e68244-c84d-4b64-95c2-bc50489c32ba-istio-envoy\") pod \"e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk\" (UID: \"20e68244-c84d-4b64-95c2-bc50489c32ba\") " pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" Aug 04 15:42:37.952560 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:37.952383 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20e68244-c84d-4b64-95c2-bc50489c32ba-istio-token\") pod \"e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk\" (UID: \"20e68244-c84d-4b64-95c2-bc50489c32ba\") " pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" Aug 04 15:42:37.952560 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:37.952424 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20e68244-c84d-4b64-95c2-bc50489c32ba-istiod-ca-cert\") pod \"e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk\" (UID: \"20e68244-c84d-4b64-95c2-bc50489c32ba\") " pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" Aug 04 15:42:38.053914 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:38.053874 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20e68244-c84d-4b64-95c2-bc50489c32ba-workload-certs\") pod \"e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk\" (UID: \"20e68244-c84d-4b64-95c2-bc50489c32ba\") " pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" Aug 04 15:42:38.053914 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:38.053918 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20e68244-c84d-4b64-95c2-bc50489c32ba-workload-socket\") pod \"e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk\" (UID: \"20e68244-c84d-4b64-95c2-bc50489c32ba\") " pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" Aug 04 15:42:38.054169 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:38.053942 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20e68244-c84d-4b64-95c2-bc50489c32ba-istio-podinfo\") pod \"e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk\" (UID: \"20e68244-c84d-4b64-95c2-bc50489c32ba\") " pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" Aug 04 15:42:38.054169 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:38.053973 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20e68244-c84d-4b64-95c2-bc50489c32ba-credential-socket\") pod \"e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk\" (UID: \"20e68244-c84d-4b64-95c2-bc50489c32ba\") " pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" Aug 04 15:42:38.054169 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:38.054005 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8g4w\" (UniqueName: \"kubernetes.io/projected/20e68244-c84d-4b64-95c2-bc50489c32ba-kube-api-access-k8g4w\") pod \"e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk\" (UID: \"20e68244-c84d-4b64-95c2-bc50489c32ba\") " pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" Aug 04 15:42:38.054169 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:38.054035 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20e68244-c84d-4b64-95c2-bc50489c32ba-istio-envoy\") pod \"e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk\" (UID: \"20e68244-c84d-4b64-95c2-bc50489c32ba\") " pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" Aug 04 15:42:38.054435 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:38.054185 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20e68244-c84d-4b64-95c2-bc50489c32ba-istio-token\") pod \"e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk\" (UID: \"20e68244-c84d-4b64-95c2-bc50489c32ba\") " pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" Aug 04 15:42:38.054435 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:38.054230 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20e68244-c84d-4b64-95c2-bc50489c32ba-istiod-ca-cert\") pod \"e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk\" (UID: \"20e68244-c84d-4b64-95c2-bc50489c32ba\") " pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" Aug 04 15:42:38.054435 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:38.054341 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20e68244-c84d-4b64-95c2-bc50489c32ba-workload-socket\") pod \"e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk\" (UID: \"20e68244-c84d-4b64-95c2-bc50489c32ba\") " pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" Aug 04 15:42:38.054435 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:38.054422 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20e68244-c84d-4b64-95c2-bc50489c32ba-workload-certs\") pod \"e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk\" (UID: \"20e68244-c84d-4b64-95c2-bc50489c32ba\") " pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" Aug 04 15:42:38.054681 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:38.054423 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20e68244-c84d-4b64-95c2-bc50489c32ba-istio-data\") pod \"e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk\" (UID: \"20e68244-c84d-4b64-95c2-bc50489c32ba\") " pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" Aug 04 15:42:38.054681 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:38.054469 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20e68244-c84d-4b64-95c2-bc50489c32ba-credential-socket\") pod \"e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk\" (UID: \"20e68244-c84d-4b64-95c2-bc50489c32ba\") " pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" Aug 04 15:42:38.054759 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:38.054716 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20e68244-c84d-4b64-95c2-bc50489c32ba-istio-data\") pod \"e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk\" (UID: \"20e68244-c84d-4b64-95c2-bc50489c32ba\") " pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" Aug 04 15:42:38.054883 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:38.054865 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20e68244-c84d-4b64-95c2-bc50489c32ba-istiod-ca-cert\") pod \"e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk\" (UID: \"20e68244-c84d-4b64-95c2-bc50489c32ba\") " pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" Aug 04 15:42:38.056502 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:38.056482 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20e68244-c84d-4b64-95c2-bc50489c32ba-istio-envoy\") pod \"e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk\" (UID: \"20e68244-c84d-4b64-95c2-bc50489c32ba\") " pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" Aug 04 15:42:38.056583 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:38.056566 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20e68244-c84d-4b64-95c2-bc50489c32ba-istio-podinfo\") pod \"e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk\" (UID: \"20e68244-c84d-4b64-95c2-bc50489c32ba\") " pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" Aug 04 15:42:38.066119 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:38.066045 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20e68244-c84d-4b64-95c2-bc50489c32ba-istio-token\") pod \"e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk\" (UID: \"20e68244-c84d-4b64-95c2-bc50489c32ba\") " pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" Aug 04 15:42:38.066578 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:38.066560 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8g4w\" (UniqueName: \"kubernetes.io/projected/20e68244-c84d-4b64-95c2-bc50489c32ba-kube-api-access-k8g4w\") pod \"e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk\" (UID: \"20e68244-c84d-4b64-95c2-bc50489c32ba\") " pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" Aug 04 15:42:38.122862 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:38.122825 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" Aug 04 15:42:38.278378 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:38.278350 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk"] Aug 04 15:42:38.279360 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:42:38.279339 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20e68244_c84d_4b64_95c2_bc50489c32ba.slice/crio-19b1c0e7217a582280b925b3ce79a5210fcee726ea1bf66b3f4f228be996a0b3 WatchSource:0}: Error finding container 19b1c0e7217a582280b925b3ce79a5210fcee726ea1bf66b3f4f228be996a0b3: Status 404 returned error can't find the container with id 19b1c0e7217a582280b925b3ce79a5210fcee726ea1bf66b3f4f228be996a0b3 Aug 04 15:42:38.281480 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:38.281445 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:42:38.281584 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:38.281529 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:42:38.281584 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:38.281573 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:42:38.626205 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:38.626100 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" event={"ID":"20e68244-c84d-4b64-95c2-bc50489c32ba","Type":"ContainerStarted","Data":"3de404beba2b6f648753f277301cedc18168603a9afaaa9e7a6c1e7ee2221dba"} Aug 04 15:42:38.626205 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:38.626173 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" event={"ID":"20e68244-c84d-4b64-95c2-bc50489c32ba","Type":"ContainerStarted","Data":"19b1c0e7217a582280b925b3ce79a5210fcee726ea1bf66b3f4f228be996a0b3"} Aug 04 15:42:38.662469 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:38.662415 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" podStartSLOduration=1.662382091 podStartE2EDuration="1.662382091s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:42:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:42:38.660009904 +0000 UTC m=+2182.119882234" watchObservedRunningTime="2026-08-04 15:42:38.662382091 +0000 UTC m=+2182.122254473" Aug 04 15:42:39.123457 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:39.123413 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" Aug 04 15:42:39.128767 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:39.128736 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" Aug 04 15:42:39.632499 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:39.632462 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" Aug 04 15:42:39.633515 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:39.633491 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" Aug 04 15:42:47.167724 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:47.167687 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:42:54.671656 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:54.671613 2574 patch_prober.go:28] interesting pod/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": read tcp 10.132.0.2:46234->10.132.0.58:15021: read: connection reset by peer" start-of-body= Aug 04 15:42:54.672134 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:54.671706 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" podUID="20e68244-c84d-4b64-95c2-bc50489c32ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": read tcp 10.132.0.2:46234->10.132.0.58:15021: read: connection reset by peer" Aug 04 15:42:54.718548 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:54.718519 2574 generic.go:356] "Generic (PLEG): container finished" podID="20e68244-c84d-4b64-95c2-bc50489c32ba" containerID="3de404beba2b6f648753f277301cedc18168603a9afaaa9e7a6c1e7ee2221dba" exitCode=0 Aug 04 15:42:54.718645 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:54.718587 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" event={"ID":"20e68244-c84d-4b64-95c2-bc50489c32ba","Type":"ContainerDied","Data":"3de404beba2b6f648753f277301cedc18168603a9afaaa9e7a6c1e7ee2221dba"} Aug 04 15:42:54.718960 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:54.718947 2574 scope.go:117] "RemoveContainer" containerID="3de404beba2b6f648753f277301cedc18168603a9afaaa9e7a6c1e7ee2221dba" Aug 04 15:42:54.719903 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:54.719875 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:42:54.719972 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:54.719943 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:42:54.720019 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:54.719983 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:42:55.728134 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:55.728089 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" event={"ID":"20e68244-c84d-4b64-95c2-bc50489c32ba","Type":"ContainerStarted","Data":"8d82370b44484615151b57cdcf27a1cd202e5ec56a824157945c9fad81220ec1"} Aug 04 15:42:56.123957 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:56.123919 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" Aug 04 15:42:56.124177 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:56.123972 2574 patch_prober.go:28] interesting pod/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Aug 04 15:42:56.124177 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:56.124043 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" podUID="20e68244-c84d-4b64-95c2-bc50489c32ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Aug 04 15:42:57.133654 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:57.133617 2574 patch_prober.go:28] interesting pod/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Aug 04 15:42:57.134236 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:57.134201 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" podUID="20e68244-c84d-4b64-95c2-bc50489c32ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Aug 04 15:42:58.123519 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:58.123479 2574 patch_prober.go:28] interesting pod/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Aug 04 15:42:58.123735 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:58.123548 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" podUID="20e68244-c84d-4b64-95c2-bc50489c32ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Aug 04 15:42:59.123510 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:59.123456 2574 patch_prober.go:28] interesting pod/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Aug 04 15:42:59.124028 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:42:59.123536 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" podUID="20e68244-c84d-4b64-95c2-bc50489c32ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Aug 04 15:43:00.126699 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:00.126667 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" Aug 04 15:43:00.127117 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:00.126917 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" Aug 04 15:43:00.127658 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:00.127643 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" Aug 04 15:43:26.614399 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:26.614348 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:43:30.672820 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:30.672777 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk"] Aug 04 15:43:30.673223 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:30.673038 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" podUID="20e68244-c84d-4b64-95c2-bc50489c32ba" containerName="istio-proxy" containerID="cri-o://8d82370b44484615151b57cdcf27a1cd202e5ec56a824157945c9fad81220ec1" gracePeriod=30 Aug 04 15:43:35.932774 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:35.932748 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" Aug 04 15:43:35.962733 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:35.962666 2574 generic.go:356] "Generic (PLEG): container finished" podID="20e68244-c84d-4b64-95c2-bc50489c32ba" containerID="8d82370b44484615151b57cdcf27a1cd202e5ec56a824157945c9fad81220ec1" exitCode=0 Aug 04 15:43:35.962733 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:35.962705 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" event={"ID":"20e68244-c84d-4b64-95c2-bc50489c32ba","Type":"ContainerDied","Data":"8d82370b44484615151b57cdcf27a1cd202e5ec56a824157945c9fad81220ec1"} Aug 04 15:43:35.962733 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:35.962728 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" Aug 04 15:43:35.963018 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:35.962747 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk" event={"ID":"20e68244-c84d-4b64-95c2-bc50489c32ba","Type":"ContainerDied","Data":"19b1c0e7217a582280b925b3ce79a5210fcee726ea1bf66b3f4f228be996a0b3"} Aug 04 15:43:35.963018 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:35.962773 2574 scope.go:117] "RemoveContainer" containerID="8d82370b44484615151b57cdcf27a1cd202e5ec56a824157945c9fad81220ec1" Aug 04 15:43:35.971861 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:35.971839 2574 scope.go:117] "RemoveContainer" containerID="3de404beba2b6f648753f277301cedc18168603a9afaaa9e7a6c1e7ee2221dba" Aug 04 15:43:35.980651 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:35.980630 2574 scope.go:117] "RemoveContainer" containerID="8d82370b44484615151b57cdcf27a1cd202e5ec56a824157945c9fad81220ec1" Aug 04 15:43:35.980921 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:43:35.980900 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d82370b44484615151b57cdcf27a1cd202e5ec56a824157945c9fad81220ec1\": container with ID starting with 8d82370b44484615151b57cdcf27a1cd202e5ec56a824157945c9fad81220ec1 not found: ID does not exist" containerID="8d82370b44484615151b57cdcf27a1cd202e5ec56a824157945c9fad81220ec1" Aug 04 15:43:35.981003 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:35.980928 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d82370b44484615151b57cdcf27a1cd202e5ec56a824157945c9fad81220ec1"} err="failed to get container status \"8d82370b44484615151b57cdcf27a1cd202e5ec56a824157945c9fad81220ec1\": rpc error: code = NotFound desc = could not find container \"8d82370b44484615151b57cdcf27a1cd202e5ec56a824157945c9fad81220ec1\": container with ID starting with 8d82370b44484615151b57cdcf27a1cd202e5ec56a824157945c9fad81220ec1 not found: ID does not exist" Aug 04 15:43:35.981003 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:35.980944 2574 scope.go:117] "RemoveContainer" containerID="3de404beba2b6f648753f277301cedc18168603a9afaaa9e7a6c1e7ee2221dba" Aug 04 15:43:35.981183 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:35.981163 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20e68244-c84d-4b64-95c2-bc50489c32ba-istio-data\") pod \"20e68244-c84d-4b64-95c2-bc50489c32ba\" (UID: \"20e68244-c84d-4b64-95c2-bc50489c32ba\") " Aug 04 15:43:35.981246 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:43:35.981184 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3de404beba2b6f648753f277301cedc18168603a9afaaa9e7a6c1e7ee2221dba\": container with ID starting with 3de404beba2b6f648753f277301cedc18168603a9afaaa9e7a6c1e7ee2221dba not found: ID does not exist" containerID="3de404beba2b6f648753f277301cedc18168603a9afaaa9e7a6c1e7ee2221dba" Aug 04 15:43:35.981246 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:35.981205 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20e68244-c84d-4b64-95c2-bc50489c32ba-credential-socket\") pod \"20e68244-c84d-4b64-95c2-bc50489c32ba\" (UID: \"20e68244-c84d-4b64-95c2-bc50489c32ba\") " Aug 04 15:43:35.981246 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:35.981214 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3de404beba2b6f648753f277301cedc18168603a9afaaa9e7a6c1e7ee2221dba"} err="failed to get container status \"3de404beba2b6f648753f277301cedc18168603a9afaaa9e7a6c1e7ee2221dba\": rpc error: code = NotFound desc = could not find container \"3de404beba2b6f648753f277301cedc18168603a9afaaa9e7a6c1e7ee2221dba\": container with ID starting with 3de404beba2b6f648753f277301cedc18168603a9afaaa9e7a6c1e7ee2221dba not found: ID does not exist" Aug 04 15:43:35.981246 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:35.981240 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20e68244-c84d-4b64-95c2-bc50489c32ba-istiod-ca-cert\") pod \"20e68244-c84d-4b64-95c2-bc50489c32ba\" (UID: \"20e68244-c84d-4b64-95c2-bc50489c32ba\") " Aug 04 15:43:35.981485 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:35.981273 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20e68244-c84d-4b64-95c2-bc50489c32ba-workload-certs\") pod \"20e68244-c84d-4b64-95c2-bc50489c32ba\" (UID: \"20e68244-c84d-4b64-95c2-bc50489c32ba\") " Aug 04 15:43:35.981485 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:35.981309 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20e68244-c84d-4b64-95c2-bc50489c32ba-istio-envoy\") pod \"20e68244-c84d-4b64-95c2-bc50489c32ba\" (UID: \"20e68244-c84d-4b64-95c2-bc50489c32ba\") " Aug 04 15:43:35.981485 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:35.981335 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k8g4w\" (UniqueName: \"kubernetes.io/projected/20e68244-c84d-4b64-95c2-bc50489c32ba-kube-api-access-k8g4w\") pod \"20e68244-c84d-4b64-95c2-bc50489c32ba\" (UID: \"20e68244-c84d-4b64-95c2-bc50489c32ba\") " Aug 04 15:43:35.981485 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:35.981413 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20e68244-c84d-4b64-95c2-bc50489c32ba-workload-socket\") pod \"20e68244-c84d-4b64-95c2-bc50489c32ba\" (UID: \"20e68244-c84d-4b64-95c2-bc50489c32ba\") " Aug 04 15:43:35.981485 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:35.981441 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20e68244-c84d-4b64-95c2-bc50489c32ba-istio-podinfo\") pod \"20e68244-c84d-4b64-95c2-bc50489c32ba\" (UID: \"20e68244-c84d-4b64-95c2-bc50489c32ba\") " Aug 04 15:43:35.981485 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:35.981468 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20e68244-c84d-4b64-95c2-bc50489c32ba-istio-token\") pod \"20e68244-c84d-4b64-95c2-bc50489c32ba\" (UID: \"20e68244-c84d-4b64-95c2-bc50489c32ba\") " Aug 04 15:43:35.981794 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:35.981478 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20e68244-c84d-4b64-95c2-bc50489c32ba-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "20e68244-c84d-4b64-95c2-bc50489c32ba" (UID: "20e68244-c84d-4b64-95c2-bc50489c32ba"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:43:35.981794 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:35.981627 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20e68244-c84d-4b64-95c2-bc50489c32ba-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "20e68244-c84d-4b64-95c2-bc50489c32ba" (UID: "20e68244-c84d-4b64-95c2-bc50489c32ba"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:43:35.981794 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:35.981705 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20e68244-c84d-4b64-95c2-bc50489c32ba-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "20e68244-c84d-4b64-95c2-bc50489c32ba" (UID: "20e68244-c84d-4b64-95c2-bc50489c32ba"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:43:35.981794 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:35.981745 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20e68244-c84d-4b64-95c2-bc50489c32ba-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "20e68244-c84d-4b64-95c2-bc50489c32ba" (UID: "20e68244-c84d-4b64-95c2-bc50489c32ba"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 15:43:35.982002 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:35.981803 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20e68244-c84d-4b64-95c2-bc50489c32ba-credential-socket\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:43:35.982002 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:35.981816 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20e68244-c84d-4b64-95c2-bc50489c32ba-workload-certs\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:43:35.982002 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:35.981827 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20e68244-c84d-4b64-95c2-bc50489c32ba-workload-socket\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:43:35.982002 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:35.981952 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20e68244-c84d-4b64-95c2-bc50489c32ba-istio-data" (OuterVolumeSpecName: "istio-data") pod "20e68244-c84d-4b64-95c2-bc50489c32ba" (UID: "20e68244-c84d-4b64-95c2-bc50489c32ba"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:43:35.983775 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:35.983752 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20e68244-c84d-4b64-95c2-bc50489c32ba-istio-token" (OuterVolumeSpecName: "istio-token") pod "20e68244-c84d-4b64-95c2-bc50489c32ba" (UID: "20e68244-c84d-4b64-95c2-bc50489c32ba"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 15:43:35.984053 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:35.984032 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20e68244-c84d-4b64-95c2-bc50489c32ba-kube-api-access-k8g4w" (OuterVolumeSpecName: "kube-api-access-k8g4w") pod "20e68244-c84d-4b64-95c2-bc50489c32ba" (UID: "20e68244-c84d-4b64-95c2-bc50489c32ba"). InnerVolumeSpecName "kube-api-access-k8g4w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 15:43:35.984136 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:35.984050 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/20e68244-c84d-4b64-95c2-bc50489c32ba-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "20e68244-c84d-4b64-95c2-bc50489c32ba" (UID: "20e68244-c84d-4b64-95c2-bc50489c32ba"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 15:43:35.984192 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:35.984142 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20e68244-c84d-4b64-95c2-bc50489c32ba-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "20e68244-c84d-4b64-95c2-bc50489c32ba" (UID: "20e68244-c84d-4b64-95c2-bc50489c32ba"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:43:36.082823 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:36.082789 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20e68244-c84d-4b64-95c2-bc50489c32ba-istio-podinfo\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:43:36.082823 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:36.082815 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20e68244-c84d-4b64-95c2-bc50489c32ba-istio-token\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:43:36.082823 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:36.082828 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20e68244-c84d-4b64-95c2-bc50489c32ba-istio-data\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:43:36.083078 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:36.082839 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20e68244-c84d-4b64-95c2-bc50489c32ba-istiod-ca-cert\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:43:36.083078 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:36.082851 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20e68244-c84d-4b64-95c2-bc50489c32ba-istio-envoy\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:43:36.083078 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:36.082862 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8g4w\" (UniqueName: \"kubernetes.io/projected/20e68244-c84d-4b64-95c2-bc50489c32ba-kube-api-access-k8g4w\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:43:36.288618 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:36.288584 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk"] Aug 04 15:43:36.292909 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:36.292886 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-96b2e1f8-openshift-default-54cdf7b488-pbbqk"] Aug 04 15:43:37.053278 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:43:37.053240 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20e68244-c84d-4b64-95c2-bc50489c32ba" path="/var/lib/kubelet/pods/20e68244-c84d-4b64-95c2-bc50489c32ba/volumes" Aug 04 15:44:04.025312 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:44:04.025276 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:45:46.737121 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:46.737085 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm"] Aug 04 15:45:46.737601 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:46.737492 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20e68244-c84d-4b64-95c2-bc50489c32ba" containerName="istio-proxy" Aug 04 15:45:46.737601 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:46.737503 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="20e68244-c84d-4b64-95c2-bc50489c32ba" containerName="istio-proxy" Aug 04 15:45:46.737601 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:46.737512 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20e68244-c84d-4b64-95c2-bc50489c32ba" containerName="istio-proxy" Aug 04 15:45:46.737601 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:46.737517 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="20e68244-c84d-4b64-95c2-bc50489c32ba" containerName="istio-proxy" Aug 04 15:45:46.737601 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:46.737578 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6c8d44c-0d49-493a-b995-f8eb8236d663" containerName="istio-proxy" Aug 04 15:45:46.737601 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:46.737586 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="20e68244-c84d-4b64-95c2-bc50489c32ba" containerName="istio-proxy" Aug 04 15:45:46.737601 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:46.737593 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="74570765-7077-44c1-8d7a-9a19b5e3d1a8" containerName="istio-proxy" Aug 04 15:45:46.737601 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:46.737599 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="20e68244-c84d-4b64-95c2-bc50489c32ba" containerName="istio-proxy" Aug 04 15:45:46.740628 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:46.740610 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" Aug 04 15:45:46.743468 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:46.743440 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-01d7a1ad-openshift-default-dockercfg-4tjwx\"" Aug 04 15:45:46.754367 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:46.754345 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm"] Aug 04 15:45:46.802558 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:46.802528 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2424c74e-70bf-4e19-8966-16b2d752ac3f-istiod-ca-cert\") pod \"e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm\" (UID: \"2424c74e-70bf-4e19-8966-16b2d752ac3f\") " pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" Aug 04 15:45:46.802558 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:46.802559 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2424c74e-70bf-4e19-8966-16b2d752ac3f-istio-data\") pod \"e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm\" (UID: \"2424c74e-70bf-4e19-8966-16b2d752ac3f\") " pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" Aug 04 15:45:46.802741 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:46.802583 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntctp\" (UniqueName: \"kubernetes.io/projected/2424c74e-70bf-4e19-8966-16b2d752ac3f-kube-api-access-ntctp\") pod \"e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm\" (UID: \"2424c74e-70bf-4e19-8966-16b2d752ac3f\") " pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" Aug 04 15:45:46.802741 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:46.802654 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2424c74e-70bf-4e19-8966-16b2d752ac3f-istio-envoy\") pod \"e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm\" (UID: \"2424c74e-70bf-4e19-8966-16b2d752ac3f\") " pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" Aug 04 15:45:46.802741 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:46.802683 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2424c74e-70bf-4e19-8966-16b2d752ac3f-istio-token\") pod \"e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm\" (UID: \"2424c74e-70bf-4e19-8966-16b2d752ac3f\") " pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" Aug 04 15:45:46.802741 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:46.802725 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2424c74e-70bf-4e19-8966-16b2d752ac3f-credential-socket\") pod \"e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm\" (UID: \"2424c74e-70bf-4e19-8966-16b2d752ac3f\") " pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" Aug 04 15:45:46.802874 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:46.802782 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2424c74e-70bf-4e19-8966-16b2d752ac3f-istio-podinfo\") pod \"e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm\" (UID: \"2424c74e-70bf-4e19-8966-16b2d752ac3f\") " pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" Aug 04 15:45:46.802874 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:46.802801 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2424c74e-70bf-4e19-8966-16b2d752ac3f-workload-certs\") pod \"e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm\" (UID: \"2424c74e-70bf-4e19-8966-16b2d752ac3f\") " pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" Aug 04 15:45:46.802874 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:46.802827 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2424c74e-70bf-4e19-8966-16b2d752ac3f-workload-socket\") pod \"e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm\" (UID: \"2424c74e-70bf-4e19-8966-16b2d752ac3f\") " pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" Aug 04 15:45:46.904090 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:46.904038 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntctp\" (UniqueName: \"kubernetes.io/projected/2424c74e-70bf-4e19-8966-16b2d752ac3f-kube-api-access-ntctp\") pod \"e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm\" (UID: \"2424c74e-70bf-4e19-8966-16b2d752ac3f\") " pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" Aug 04 15:45:46.904090 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:46.904099 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2424c74e-70bf-4e19-8966-16b2d752ac3f-istio-envoy\") pod \"e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm\" (UID: \"2424c74e-70bf-4e19-8966-16b2d752ac3f\") " pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" Aug 04 15:45:46.904341 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:46.904120 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2424c74e-70bf-4e19-8966-16b2d752ac3f-istio-token\") pod \"e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm\" (UID: \"2424c74e-70bf-4e19-8966-16b2d752ac3f\") " pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" Aug 04 15:45:46.904341 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:46.904151 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2424c74e-70bf-4e19-8966-16b2d752ac3f-credential-socket\") pod \"e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm\" (UID: \"2424c74e-70bf-4e19-8966-16b2d752ac3f\") " pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" Aug 04 15:45:46.904341 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:46.904189 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2424c74e-70bf-4e19-8966-16b2d752ac3f-istio-podinfo\") pod \"e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm\" (UID: \"2424c74e-70bf-4e19-8966-16b2d752ac3f\") " pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" Aug 04 15:45:46.904341 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:46.904214 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2424c74e-70bf-4e19-8966-16b2d752ac3f-workload-certs\") pod \"e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm\" (UID: \"2424c74e-70bf-4e19-8966-16b2d752ac3f\") " pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" Aug 04 15:45:46.904341 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:46.904249 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2424c74e-70bf-4e19-8966-16b2d752ac3f-workload-socket\") pod \"e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm\" (UID: \"2424c74e-70bf-4e19-8966-16b2d752ac3f\") " pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" Aug 04 15:45:46.904341 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:46.904299 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2424c74e-70bf-4e19-8966-16b2d752ac3f-istiod-ca-cert\") pod \"e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm\" (UID: \"2424c74e-70bf-4e19-8966-16b2d752ac3f\") " pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" Aug 04 15:45:46.904341 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:46.904320 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2424c74e-70bf-4e19-8966-16b2d752ac3f-istio-data\") pod \"e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm\" (UID: \"2424c74e-70bf-4e19-8966-16b2d752ac3f\") " pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" Aug 04 15:45:46.904775 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:46.904700 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2424c74e-70bf-4e19-8966-16b2d752ac3f-istio-data\") pod \"e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm\" (UID: \"2424c74e-70bf-4e19-8966-16b2d752ac3f\") " pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" Aug 04 15:45:46.904775 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:46.904728 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2424c74e-70bf-4e19-8966-16b2d752ac3f-workload-certs\") pod \"e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm\" (UID: \"2424c74e-70bf-4e19-8966-16b2d752ac3f\") " pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" Aug 04 15:45:46.904862 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:46.904798 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2424c74e-70bf-4e19-8966-16b2d752ac3f-workload-socket\") pod \"e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm\" (UID: \"2424c74e-70bf-4e19-8966-16b2d752ac3f\") " pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" Aug 04 15:45:46.904906 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:46.904871 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2424c74e-70bf-4e19-8966-16b2d752ac3f-credential-socket\") pod \"e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm\" (UID: \"2424c74e-70bf-4e19-8966-16b2d752ac3f\") " pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" Aug 04 15:45:46.905077 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:46.905058 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2424c74e-70bf-4e19-8966-16b2d752ac3f-istiod-ca-cert\") pod \"e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm\" (UID: \"2424c74e-70bf-4e19-8966-16b2d752ac3f\") " pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" Aug 04 15:45:46.906982 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:46.906961 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2424c74e-70bf-4e19-8966-16b2d752ac3f-istio-envoy\") pod \"e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm\" (UID: \"2424c74e-70bf-4e19-8966-16b2d752ac3f\") " pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" Aug 04 15:45:46.907112 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:46.907095 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2424c74e-70bf-4e19-8966-16b2d752ac3f-istio-podinfo\") pod \"e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm\" (UID: \"2424c74e-70bf-4e19-8966-16b2d752ac3f\") " pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" Aug 04 15:45:46.912933 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:46.912910 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntctp\" (UniqueName: \"kubernetes.io/projected/2424c74e-70bf-4e19-8966-16b2d752ac3f-kube-api-access-ntctp\") pod \"e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm\" (UID: \"2424c74e-70bf-4e19-8966-16b2d752ac3f\") " pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" Aug 04 15:45:46.913149 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:46.913129 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2424c74e-70bf-4e19-8966-16b2d752ac3f-istio-token\") pod \"e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm\" (UID: \"2424c74e-70bf-4e19-8966-16b2d752ac3f\") " pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" Aug 04 15:45:47.050582 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:47.050504 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" Aug 04 15:45:47.198618 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:47.198594 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm"] Aug 04 15:45:47.201055 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:45:47.201031 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2424c74e_70bf_4e19_8966_16b2d752ac3f.slice/crio-b57848922ad58b0909c928e46171ec09008b06df660500324210ae8ea522ae45 WatchSource:0}: Error finding container b57848922ad58b0909c928e46171ec09008b06df660500324210ae8ea522ae45: Status 404 returned error can't find the container with id b57848922ad58b0909c928e46171ec09008b06df660500324210ae8ea522ae45 Aug 04 15:45:47.203173 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:47.203135 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:45:47.203262 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:47.203206 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:45:47.203262 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:47.203236 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:45:47.678595 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:47.678553 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" event={"ID":"2424c74e-70bf-4e19-8966-16b2d752ac3f","Type":"ContainerStarted","Data":"c74607cf5d05f861fa6d9463639910557da601ad99219f7e88c3bf3be58c6fd2"} Aug 04 15:45:47.678767 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:47.678620 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" event={"ID":"2424c74e-70bf-4e19-8966-16b2d752ac3f","Type":"ContainerStarted","Data":"b57848922ad58b0909c928e46171ec09008b06df660500324210ae8ea522ae45"} Aug 04 15:45:47.707408 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:47.706201 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" podStartSLOduration=1.706181935 podStartE2EDuration="1.706181935s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:45:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:45:47.701007854 +0000 UTC m=+2371.160880204" watchObservedRunningTime="2026-08-04 15:45:47.706181935 +0000 UTC m=+2371.166054285" Aug 04 15:45:48.051282 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:48.051245 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" Aug 04 15:45:48.056595 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:48.056572 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" Aug 04 15:45:48.685753 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:48.685722 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" Aug 04 15:45:48.686690 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:48.686670 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" Aug 04 15:45:50.122958 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:50.122920 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:45:59.248447 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:45:59.248409 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:46:03.687428 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:46:03.687308 2574 patch_prober.go:28] interesting pod/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Aug 04 15:46:03.687428 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:46:03.687376 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" podUID="2424c74e-70bf-4e19-8966-16b2d752ac3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Aug 04 15:46:03.781222 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:46:03.781187 2574 generic.go:356] "Generic (PLEG): container finished" podID="2424c74e-70bf-4e19-8966-16b2d752ac3f" containerID="c74607cf5d05f861fa6d9463639910557da601ad99219f7e88c3bf3be58c6fd2" exitCode=0 Aug 04 15:46:03.781382 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:46:03.781269 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" event={"ID":"2424c74e-70bf-4e19-8966-16b2d752ac3f","Type":"ContainerDied","Data":"c74607cf5d05f861fa6d9463639910557da601ad99219f7e88c3bf3be58c6fd2"} Aug 04 15:46:03.781677 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:46:03.781663 2574 scope.go:117] "RemoveContainer" containerID="c74607cf5d05f861fa6d9463639910557da601ad99219f7e88c3bf3be58c6fd2" Aug 04 15:46:03.782651 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:46:03.782622 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:46:03.782715 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:46:03.782697 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:46:03.782764 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:46:03.782751 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:46:04.788760 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:46:04.788726 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" event={"ID":"2424c74e-70bf-4e19-8966-16b2d752ac3f","Type":"ContainerStarted","Data":"fe5b652462ce27952f7515cf89512218ab64122ce88e8cc4ce60869918f3a550"} Aug 04 15:46:05.052090 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:46:05.051942 2574 patch_prober.go:28] interesting pod/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Aug 04 15:46:05.052090 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:46:05.052034 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" podUID="2424c74e-70bf-4e19-8966-16b2d752ac3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Aug 04 15:46:05.054974 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:46:05.054949 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" Aug 04 15:46:06.051048 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:46:06.051011 2574 patch_prober.go:28] interesting pod/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Aug 04 15:46:06.051461 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:46:06.051077 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" podUID="2424c74e-70bf-4e19-8966-16b2d752ac3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Aug 04 15:46:07.051081 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:46:07.051043 2574 patch_prober.go:28] interesting pod/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Aug 04 15:46:07.051546 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:46:07.051099 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" podUID="2424c74e-70bf-4e19-8966-16b2d752ac3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Aug 04 15:46:08.051895 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:46:08.051861 2574 patch_prober.go:28] interesting pod/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Aug 04 15:46:08.052325 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:46:08.051917 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" podUID="2424c74e-70bf-4e19-8966-16b2d752ac3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Aug 04 15:46:09.050989 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:46:09.050945 2574 patch_prober.go:28] interesting pod/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Aug 04 15:46:09.051190 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:46:09.051005 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" podUID="2424c74e-70bf-4e19-8966-16b2d752ac3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Aug 04 15:46:10.054558 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:46:10.054521 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" Aug 04 15:46:10.054928 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:46:10.054758 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" Aug 04 15:46:10.055555 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:46:10.055537 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" Aug 04 15:46:17.222356 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:46:17.222325 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-r8wkp_0580fdab-0381-4d8e-be5f-bde5eb44e0e6/console-operator/1.log" Aug 04 15:46:17.226902 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:46:17.226877 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8h6k_9c62e3b1-c2dc-4b2a-a61d-7d80049f355f/ovn-acl-logging/0.log" Aug 04 15:46:17.230239 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:46:17.230216 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-r8wkp_0580fdab-0381-4d8e-be5f-bde5eb44e0e6/console-operator/1.log" Aug 04 15:46:17.234483 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:46:17.234466 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8h6k_9c62e3b1-c2dc-4b2a-a61d-7d80049f355f/ovn-acl-logging/0.log" Aug 04 15:46:32.019412 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:46:32.019365 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:46:46.721540 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:46:46.721458 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:47:18.616445 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:47:18.616405 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:47:29.714622 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:47:29.714589 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:48:12.690844 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:12.690765 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm"] Aug 04 15:48:12.691325 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:12.691009 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" podUID="2424c74e-70bf-4e19-8966-16b2d752ac3f" containerName="istio-proxy" containerID="cri-o://fe5b652462ce27952f7515cf89512218ab64122ce88e8cc4ce60869918f3a550" gracePeriod=30 Aug 04 15:48:15.425805 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:15.425767 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:48:17.941400 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:17.941362 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" Aug 04 15:48:18.038331 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:18.038303 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2424c74e-70bf-4e19-8966-16b2d752ac3f-istio-token\") pod \"2424c74e-70bf-4e19-8966-16b2d752ac3f\" (UID: \"2424c74e-70bf-4e19-8966-16b2d752ac3f\") " Aug 04 15:48:18.038504 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:18.038365 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2424c74e-70bf-4e19-8966-16b2d752ac3f-credential-socket\") pod \"2424c74e-70bf-4e19-8966-16b2d752ac3f\" (UID: \"2424c74e-70bf-4e19-8966-16b2d752ac3f\") " Aug 04 15:48:18.038504 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:18.038430 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2424c74e-70bf-4e19-8966-16b2d752ac3f-istiod-ca-cert\") pod \"2424c74e-70bf-4e19-8966-16b2d752ac3f\" (UID: \"2424c74e-70bf-4e19-8966-16b2d752ac3f\") " Aug 04 15:48:18.038504 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:18.038454 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2424c74e-70bf-4e19-8966-16b2d752ac3f-workload-socket\") pod \"2424c74e-70bf-4e19-8966-16b2d752ac3f\" (UID: \"2424c74e-70bf-4e19-8966-16b2d752ac3f\") " Aug 04 15:48:18.038504 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:18.038476 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2424c74e-70bf-4e19-8966-16b2d752ac3f-istio-data\") pod \"2424c74e-70bf-4e19-8966-16b2d752ac3f\" (UID: \"2424c74e-70bf-4e19-8966-16b2d752ac3f\") " Aug 04 15:48:18.038504 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:18.038498 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ntctp\" (UniqueName: \"kubernetes.io/projected/2424c74e-70bf-4e19-8966-16b2d752ac3f-kube-api-access-ntctp\") pod \"2424c74e-70bf-4e19-8966-16b2d752ac3f\" (UID: \"2424c74e-70bf-4e19-8966-16b2d752ac3f\") " Aug 04 15:48:18.038751 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:18.038531 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2424c74e-70bf-4e19-8966-16b2d752ac3f-istio-envoy\") pod \"2424c74e-70bf-4e19-8966-16b2d752ac3f\" (UID: \"2424c74e-70bf-4e19-8966-16b2d752ac3f\") " Aug 04 15:48:18.038751 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:18.038641 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2424c74e-70bf-4e19-8966-16b2d752ac3f-workload-certs\") pod \"2424c74e-70bf-4e19-8966-16b2d752ac3f\" (UID: \"2424c74e-70bf-4e19-8966-16b2d752ac3f\") " Aug 04 15:48:18.038751 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:18.038685 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2424c74e-70bf-4e19-8966-16b2d752ac3f-istio-podinfo\") pod \"2424c74e-70bf-4e19-8966-16b2d752ac3f\" (UID: \"2424c74e-70bf-4e19-8966-16b2d752ac3f\") " Aug 04 15:48:18.038751 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:18.038664 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2424c74e-70bf-4e19-8966-16b2d752ac3f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2424c74e-70bf-4e19-8966-16b2d752ac3f" (UID: "2424c74e-70bf-4e19-8966-16b2d752ac3f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:48:18.038751 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:18.038738 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2424c74e-70bf-4e19-8966-16b2d752ac3f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2424c74e-70bf-4e19-8966-16b2d752ac3f" (UID: "2424c74e-70bf-4e19-8966-16b2d752ac3f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:48:18.039027 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:18.039000 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2424c74e-70bf-4e19-8966-16b2d752ac3f-credential-socket\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:48:18.039085 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:18.039017 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2424c74e-70bf-4e19-8966-16b2d752ac3f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2424c74e-70bf-4e19-8966-16b2d752ac3f" (UID: "2424c74e-70bf-4e19-8966-16b2d752ac3f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:48:18.039085 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:18.039029 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2424c74e-70bf-4e19-8966-16b2d752ac3f-workload-socket\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:48:18.039184 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:18.039132 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2424c74e-70bf-4e19-8966-16b2d752ac3f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2424c74e-70bf-4e19-8966-16b2d752ac3f" (UID: "2424c74e-70bf-4e19-8966-16b2d752ac3f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 15:48:18.039563 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:18.039540 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2424c74e-70bf-4e19-8966-16b2d752ac3f-istio-data" (OuterVolumeSpecName: "istio-data") pod "2424c74e-70bf-4e19-8966-16b2d752ac3f" (UID: "2424c74e-70bf-4e19-8966-16b2d752ac3f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:48:18.040891 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:18.040871 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2424c74e-70bf-4e19-8966-16b2d752ac3f-istio-token" (OuterVolumeSpecName: "istio-token") pod "2424c74e-70bf-4e19-8966-16b2d752ac3f" (UID: "2424c74e-70bf-4e19-8966-16b2d752ac3f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 15:48:18.040979 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:18.040934 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2424c74e-70bf-4e19-8966-16b2d752ac3f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2424c74e-70bf-4e19-8966-16b2d752ac3f" (UID: "2424c74e-70bf-4e19-8966-16b2d752ac3f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:48:18.041120 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:18.041099 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2424c74e-70bf-4e19-8966-16b2d752ac3f-kube-api-access-ntctp" (OuterVolumeSpecName: "kube-api-access-ntctp") pod "2424c74e-70bf-4e19-8966-16b2d752ac3f" (UID: "2424c74e-70bf-4e19-8966-16b2d752ac3f"). InnerVolumeSpecName "kube-api-access-ntctp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 15:48:18.041120 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:18.041100 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2424c74e-70bf-4e19-8966-16b2d752ac3f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2424c74e-70bf-4e19-8966-16b2d752ac3f" (UID: "2424c74e-70bf-4e19-8966-16b2d752ac3f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 15:48:18.140068 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:18.140038 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2424c74e-70bf-4e19-8966-16b2d752ac3f-istiod-ca-cert\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:48:18.140068 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:18.140067 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2424c74e-70bf-4e19-8966-16b2d752ac3f-istio-data\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:48:18.140251 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:18.140080 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ntctp\" (UniqueName: \"kubernetes.io/projected/2424c74e-70bf-4e19-8966-16b2d752ac3f-kube-api-access-ntctp\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:48:18.140251 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:18.140089 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2424c74e-70bf-4e19-8966-16b2d752ac3f-istio-envoy\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:48:18.140251 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:18.140098 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2424c74e-70bf-4e19-8966-16b2d752ac3f-workload-certs\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:48:18.140251 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:18.140106 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2424c74e-70bf-4e19-8966-16b2d752ac3f-istio-podinfo\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:48:18.140251 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:18.140115 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2424c74e-70bf-4e19-8966-16b2d752ac3f-istio-token\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:48:18.516642 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:18.516608 2574 generic.go:356] "Generic (PLEG): container finished" podID="2424c74e-70bf-4e19-8966-16b2d752ac3f" containerID="fe5b652462ce27952f7515cf89512218ab64122ce88e8cc4ce60869918f3a550" exitCode=0 Aug 04 15:48:18.516808 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:18.516657 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" Aug 04 15:48:18.516808 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:18.516672 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" event={"ID":"2424c74e-70bf-4e19-8966-16b2d752ac3f","Type":"ContainerDied","Data":"fe5b652462ce27952f7515cf89512218ab64122ce88e8cc4ce60869918f3a550"} Aug 04 15:48:18.516808 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:18.516719 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm" event={"ID":"2424c74e-70bf-4e19-8966-16b2d752ac3f","Type":"ContainerDied","Data":"b57848922ad58b0909c928e46171ec09008b06df660500324210ae8ea522ae45"} Aug 04 15:48:18.516808 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:18.516743 2574 scope.go:117] "RemoveContainer" containerID="fe5b652462ce27952f7515cf89512218ab64122ce88e8cc4ce60869918f3a550" Aug 04 15:48:18.529223 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:18.529199 2574 scope.go:117] "RemoveContainer" containerID="c74607cf5d05f861fa6d9463639910557da601ad99219f7e88c3bf3be58c6fd2" Aug 04 15:48:18.540613 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:18.540589 2574 scope.go:117] "RemoveContainer" containerID="fe5b652462ce27952f7515cf89512218ab64122ce88e8cc4ce60869918f3a550" Aug 04 15:48:18.540855 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:48:18.540836 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe5b652462ce27952f7515cf89512218ab64122ce88e8cc4ce60869918f3a550\": container with ID starting with fe5b652462ce27952f7515cf89512218ab64122ce88e8cc4ce60869918f3a550 not found: ID does not exist" containerID="fe5b652462ce27952f7515cf89512218ab64122ce88e8cc4ce60869918f3a550" Aug 04 15:48:18.540895 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:18.540864 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe5b652462ce27952f7515cf89512218ab64122ce88e8cc4ce60869918f3a550"} err="failed to get container status \"fe5b652462ce27952f7515cf89512218ab64122ce88e8cc4ce60869918f3a550\": rpc error: code = NotFound desc = could not find container \"fe5b652462ce27952f7515cf89512218ab64122ce88e8cc4ce60869918f3a550\": container with ID starting with fe5b652462ce27952f7515cf89512218ab64122ce88e8cc4ce60869918f3a550 not found: ID does not exist" Aug 04 15:48:18.540895 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:18.540881 2574 scope.go:117] "RemoveContainer" containerID="c74607cf5d05f861fa6d9463639910557da601ad99219f7e88c3bf3be58c6fd2" Aug 04 15:48:18.541104 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:48:18.541084 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c74607cf5d05f861fa6d9463639910557da601ad99219f7e88c3bf3be58c6fd2\": container with ID starting with c74607cf5d05f861fa6d9463639910557da601ad99219f7e88c3bf3be58c6fd2 not found: ID does not exist" containerID="c74607cf5d05f861fa6d9463639910557da601ad99219f7e88c3bf3be58c6fd2" Aug 04 15:48:18.541138 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:18.541111 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c74607cf5d05f861fa6d9463639910557da601ad99219f7e88c3bf3be58c6fd2"} err="failed to get container status \"c74607cf5d05f861fa6d9463639910557da601ad99219f7e88c3bf3be58c6fd2\": rpc error: code = NotFound desc = could not find container \"c74607cf5d05f861fa6d9463639910557da601ad99219f7e88c3bf3be58c6fd2\": container with ID starting with c74607cf5d05f861fa6d9463639910557da601ad99219f7e88c3bf3be58c6fd2 not found: ID does not exist" Aug 04 15:48:18.546813 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:18.546792 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm"] Aug 04 15:48:18.550924 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:18.550904 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-01d7a1ad-openshift-default-7f65b7464-tf2tm"] Aug 04 15:48:19.054368 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:19.054333 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2424c74e-70bf-4e19-8966-16b2d752ac3f" path="/var/lib/kubelet/pods/2424c74e-70bf-4e19-8966-16b2d752ac3f/volumes" Aug 04 15:48:39.233029 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:39.232989 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:48:50.023201 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:48:50.023164 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:49:30.609497 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:30.609461 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79"] Aug 04 15:49:30.609957 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:30.609844 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2424c74e-70bf-4e19-8966-16b2d752ac3f" containerName="istio-proxy" Aug 04 15:49:30.609957 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:30.609857 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2424c74e-70bf-4e19-8966-16b2d752ac3f" containerName="istio-proxy" Aug 04 15:49:30.609957 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:30.609872 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2424c74e-70bf-4e19-8966-16b2d752ac3f" containerName="istio-proxy" Aug 04 15:49:30.609957 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:30.609878 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2424c74e-70bf-4e19-8966-16b2d752ac3f" containerName="istio-proxy" Aug 04 15:49:30.609957 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:30.609942 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="2424c74e-70bf-4e19-8966-16b2d752ac3f" containerName="istio-proxy" Aug 04 15:49:30.612739 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:30.612720 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" Aug 04 15:49:30.616879 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:30.616857 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-c2a99b-openshift-default-dockercfg-t7rqg\"" Aug 04 15:49:30.637512 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:30.637488 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79"] Aug 04 15:49:30.676359 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:30.676259 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db88d5f4-c855-48ac-9c0e-3dcd3305762a-credential-socket\") pod \"e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79\" (UID: \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\") " pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" Aug 04 15:49:30.676359 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:30.676318 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db88d5f4-c855-48ac-9c0e-3dcd3305762a-istio-data\") pod \"e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79\" (UID: \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\") " pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" Aug 04 15:49:30.676359 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:30.676359 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db88d5f4-c855-48ac-9c0e-3dcd3305762a-istio-envoy\") pod \"e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79\" (UID: \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\") " pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" Aug 04 15:49:30.676666 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:30.676465 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4brl\" (UniqueName: \"kubernetes.io/projected/db88d5f4-c855-48ac-9c0e-3dcd3305762a-kube-api-access-q4brl\") pod \"e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79\" (UID: \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\") " pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" Aug 04 15:49:30.676666 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:30.676546 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db88d5f4-c855-48ac-9c0e-3dcd3305762a-workload-socket\") pod \"e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79\" (UID: \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\") " pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" Aug 04 15:49:30.676666 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:30.676584 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db88d5f4-c855-48ac-9c0e-3dcd3305762a-istio-podinfo\") pod \"e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79\" (UID: \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\") " pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" Aug 04 15:49:30.676666 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:30.676614 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db88d5f4-c855-48ac-9c0e-3dcd3305762a-workload-certs\") pod \"e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79\" (UID: \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\") " pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" Aug 04 15:49:30.676856 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:30.676691 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db88d5f4-c855-48ac-9c0e-3dcd3305762a-istio-token\") pod \"e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79\" (UID: \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\") " pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" Aug 04 15:49:30.676856 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:30.676715 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db88d5f4-c855-48ac-9c0e-3dcd3305762a-istiod-ca-cert\") pod \"e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79\" (UID: \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\") " pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" Aug 04 15:49:30.777162 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:30.777128 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4brl\" (UniqueName: \"kubernetes.io/projected/db88d5f4-c855-48ac-9c0e-3dcd3305762a-kube-api-access-q4brl\") pod \"e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79\" (UID: \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\") " pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" Aug 04 15:49:30.777162 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:30.777167 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db88d5f4-c855-48ac-9c0e-3dcd3305762a-workload-socket\") pod \"e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79\" (UID: \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\") " pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" Aug 04 15:49:30.777369 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:30.777186 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db88d5f4-c855-48ac-9c0e-3dcd3305762a-istio-podinfo\") pod \"e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79\" (UID: \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\") " pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" Aug 04 15:49:30.777369 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:30.777318 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db88d5f4-c855-48ac-9c0e-3dcd3305762a-workload-certs\") pod \"e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79\" (UID: \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\") " pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" Aug 04 15:49:30.777555 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:30.777410 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db88d5f4-c855-48ac-9c0e-3dcd3305762a-istio-token\") pod \"e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79\" (UID: \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\") " pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" Aug 04 15:49:30.777555 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:30.777444 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db88d5f4-c855-48ac-9c0e-3dcd3305762a-istiod-ca-cert\") pod \"e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79\" (UID: \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\") " pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" Aug 04 15:49:30.777555 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:30.777505 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db88d5f4-c855-48ac-9c0e-3dcd3305762a-credential-socket\") pod \"e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79\" (UID: \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\") " pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" Aug 04 15:49:30.777555 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:30.777531 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db88d5f4-c855-48ac-9c0e-3dcd3305762a-istio-data\") pod \"e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79\" (UID: \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\") " pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" Aug 04 15:49:30.777763 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:30.777574 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db88d5f4-c855-48ac-9c0e-3dcd3305762a-istio-envoy\") pod \"e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79\" (UID: \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\") " pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" Aug 04 15:49:30.777763 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:30.777622 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db88d5f4-c855-48ac-9c0e-3dcd3305762a-workload-socket\") pod \"e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79\" (UID: \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\") " pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" Aug 04 15:49:30.777763 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:30.777679 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db88d5f4-c855-48ac-9c0e-3dcd3305762a-workload-certs\") pod \"e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79\" (UID: \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\") " pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" Aug 04 15:49:30.777922 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:30.777897 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db88d5f4-c855-48ac-9c0e-3dcd3305762a-credential-socket\") pod \"e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79\" (UID: \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\") " pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" Aug 04 15:49:30.777996 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:30.777979 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db88d5f4-c855-48ac-9c0e-3dcd3305762a-istio-data\") pod \"e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79\" (UID: \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\") " pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" Aug 04 15:49:30.778093 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:30.778077 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db88d5f4-c855-48ac-9c0e-3dcd3305762a-istiod-ca-cert\") pod \"e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79\" (UID: \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\") " pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" Aug 04 15:49:30.779664 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:30.779643 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db88d5f4-c855-48ac-9c0e-3dcd3305762a-istio-envoy\") pod \"e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79\" (UID: \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\") " pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" Aug 04 15:49:30.779770 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:30.779732 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db88d5f4-c855-48ac-9c0e-3dcd3305762a-istio-podinfo\") pod \"e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79\" (UID: \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\") " pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" Aug 04 15:49:30.786241 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:30.786218 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4brl\" (UniqueName: \"kubernetes.io/projected/db88d5f4-c855-48ac-9c0e-3dcd3305762a-kube-api-access-q4brl\") pod \"e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79\" (UID: \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\") " pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" Aug 04 15:49:30.787672 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:30.787644 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db88d5f4-c855-48ac-9c0e-3dcd3305762a-istio-token\") pod \"e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79\" (UID: \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\") " pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" Aug 04 15:49:30.923002 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:30.922929 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" Aug 04 15:49:31.064368 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:31.064342 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79"] Aug 04 15:49:31.066787 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:49:31.066760 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb88d5f4_c855_48ac_9c0e_3dcd3305762a.slice/crio-cee23b7bdc57eca2031230e9447e93732efe06689663f7cb0756a04f79ea3e5e WatchSource:0}: Error finding container cee23b7bdc57eca2031230e9447e93732efe06689663f7cb0756a04f79ea3e5e: Status 404 returned error can't find the container with id cee23b7bdc57eca2031230e9447e93732efe06689663f7cb0756a04f79ea3e5e Aug 04 15:49:31.069369 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:31.069336 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:49:31.069486 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:31.069470 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:49:31.069528 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:31.069505 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:49:31.909727 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:31.909689 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" event={"ID":"db88d5f4-c855-48ac-9c0e-3dcd3305762a","Type":"ContainerStarted","Data":"a46d501a23b032bede07f4a10488db0ebbed09e5a8b053e6a78f472a32446320"} Aug 04 15:49:31.910113 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:31.909772 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" event={"ID":"db88d5f4-c855-48ac-9c0e-3dcd3305762a","Type":"ContainerStarted","Data":"cee23b7bdc57eca2031230e9447e93732efe06689663f7cb0756a04f79ea3e5e"} Aug 04 15:49:31.923468 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:31.923446 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" Aug 04 15:49:31.935791 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:31.935749 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" podStartSLOduration=1.935735526 podStartE2EDuration="1.935735526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:49:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:49:31.933350119 +0000 UTC m=+2595.393222469" watchObservedRunningTime="2026-08-04 15:49:31.935735526 +0000 UTC m=+2595.395607879" Aug 04 15:49:32.928575 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:32.928543 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" Aug 04 15:49:33.920653 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:33.920621 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" Aug 04 15:49:33.921578 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:33.921558 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" Aug 04 15:49:39.117383 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:39.117345 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:49:48.002894 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:48.002860 2574 generic.go:356] "Generic (PLEG): container finished" podID="db88d5f4-c855-48ac-9c0e-3dcd3305762a" containerID="a46d501a23b032bede07f4a10488db0ebbed09e5a8b053e6a78f472a32446320" exitCode=0 Aug 04 15:49:48.003299 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:48.002920 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" event={"ID":"db88d5f4-c855-48ac-9c0e-3dcd3305762a","Type":"ContainerDied","Data":"a46d501a23b032bede07f4a10488db0ebbed09e5a8b053e6a78f472a32446320"} Aug 04 15:49:48.003342 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:48.003302 2574 scope.go:117] "RemoveContainer" containerID="a46d501a23b032bede07f4a10488db0ebbed09e5a8b053e6a78f472a32446320" Aug 04 15:49:48.004315 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:48.004277 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:49:48.004462 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:48.004344 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:49:48.004462 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:48.004376 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:49:48.920734 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:48.920688 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" Aug 04 15:49:49.017329 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:49.017296 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" event={"ID":"db88d5f4-c855-48ac-9c0e-3dcd3305762a","Type":"ContainerStarted","Data":"ae16dd3f770dcff14c55235661e358d9e36e594b4aa1ca0eefd6b0b9c39cfe0d"} Aug 04 15:49:49.924282 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:49.924197 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" Aug 04 15:49:49.925289 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:49.925256 2574 patch_prober.go:28] interesting pod/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Aug 04 15:49:49.925436 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:49.925331 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" podUID="db88d5f4-c855-48ac-9c0e-3dcd3305762a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Aug 04 15:49:50.923571 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:50.923532 2574 patch_prober.go:28] interesting pod/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Aug 04 15:49:50.924070 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:50.923616 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" podUID="db88d5f4-c855-48ac-9c0e-3dcd3305762a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Aug 04 15:49:51.923312 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:51.923275 2574 patch_prober.go:28] interesting pod/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Aug 04 15:49:51.923530 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:51.923338 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" podUID="db88d5f4-c855-48ac-9c0e-3dcd3305762a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Aug 04 15:49:52.923588 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:52.923553 2574 patch_prober.go:28] interesting pod/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Aug 04 15:49:52.923984 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:52.923605 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" podUID="db88d5f4-c855-48ac-9c0e-3dcd3305762a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Aug 04 15:49:53.927653 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:53.927622 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" Aug 04 15:49:53.928135 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:53.928114 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" Aug 04 15:49:53.928846 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:53.928829 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" Aug 04 15:49:59.608249 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:59.608212 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls"] Aug 04 15:49:59.608934 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:59.608917 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="2424c74e-70bf-4e19-8966-16b2d752ac3f" containerName="istio-proxy" Aug 04 15:49:59.612061 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:59.612043 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" Aug 04 15:49:59.615159 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:59.615141 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-926308-openshift-default-dockercfg-kff84\"" Aug 04 15:49:59.624990 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:59.624960 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls"] Aug 04 15:49:59.746111 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:59.746074 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z56tw\" (UniqueName: \"kubernetes.io/projected/8bd99234-2a29-4e67-b0bc-8e061bb1872c-kube-api-access-z56tw\") pod \"e2e-inf-b-926308-openshift-default-78dc965d48-lbmls\" (UID: \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\") " pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" Aug 04 15:49:59.746111 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:59.746114 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bd99234-2a29-4e67-b0bc-8e061bb1872c-credential-socket\") pod \"e2e-inf-b-926308-openshift-default-78dc965d48-lbmls\" (UID: \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\") " pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" Aug 04 15:49:59.746314 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:59.746144 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bd99234-2a29-4e67-b0bc-8e061bb1872c-istio-token\") pod \"e2e-inf-b-926308-openshift-default-78dc965d48-lbmls\" (UID: \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\") " pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" Aug 04 15:49:59.746314 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:59.746194 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bd99234-2a29-4e67-b0bc-8e061bb1872c-istio-data\") pod \"e2e-inf-b-926308-openshift-default-78dc965d48-lbmls\" (UID: \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\") " pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" Aug 04 15:49:59.746314 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:59.746218 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bd99234-2a29-4e67-b0bc-8e061bb1872c-workload-certs\") pod \"e2e-inf-b-926308-openshift-default-78dc965d48-lbmls\" (UID: \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\") " pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" Aug 04 15:49:59.746314 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:59.746254 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bd99234-2a29-4e67-b0bc-8e061bb1872c-workload-socket\") pod \"e2e-inf-b-926308-openshift-default-78dc965d48-lbmls\" (UID: \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\") " pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" Aug 04 15:49:59.746314 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:59.746271 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bd99234-2a29-4e67-b0bc-8e061bb1872c-istio-podinfo\") pod \"e2e-inf-b-926308-openshift-default-78dc965d48-lbmls\" (UID: \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\") " pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" Aug 04 15:49:59.746543 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:59.746347 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bd99234-2a29-4e67-b0bc-8e061bb1872c-istiod-ca-cert\") pod \"e2e-inf-b-926308-openshift-default-78dc965d48-lbmls\" (UID: \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\") " pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" Aug 04 15:49:59.746543 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:59.746371 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bd99234-2a29-4e67-b0bc-8e061bb1872c-istio-envoy\") pod \"e2e-inf-b-926308-openshift-default-78dc965d48-lbmls\" (UID: \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\") " pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" Aug 04 15:49:59.847266 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:59.847233 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bd99234-2a29-4e67-b0bc-8e061bb1872c-workload-socket\") pod \"e2e-inf-b-926308-openshift-default-78dc965d48-lbmls\" (UID: \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\") " pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" Aug 04 15:49:59.847266 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:59.847269 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bd99234-2a29-4e67-b0bc-8e061bb1872c-istio-podinfo\") pod \"e2e-inf-b-926308-openshift-default-78dc965d48-lbmls\" (UID: \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\") " pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" Aug 04 15:49:59.847526 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:59.847325 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bd99234-2a29-4e67-b0bc-8e061bb1872c-istiod-ca-cert\") pod \"e2e-inf-b-926308-openshift-default-78dc965d48-lbmls\" (UID: \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\") " pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" Aug 04 15:49:59.847526 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:59.847350 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bd99234-2a29-4e67-b0bc-8e061bb1872c-istio-envoy\") pod \"e2e-inf-b-926308-openshift-default-78dc965d48-lbmls\" (UID: \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\") " pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" Aug 04 15:49:59.847526 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:59.847423 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z56tw\" (UniqueName: \"kubernetes.io/projected/8bd99234-2a29-4e67-b0bc-8e061bb1872c-kube-api-access-z56tw\") pod \"e2e-inf-b-926308-openshift-default-78dc965d48-lbmls\" (UID: \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\") " pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" Aug 04 15:49:59.847526 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:59.847456 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bd99234-2a29-4e67-b0bc-8e061bb1872c-credential-socket\") pod \"e2e-inf-b-926308-openshift-default-78dc965d48-lbmls\" (UID: \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\") " pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" Aug 04 15:49:59.847526 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:59.847494 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bd99234-2a29-4e67-b0bc-8e061bb1872c-istio-token\") pod \"e2e-inf-b-926308-openshift-default-78dc965d48-lbmls\" (UID: \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\") " pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" Aug 04 15:49:59.847790 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:59.847527 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bd99234-2a29-4e67-b0bc-8e061bb1872c-istio-data\") pod \"e2e-inf-b-926308-openshift-default-78dc965d48-lbmls\" (UID: \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\") " pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" Aug 04 15:49:59.847790 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:59.847603 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bd99234-2a29-4e67-b0bc-8e061bb1872c-workload-certs\") pod \"e2e-inf-b-926308-openshift-default-78dc965d48-lbmls\" (UID: \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\") " pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" Aug 04 15:49:59.847790 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:59.847727 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bd99234-2a29-4e67-b0bc-8e061bb1872c-workload-socket\") pod \"e2e-inf-b-926308-openshift-default-78dc965d48-lbmls\" (UID: \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\") " pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" Aug 04 15:49:59.847975 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:59.847811 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bd99234-2a29-4e67-b0bc-8e061bb1872c-credential-socket\") pod \"e2e-inf-b-926308-openshift-default-78dc965d48-lbmls\" (UID: \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\") " pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" Aug 04 15:49:59.847975 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:59.847887 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bd99234-2a29-4e67-b0bc-8e061bb1872c-istio-data\") pod \"e2e-inf-b-926308-openshift-default-78dc965d48-lbmls\" (UID: \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\") " pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" Aug 04 15:49:59.847975 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:59.847937 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bd99234-2a29-4e67-b0bc-8e061bb1872c-workload-certs\") pod \"e2e-inf-b-926308-openshift-default-78dc965d48-lbmls\" (UID: \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\") " pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" Aug 04 15:49:59.848131 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:59.848114 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bd99234-2a29-4e67-b0bc-8e061bb1872c-istiod-ca-cert\") pod \"e2e-inf-b-926308-openshift-default-78dc965d48-lbmls\" (UID: \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\") " pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" Aug 04 15:49:59.849914 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:59.849886 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bd99234-2a29-4e67-b0bc-8e061bb1872c-istio-envoy\") pod \"e2e-inf-b-926308-openshift-default-78dc965d48-lbmls\" (UID: \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\") " pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" Aug 04 15:49:59.850038 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:59.850018 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bd99234-2a29-4e67-b0bc-8e061bb1872c-istio-podinfo\") pod \"e2e-inf-b-926308-openshift-default-78dc965d48-lbmls\" (UID: \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\") " pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" Aug 04 15:49:59.856505 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:59.856485 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bd99234-2a29-4e67-b0bc-8e061bb1872c-istio-token\") pod \"e2e-inf-b-926308-openshift-default-78dc965d48-lbmls\" (UID: \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\") " pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" Aug 04 15:49:59.859992 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:59.859939 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z56tw\" (UniqueName: \"kubernetes.io/projected/8bd99234-2a29-4e67-b0bc-8e061bb1872c-kube-api-access-z56tw\") pod \"e2e-inf-b-926308-openshift-default-78dc965d48-lbmls\" (UID: \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\") " pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" Aug 04 15:49:59.923613 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:49:59.923589 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" Aug 04 15:50:00.066134 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:00.066108 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls"] Aug 04 15:50:00.067763 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:50:00.067727 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bd99234_2a29_4e67_b0bc_8e061bb1872c.slice/crio-47d86629b079237ac16afaba6586378667e5a3418978857da7128f2b188abe4b WatchSource:0}: Error finding container 47d86629b079237ac16afaba6586378667e5a3418978857da7128f2b188abe4b: Status 404 returned error can't find the container with id 47d86629b079237ac16afaba6586378667e5a3418978857da7128f2b188abe4b Aug 04 15:50:00.069703 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:00.069664 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:50:00.069825 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:00.069740 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:50:00.069825 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:00.069782 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:50:00.091706 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:00.091681 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" event={"ID":"8bd99234-2a29-4e67-b0bc-8e061bb1872c","Type":"ContainerStarted","Data":"47d86629b079237ac16afaba6586378667e5a3418978857da7128f2b188abe4b"} Aug 04 15:50:01.098698 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:01.098659 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" event={"ID":"8bd99234-2a29-4e67-b0bc-8e061bb1872c","Type":"ContainerStarted","Data":"90e21dcea7384b2be81a8c3268ca1fb31513247e04a53c06574fba680e38a2d4"} Aug 04 15:50:01.120341 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:01.120286 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" podStartSLOduration=2.120270953 podStartE2EDuration="2.120270953s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:49:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:50:01.119686931 +0000 UTC m=+2624.579559293" watchObservedRunningTime="2026-08-04 15:50:01.120270953 +0000 UTC m=+2624.580143302" Aug 04 15:50:01.924809 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:01.924762 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" Aug 04 15:50:01.930071 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:01.930043 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" Aug 04 15:50:02.104768 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:02.104731 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" Aug 04 15:50:02.105932 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:02.105911 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" Aug 04 15:50:10.310217 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:10.310182 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:50:20.202560 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:20.202526 2574 generic.go:356] "Generic (PLEG): container finished" podID="8bd99234-2a29-4e67-b0bc-8e061bb1872c" containerID="90e21dcea7384b2be81a8c3268ca1fb31513247e04a53c06574fba680e38a2d4" exitCode=0 Aug 04 15:50:20.202952 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:20.202600 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" event={"ID":"8bd99234-2a29-4e67-b0bc-8e061bb1872c","Type":"ContainerDied","Data":"90e21dcea7384b2be81a8c3268ca1fb31513247e04a53c06574fba680e38a2d4"} Aug 04 15:50:20.203087 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:20.203070 2574 scope.go:117] "RemoveContainer" containerID="90e21dcea7384b2be81a8c3268ca1fb31513247e04a53c06574fba680e38a2d4" Aug 04 15:50:20.204312 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:20.204278 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:50:20.204409 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:20.204350 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:50:20.204459 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:20.204409 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:50:21.210200 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:21.210164 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" event={"ID":"8bd99234-2a29-4e67-b0bc-8e061bb1872c","Type":"ContainerStarted","Data":"f7301c0d61067a26181f802919516c6414d751660d2f81ed6aacace1b9d5e226"} Aug 04 15:50:21.924217 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:21.924177 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" Aug 04 15:50:21.924466 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:21.924437 2574 patch_prober.go:28] interesting pod/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Aug 04 15:50:21.924559 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:21.924501 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" podUID="8bd99234-2a29-4e67-b0bc-8e061bb1872c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Aug 04 15:50:22.924875 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:22.924830 2574 patch_prober.go:28] interesting pod/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Aug 04 15:50:22.925407 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:22.924937 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" podUID="8bd99234-2a29-4e67-b0bc-8e061bb1872c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Aug 04 15:50:23.924213 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:23.924165 2574 patch_prober.go:28] interesting pod/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Aug 04 15:50:23.924454 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:23.924267 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" podUID="8bd99234-2a29-4e67-b0bc-8e061bb1872c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Aug 04 15:50:24.925085 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:24.925035 2574 patch_prober.go:28] interesting pod/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Aug 04 15:50:24.925708 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:24.925653 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" podUID="8bd99234-2a29-4e67-b0bc-8e061bb1872c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Aug 04 15:50:25.927545 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:25.927507 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" Aug 04 15:50:25.928006 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:25.927759 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" Aug 04 15:50:25.928432 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:25.928416 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" Aug 04 15:50:34.297937 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:34.297904 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-c2a99b/test-model-c2a99b-kserve-84f7bcd65c-58kzn"] Aug 04 15:50:34.301596 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:34.301578 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c2a99b/test-model-c2a99b-kserve-84f7bcd65c-58kzn" Aug 04 15:50:34.304663 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:34.304638 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-c2a99b\"/\"kube-root-ca.crt\"" Aug 04 15:50:34.305647 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:34.305626 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-c2a99b\"/\"openshift-service-ca.crt\"" Aug 04 15:50:34.305754 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:34.305642 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-c2a99b\"/\"default-dockercfg-xn4tb\"" Aug 04 15:50:34.305754 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:34.305698 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-c2a99b\"/\"test-model-c2a99b-kserve-self-signed-certs\"" Aug 04 15:50:34.312169 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:34.312145 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-c2a99b/test-model-c2a99b-kserve-84f7bcd65c-58kzn"] Aug 04 15:50:34.466851 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:34.466817 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca-dshm\") pod \"test-model-c2a99b-kserve-84f7bcd65c-58kzn\" (UID: \"5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca\") " pod="ai-tenant-e2e-inf-a-c2a99b/test-model-c2a99b-kserve-84f7bcd65c-58kzn" Aug 04 15:50:34.466851 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:34.466852 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca-tls-certs\") pod \"test-model-c2a99b-kserve-84f7bcd65c-58kzn\" (UID: \"5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca\") " pod="ai-tenant-e2e-inf-a-c2a99b/test-model-c2a99b-kserve-84f7bcd65c-58kzn" Aug 04 15:50:34.467041 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:34.466869 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca-home\") pod \"test-model-c2a99b-kserve-84f7bcd65c-58kzn\" (UID: \"5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca\") " pod="ai-tenant-e2e-inf-a-c2a99b/test-model-c2a99b-kserve-84f7bcd65c-58kzn" Aug 04 15:50:34.467041 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:34.466889 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca-tmp-dir\") pod \"test-model-c2a99b-kserve-84f7bcd65c-58kzn\" (UID: \"5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca\") " pod="ai-tenant-e2e-inf-a-c2a99b/test-model-c2a99b-kserve-84f7bcd65c-58kzn" Aug 04 15:50:34.467041 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:34.467016 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca-model-cache\") pod \"test-model-c2a99b-kserve-84f7bcd65c-58kzn\" (UID: \"5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca\") " pod="ai-tenant-e2e-inf-a-c2a99b/test-model-c2a99b-kserve-84f7bcd65c-58kzn" Aug 04 15:50:34.467144 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:34.467051 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6msdg\" (UniqueName: \"kubernetes.io/projected/5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca-kube-api-access-6msdg\") pod \"test-model-c2a99b-kserve-84f7bcd65c-58kzn\" (UID: \"5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca\") " pod="ai-tenant-e2e-inf-a-c2a99b/test-model-c2a99b-kserve-84f7bcd65c-58kzn" Aug 04 15:50:34.568280 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:34.568194 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca-model-cache\") pod \"test-model-c2a99b-kserve-84f7bcd65c-58kzn\" (UID: \"5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca\") " pod="ai-tenant-e2e-inf-a-c2a99b/test-model-c2a99b-kserve-84f7bcd65c-58kzn" Aug 04 15:50:34.568280 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:34.568239 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6msdg\" (UniqueName: \"kubernetes.io/projected/5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca-kube-api-access-6msdg\") pod \"test-model-c2a99b-kserve-84f7bcd65c-58kzn\" (UID: \"5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca\") " pod="ai-tenant-e2e-inf-a-c2a99b/test-model-c2a99b-kserve-84f7bcd65c-58kzn" Aug 04 15:50:34.568525 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:34.568295 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca-dshm\") pod \"test-model-c2a99b-kserve-84f7bcd65c-58kzn\" (UID: \"5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca\") " pod="ai-tenant-e2e-inf-a-c2a99b/test-model-c2a99b-kserve-84f7bcd65c-58kzn" Aug 04 15:50:34.568525 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:34.568319 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca-tls-certs\") pod \"test-model-c2a99b-kserve-84f7bcd65c-58kzn\" (UID: \"5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca\") " pod="ai-tenant-e2e-inf-a-c2a99b/test-model-c2a99b-kserve-84f7bcd65c-58kzn" Aug 04 15:50:34.568525 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:34.568341 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca-home\") pod \"test-model-c2a99b-kserve-84f7bcd65c-58kzn\" (UID: \"5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca\") " pod="ai-tenant-e2e-inf-a-c2a99b/test-model-c2a99b-kserve-84f7bcd65c-58kzn" Aug 04 15:50:34.568525 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:34.568370 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca-tmp-dir\") pod \"test-model-c2a99b-kserve-84f7bcd65c-58kzn\" (UID: \"5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca\") " pod="ai-tenant-e2e-inf-a-c2a99b/test-model-c2a99b-kserve-84f7bcd65c-58kzn" Aug 04 15:50:34.568727 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:34.568655 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca-model-cache\") pod \"test-model-c2a99b-kserve-84f7bcd65c-58kzn\" (UID: \"5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca\") " pod="ai-tenant-e2e-inf-a-c2a99b/test-model-c2a99b-kserve-84f7bcd65c-58kzn" Aug 04 15:50:34.568727 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:34.568666 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca-home\") pod \"test-model-c2a99b-kserve-84f7bcd65c-58kzn\" (UID: \"5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca\") " pod="ai-tenant-e2e-inf-a-c2a99b/test-model-c2a99b-kserve-84f7bcd65c-58kzn" Aug 04 15:50:34.568827 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:34.568784 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca-tmp-dir\") pod \"test-model-c2a99b-kserve-84f7bcd65c-58kzn\" (UID: \"5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca\") " pod="ai-tenant-e2e-inf-a-c2a99b/test-model-c2a99b-kserve-84f7bcd65c-58kzn" Aug 04 15:50:34.570734 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:34.570707 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca-dshm\") pod \"test-model-c2a99b-kserve-84f7bcd65c-58kzn\" (UID: \"5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca\") " pod="ai-tenant-e2e-inf-a-c2a99b/test-model-c2a99b-kserve-84f7bcd65c-58kzn" Aug 04 15:50:34.570832 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:34.570767 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca-tls-certs\") pod \"test-model-c2a99b-kserve-84f7bcd65c-58kzn\" (UID: \"5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca\") " pod="ai-tenant-e2e-inf-a-c2a99b/test-model-c2a99b-kserve-84f7bcd65c-58kzn" Aug 04 15:50:34.577092 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:34.577065 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6msdg\" (UniqueName: \"kubernetes.io/projected/5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca-kube-api-access-6msdg\") pod \"test-model-c2a99b-kserve-84f7bcd65c-58kzn\" (UID: \"5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca\") " pod="ai-tenant-e2e-inf-a-c2a99b/test-model-c2a99b-kserve-84f7bcd65c-58kzn" Aug 04 15:50:34.611727 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:34.611687 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c2a99b/test-model-c2a99b-kserve-84f7bcd65c-58kzn" Aug 04 15:50:34.751724 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:34.751686 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-c2a99b/test-model-c2a99b-kserve-84f7bcd65c-58kzn"] Aug 04 15:50:34.752713 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:50:34.752689 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5cdde67b_d913_4f12_a04c_e5e2c7bbc9ca.slice/crio-f8dcc916e6d44862f291e4985927a52c29a500cc4ad83b270cfc17d452298f4b WatchSource:0}: Error finding container f8dcc916e6d44862f291e4985927a52c29a500cc4ad83b270cfc17d452298f4b: Status 404 returned error can't find the container with id f8dcc916e6d44862f291e4985927a52c29a500cc4ad83b270cfc17d452298f4b Aug 04 15:50:35.301286 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:35.301249 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c2a99b/test-model-c2a99b-kserve-84f7bcd65c-58kzn" event={"ID":"5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca","Type":"ContainerStarted","Data":"af36277909c781eb17c38d3c82d94ee54c218b5c4b47f411735fa8f8760e5994"} Aug 04 15:50:35.301733 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:35.301297 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c2a99b/test-model-c2a99b-kserve-84f7bcd65c-58kzn" event={"ID":"5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca","Type":"ContainerStarted","Data":"f8dcc916e6d44862f291e4985927a52c29a500cc4ad83b270cfc17d452298f4b"} Aug 04 15:50:35.321261 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:35.321213 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-c2a99b/test-model-c2a99b-kserve-84f7bcd65c-58kzn" podStartSLOduration=1.321200446 podStartE2EDuration="1.321200446s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:50:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:50:35.319051733 +0000 UTC m=+2658.778924083" watchObservedRunningTime="2026-08-04 15:50:35.321200446 +0000 UTC m=+2658.781072801" Aug 04 15:50:37.135185 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:37.135154 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:50:44.612057 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:44.612023 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-c2a99b/test-model-c2a99b-kserve-84f7bcd65c-58kzn" Aug 04 15:50:44.612611 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:44.612276 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-c2a99b/test-model-c2a99b-kserve-84f7bcd65c-58kzn" Aug 04 15:50:44.625800 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:44.625775 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-c2a99b/test-model-c2a99b-kserve-84f7bcd65c-58kzn" Aug 04 15:50:45.375221 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:45.375191 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-c2a99b/test-model-c2a99b-kserve-84f7bcd65c-58kzn" Aug 04 15:50:49.217469 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:50:49.217433 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:51:11.117559 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:51:11.117477 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:51:17.260797 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:51:17.260764 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-r8wkp_0580fdab-0381-4d8e-be5f-bde5eb44e0e6/console-operator/1.log" Aug 04 15:51:17.264838 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:51:17.264814 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8h6k_9c62e3b1-c2dc-4b2a-a61d-7d80049f355f/ovn-acl-logging/0.log" Aug 04 15:51:17.268445 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:51:17.268423 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-r8wkp_0580fdab-0381-4d8e-be5f-bde5eb44e0e6/console-operator/1.log" Aug 04 15:51:17.272339 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:51:17.272321 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8h6k_9c62e3b1-c2dc-4b2a-a61d-7d80049f355f/ovn-acl-logging/0.log" Aug 04 15:51:27.919785 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:51:27.919743 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:51:48.273321 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:51:48.273289 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-c2a99b/test-model-c2a99b-kserve-84f7bcd65c-58kzn"] Aug 04 15:51:52.615010 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:51:52.614977 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:52:03.292494 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:03.292448 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-c2a99b/test-model-c2a99b-kserve-84f7bcd65c-58kzn" podUID="5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca" containerName="main" containerID="cri-o://af36277909c781eb17c38d3c82d94ee54c218b5c4b47f411735fa8f8760e5994" gracePeriod=45 Aug 04 15:52:03.550009 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:03.549947 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c2a99b/test-model-c2a99b-kserve-84f7bcd65c-58kzn" Aug 04 15:52:03.691028 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:03.690995 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca-dshm\") pod \"5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca\" (UID: \"5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca\") " Aug 04 15:52:03.691191 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:03.691046 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6msdg\" (UniqueName: \"kubernetes.io/projected/5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca-kube-api-access-6msdg\") pod \"5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca\" (UID: \"5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca\") " Aug 04 15:52:03.691191 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:03.691076 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca-model-cache\") pod \"5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca\" (UID: \"5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca\") " Aug 04 15:52:03.691191 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:03.691100 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca-tls-certs\") pod \"5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca\" (UID: \"5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca\") " Aug 04 15:52:03.691191 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:03.691120 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca-tmp-dir\") pod \"5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca\" (UID: \"5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca\") " Aug 04 15:52:03.691191 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:03.691179 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca-home\") pod \"5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca\" (UID: \"5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca\") " Aug 04 15:52:03.691487 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:03.691373 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca-model-cache" (OuterVolumeSpecName: "model-cache") pod "5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca" (UID: "5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:52:03.691487 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:03.691426 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca" (UID: "5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:52:03.691598 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:03.691542 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca-home" (OuterVolumeSpecName: "home") pod "5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca" (UID: "5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:52:03.693474 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:03.693443 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca" (UID: "5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 15:52:03.693474 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:03.693453 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca-dshm" (OuterVolumeSpecName: "dshm") pod "5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca" (UID: "5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:52:03.693610 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:03.693501 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca-kube-api-access-6msdg" (OuterVolumeSpecName: "kube-api-access-6msdg") pod "5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca" (UID: "5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca"). InnerVolumeSpecName "kube-api-access-6msdg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 15:52:03.795366 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:03.795236 2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca-home\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:52:03.795366 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:03.795363 2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca-dshm\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:52:03.795593 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:03.795424 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6msdg\" (UniqueName: \"kubernetes.io/projected/5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca-kube-api-access-6msdg\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:52:03.795593 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:03.795441 2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca-model-cache\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:52:03.795593 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:03.795455 2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca-tls-certs\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:52:03.795593 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:03.795466 2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca-tmp-dir\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:52:03.799348 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:03.799321 2574 generic.go:356] "Generic (PLEG): container finished" podID="5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca" containerID="af36277909c781eb17c38d3c82d94ee54c218b5c4b47f411735fa8f8760e5994" exitCode=0 Aug 04 15:52:03.799478 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:03.799377 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-c2a99b/test-model-c2a99b-kserve-84f7bcd65c-58kzn" Aug 04 15:52:03.799478 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:03.799415 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c2a99b/test-model-c2a99b-kserve-84f7bcd65c-58kzn" event={"ID":"5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca","Type":"ContainerDied","Data":"af36277909c781eb17c38d3c82d94ee54c218b5c4b47f411735fa8f8760e5994"} Aug 04 15:52:03.799478 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:03.799463 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-c2a99b/test-model-c2a99b-kserve-84f7bcd65c-58kzn" event={"ID":"5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca","Type":"ContainerDied","Data":"f8dcc916e6d44862f291e4985927a52c29a500cc4ad83b270cfc17d452298f4b"} Aug 04 15:52:03.799594 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:03.799480 2574 scope.go:117] "RemoveContainer" containerID="af36277909c781eb17c38d3c82d94ee54c218b5c4b47f411735fa8f8760e5994" Aug 04 15:52:03.811614 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:03.811597 2574 scope.go:117] "RemoveContainer" containerID="af36277909c781eb17c38d3c82d94ee54c218b5c4b47f411735fa8f8760e5994" Aug 04 15:52:03.811858 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:52:03.811839 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af36277909c781eb17c38d3c82d94ee54c218b5c4b47f411735fa8f8760e5994\": container with ID starting with af36277909c781eb17c38d3c82d94ee54c218b5c4b47f411735fa8f8760e5994 not found: ID does not exist" containerID="af36277909c781eb17c38d3c82d94ee54c218b5c4b47f411735fa8f8760e5994" Aug 04 15:52:03.811901 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:03.811865 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af36277909c781eb17c38d3c82d94ee54c218b5c4b47f411735fa8f8760e5994"} err="failed to get container status \"af36277909c781eb17c38d3c82d94ee54c218b5c4b47f411735fa8f8760e5994\": rpc error: code = NotFound desc = could not find container \"af36277909c781eb17c38d3c82d94ee54c218b5c4b47f411735fa8f8760e5994\": container with ID starting with af36277909c781eb17c38d3c82d94ee54c218b5c4b47f411735fa8f8760e5994 not found: ID does not exist" Aug 04 15:52:03.825668 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:03.825645 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-c2a99b/test-model-c2a99b-kserve-84f7bcd65c-58kzn"] Aug 04 15:52:03.831016 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:03.830996 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-c2a99b/test-model-c2a99b-kserve-84f7bcd65c-58kzn"] Aug 04 15:52:05.053916 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:05.053880 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca" path="/var/lib/kubelet/pods/5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca/volumes" Aug 04 15:52:05.605817 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:05.605777 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:52:07.014710 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:07.014676 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79"] Aug 04 15:52:07.015146 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:07.014899 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" podUID="db88d5f4-c855-48ac-9c0e-3dcd3305762a" containerName="istio-proxy" containerID="cri-o://ae16dd3f770dcff14c55235661e358d9e36e594b4aa1ca0eefd6b0b9c39cfe0d" gracePeriod=30 Aug 04 15:52:08.930026 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:08.929992 2574 patch_prober.go:28] interesting pod/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 04 15:52:08.930422 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:08.930060 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" podUID="db88d5f4-c855-48ac-9c0e-3dcd3305762a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 15:52:12.280309 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:12.280252 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" Aug 04 15:52:12.469525 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:12.469479 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db88d5f4-c855-48ac-9c0e-3dcd3305762a-istio-envoy\") pod \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\" (UID: \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\") " Aug 04 15:52:12.469525 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:12.469536 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db88d5f4-c855-48ac-9c0e-3dcd3305762a-workload-socket\") pod \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\" (UID: \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\") " Aug 04 15:52:12.469778 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:12.469570 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db88d5f4-c855-48ac-9c0e-3dcd3305762a-istio-podinfo\") pod \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\" (UID: \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\") " Aug 04 15:52:12.469778 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:12.469591 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db88d5f4-c855-48ac-9c0e-3dcd3305762a-istio-data\") pod \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\" (UID: \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\") " Aug 04 15:52:12.469778 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:12.469621 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db88d5f4-c855-48ac-9c0e-3dcd3305762a-credential-socket\") pod \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\" (UID: \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\") " Aug 04 15:52:12.469778 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:12.469679 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db88d5f4-c855-48ac-9c0e-3dcd3305762a-istiod-ca-cert\") pod \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\" (UID: \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\") " Aug 04 15:52:12.469778 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:12.469728 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q4brl\" (UniqueName: \"kubernetes.io/projected/db88d5f4-c855-48ac-9c0e-3dcd3305762a-kube-api-access-q4brl\") pod \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\" (UID: \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\") " Aug 04 15:52:12.470030 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:12.469781 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db88d5f4-c855-48ac-9c0e-3dcd3305762a-istio-token\") pod \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\" (UID: \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\") " Aug 04 15:52:12.470030 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:12.469823 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db88d5f4-c855-48ac-9c0e-3dcd3305762a-workload-certs\") pod \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\" (UID: \"db88d5f4-c855-48ac-9c0e-3dcd3305762a\") " Aug 04 15:52:12.470030 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:12.469855 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db88d5f4-c855-48ac-9c0e-3dcd3305762a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "db88d5f4-c855-48ac-9c0e-3dcd3305762a" (UID: "db88d5f4-c855-48ac-9c0e-3dcd3305762a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:52:12.470030 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:12.469979 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db88d5f4-c855-48ac-9c0e-3dcd3305762a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "db88d5f4-c855-48ac-9c0e-3dcd3305762a" (UID: "db88d5f4-c855-48ac-9c0e-3dcd3305762a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:52:12.470244 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:12.470117 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db88d5f4-c855-48ac-9c0e-3dcd3305762a-workload-socket\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:52:12.470244 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:12.470137 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db88d5f4-c855-48ac-9c0e-3dcd3305762a-credential-socket\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:52:12.470244 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:12.470199 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db88d5f4-c855-48ac-9c0e-3dcd3305762a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "db88d5f4-c855-48ac-9c0e-3dcd3305762a" (UID: "db88d5f4-c855-48ac-9c0e-3dcd3305762a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:52:12.470428 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:12.470351 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db88d5f4-c855-48ac-9c0e-3dcd3305762a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "db88d5f4-c855-48ac-9c0e-3dcd3305762a" (UID: "db88d5f4-c855-48ac-9c0e-3dcd3305762a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 15:52:12.470865 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:12.470845 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db88d5f4-c855-48ac-9c0e-3dcd3305762a-istio-data" (OuterVolumeSpecName: "istio-data") pod "db88d5f4-c855-48ac-9c0e-3dcd3305762a" (UID: "db88d5f4-c855-48ac-9c0e-3dcd3305762a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:52:12.471804 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:12.471776 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/db88d5f4-c855-48ac-9c0e-3dcd3305762a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "db88d5f4-c855-48ac-9c0e-3dcd3305762a" (UID: "db88d5f4-c855-48ac-9c0e-3dcd3305762a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 15:52:12.472038 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:12.472017 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db88d5f4-c855-48ac-9c0e-3dcd3305762a-kube-api-access-q4brl" (OuterVolumeSpecName: "kube-api-access-q4brl") pod "db88d5f4-c855-48ac-9c0e-3dcd3305762a" (UID: "db88d5f4-c855-48ac-9c0e-3dcd3305762a"). InnerVolumeSpecName "kube-api-access-q4brl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 15:52:12.472193 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:12.472177 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db88d5f4-c855-48ac-9c0e-3dcd3305762a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "db88d5f4-c855-48ac-9c0e-3dcd3305762a" (UID: "db88d5f4-c855-48ac-9c0e-3dcd3305762a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:52:12.472272 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:12.472211 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db88d5f4-c855-48ac-9c0e-3dcd3305762a-istio-token" (OuterVolumeSpecName: "istio-token") pod "db88d5f4-c855-48ac-9c0e-3dcd3305762a" (UID: "db88d5f4-c855-48ac-9c0e-3dcd3305762a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 15:52:12.570632 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:12.570562 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db88d5f4-c855-48ac-9c0e-3dcd3305762a-workload-certs\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:52:12.570632 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:12.570584 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db88d5f4-c855-48ac-9c0e-3dcd3305762a-istio-envoy\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:52:12.570632 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:12.570594 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db88d5f4-c855-48ac-9c0e-3dcd3305762a-istio-podinfo\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:52:12.570632 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:12.570602 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db88d5f4-c855-48ac-9c0e-3dcd3305762a-istio-data\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:52:12.570632 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:12.570610 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db88d5f4-c855-48ac-9c0e-3dcd3305762a-istiod-ca-cert\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:52:12.570632 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:12.570618 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q4brl\" (UniqueName: \"kubernetes.io/projected/db88d5f4-c855-48ac-9c0e-3dcd3305762a-kube-api-access-q4brl\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:52:12.570632 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:12.570627 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db88d5f4-c855-48ac-9c0e-3dcd3305762a-istio-token\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:52:12.852614 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:12.852525 2574 generic.go:356] "Generic (PLEG): container finished" podID="db88d5f4-c855-48ac-9c0e-3dcd3305762a" containerID="ae16dd3f770dcff14c55235661e358d9e36e594b4aa1ca0eefd6b0b9c39cfe0d" exitCode=0 Aug 04 15:52:12.852614 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:12.852593 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" Aug 04 15:52:12.852614 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:12.852593 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" event={"ID":"db88d5f4-c855-48ac-9c0e-3dcd3305762a","Type":"ContainerDied","Data":"ae16dd3f770dcff14c55235661e358d9e36e594b4aa1ca0eefd6b0b9c39cfe0d"} Aug 04 15:52:12.852847 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:12.852650 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79" event={"ID":"db88d5f4-c855-48ac-9c0e-3dcd3305762a","Type":"ContainerDied","Data":"cee23b7bdc57eca2031230e9447e93732efe06689663f7cb0756a04f79ea3e5e"} Aug 04 15:52:12.852847 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:12.852673 2574 scope.go:117] "RemoveContainer" containerID="ae16dd3f770dcff14c55235661e358d9e36e594b4aa1ca0eefd6b0b9c39cfe0d" Aug 04 15:52:12.862522 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:12.862501 2574 scope.go:117] "RemoveContainer" containerID="a46d501a23b032bede07f4a10488db0ebbed09e5a8b053e6a78f472a32446320" Aug 04 15:52:12.870608 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:12.870584 2574 scope.go:117] "RemoveContainer" containerID="ae16dd3f770dcff14c55235661e358d9e36e594b4aa1ca0eefd6b0b9c39cfe0d" Aug 04 15:52:12.870859 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:52:12.870836 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae16dd3f770dcff14c55235661e358d9e36e594b4aa1ca0eefd6b0b9c39cfe0d\": container with ID starting with ae16dd3f770dcff14c55235661e358d9e36e594b4aa1ca0eefd6b0b9c39cfe0d not found: ID does not exist" containerID="ae16dd3f770dcff14c55235661e358d9e36e594b4aa1ca0eefd6b0b9c39cfe0d" Aug 04 15:52:12.870906 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:12.870867 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae16dd3f770dcff14c55235661e358d9e36e594b4aa1ca0eefd6b0b9c39cfe0d"} err="failed to get container status \"ae16dd3f770dcff14c55235661e358d9e36e594b4aa1ca0eefd6b0b9c39cfe0d\": rpc error: code = NotFound desc = could not find container \"ae16dd3f770dcff14c55235661e358d9e36e594b4aa1ca0eefd6b0b9c39cfe0d\": container with ID starting with ae16dd3f770dcff14c55235661e358d9e36e594b4aa1ca0eefd6b0b9c39cfe0d not found: ID does not exist" Aug 04 15:52:12.870906 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:12.870885 2574 scope.go:117] "RemoveContainer" containerID="a46d501a23b032bede07f4a10488db0ebbed09e5a8b053e6a78f472a32446320" Aug 04 15:52:12.871082 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:52:12.871068 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a46d501a23b032bede07f4a10488db0ebbed09e5a8b053e6a78f472a32446320\": container with ID starting with a46d501a23b032bede07f4a10488db0ebbed09e5a8b053e6a78f472a32446320 not found: ID does not exist" containerID="a46d501a23b032bede07f4a10488db0ebbed09e5a8b053e6a78f472a32446320" Aug 04 15:52:12.871133 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:12.871085 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a46d501a23b032bede07f4a10488db0ebbed09e5a8b053e6a78f472a32446320"} err="failed to get container status \"a46d501a23b032bede07f4a10488db0ebbed09e5a8b053e6a78f472a32446320\": rpc error: code = NotFound desc = could not find container \"a46d501a23b032bede07f4a10488db0ebbed09e5a8b053e6a78f472a32446320\": container with ID starting with a46d501a23b032bede07f4a10488db0ebbed09e5a8b053e6a78f472a32446320 not found: ID does not exist" Aug 04 15:52:12.877283 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:12.877263 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79"] Aug 04 15:52:12.882263 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:12.882243 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-c2a99b-openshift-default-6dd798df94-j4z79"] Aug 04 15:52:13.053701 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:13.053667 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db88d5f4-c855-48ac-9c0e-3dcd3305762a" path="/var/lib/kubelet/pods/db88d5f4-c855-48ac-9c0e-3dcd3305762a/volumes" Aug 04 15:52:20.912866 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:20.912835 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:52:25.176221 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:25.176178 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls"] Aug 04 15:52:25.176818 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:25.176408 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" podUID="8bd99234-2a29-4e67-b0bc-8e061bb1872c" containerName="istio-proxy" containerID="cri-o://f7301c0d61067a26181f802919516c6414d751660d2f81ed6aacace1b9d5e226" gracePeriod=30 Aug 04 15:52:25.929970 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:25.929937 2574 patch_prober.go:28] interesting pod/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 04 15:52:25.930135 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:25.929997 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" podUID="8bd99234-2a29-4e67-b0bc-8e061bb1872c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 15:52:26.110302 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.110269 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh"] Aug 04 15:52:26.110738 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.110719 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db88d5f4-c855-48ac-9c0e-3dcd3305762a" containerName="istio-proxy" Aug 04 15:52:26.110831 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.110739 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="db88d5f4-c855-48ac-9c0e-3dcd3305762a" containerName="istio-proxy" Aug 04 15:52:26.110831 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.110771 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db88d5f4-c855-48ac-9c0e-3dcd3305762a" containerName="istio-proxy" Aug 04 15:52:26.110831 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.110780 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="db88d5f4-c855-48ac-9c0e-3dcd3305762a" containerName="istio-proxy" Aug 04 15:52:26.110831 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.110798 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca" containerName="main" Aug 04 15:52:26.110831 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.110806 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca" containerName="main" Aug 04 15:52:26.111084 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.110932 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="db88d5f4-c855-48ac-9c0e-3dcd3305762a" containerName="istio-proxy" Aug 04 15:52:26.111084 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.110947 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="db88d5f4-c855-48ac-9c0e-3dcd3305762a" containerName="istio-proxy" Aug 04 15:52:26.111084 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.110962 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cdde67b-d913-4f12-a04c-e5e2c7bbc9ca" containerName="main" Aug 04 15:52:26.114006 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.113985 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" Aug 04 15:52:26.117142 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.117122 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-bac88d-openshift-default-dockercfg-56p44\"" Aug 04 15:52:26.126083 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.126060 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh"] Aug 04 15:52:26.195938 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.195870 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56ceb5c8-199f-41d9-9fa1-3501db74a86b-istio-podinfo\") pod \"e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh\" (UID: \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\") " pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" Aug 04 15:52:26.195938 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.195905 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56ceb5c8-199f-41d9-9fa1-3501db74a86b-workload-certs\") pod \"e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh\" (UID: \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\") " pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" Aug 04 15:52:26.195938 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.195925 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56ceb5c8-199f-41d9-9fa1-3501db74a86b-istiod-ca-cert\") pod \"e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh\" (UID: \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\") " pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" Aug 04 15:52:26.196429 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.195948 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56ceb5c8-199f-41d9-9fa1-3501db74a86b-istio-envoy\") pod \"e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh\" (UID: \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\") " pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" Aug 04 15:52:26.196429 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.196012 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56ceb5c8-199f-41d9-9fa1-3501db74a86b-credential-socket\") pod \"e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh\" (UID: \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\") " pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" Aug 04 15:52:26.196429 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.196050 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56ceb5c8-199f-41d9-9fa1-3501db74a86b-workload-socket\") pod \"e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh\" (UID: \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\") " pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" Aug 04 15:52:26.196429 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.196077 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56ceb5c8-199f-41d9-9fa1-3501db74a86b-istio-token\") pod \"e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh\" (UID: \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\") " pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" Aug 04 15:52:26.196429 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.196160 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xwth\" (UniqueName: \"kubernetes.io/projected/56ceb5c8-199f-41d9-9fa1-3501db74a86b-kube-api-access-5xwth\") pod \"e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh\" (UID: \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\") " pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" Aug 04 15:52:26.196429 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.196203 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56ceb5c8-199f-41d9-9fa1-3501db74a86b-istio-data\") pod \"e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh\" (UID: \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\") " pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" Aug 04 15:52:26.296745 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.296711 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56ceb5c8-199f-41d9-9fa1-3501db74a86b-istio-podinfo\") pod \"e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh\" (UID: \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\") " pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" Aug 04 15:52:26.296918 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.296759 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56ceb5c8-199f-41d9-9fa1-3501db74a86b-workload-certs\") pod \"e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh\" (UID: \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\") " pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" Aug 04 15:52:26.296918 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.296872 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56ceb5c8-199f-41d9-9fa1-3501db74a86b-istiod-ca-cert\") pod \"e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh\" (UID: \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\") " pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" Aug 04 15:52:26.297000 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.296921 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56ceb5c8-199f-41d9-9fa1-3501db74a86b-istio-envoy\") pod \"e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh\" (UID: \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\") " pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" Aug 04 15:52:26.297000 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.296976 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56ceb5c8-199f-41d9-9fa1-3501db74a86b-credential-socket\") pod \"e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh\" (UID: \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\") " pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" Aug 04 15:52:26.297083 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.297011 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56ceb5c8-199f-41d9-9fa1-3501db74a86b-workload-socket\") pod \"e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh\" (UID: \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\") " pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" Aug 04 15:52:26.297083 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.297048 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56ceb5c8-199f-41d9-9fa1-3501db74a86b-istio-token\") pod \"e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh\" (UID: \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\") " pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" Aug 04 15:52:26.297172 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.297090 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xwth\" (UniqueName: \"kubernetes.io/projected/56ceb5c8-199f-41d9-9fa1-3501db74a86b-kube-api-access-5xwth\") pod \"e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh\" (UID: \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\") " pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" Aug 04 15:52:26.297172 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.297118 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56ceb5c8-199f-41d9-9fa1-3501db74a86b-istio-data\") pod \"e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh\" (UID: \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\") " pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" Aug 04 15:52:26.297172 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.297138 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56ceb5c8-199f-41d9-9fa1-3501db74a86b-workload-certs\") pod \"e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh\" (UID: \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\") " pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" Aug 04 15:52:26.297346 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.297320 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56ceb5c8-199f-41d9-9fa1-3501db74a86b-credential-socket\") pod \"e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh\" (UID: \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\") " pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" Aug 04 15:52:26.297571 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.297548 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56ceb5c8-199f-41d9-9fa1-3501db74a86b-istio-data\") pod \"e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh\" (UID: \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\") " pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" Aug 04 15:52:26.297690 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.297545 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56ceb5c8-199f-41d9-9fa1-3501db74a86b-workload-socket\") pod \"e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh\" (UID: \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\") " pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" Aug 04 15:52:26.297690 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.297576 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56ceb5c8-199f-41d9-9fa1-3501db74a86b-istiod-ca-cert\") pod \"e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh\" (UID: \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\") " pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" Aug 04 15:52:26.299708 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.299682 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56ceb5c8-199f-41d9-9fa1-3501db74a86b-istio-envoy\") pod \"e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh\" (UID: \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\") " pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" Aug 04 15:52:26.299812 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.299797 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56ceb5c8-199f-41d9-9fa1-3501db74a86b-istio-podinfo\") pod \"e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh\" (UID: \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\") " pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" Aug 04 15:52:26.305482 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.305462 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xwth\" (UniqueName: \"kubernetes.io/projected/56ceb5c8-199f-41d9-9fa1-3501db74a86b-kube-api-access-5xwth\") pod \"e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh\" (UID: \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\") " pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" Aug 04 15:52:26.306068 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.306048 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56ceb5c8-199f-41d9-9fa1-3501db74a86b-istio-token\") pod \"e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh\" (UID: \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\") " pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" Aug 04 15:52:26.424231 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.424198 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" Aug 04 15:52:26.571885 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.571851 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh"] Aug 04 15:52:26.582890 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.582843 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:52:26.582992 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.582938 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:52:26.583037 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.583010 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:52:26.934140 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.934101 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" event={"ID":"56ceb5c8-199f-41d9-9fa1-3501db74a86b","Type":"ContainerStarted","Data":"2687ef81b1d608de5abba3928a16f2df7bd1ec536936e0962914b3163b09c7b4"} Aug 04 15:52:26.934343 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.934157 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" event={"ID":"56ceb5c8-199f-41d9-9fa1-3501db74a86b","Type":"ContainerStarted","Data":"7c0208aa3deda7bdd4b696c602452a0c6e93334c6fba21d7cfb3a3dcc83e0fed"} Aug 04 15:52:26.985423 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:26.985354 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" podStartSLOduration=0.98533873 podStartE2EDuration="985.33873ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:52:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:52:26.981120799 +0000 UTC m=+2770.440993148" watchObservedRunningTime="2026-08-04 15:52:26.98533873 +0000 UTC m=+2770.445211151" Aug 04 15:52:27.424654 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:27.424615 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" Aug 04 15:52:27.429866 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:27.429842 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" Aug 04 15:52:27.939950 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:27.939920 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" Aug 04 15:52:27.940781 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:27.940763 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" Aug 04 15:52:30.445380 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:30.445311 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" Aug 04 15:52:30.539999 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:30.539960 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bd99234-2a29-4e67-b0bc-8e061bb1872c-istio-token\") pod \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\" (UID: \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\") " Aug 04 15:52:30.540196 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:30.540072 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bd99234-2a29-4e67-b0bc-8e061bb1872c-istio-data\") pod \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\" (UID: \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\") " Aug 04 15:52:30.540196 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:30.540094 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bd99234-2a29-4e67-b0bc-8e061bb1872c-istio-envoy\") pod \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\" (UID: \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\") " Aug 04 15:52:30.540196 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:30.540130 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bd99234-2a29-4e67-b0bc-8e061bb1872c-workload-socket\") pod \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\" (UID: \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\") " Aug 04 15:52:30.540196 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:30.540180 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bd99234-2a29-4e67-b0bc-8e061bb1872c-istiod-ca-cert\") pod \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\" (UID: \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\") " Aug 04 15:52:30.540448 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:30.540208 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bd99234-2a29-4e67-b0bc-8e061bb1872c-credential-socket\") pod \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\" (UID: \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\") " Aug 04 15:52:30.540448 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:30.540249 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bd99234-2a29-4e67-b0bc-8e061bb1872c-workload-certs\") pod \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\" (UID: \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\") " Aug 04 15:52:30.540448 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:30.540276 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z56tw\" (UniqueName: \"kubernetes.io/projected/8bd99234-2a29-4e67-b0bc-8e061bb1872c-kube-api-access-z56tw\") pod \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\" (UID: \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\") " Aug 04 15:52:30.540448 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:30.540303 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bd99234-2a29-4e67-b0bc-8e061bb1872c-istio-podinfo\") pod \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\" (UID: \"8bd99234-2a29-4e67-b0bc-8e061bb1872c\") " Aug 04 15:52:30.541417 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:30.541255 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bd99234-2a29-4e67-b0bc-8e061bb1872c-istio-data" (OuterVolumeSpecName: "istio-data") pod "8bd99234-2a29-4e67-b0bc-8e061bb1872c" (UID: "8bd99234-2a29-4e67-b0bc-8e061bb1872c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:52:30.541553 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:30.541469 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bd99234-2a29-4e67-b0bc-8e061bb1872c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8bd99234-2a29-4e67-b0bc-8e061bb1872c" (UID: "8bd99234-2a29-4e67-b0bc-8e061bb1872c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:52:30.541627 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:30.541559 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bd99234-2a29-4e67-b0bc-8e061bb1872c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8bd99234-2a29-4e67-b0bc-8e061bb1872c" (UID: "8bd99234-2a29-4e67-b0bc-8e061bb1872c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:52:30.541627 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:30.541608 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bd99234-2a29-4e67-b0bc-8e061bb1872c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8bd99234-2a29-4e67-b0bc-8e061bb1872c" (UID: "8bd99234-2a29-4e67-b0bc-8e061bb1872c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:52:30.542165 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:30.542128 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bd99234-2a29-4e67-b0bc-8e061bb1872c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8bd99234-2a29-4e67-b0bc-8e061bb1872c" (UID: "8bd99234-2a29-4e67-b0bc-8e061bb1872c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 15:52:30.543335 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:30.543081 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bd99234-2a29-4e67-b0bc-8e061bb1872c-istio-token" (OuterVolumeSpecName: "istio-token") pod "8bd99234-2a29-4e67-b0bc-8e061bb1872c" (UID: "8bd99234-2a29-4e67-b0bc-8e061bb1872c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 15:52:30.543770 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:30.543533 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8bd99234-2a29-4e67-b0bc-8e061bb1872c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8bd99234-2a29-4e67-b0bc-8e061bb1872c" (UID: "8bd99234-2a29-4e67-b0bc-8e061bb1872c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 15:52:30.543770 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:30.543630 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bd99234-2a29-4e67-b0bc-8e061bb1872c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8bd99234-2a29-4e67-b0bc-8e061bb1872c" (UID: "8bd99234-2a29-4e67-b0bc-8e061bb1872c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:52:30.543884 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:30.543777 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bd99234-2a29-4e67-b0bc-8e061bb1872c-kube-api-access-z56tw" (OuterVolumeSpecName: "kube-api-access-z56tw") pod "8bd99234-2a29-4e67-b0bc-8e061bb1872c" (UID: "8bd99234-2a29-4e67-b0bc-8e061bb1872c"). InnerVolumeSpecName "kube-api-access-z56tw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 15:52:30.641673 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:30.641639 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8bd99234-2a29-4e67-b0bc-8e061bb1872c-istio-data\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:52:30.641673 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:30.641672 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8bd99234-2a29-4e67-b0bc-8e061bb1872c-istio-envoy\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:52:30.641892 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:30.641684 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bd99234-2a29-4e67-b0bc-8e061bb1872c-workload-socket\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:52:30.641892 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:30.641699 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8bd99234-2a29-4e67-b0bc-8e061bb1872c-istiod-ca-cert\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:52:30.641892 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:30.641711 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8bd99234-2a29-4e67-b0bc-8e061bb1872c-credential-socket\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:52:30.641892 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:30.641724 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8bd99234-2a29-4e67-b0bc-8e061bb1872c-workload-certs\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:52:30.641892 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:30.641735 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z56tw\" (UniqueName: \"kubernetes.io/projected/8bd99234-2a29-4e67-b0bc-8e061bb1872c-kube-api-access-z56tw\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:52:30.641892 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:30.641746 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8bd99234-2a29-4e67-b0bc-8e061bb1872c-istio-podinfo\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:52:30.641892 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:30.641757 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8bd99234-2a29-4e67-b0bc-8e061bb1872c-istio-token\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:52:30.959249 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:30.959212 2574 generic.go:356] "Generic (PLEG): container finished" podID="8bd99234-2a29-4e67-b0bc-8e061bb1872c" containerID="f7301c0d61067a26181f802919516c6414d751660d2f81ed6aacace1b9d5e226" exitCode=0 Aug 04 15:52:30.959467 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:30.959262 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" Aug 04 15:52:30.959467 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:30.959294 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" event={"ID":"8bd99234-2a29-4e67-b0bc-8e061bb1872c","Type":"ContainerDied","Data":"f7301c0d61067a26181f802919516c6414d751660d2f81ed6aacace1b9d5e226"} Aug 04 15:52:30.959467 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:30.959347 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls" event={"ID":"8bd99234-2a29-4e67-b0bc-8e061bb1872c","Type":"ContainerDied","Data":"47d86629b079237ac16afaba6586378667e5a3418978857da7128f2b188abe4b"} Aug 04 15:52:30.959467 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:30.959369 2574 scope.go:117] "RemoveContainer" containerID="f7301c0d61067a26181f802919516c6414d751660d2f81ed6aacace1b9d5e226" Aug 04 15:52:30.969362 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:30.969342 2574 scope.go:117] "RemoveContainer" containerID="90e21dcea7384b2be81a8c3268ca1fb31513247e04a53c06574fba680e38a2d4" Aug 04 15:52:30.977674 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:30.977655 2574 scope.go:117] "RemoveContainer" containerID="f7301c0d61067a26181f802919516c6414d751660d2f81ed6aacace1b9d5e226" Aug 04 15:52:30.977909 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:52:30.977889 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7301c0d61067a26181f802919516c6414d751660d2f81ed6aacace1b9d5e226\": container with ID starting with f7301c0d61067a26181f802919516c6414d751660d2f81ed6aacace1b9d5e226 not found: ID does not exist" containerID="f7301c0d61067a26181f802919516c6414d751660d2f81ed6aacace1b9d5e226" Aug 04 15:52:30.977957 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:30.977917 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7301c0d61067a26181f802919516c6414d751660d2f81ed6aacace1b9d5e226"} err="failed to get container status \"f7301c0d61067a26181f802919516c6414d751660d2f81ed6aacace1b9d5e226\": rpc error: code = NotFound desc = could not find container \"f7301c0d61067a26181f802919516c6414d751660d2f81ed6aacace1b9d5e226\": container with ID starting with f7301c0d61067a26181f802919516c6414d751660d2f81ed6aacace1b9d5e226 not found: ID does not exist" Aug 04 15:52:30.977957 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:30.977933 2574 scope.go:117] "RemoveContainer" containerID="90e21dcea7384b2be81a8c3268ca1fb31513247e04a53c06574fba680e38a2d4" Aug 04 15:52:30.978168 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:52:30.978150 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90e21dcea7384b2be81a8c3268ca1fb31513247e04a53c06574fba680e38a2d4\": container with ID starting with 90e21dcea7384b2be81a8c3268ca1fb31513247e04a53c06574fba680e38a2d4 not found: ID does not exist" containerID="90e21dcea7384b2be81a8c3268ca1fb31513247e04a53c06574fba680e38a2d4" Aug 04 15:52:30.978226 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:30.978178 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90e21dcea7384b2be81a8c3268ca1fb31513247e04a53c06574fba680e38a2d4"} err="failed to get container status \"90e21dcea7384b2be81a8c3268ca1fb31513247e04a53c06574fba680e38a2d4\": rpc error: code = NotFound desc = could not find container \"90e21dcea7384b2be81a8c3268ca1fb31513247e04a53c06574fba680e38a2d4\": container with ID starting with 90e21dcea7384b2be81a8c3268ca1fb31513247e04a53c06574fba680e38a2d4 not found: ID does not exist" Aug 04 15:52:30.984463 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:30.984443 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls"] Aug 04 15:52:30.997105 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:30.994117 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-926308-openshift-default-78dc965d48-lbmls"] Aug 04 15:52:31.053730 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:31.053698 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bd99234-2a29-4e67-b0bc-8e061bb1872c" path="/var/lib/kubelet/pods/8bd99234-2a29-4e67-b0bc-8e061bb1872c/volumes" Aug 04 15:52:35.411796 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:35.411760 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:52:43.316209 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:43.316170 2574 patch_prober.go:28] interesting pod/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": read tcp 10.132.0.2:54290->10.132.0.63:15021: read: connection reset by peer" start-of-body= Aug 04 15:52:43.316614 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:43.316258 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" podUID="56ceb5c8-199f-41d9-9fa1-3501db74a86b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": read tcp 10.132.0.2:54290->10.132.0.63:15021: read: connection reset by peer" Aug 04 15:52:44.043961 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:44.043916 2574 generic.go:356] "Generic (PLEG): container finished" podID="56ceb5c8-199f-41d9-9fa1-3501db74a86b" containerID="2687ef81b1d608de5abba3928a16f2df7bd1ec536936e0962914b3163b09c7b4" exitCode=0 Aug 04 15:52:44.044138 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:44.043991 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" event={"ID":"56ceb5c8-199f-41d9-9fa1-3501db74a86b","Type":"ContainerDied","Data":"2687ef81b1d608de5abba3928a16f2df7bd1ec536936e0962914b3163b09c7b4"} Aug 04 15:52:44.044477 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:44.044463 2574 scope.go:117] "RemoveContainer" containerID="2687ef81b1d608de5abba3928a16f2df7bd1ec536936e0962914b3163b09c7b4" Aug 04 15:52:44.045514 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:44.045487 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:52:44.045576 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:44.045562 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:52:44.045614 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:44.045597 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:52:45.054957 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:45.054919 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" event={"ID":"56ceb5c8-199f-41d9-9fa1-3501db74a86b","Type":"ContainerStarted","Data":"731769d4e5feaf5ced8c06b916e51e4f632a5e59cfaf8e455f8ce2a1994d41af"} Aug 04 15:52:45.425530 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:45.425432 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" Aug 04 15:52:45.425723 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:45.425562 2574 patch_prober.go:28] interesting pod/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Aug 04 15:52:45.425723 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:45.425605 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" podUID="56ceb5c8-199f-41d9-9fa1-3501db74a86b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Aug 04 15:52:46.425300 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:46.425256 2574 patch_prober.go:28] interesting pod/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Aug 04 15:52:46.425845 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:46.425329 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" podUID="56ceb5c8-199f-41d9-9fa1-3501db74a86b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Aug 04 15:52:47.424716 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:47.424672 2574 patch_prober.go:28] interesting pod/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Aug 04 15:52:47.424934 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:47.424757 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" podUID="56ceb5c8-199f-41d9-9fa1-3501db74a86b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Aug 04 15:52:48.425631 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:48.425590 2574 patch_prober.go:28] interesting pod/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Aug 04 15:52:48.426167 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:48.425663 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" podUID="56ceb5c8-199f-41d9-9fa1-3501db74a86b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Aug 04 15:52:49.428544 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:49.428505 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" Aug 04 15:52:49.428935 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:49.428894 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" Aug 04 15:52:49.429671 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:49.429651 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" Aug 04 15:52:54.860028 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:54.859998 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm"] Aug 04 15:52:54.860444 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:54.860430 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bd99234-2a29-4e67-b0bc-8e061bb1872c" containerName="istio-proxy" Aug 04 15:52:54.860499 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:54.860445 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bd99234-2a29-4e67-b0bc-8e061bb1872c" containerName="istio-proxy" Aug 04 15:52:54.860499 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:54.860456 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bd99234-2a29-4e67-b0bc-8e061bb1872c" containerName="istio-proxy" Aug 04 15:52:54.860499 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:54.860461 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bd99234-2a29-4e67-b0bc-8e061bb1872c" containerName="istio-proxy" Aug 04 15:52:54.860599 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:54.860538 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bd99234-2a29-4e67-b0bc-8e061bb1872c" containerName="istio-proxy" Aug 04 15:52:54.860599 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:54.860546 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bd99234-2a29-4e67-b0bc-8e061bb1872c" containerName="istio-proxy" Aug 04 15:52:54.862647 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:54.862629 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" Aug 04 15:52:54.865499 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:54.865481 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-df124e-openshift-default-dockercfg-z85jj\"" Aug 04 15:52:54.877724 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:54.877697 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm"] Aug 04 15:52:54.972638 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:54.972602 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/996aa267-f951-40b0-bfda-95a4fe53237e-credential-socket\") pod \"e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm\" (UID: \"996aa267-f951-40b0-bfda-95a4fe53237e\") " pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" Aug 04 15:52:54.972823 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:54.972653 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/996aa267-f951-40b0-bfda-95a4fe53237e-istio-podinfo\") pod \"e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm\" (UID: \"996aa267-f951-40b0-bfda-95a4fe53237e\") " pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" Aug 04 15:52:54.972823 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:54.972680 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/996aa267-f951-40b0-bfda-95a4fe53237e-istiod-ca-cert\") pod \"e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm\" (UID: \"996aa267-f951-40b0-bfda-95a4fe53237e\") " pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" Aug 04 15:52:54.972823 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:54.972747 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/996aa267-f951-40b0-bfda-95a4fe53237e-workload-socket\") pod \"e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm\" (UID: \"996aa267-f951-40b0-bfda-95a4fe53237e\") " pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" Aug 04 15:52:54.972823 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:54.972782 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/996aa267-f951-40b0-bfda-95a4fe53237e-istio-token\") pod \"e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm\" (UID: \"996aa267-f951-40b0-bfda-95a4fe53237e\") " pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" Aug 04 15:52:54.972998 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:54.972869 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/996aa267-f951-40b0-bfda-95a4fe53237e-workload-certs\") pod \"e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm\" (UID: \"996aa267-f951-40b0-bfda-95a4fe53237e\") " pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" Aug 04 15:52:54.972998 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:54.972916 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/996aa267-f951-40b0-bfda-95a4fe53237e-istio-data\") pod \"e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm\" (UID: \"996aa267-f951-40b0-bfda-95a4fe53237e\") " pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" Aug 04 15:52:54.972998 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:54.972981 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fm98m\" (UniqueName: \"kubernetes.io/projected/996aa267-f951-40b0-bfda-95a4fe53237e-kube-api-access-fm98m\") pod \"e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm\" (UID: \"996aa267-f951-40b0-bfda-95a4fe53237e\") " pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" Aug 04 15:52:54.973109 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:54.973022 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/996aa267-f951-40b0-bfda-95a4fe53237e-istio-envoy\") pod \"e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm\" (UID: \"996aa267-f951-40b0-bfda-95a4fe53237e\") " pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" Aug 04 15:52:55.073492 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:55.073460 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/996aa267-f951-40b0-bfda-95a4fe53237e-credential-socket\") pod \"e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm\" (UID: \"996aa267-f951-40b0-bfda-95a4fe53237e\") " pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" Aug 04 15:52:55.073651 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:55.073499 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/996aa267-f951-40b0-bfda-95a4fe53237e-istio-podinfo\") pod \"e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm\" (UID: \"996aa267-f951-40b0-bfda-95a4fe53237e\") " pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" Aug 04 15:52:55.073651 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:55.073517 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/996aa267-f951-40b0-bfda-95a4fe53237e-istiod-ca-cert\") pod \"e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm\" (UID: \"996aa267-f951-40b0-bfda-95a4fe53237e\") " pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" Aug 04 15:52:55.073651 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:55.073541 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/996aa267-f951-40b0-bfda-95a4fe53237e-workload-socket\") pod \"e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm\" (UID: \"996aa267-f951-40b0-bfda-95a4fe53237e\") " pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" Aug 04 15:52:55.073651 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:55.073562 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/996aa267-f951-40b0-bfda-95a4fe53237e-istio-token\") pod \"e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm\" (UID: \"996aa267-f951-40b0-bfda-95a4fe53237e\") " pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" Aug 04 15:52:55.073651 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:55.073602 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/996aa267-f951-40b0-bfda-95a4fe53237e-workload-certs\") pod \"e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm\" (UID: \"996aa267-f951-40b0-bfda-95a4fe53237e\") " pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" Aug 04 15:52:55.073651 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:55.073632 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/996aa267-f951-40b0-bfda-95a4fe53237e-istio-data\") pod \"e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm\" (UID: \"996aa267-f951-40b0-bfda-95a4fe53237e\") " pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" Aug 04 15:52:55.074010 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:55.073678 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fm98m\" (UniqueName: \"kubernetes.io/projected/996aa267-f951-40b0-bfda-95a4fe53237e-kube-api-access-fm98m\") pod \"e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm\" (UID: \"996aa267-f951-40b0-bfda-95a4fe53237e\") " pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" Aug 04 15:52:55.074010 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:55.073716 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/996aa267-f951-40b0-bfda-95a4fe53237e-istio-envoy\") pod \"e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm\" (UID: \"996aa267-f951-40b0-bfda-95a4fe53237e\") " pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" Aug 04 15:52:55.074010 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:55.073928 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/996aa267-f951-40b0-bfda-95a4fe53237e-credential-socket\") pod \"e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm\" (UID: \"996aa267-f951-40b0-bfda-95a4fe53237e\") " pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" Aug 04 15:52:55.074184 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:55.074015 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/996aa267-f951-40b0-bfda-95a4fe53237e-workload-socket\") pod \"e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm\" (UID: \"996aa267-f951-40b0-bfda-95a4fe53237e\") " pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" Aug 04 15:52:55.074184 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:55.074030 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/996aa267-f951-40b0-bfda-95a4fe53237e-workload-certs\") pod \"e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm\" (UID: \"996aa267-f951-40b0-bfda-95a4fe53237e\") " pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" Aug 04 15:52:55.074276 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:55.074250 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/996aa267-f951-40b0-bfda-95a4fe53237e-istio-data\") pod \"e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm\" (UID: \"996aa267-f951-40b0-bfda-95a4fe53237e\") " pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" Aug 04 15:52:55.074471 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:55.074448 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/996aa267-f951-40b0-bfda-95a4fe53237e-istiod-ca-cert\") pod \"e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm\" (UID: \"996aa267-f951-40b0-bfda-95a4fe53237e\") " pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" Aug 04 15:52:55.076135 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:55.076108 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/996aa267-f951-40b0-bfda-95a4fe53237e-istio-envoy\") pod \"e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm\" (UID: \"996aa267-f951-40b0-bfda-95a4fe53237e\") " pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" Aug 04 15:52:55.076350 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:55.076331 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/996aa267-f951-40b0-bfda-95a4fe53237e-istio-podinfo\") pod \"e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm\" (UID: \"996aa267-f951-40b0-bfda-95a4fe53237e\") " pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" Aug 04 15:52:55.094473 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:55.094450 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/996aa267-f951-40b0-bfda-95a4fe53237e-istio-token\") pod \"e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm\" (UID: \"996aa267-f951-40b0-bfda-95a4fe53237e\") " pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" Aug 04 15:52:55.094581 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:55.094550 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fm98m\" (UniqueName: \"kubernetes.io/projected/996aa267-f951-40b0-bfda-95a4fe53237e-kube-api-access-fm98m\") pod \"e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm\" (UID: \"996aa267-f951-40b0-bfda-95a4fe53237e\") " pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" Aug 04 15:52:55.173173 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:55.173075 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" Aug 04 15:52:55.311029 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:55.311004 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm"] Aug 04 15:52:55.312636 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:52:55.312601 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod996aa267_f951_40b0_bfda_95a4fe53237e.slice/crio-6e1363ed7c4cc8405f621f98100d51403e1b8644ed34caf3068702a50a95be57 WatchSource:0}: Error finding container 6e1363ed7c4cc8405f621f98100d51403e1b8644ed34caf3068702a50a95be57: Status 404 returned error can't find the container with id 6e1363ed7c4cc8405f621f98100d51403e1b8644ed34caf3068702a50a95be57 Aug 04 15:52:55.314720 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:55.314685 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:52:55.314814 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:55.314764 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:52:55.314877 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:55.314812 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:52:56.117191 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:56.117154 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" event={"ID":"996aa267-f951-40b0-bfda-95a4fe53237e","Type":"ContainerStarted","Data":"8bfc23151ca1b6285c54f5c3c9d3095791803562880af621b7e1e9a9096d780d"} Aug 04 15:52:56.117671 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:56.117215 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" event={"ID":"996aa267-f951-40b0-bfda-95a4fe53237e","Type":"ContainerStarted","Data":"6e1363ed7c4cc8405f621f98100d51403e1b8644ed34caf3068702a50a95be57"} Aug 04 15:52:56.143061 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:56.143015 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" podStartSLOduration=2.143000028 podStartE2EDuration="2.143000028s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:52:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:52:56.141080405 +0000 UTC m=+2799.600952754" watchObservedRunningTime="2026-08-04 15:52:56.143000028 +0000 UTC m=+2799.602872377" Aug 04 15:52:56.173226 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:56.173196 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" Aug 04 15:52:56.178882 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:56.178859 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" Aug 04 15:52:57.122825 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:57.122795 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" Aug 04 15:52:57.124154 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:52:57.124129 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" Aug 04 15:53:04.117674 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:04.117636 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:53:12.124715 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:12.124679 2574 patch_prober.go:28] interesting pod/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Aug 04 15:53:12.125128 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:12.124746 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" podUID="996aa267-f951-40b0-bfda-95a4fe53237e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Aug 04 15:53:12.211205 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:12.211166 2574 generic.go:356] "Generic (PLEG): container finished" podID="996aa267-f951-40b0-bfda-95a4fe53237e" containerID="8bfc23151ca1b6285c54f5c3c9d3095791803562880af621b7e1e9a9096d780d" exitCode=0 Aug 04 15:53:12.211205 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:12.211189 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" event={"ID":"996aa267-f951-40b0-bfda-95a4fe53237e","Type":"ContainerDied","Data":"8bfc23151ca1b6285c54f5c3c9d3095791803562880af621b7e1e9a9096d780d"} Aug 04 15:53:12.211648 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:12.211632 2574 scope.go:117] "RemoveContainer" containerID="8bfc23151ca1b6285c54f5c3c9d3095791803562880af621b7e1e9a9096d780d" Aug 04 15:53:12.212790 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:12.212750 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:53:12.212855 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:12.212841 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:53:12.212897 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:12.212876 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:53:13.218244 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:13.218210 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" event={"ID":"996aa267-f951-40b0-bfda-95a4fe53237e","Type":"ContainerStarted","Data":"c05bb3d79c31655918d114492ef6cae44053540bce690a3504b940df5bc83932"} Aug 04 15:53:14.173553 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:14.173504 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" Aug 04 15:53:14.173776 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:14.173752 2574 patch_prober.go:28] interesting pod/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Aug 04 15:53:14.173833 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:14.173814 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" podUID="996aa267-f951-40b0-bfda-95a4fe53237e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Aug 04 15:53:15.173738 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:15.173701 2574 patch_prober.go:28] interesting pod/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Aug 04 15:53:15.174133 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:15.173754 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" podUID="996aa267-f951-40b0-bfda-95a4fe53237e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Aug 04 15:53:16.174448 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:16.174377 2574 patch_prober.go:28] interesting pod/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Aug 04 15:53:16.174932 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:16.174477 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" podUID="996aa267-f951-40b0-bfda-95a4fe53237e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Aug 04 15:53:17.188288 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:17.188259 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" Aug 04 15:53:17.188795 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:17.188625 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" Aug 04 15:53:17.189238 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:17.189220 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" Aug 04 15:53:44.213091 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:44.213057 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:53:48.784358 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:48.784324 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh"] Aug 04 15:53:48.784841 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:48.784592 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" podUID="56ceb5c8-199f-41d9-9fa1-3501db74a86b" containerName="istio-proxy" containerID="cri-o://731769d4e5feaf5ced8c06b916e51e4f632a5e59cfaf8e455f8ce2a1994d41af" gracePeriod=30 Aug 04 15:53:49.431877 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:49.431841 2574 patch_prober.go:28] interesting pod/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 04 15:53:49.432056 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:49.431907 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" podUID="56ceb5c8-199f-41d9-9fa1-3501db74a86b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 15:53:54.038830 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:54.038803 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" Aug 04 15:53:54.123877 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:54.123803 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56ceb5c8-199f-41d9-9fa1-3501db74a86b-istio-token\") pod \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\" (UID: \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\") " Aug 04 15:53:54.123877 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:54.123832 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5xwth\" (UniqueName: \"kubernetes.io/projected/56ceb5c8-199f-41d9-9fa1-3501db74a86b-kube-api-access-5xwth\") pod \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\" (UID: \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\") " Aug 04 15:53:54.123877 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:54.123850 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56ceb5c8-199f-41d9-9fa1-3501db74a86b-istio-data\") pod \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\" (UID: \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\") " Aug 04 15:53:54.123877 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:54.123866 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56ceb5c8-199f-41d9-9fa1-3501db74a86b-istio-envoy\") pod \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\" (UID: \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\") " Aug 04 15:53:54.124186 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:54.123885 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56ceb5c8-199f-41d9-9fa1-3501db74a86b-workload-certs\") pod \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\" (UID: \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\") " Aug 04 15:53:54.124186 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:54.123950 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56ceb5c8-199f-41d9-9fa1-3501db74a86b-istio-podinfo\") pod \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\" (UID: \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\") " Aug 04 15:53:54.124186 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:54.123982 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56ceb5c8-199f-41d9-9fa1-3501db74a86b-workload-socket\") pod \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\" (UID: \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\") " Aug 04 15:53:54.124186 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:54.124012 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56ceb5c8-199f-41d9-9fa1-3501db74a86b-istiod-ca-cert\") pod \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\" (UID: \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\") " Aug 04 15:53:54.124186 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:54.124045 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56ceb5c8-199f-41d9-9fa1-3501db74a86b-credential-socket\") pod \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\" (UID: \"56ceb5c8-199f-41d9-9fa1-3501db74a86b\") " Aug 04 15:53:54.124498 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:54.124282 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56ceb5c8-199f-41d9-9fa1-3501db74a86b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "56ceb5c8-199f-41d9-9fa1-3501db74a86b" (UID: "56ceb5c8-199f-41d9-9fa1-3501db74a86b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:53:54.124570 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:54.124499 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56ceb5c8-199f-41d9-9fa1-3501db74a86b-workload-certs\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:53:54.124570 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:54.124516 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56ceb5c8-199f-41d9-9fa1-3501db74a86b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "56ceb5c8-199f-41d9-9fa1-3501db74a86b" (UID: "56ceb5c8-199f-41d9-9fa1-3501db74a86b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:53:54.124784 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:54.124763 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56ceb5c8-199f-41d9-9fa1-3501db74a86b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "56ceb5c8-199f-41d9-9fa1-3501db74a86b" (UID: "56ceb5c8-199f-41d9-9fa1-3501db74a86b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 15:53:54.124934 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:54.124911 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56ceb5c8-199f-41d9-9fa1-3501db74a86b-istio-data" (OuterVolumeSpecName: "istio-data") pod "56ceb5c8-199f-41d9-9fa1-3501db74a86b" (UID: "56ceb5c8-199f-41d9-9fa1-3501db74a86b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:53:54.125013 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:54.124951 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56ceb5c8-199f-41d9-9fa1-3501db74a86b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "56ceb5c8-199f-41d9-9fa1-3501db74a86b" (UID: "56ceb5c8-199f-41d9-9fa1-3501db74a86b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:53:54.126205 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:54.126179 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56ceb5c8-199f-41d9-9fa1-3501db74a86b-kube-api-access-5xwth" (OuterVolumeSpecName: "kube-api-access-5xwth") pod "56ceb5c8-199f-41d9-9fa1-3501db74a86b" (UID: "56ceb5c8-199f-41d9-9fa1-3501db74a86b"). InnerVolumeSpecName "kube-api-access-5xwth". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 15:53:54.126283 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:54.126238 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/56ceb5c8-199f-41d9-9fa1-3501db74a86b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "56ceb5c8-199f-41d9-9fa1-3501db74a86b" (UID: "56ceb5c8-199f-41d9-9fa1-3501db74a86b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 15:53:54.126466 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:54.126441 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56ceb5c8-199f-41d9-9fa1-3501db74a86b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "56ceb5c8-199f-41d9-9fa1-3501db74a86b" (UID: "56ceb5c8-199f-41d9-9fa1-3501db74a86b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:53:54.126637 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:54.126620 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56ceb5c8-199f-41d9-9fa1-3501db74a86b-istio-token" (OuterVolumeSpecName: "istio-token") pod "56ceb5c8-199f-41d9-9fa1-3501db74a86b" (UID: "56ceb5c8-199f-41d9-9fa1-3501db74a86b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 15:53:54.225218 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:54.225182 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56ceb5c8-199f-41d9-9fa1-3501db74a86b-istiod-ca-cert\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:53:54.225218 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:54.225210 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56ceb5c8-199f-41d9-9fa1-3501db74a86b-credential-socket\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:53:54.225438 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:54.225223 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56ceb5c8-199f-41d9-9fa1-3501db74a86b-istio-token\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:53:54.225438 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:54.225237 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5xwth\" (UniqueName: \"kubernetes.io/projected/56ceb5c8-199f-41d9-9fa1-3501db74a86b-kube-api-access-5xwth\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:53:54.225438 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:54.225248 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56ceb5c8-199f-41d9-9fa1-3501db74a86b-istio-data\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:53:54.225438 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:54.225262 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56ceb5c8-199f-41d9-9fa1-3501db74a86b-istio-envoy\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:53:54.225438 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:54.225272 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56ceb5c8-199f-41d9-9fa1-3501db74a86b-istio-podinfo\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:53:54.225438 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:54.225283 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56ceb5c8-199f-41d9-9fa1-3501db74a86b-workload-socket\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:53:54.465377 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:54.465283 2574 generic.go:356] "Generic (PLEG): container finished" podID="56ceb5c8-199f-41d9-9fa1-3501db74a86b" containerID="731769d4e5feaf5ced8c06b916e51e4f632a5e59cfaf8e455f8ce2a1994d41af" exitCode=0 Aug 04 15:53:54.465377 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:54.465331 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" Aug 04 15:53:54.465598 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:54.465372 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" event={"ID":"56ceb5c8-199f-41d9-9fa1-3501db74a86b","Type":"ContainerDied","Data":"731769d4e5feaf5ced8c06b916e51e4f632a5e59cfaf8e455f8ce2a1994d41af"} Aug 04 15:53:54.465598 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:54.465452 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh" event={"ID":"56ceb5c8-199f-41d9-9fa1-3501db74a86b","Type":"ContainerDied","Data":"7c0208aa3deda7bdd4b696c602452a0c6e93334c6fba21d7cfb3a3dcc83e0fed"} Aug 04 15:53:54.465598 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:54.465479 2574 scope.go:117] "RemoveContainer" containerID="731769d4e5feaf5ced8c06b916e51e4f632a5e59cfaf8e455f8ce2a1994d41af" Aug 04 15:53:54.475261 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:54.475246 2574 scope.go:117] "RemoveContainer" containerID="2687ef81b1d608de5abba3928a16f2df7bd1ec536936e0962914b3163b09c7b4" Aug 04 15:53:54.483306 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:54.483288 2574 scope.go:117] "RemoveContainer" containerID="731769d4e5feaf5ced8c06b916e51e4f632a5e59cfaf8e455f8ce2a1994d41af" Aug 04 15:53:54.483565 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:53:54.483547 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"731769d4e5feaf5ced8c06b916e51e4f632a5e59cfaf8e455f8ce2a1994d41af\": container with ID starting with 731769d4e5feaf5ced8c06b916e51e4f632a5e59cfaf8e455f8ce2a1994d41af not found: ID does not exist" containerID="731769d4e5feaf5ced8c06b916e51e4f632a5e59cfaf8e455f8ce2a1994d41af" Aug 04 15:53:54.483623 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:54.483573 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"731769d4e5feaf5ced8c06b916e51e4f632a5e59cfaf8e455f8ce2a1994d41af"} err="failed to get container status \"731769d4e5feaf5ced8c06b916e51e4f632a5e59cfaf8e455f8ce2a1994d41af\": rpc error: code = NotFound desc = could not find container \"731769d4e5feaf5ced8c06b916e51e4f632a5e59cfaf8e455f8ce2a1994d41af\": container with ID starting with 731769d4e5feaf5ced8c06b916e51e4f632a5e59cfaf8e455f8ce2a1994d41af not found: ID does not exist" Aug 04 15:53:54.483623 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:54.483589 2574 scope.go:117] "RemoveContainer" containerID="2687ef81b1d608de5abba3928a16f2df7bd1ec536936e0962914b3163b09c7b4" Aug 04 15:53:54.483818 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:53:54.483799 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2687ef81b1d608de5abba3928a16f2df7bd1ec536936e0962914b3163b09c7b4\": container with ID starting with 2687ef81b1d608de5abba3928a16f2df7bd1ec536936e0962914b3163b09c7b4 not found: ID does not exist" containerID="2687ef81b1d608de5abba3928a16f2df7bd1ec536936e0962914b3163b09c7b4" Aug 04 15:53:54.483859 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:54.483824 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2687ef81b1d608de5abba3928a16f2df7bd1ec536936e0962914b3163b09c7b4"} err="failed to get container status \"2687ef81b1d608de5abba3928a16f2df7bd1ec536936e0962914b3163b09c7b4\": rpc error: code = NotFound desc = could not find container \"2687ef81b1d608de5abba3928a16f2df7bd1ec536936e0962914b3163b09c7b4\": container with ID starting with 2687ef81b1d608de5abba3928a16f2df7bd1ec536936e0962914b3163b09c7b4 not found: ID does not exist" Aug 04 15:53:54.490770 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:54.490749 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh"] Aug 04 15:53:54.495345 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:54.495320 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-bac88d-openshift-default-568d69f9b8-d72bh"] Aug 04 15:53:55.054504 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:53:55.054472 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56ceb5c8-199f-41d9-9fa1-3501db74a86b" path="/var/lib/kubelet/pods/56ceb5c8-199f-41d9-9fa1-3501db74a86b/volumes" Aug 04 15:54:09.514030 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:09.513952 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:54:11.682018 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:11.681978 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm"] Aug 04 15:54:11.682449 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:11.682211 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" podUID="996aa267-f951-40b0-bfda-95a4fe53237e" containerName="istio-proxy" containerID="cri-o://c05bb3d79c31655918d114492ef6cae44053540bce690a3504b940df5bc83932" gracePeriod=30 Aug 04 15:54:12.633096 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:12.633064 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg"] Aug 04 15:54:12.633480 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:12.633466 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56ceb5c8-199f-41d9-9fa1-3501db74a86b" containerName="istio-proxy" Aug 04 15:54:12.633480 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:12.633481 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="56ceb5c8-199f-41d9-9fa1-3501db74a86b" containerName="istio-proxy" Aug 04 15:54:12.633593 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:12.633496 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56ceb5c8-199f-41d9-9fa1-3501db74a86b" containerName="istio-proxy" Aug 04 15:54:12.633593 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:12.633505 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="56ceb5c8-199f-41d9-9fa1-3501db74a86b" containerName="istio-proxy" Aug 04 15:54:12.633689 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:12.633611 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="56ceb5c8-199f-41d9-9fa1-3501db74a86b" containerName="istio-proxy" Aug 04 15:54:12.633689 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:12.633625 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="56ceb5c8-199f-41d9-9fa1-3501db74a86b" containerName="istio-proxy" Aug 04 15:54:12.636910 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:12.636892 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" Aug 04 15:54:12.640632 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:12.640613 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-05b2bc-openshift-default-dockercfg-xxfth\"" Aug 04 15:54:12.652706 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:12.652682 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg"] Aug 04 15:54:12.694531 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:12.694492 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c977f26-4e15-4788-ac7e-b0933b030ac2-workload-socket\") pod \"e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg\" (UID: \"3c977f26-4e15-4788-ac7e-b0933b030ac2\") " pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" Aug 04 15:54:12.694531 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:12.694536 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgdvf\" (UniqueName: \"kubernetes.io/projected/3c977f26-4e15-4788-ac7e-b0933b030ac2-kube-api-access-zgdvf\") pod \"e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg\" (UID: \"3c977f26-4e15-4788-ac7e-b0933b030ac2\") " pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" Aug 04 15:54:12.694953 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:12.694564 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c977f26-4e15-4788-ac7e-b0933b030ac2-istiod-ca-cert\") pod \"e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg\" (UID: \"3c977f26-4e15-4788-ac7e-b0933b030ac2\") " pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" Aug 04 15:54:12.694953 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:12.694631 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c977f26-4e15-4788-ac7e-b0933b030ac2-istio-token\") pod \"e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg\" (UID: \"3c977f26-4e15-4788-ac7e-b0933b030ac2\") " pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" Aug 04 15:54:12.694953 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:12.694701 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c977f26-4e15-4788-ac7e-b0933b030ac2-credential-socket\") pod \"e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg\" (UID: \"3c977f26-4e15-4788-ac7e-b0933b030ac2\") " pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" Aug 04 15:54:12.694953 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:12.694761 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c977f26-4e15-4788-ac7e-b0933b030ac2-istio-podinfo\") pod \"e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg\" (UID: \"3c977f26-4e15-4788-ac7e-b0933b030ac2\") " pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" Aug 04 15:54:12.694953 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:12.694806 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c977f26-4e15-4788-ac7e-b0933b030ac2-istio-envoy\") pod \"e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg\" (UID: \"3c977f26-4e15-4788-ac7e-b0933b030ac2\") " pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" Aug 04 15:54:12.694953 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:12.694832 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c977f26-4e15-4788-ac7e-b0933b030ac2-workload-certs\") pod \"e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg\" (UID: \"3c977f26-4e15-4788-ac7e-b0933b030ac2\") " pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" Aug 04 15:54:12.694953 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:12.694852 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c977f26-4e15-4788-ac7e-b0933b030ac2-istio-data\") pod \"e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg\" (UID: \"3c977f26-4e15-4788-ac7e-b0933b030ac2\") " pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" Aug 04 15:54:12.795585 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:12.795549 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c977f26-4e15-4788-ac7e-b0933b030ac2-istio-envoy\") pod \"e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg\" (UID: \"3c977f26-4e15-4788-ac7e-b0933b030ac2\") " pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" Aug 04 15:54:12.795585 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:12.795592 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c977f26-4e15-4788-ac7e-b0933b030ac2-workload-certs\") pod \"e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg\" (UID: \"3c977f26-4e15-4788-ac7e-b0933b030ac2\") " pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" Aug 04 15:54:12.795806 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:12.795618 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c977f26-4e15-4788-ac7e-b0933b030ac2-istio-data\") pod \"e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg\" (UID: \"3c977f26-4e15-4788-ac7e-b0933b030ac2\") " pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" Aug 04 15:54:12.795806 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:12.795647 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c977f26-4e15-4788-ac7e-b0933b030ac2-workload-socket\") pod \"e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg\" (UID: \"3c977f26-4e15-4788-ac7e-b0933b030ac2\") " pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" Aug 04 15:54:12.795806 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:12.795677 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgdvf\" (UniqueName: \"kubernetes.io/projected/3c977f26-4e15-4788-ac7e-b0933b030ac2-kube-api-access-zgdvf\") pod \"e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg\" (UID: \"3c977f26-4e15-4788-ac7e-b0933b030ac2\") " pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" Aug 04 15:54:12.795806 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:12.795712 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c977f26-4e15-4788-ac7e-b0933b030ac2-istiod-ca-cert\") pod \"e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg\" (UID: \"3c977f26-4e15-4788-ac7e-b0933b030ac2\") " pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" Aug 04 15:54:12.795806 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:12.795737 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c977f26-4e15-4788-ac7e-b0933b030ac2-istio-token\") pod \"e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg\" (UID: \"3c977f26-4e15-4788-ac7e-b0933b030ac2\") " pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" Aug 04 15:54:12.795806 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:12.795794 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c977f26-4e15-4788-ac7e-b0933b030ac2-credential-socket\") pod \"e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg\" (UID: \"3c977f26-4e15-4788-ac7e-b0933b030ac2\") " pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" Aug 04 15:54:12.796138 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:12.795853 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c977f26-4e15-4788-ac7e-b0933b030ac2-istio-podinfo\") pod \"e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg\" (UID: \"3c977f26-4e15-4788-ac7e-b0933b030ac2\") " pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" Aug 04 15:54:12.796138 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:12.796019 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c977f26-4e15-4788-ac7e-b0933b030ac2-workload-certs\") pod \"e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg\" (UID: \"3c977f26-4e15-4788-ac7e-b0933b030ac2\") " pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" Aug 04 15:54:12.796138 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:12.796078 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c977f26-4e15-4788-ac7e-b0933b030ac2-workload-socket\") pod \"e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg\" (UID: \"3c977f26-4e15-4788-ac7e-b0933b030ac2\") " pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" Aug 04 15:54:12.796138 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:12.796113 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c977f26-4e15-4788-ac7e-b0933b030ac2-istio-data\") pod \"e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg\" (UID: \"3c977f26-4e15-4788-ac7e-b0933b030ac2\") " pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" Aug 04 15:54:12.796351 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:12.796295 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c977f26-4e15-4788-ac7e-b0933b030ac2-credential-socket\") pod \"e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg\" (UID: \"3c977f26-4e15-4788-ac7e-b0933b030ac2\") " pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" Aug 04 15:54:12.796532 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:12.796509 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c977f26-4e15-4788-ac7e-b0933b030ac2-istiod-ca-cert\") pod \"e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg\" (UID: \"3c977f26-4e15-4788-ac7e-b0933b030ac2\") " pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" Aug 04 15:54:12.797941 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:12.797921 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c977f26-4e15-4788-ac7e-b0933b030ac2-istio-envoy\") pod \"e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg\" (UID: \"3c977f26-4e15-4788-ac7e-b0933b030ac2\") " pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" Aug 04 15:54:12.798157 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:12.798139 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c977f26-4e15-4788-ac7e-b0933b030ac2-istio-podinfo\") pod \"e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg\" (UID: \"3c977f26-4e15-4788-ac7e-b0933b030ac2\") " pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" Aug 04 15:54:12.804832 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:12.804811 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgdvf\" (UniqueName: \"kubernetes.io/projected/3c977f26-4e15-4788-ac7e-b0933b030ac2-kube-api-access-zgdvf\") pod \"e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg\" (UID: \"3c977f26-4e15-4788-ac7e-b0933b030ac2\") " pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" Aug 04 15:54:12.808259 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:12.808232 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c977f26-4e15-4788-ac7e-b0933b030ac2-istio-token\") pod \"e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg\" (UID: \"3c977f26-4e15-4788-ac7e-b0933b030ac2\") " pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" Aug 04 15:54:12.947024 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:12.946937 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" Aug 04 15:54:13.090162 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:13.090137 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg"] Aug 04 15:54:13.091589 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:54:13.091561 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c977f26_4e15_4788_ac7e_b0933b030ac2.slice/crio-88d8b5f1b7649e28b1830634d649161fcc8e3e300ddd90b1f456509a9963df4a WatchSource:0}: Error finding container 88d8b5f1b7649e28b1830634d649161fcc8e3e300ddd90b1f456509a9963df4a: Status 404 returned error can't find the container with id 88d8b5f1b7649e28b1830634d649161fcc8e3e300ddd90b1f456509a9963df4a Aug 04 15:54:13.093550 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:13.093521 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:54:13.093615 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:13.093588 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:54:13.093653 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:13.093618 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:54:13.570662 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:13.570624 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" event={"ID":"3c977f26-4e15-4788-ac7e-b0933b030ac2","Type":"ContainerStarted","Data":"91f819113cef6db375d6af01f1ce282b7078fc5dbff2e612fb150ed1b55f3461"} Aug 04 15:54:13.570861 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:13.570679 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" event={"ID":"3c977f26-4e15-4788-ac7e-b0933b030ac2","Type":"ContainerStarted","Data":"88d8b5f1b7649e28b1830634d649161fcc8e3e300ddd90b1f456509a9963df4a"} Aug 04 15:54:13.602348 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:13.602296 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" podStartSLOduration=1.602279966 podStartE2EDuration="1.602279966s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:54:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:54:13.599947219 +0000 UTC m=+2877.059819565" watchObservedRunningTime="2026-08-04 15:54:13.602279966 +0000 UTC m=+2877.062152315" Aug 04 15:54:13.947255 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:13.947175 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" Aug 04 15:54:14.952749 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:14.952712 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" Aug 04 15:54:15.582312 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:15.582282 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" Aug 04 15:54:15.583167 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:15.583146 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" Aug 04 15:54:16.942033 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:16.941970 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" Aug 04 15:54:17.034283 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:17.034252 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/996aa267-f951-40b0-bfda-95a4fe53237e-credential-socket\") pod \"996aa267-f951-40b0-bfda-95a4fe53237e\" (UID: \"996aa267-f951-40b0-bfda-95a4fe53237e\") " Aug 04 15:54:17.034506 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:17.034290 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/996aa267-f951-40b0-bfda-95a4fe53237e-workload-certs\") pod \"996aa267-f951-40b0-bfda-95a4fe53237e\" (UID: \"996aa267-f951-40b0-bfda-95a4fe53237e\") " Aug 04 15:54:17.034506 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:17.034314 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/996aa267-f951-40b0-bfda-95a4fe53237e-workload-socket\") pod \"996aa267-f951-40b0-bfda-95a4fe53237e\" (UID: \"996aa267-f951-40b0-bfda-95a4fe53237e\") " Aug 04 15:54:17.034506 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:17.034362 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/996aa267-f951-40b0-bfda-95a4fe53237e-istio-podinfo\") pod \"996aa267-f951-40b0-bfda-95a4fe53237e\" (UID: \"996aa267-f951-40b0-bfda-95a4fe53237e\") " Aug 04 15:54:17.034506 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:17.034435 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/996aa267-f951-40b0-bfda-95a4fe53237e-istio-data\") pod \"996aa267-f951-40b0-bfda-95a4fe53237e\" (UID: \"996aa267-f951-40b0-bfda-95a4fe53237e\") " Aug 04 15:54:17.035079 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:17.034567 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/996aa267-f951-40b0-bfda-95a4fe53237e-istio-token\") pod \"996aa267-f951-40b0-bfda-95a4fe53237e\" (UID: \"996aa267-f951-40b0-bfda-95a4fe53237e\") " Aug 04 15:54:17.035079 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:17.034598 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/996aa267-f951-40b0-bfda-95a4fe53237e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "996aa267-f951-40b0-bfda-95a4fe53237e" (UID: "996aa267-f951-40b0-bfda-95a4fe53237e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:54:17.035079 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:17.034633 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/996aa267-f951-40b0-bfda-95a4fe53237e-istio-envoy\") pod \"996aa267-f951-40b0-bfda-95a4fe53237e\" (UID: \"996aa267-f951-40b0-bfda-95a4fe53237e\") " Aug 04 15:54:17.035079 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:17.034638 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/996aa267-f951-40b0-bfda-95a4fe53237e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "996aa267-f951-40b0-bfda-95a4fe53237e" (UID: "996aa267-f951-40b0-bfda-95a4fe53237e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:54:17.035079 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:17.034666 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/996aa267-f951-40b0-bfda-95a4fe53237e-istiod-ca-cert\") pod \"996aa267-f951-40b0-bfda-95a4fe53237e\" (UID: \"996aa267-f951-40b0-bfda-95a4fe53237e\") " Aug 04 15:54:17.035079 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:17.034700 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fm98m\" (UniqueName: \"kubernetes.io/projected/996aa267-f951-40b0-bfda-95a4fe53237e-kube-api-access-fm98m\") pod \"996aa267-f951-40b0-bfda-95a4fe53237e\" (UID: \"996aa267-f951-40b0-bfda-95a4fe53237e\") " Aug 04 15:54:17.035079 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:17.034875 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/996aa267-f951-40b0-bfda-95a4fe53237e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "996aa267-f951-40b0-bfda-95a4fe53237e" (UID: "996aa267-f951-40b0-bfda-95a4fe53237e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:54:17.035502 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:17.035213 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/996aa267-f951-40b0-bfda-95a4fe53237e-credential-socket\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:54:17.035502 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:17.035252 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/996aa267-f951-40b0-bfda-95a4fe53237e-workload-certs\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:54:17.035502 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:17.035253 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/996aa267-f951-40b0-bfda-95a4fe53237e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "996aa267-f951-40b0-bfda-95a4fe53237e" (UID: "996aa267-f951-40b0-bfda-95a4fe53237e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 15:54:17.035502 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:17.035267 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/996aa267-f951-40b0-bfda-95a4fe53237e-workload-socket\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:54:17.035713 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:17.035586 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/996aa267-f951-40b0-bfda-95a4fe53237e-istio-data" (OuterVolumeSpecName: "istio-data") pod "996aa267-f951-40b0-bfda-95a4fe53237e" (UID: "996aa267-f951-40b0-bfda-95a4fe53237e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:54:17.037257 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:17.037235 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/996aa267-f951-40b0-bfda-95a4fe53237e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "996aa267-f951-40b0-bfda-95a4fe53237e" (UID: "996aa267-f951-40b0-bfda-95a4fe53237e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 15:54:17.037466 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:17.037444 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/996aa267-f951-40b0-bfda-95a4fe53237e-kube-api-access-fm98m" (OuterVolumeSpecName: "kube-api-access-fm98m") pod "996aa267-f951-40b0-bfda-95a4fe53237e" (UID: "996aa267-f951-40b0-bfda-95a4fe53237e"). InnerVolumeSpecName "kube-api-access-fm98m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 15:54:17.037548 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:17.037449 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/996aa267-f951-40b0-bfda-95a4fe53237e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "996aa267-f951-40b0-bfda-95a4fe53237e" (UID: "996aa267-f951-40b0-bfda-95a4fe53237e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 15:54:17.037548 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:17.037484 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/996aa267-f951-40b0-bfda-95a4fe53237e-istio-token" (OuterVolumeSpecName: "istio-token") pod "996aa267-f951-40b0-bfda-95a4fe53237e" (UID: "996aa267-f951-40b0-bfda-95a4fe53237e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 15:54:17.136626 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:17.136594 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/996aa267-f951-40b0-bfda-95a4fe53237e-istio-podinfo\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:54:17.136626 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:17.136622 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/996aa267-f951-40b0-bfda-95a4fe53237e-istio-data\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:54:17.136626 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:17.136631 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/996aa267-f951-40b0-bfda-95a4fe53237e-istio-token\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:54:17.136824 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:17.136639 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/996aa267-f951-40b0-bfda-95a4fe53237e-istio-envoy\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:54:17.136824 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:17.136648 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/996aa267-f951-40b0-bfda-95a4fe53237e-istiod-ca-cert\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:54:17.136824 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:17.136661 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fm98m\" (UniqueName: \"kubernetes.io/projected/996aa267-f951-40b0-bfda-95a4fe53237e-kube-api-access-fm98m\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 15:54:17.594921 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:17.594883 2574 generic.go:356] "Generic (PLEG): container finished" podID="996aa267-f951-40b0-bfda-95a4fe53237e" containerID="c05bb3d79c31655918d114492ef6cae44053540bce690a3504b940df5bc83932" exitCode=0 Aug 04 15:54:17.595089 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:17.594938 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" Aug 04 15:54:17.595089 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:17.594972 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" event={"ID":"996aa267-f951-40b0-bfda-95a4fe53237e","Type":"ContainerDied","Data":"c05bb3d79c31655918d114492ef6cae44053540bce690a3504b940df5bc83932"} Aug 04 15:54:17.595089 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:17.595032 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm" event={"ID":"996aa267-f951-40b0-bfda-95a4fe53237e","Type":"ContainerDied","Data":"6e1363ed7c4cc8405f621f98100d51403e1b8644ed34caf3068702a50a95be57"} Aug 04 15:54:17.595089 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:17.595058 2574 scope.go:117] "RemoveContainer" containerID="c05bb3d79c31655918d114492ef6cae44053540bce690a3504b940df5bc83932" Aug 04 15:54:17.604291 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:17.604273 2574 scope.go:117] "RemoveContainer" containerID="8bfc23151ca1b6285c54f5c3c9d3095791803562880af621b7e1e9a9096d780d" Aug 04 15:54:17.612337 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:17.612268 2574 scope.go:117] "RemoveContainer" containerID="c05bb3d79c31655918d114492ef6cae44053540bce690a3504b940df5bc83932" Aug 04 15:54:17.612589 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:54:17.612569 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c05bb3d79c31655918d114492ef6cae44053540bce690a3504b940df5bc83932\": container with ID starting with c05bb3d79c31655918d114492ef6cae44053540bce690a3504b940df5bc83932 not found: ID does not exist" containerID="c05bb3d79c31655918d114492ef6cae44053540bce690a3504b940df5bc83932" Aug 04 15:54:17.612673 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:17.612597 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c05bb3d79c31655918d114492ef6cae44053540bce690a3504b940df5bc83932"} err="failed to get container status \"c05bb3d79c31655918d114492ef6cae44053540bce690a3504b940df5bc83932\": rpc error: code = NotFound desc = could not find container \"c05bb3d79c31655918d114492ef6cae44053540bce690a3504b940df5bc83932\": container with ID starting with c05bb3d79c31655918d114492ef6cae44053540bce690a3504b940df5bc83932 not found: ID does not exist" Aug 04 15:54:17.612673 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:17.612614 2574 scope.go:117] "RemoveContainer" containerID="8bfc23151ca1b6285c54f5c3c9d3095791803562880af621b7e1e9a9096d780d" Aug 04 15:54:17.612842 ip-10-0-132-155 kubenswrapper[2574]: E0804 15:54:17.612827 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8bfc23151ca1b6285c54f5c3c9d3095791803562880af621b7e1e9a9096d780d\": container with ID starting with 8bfc23151ca1b6285c54f5c3c9d3095791803562880af621b7e1e9a9096d780d not found: ID does not exist" containerID="8bfc23151ca1b6285c54f5c3c9d3095791803562880af621b7e1e9a9096d780d" Aug 04 15:54:17.612899 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:17.612845 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8bfc23151ca1b6285c54f5c3c9d3095791803562880af621b7e1e9a9096d780d"} err="failed to get container status \"8bfc23151ca1b6285c54f5c3c9d3095791803562880af621b7e1e9a9096d780d\": rpc error: code = NotFound desc = could not find container \"8bfc23151ca1b6285c54f5c3c9d3095791803562880af621b7e1e9a9096d780d\": container with ID starting with 8bfc23151ca1b6285c54f5c3c9d3095791803562880af621b7e1e9a9096d780d not found: ID does not exist" Aug 04 15:54:17.614547 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:17.614524 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm"] Aug 04 15:54:17.618575 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:17.618554 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-df124e-openshift-default-5cbbbb6758-kn5gm"] Aug 04 15:54:19.054366 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:19.054334 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="996aa267-f951-40b0-bfda-95a4fe53237e" path="/var/lib/kubelet/pods/996aa267-f951-40b0-bfda-95a4fe53237e/volumes" Aug 04 15:54:20.416269 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:20.416234 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:54:28.668491 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:28.668456 2574 generic.go:356] "Generic (PLEG): container finished" podID="3c977f26-4e15-4788-ac7e-b0933b030ac2" containerID="91f819113cef6db375d6af01f1ce282b7078fc5dbff2e612fb150ed1b55f3461" exitCode=0 Aug 04 15:54:28.668970 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:28.668529 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" event={"ID":"3c977f26-4e15-4788-ac7e-b0933b030ac2","Type":"ContainerDied","Data":"91f819113cef6db375d6af01f1ce282b7078fc5dbff2e612fb150ed1b55f3461"} Aug 04 15:54:28.668970 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:28.668943 2574 scope.go:117] "RemoveContainer" containerID="91f819113cef6db375d6af01f1ce282b7078fc5dbff2e612fb150ed1b55f3461" Aug 04 15:54:28.669934 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:28.669906 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:54:28.669987 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:28.669975 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:54:28.670022 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:28.670009 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:54:29.677179 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:29.677146 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" event={"ID":"3c977f26-4e15-4788-ac7e-b0933b030ac2","Type":"ContainerStarted","Data":"d0d92338fcbed7591ead42b2a8c8f672a0149b2c576b22ebb9862e57b412ae72"} Aug 04 15:54:29.947559 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:29.947448 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" Aug 04 15:54:29.948966 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:29.948930 2574 patch_prober.go:28] interesting pod/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Aug 04 15:54:29.949111 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:29.949016 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" podUID="3c977f26-4e15-4788-ac7e-b0933b030ac2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Aug 04 15:54:30.582747 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:30.582715 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" Aug 04 15:54:30.948067 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:30.947959 2574 patch_prober.go:28] interesting pod/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Aug 04 15:54:30.948067 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:30.948029 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" podUID="3c977f26-4e15-4788-ac7e-b0933b030ac2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Aug 04 15:54:31.947795 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:31.947745 2574 patch_prober.go:28] interesting pod/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Aug 04 15:54:31.948031 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:31.947815 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" podUID="3c977f26-4e15-4788-ac7e-b0933b030ac2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Aug 04 15:54:32.947793 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:32.947739 2574 patch_prober.go:28] interesting pod/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Aug 04 15:54:32.948334 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:32.947828 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" podUID="3c977f26-4e15-4788-ac7e-b0933b030ac2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Aug 04 15:54:33.951007 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:33.950976 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" Aug 04 15:54:33.952046 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:33.952024 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" Aug 04 15:54:41.362283 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:41.362244 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb"] Aug 04 15:54:41.362853 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:41.362837 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="996aa267-f951-40b0-bfda-95a4fe53237e" containerName="istio-proxy" Aug 04 15:54:41.362900 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:41.362857 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="996aa267-f951-40b0-bfda-95a4fe53237e" containerName="istio-proxy" Aug 04 15:54:41.362900 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:41.362880 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="996aa267-f951-40b0-bfda-95a4fe53237e" containerName="istio-proxy" Aug 04 15:54:41.362900 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:41.362891 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="996aa267-f951-40b0-bfda-95a4fe53237e" containerName="istio-proxy" Aug 04 15:54:41.363013 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:41.363002 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="996aa267-f951-40b0-bfda-95a4fe53237e" containerName="istio-proxy" Aug 04 15:54:41.365534 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:41.365516 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" Aug 04 15:54:41.369087 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:41.369069 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-7b41f3-openshift-default-dockercfg-64jgv\"" Aug 04 15:54:41.383359 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:41.383336 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb"] Aug 04 15:54:41.455956 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:41.455910 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5f28101-5e3f-445e-ab77-dd5d594e6b12-istio-podinfo\") pod \"e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb\" (UID: \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\") " pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" Aug 04 15:54:41.456148 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:41.455982 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5f28101-5e3f-445e-ab77-dd5d594e6b12-workload-certs\") pod \"e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb\" (UID: \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\") " pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" Aug 04 15:54:41.456148 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:41.456010 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9ck5\" (UniqueName: \"kubernetes.io/projected/d5f28101-5e3f-445e-ab77-dd5d594e6b12-kube-api-access-k9ck5\") pod \"e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb\" (UID: \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\") " pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" Aug 04 15:54:41.456148 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:41.456051 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5f28101-5e3f-445e-ab77-dd5d594e6b12-istio-data\") pod \"e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb\" (UID: \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\") " pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" Aug 04 15:54:41.456148 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:41.456076 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5f28101-5e3f-445e-ab77-dd5d594e6b12-istiod-ca-cert\") pod \"e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb\" (UID: \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\") " pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" Aug 04 15:54:41.456148 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:41.456110 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5f28101-5e3f-445e-ab77-dd5d594e6b12-workload-socket\") pod \"e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb\" (UID: \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\") " pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" Aug 04 15:54:41.456337 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:41.456191 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5f28101-5e3f-445e-ab77-dd5d594e6b12-istio-envoy\") pod \"e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb\" (UID: \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\") " pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" Aug 04 15:54:41.456337 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:41.456252 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5f28101-5e3f-445e-ab77-dd5d594e6b12-istio-token\") pod \"e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb\" (UID: \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\") " pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" Aug 04 15:54:41.456337 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:41.456295 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5f28101-5e3f-445e-ab77-dd5d594e6b12-credential-socket\") pod \"e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb\" (UID: \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\") " pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" Aug 04 15:54:41.556698 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:41.556661 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5f28101-5e3f-445e-ab77-dd5d594e6b12-credential-socket\") pod \"e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb\" (UID: \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\") " pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" Aug 04 15:54:41.556885 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:41.556705 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5f28101-5e3f-445e-ab77-dd5d594e6b12-istio-podinfo\") pod \"e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb\" (UID: \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\") " pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" Aug 04 15:54:41.556885 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:41.556749 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5f28101-5e3f-445e-ab77-dd5d594e6b12-workload-certs\") pod \"e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb\" (UID: \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\") " pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" Aug 04 15:54:41.556885 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:41.556775 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9ck5\" (UniqueName: \"kubernetes.io/projected/d5f28101-5e3f-445e-ab77-dd5d594e6b12-kube-api-access-k9ck5\") pod \"e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb\" (UID: \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\") " pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" Aug 04 15:54:41.556885 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:41.556814 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5f28101-5e3f-445e-ab77-dd5d594e6b12-istio-data\") pod \"e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb\" (UID: \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\") " pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" Aug 04 15:54:41.556885 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:41.556853 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5f28101-5e3f-445e-ab77-dd5d594e6b12-istiod-ca-cert\") pod \"e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb\" (UID: \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\") " pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" Aug 04 15:54:41.557162 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:41.556893 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5f28101-5e3f-445e-ab77-dd5d594e6b12-workload-socket\") pod \"e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb\" (UID: \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\") " pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" Aug 04 15:54:41.557162 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:41.556942 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5f28101-5e3f-445e-ab77-dd5d594e6b12-istio-envoy\") pod \"e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb\" (UID: \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\") " pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" Aug 04 15:54:41.557162 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:41.556993 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5f28101-5e3f-445e-ab77-dd5d594e6b12-istio-token\") pod \"e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb\" (UID: \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\") " pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" Aug 04 15:54:41.557309 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:41.557162 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5f28101-5e3f-445e-ab77-dd5d594e6b12-workload-certs\") pod \"e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb\" (UID: \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\") " pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" Aug 04 15:54:41.557309 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:41.557208 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5f28101-5e3f-445e-ab77-dd5d594e6b12-istio-data\") pod \"e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb\" (UID: \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\") " pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" Aug 04 15:54:41.557309 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:41.557292 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5f28101-5e3f-445e-ab77-dd5d594e6b12-workload-socket\") pod \"e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb\" (UID: \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\") " pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" Aug 04 15:54:41.557442 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:41.557328 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5f28101-5e3f-445e-ab77-dd5d594e6b12-credential-socket\") pod \"e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb\" (UID: \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\") " pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" Aug 04 15:54:41.557575 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:41.557553 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5f28101-5e3f-445e-ab77-dd5d594e6b12-istiod-ca-cert\") pod \"e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb\" (UID: \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\") " pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" Aug 04 15:54:41.559164 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:41.559141 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5f28101-5e3f-445e-ab77-dd5d594e6b12-istio-envoy\") pod \"e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb\" (UID: \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\") " pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" Aug 04 15:54:41.559261 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:41.559222 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5f28101-5e3f-445e-ab77-dd5d594e6b12-istio-podinfo\") pod \"e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb\" (UID: \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\") " pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" Aug 04 15:54:41.565529 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:41.565508 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5f28101-5e3f-445e-ab77-dd5d594e6b12-istio-token\") pod \"e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb\" (UID: \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\") " pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" Aug 04 15:54:41.565826 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:41.565807 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9ck5\" (UniqueName: \"kubernetes.io/projected/d5f28101-5e3f-445e-ab77-dd5d594e6b12-kube-api-access-k9ck5\") pod \"e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb\" (UID: \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\") " pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" Aug 04 15:54:41.676304 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:41.676228 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" Aug 04 15:54:41.819880 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:41.819850 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb"] Aug 04 15:54:41.820770 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:54:41.820730 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5f28101_5e3f_445e_ab77_dd5d594e6b12.slice/crio-3462b6a3e5a35b754334014bbcb3febafccab975acb27dcc1e5499ffdddbffee WatchSource:0}: Error finding container 3462b6a3e5a35b754334014bbcb3febafccab975acb27dcc1e5499ffdddbffee: Status 404 returned error can't find the container with id 3462b6a3e5a35b754334014bbcb3febafccab975acb27dcc1e5499ffdddbffee Aug 04 15:54:41.822916 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:41.822879 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:54:41.823037 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:41.822943 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:54:41.823037 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:41.822971 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:54:42.754165 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:42.754124 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" event={"ID":"d5f28101-5e3f-445e-ab77-dd5d594e6b12","Type":"ContainerStarted","Data":"f9d4332625914b88904dfb19ae75a79187a09810b212603545413708c36a77af"} Aug 04 15:54:42.754584 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:42.754184 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" event={"ID":"d5f28101-5e3f-445e-ab77-dd5d594e6b12","Type":"ContainerStarted","Data":"3462b6a3e5a35b754334014bbcb3febafccab975acb27dcc1e5499ffdddbffee"} Aug 04 15:54:42.776757 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:42.776714 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" podStartSLOduration=1.776699862 podStartE2EDuration="1.776699862s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:54:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:54:42.77605249 +0000 UTC m=+2906.235924840" watchObservedRunningTime="2026-08-04 15:54:42.776699862 +0000 UTC m=+2906.236572211" Aug 04 15:54:43.677409 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:43.677351 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" Aug 04 15:54:43.682473 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:43.682449 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" Aug 04 15:54:43.759638 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:43.759603 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" Aug 04 15:54:43.760633 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:43.760614 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" Aug 04 15:54:51.912128 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:51.912088 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:54:59.639811 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:59.639773 2574 patch_prober.go:28] interesting pod/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": read tcp 10.132.0.2:42492->10.132.0.66:15021: read: connection reset by peer" start-of-body= Aug 04 15:54:59.640242 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:59.639841 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" podUID="d5f28101-5e3f-445e-ab77-dd5d594e6b12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": read tcp 10.132.0.2:42492->10.132.0.66:15021: read: connection reset by peer" Aug 04 15:54:59.858951 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:59.858868 2574 generic.go:356] "Generic (PLEG): container finished" podID="d5f28101-5e3f-445e-ab77-dd5d594e6b12" containerID="f9d4332625914b88904dfb19ae75a79187a09810b212603545413708c36a77af" exitCode=0 Aug 04 15:54:59.859102 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:59.858942 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" event={"ID":"d5f28101-5e3f-445e-ab77-dd5d594e6b12","Type":"ContainerDied","Data":"f9d4332625914b88904dfb19ae75a79187a09810b212603545413708c36a77af"} Aug 04 15:54:59.859344 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:59.859330 2574 scope.go:117] "RemoveContainer" containerID="f9d4332625914b88904dfb19ae75a79187a09810b212603545413708c36a77af" Aug 04 15:54:59.860342 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:59.860316 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:54:59.860433 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:59.860413 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:54:59.860484 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:54:59.860453 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 15:55:00.867333 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:00.867299 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" event={"ID":"d5f28101-5e3f-445e-ab77-dd5d594e6b12","Type":"ContainerStarted","Data":"38e167a78eefab81081111f130bfdb4a848f1243e15efe37a84f542ef1dcad05"} Aug 04 15:55:01.676895 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:01.676856 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" Aug 04 15:55:01.677077 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:01.677051 2574 patch_prober.go:28] interesting pod/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Aug 04 15:55:01.677126 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:01.677106 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" podUID="d5f28101-5e3f-445e-ab77-dd5d594e6b12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Aug 04 15:55:02.677018 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:02.676976 2574 patch_prober.go:28] interesting pod/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Aug 04 15:55:02.677517 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:02.677038 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" podUID="d5f28101-5e3f-445e-ab77-dd5d594e6b12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Aug 04 15:55:03.676797 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:03.676758 2574 patch_prober.go:28] interesting pod/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Aug 04 15:55:03.676982 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:03.676819 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" podUID="d5f28101-5e3f-445e-ab77-dd5d594e6b12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Aug 04 15:55:04.676849 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:04.676813 2574 patch_prober.go:28] interesting pod/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Aug 04 15:55:04.677254 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:04.676877 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" podUID="d5f28101-5e3f-445e-ab77-dd5d594e6b12" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Aug 04 15:55:05.680896 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:05.680867 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" Aug 04 15:55:05.681357 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:05.681326 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" Aug 04 15:55:05.682090 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:05.682071 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" Aug 04 15:55:21.997618 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:21.997583 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-05b2bc/auth-test-model-05b2bc-kserve-79b748979-sfmcm"] Aug 04 15:55:21.998077 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:21.998058 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="996aa267-f951-40b0-bfda-95a4fe53237e" containerName="istio-proxy" Aug 04 15:55:22.000705 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:22.000687 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-05b2bc/auth-test-model-05b2bc-kserve-79b748979-sfmcm" Aug 04 15:55:22.003546 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:22.003524 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-05b2bc\"/\"kube-root-ca.crt\"" Aug 04 15:55:22.004667 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:22.004639 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-05b2bc\"/\"openshift-service-ca.crt\"" Aug 04 15:55:22.004759 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:22.004677 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-05b2bc\"/\"default-dockercfg-4nsxr\"" Aug 04 15:55:22.004826 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:22.004767 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-05b2bc\"/\"auth-test-model-05b2bc-kserve-self-signed-certs\"" Aug 04 15:55:22.011097 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:22.011074 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-05b2bc/auth-test-model-05b2bc-kserve-79b748979-sfmcm"] Aug 04 15:55:22.127144 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:22.127110 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/253fe340-d11c-4a1e-a36f-5defa59a27dd-tmp-dir\") pod \"auth-test-model-05b2bc-kserve-79b748979-sfmcm\" (UID: \"253fe340-d11c-4a1e-a36f-5defa59a27dd\") " pod="ai-tenant-e2e-shared-a-05b2bc/auth-test-model-05b2bc-kserve-79b748979-sfmcm" Aug 04 15:55:22.127318 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:22.127152 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/253fe340-d11c-4a1e-a36f-5defa59a27dd-tls-certs\") pod \"auth-test-model-05b2bc-kserve-79b748979-sfmcm\" (UID: \"253fe340-d11c-4a1e-a36f-5defa59a27dd\") " pod="ai-tenant-e2e-shared-a-05b2bc/auth-test-model-05b2bc-kserve-79b748979-sfmcm" Aug 04 15:55:22.127318 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:22.127269 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9zbh\" (UniqueName: \"kubernetes.io/projected/253fe340-d11c-4a1e-a36f-5defa59a27dd-kube-api-access-h9zbh\") pod \"auth-test-model-05b2bc-kserve-79b748979-sfmcm\" (UID: \"253fe340-d11c-4a1e-a36f-5defa59a27dd\") " pod="ai-tenant-e2e-shared-a-05b2bc/auth-test-model-05b2bc-kserve-79b748979-sfmcm" Aug 04 15:55:22.127429 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:22.127334 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/253fe340-d11c-4a1e-a36f-5defa59a27dd-dshm\") pod \"auth-test-model-05b2bc-kserve-79b748979-sfmcm\" (UID: \"253fe340-d11c-4a1e-a36f-5defa59a27dd\") " pod="ai-tenant-e2e-shared-a-05b2bc/auth-test-model-05b2bc-kserve-79b748979-sfmcm" Aug 04 15:55:22.127429 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:22.127358 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/253fe340-d11c-4a1e-a36f-5defa59a27dd-home\") pod \"auth-test-model-05b2bc-kserve-79b748979-sfmcm\" (UID: \"253fe340-d11c-4a1e-a36f-5defa59a27dd\") " pod="ai-tenant-e2e-shared-a-05b2bc/auth-test-model-05b2bc-kserve-79b748979-sfmcm" Aug 04 15:55:22.127429 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:22.127408 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/253fe340-d11c-4a1e-a36f-5defa59a27dd-model-cache\") pod \"auth-test-model-05b2bc-kserve-79b748979-sfmcm\" (UID: \"253fe340-d11c-4a1e-a36f-5defa59a27dd\") " pod="ai-tenant-e2e-shared-a-05b2bc/auth-test-model-05b2bc-kserve-79b748979-sfmcm" Aug 04 15:55:22.227976 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:22.227941 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9zbh\" (UniqueName: \"kubernetes.io/projected/253fe340-d11c-4a1e-a36f-5defa59a27dd-kube-api-access-h9zbh\") pod \"auth-test-model-05b2bc-kserve-79b748979-sfmcm\" (UID: \"253fe340-d11c-4a1e-a36f-5defa59a27dd\") " pod="ai-tenant-e2e-shared-a-05b2bc/auth-test-model-05b2bc-kserve-79b748979-sfmcm" Aug 04 15:55:22.228165 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:22.227999 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/253fe340-d11c-4a1e-a36f-5defa59a27dd-dshm\") pod \"auth-test-model-05b2bc-kserve-79b748979-sfmcm\" (UID: \"253fe340-d11c-4a1e-a36f-5defa59a27dd\") " pod="ai-tenant-e2e-shared-a-05b2bc/auth-test-model-05b2bc-kserve-79b748979-sfmcm" Aug 04 15:55:22.228165 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:22.228023 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/253fe340-d11c-4a1e-a36f-5defa59a27dd-home\") pod \"auth-test-model-05b2bc-kserve-79b748979-sfmcm\" (UID: \"253fe340-d11c-4a1e-a36f-5defa59a27dd\") " pod="ai-tenant-e2e-shared-a-05b2bc/auth-test-model-05b2bc-kserve-79b748979-sfmcm" Aug 04 15:55:22.228165 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:22.228070 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/253fe340-d11c-4a1e-a36f-5defa59a27dd-model-cache\") pod \"auth-test-model-05b2bc-kserve-79b748979-sfmcm\" (UID: \"253fe340-d11c-4a1e-a36f-5defa59a27dd\") " pod="ai-tenant-e2e-shared-a-05b2bc/auth-test-model-05b2bc-kserve-79b748979-sfmcm" Aug 04 15:55:22.228165 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:22.228108 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/253fe340-d11c-4a1e-a36f-5defa59a27dd-tmp-dir\") pod \"auth-test-model-05b2bc-kserve-79b748979-sfmcm\" (UID: \"253fe340-d11c-4a1e-a36f-5defa59a27dd\") " pod="ai-tenant-e2e-shared-a-05b2bc/auth-test-model-05b2bc-kserve-79b748979-sfmcm" Aug 04 15:55:22.228165 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:22.228148 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/253fe340-d11c-4a1e-a36f-5defa59a27dd-tls-certs\") pod \"auth-test-model-05b2bc-kserve-79b748979-sfmcm\" (UID: \"253fe340-d11c-4a1e-a36f-5defa59a27dd\") " pod="ai-tenant-e2e-shared-a-05b2bc/auth-test-model-05b2bc-kserve-79b748979-sfmcm" Aug 04 15:55:22.228574 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:22.228554 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/253fe340-d11c-4a1e-a36f-5defa59a27dd-home\") pod \"auth-test-model-05b2bc-kserve-79b748979-sfmcm\" (UID: \"253fe340-d11c-4a1e-a36f-5defa59a27dd\") " pod="ai-tenant-e2e-shared-a-05b2bc/auth-test-model-05b2bc-kserve-79b748979-sfmcm" Aug 04 15:55:22.228638 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:22.228610 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/253fe340-d11c-4a1e-a36f-5defa59a27dd-tmp-dir\") pod \"auth-test-model-05b2bc-kserve-79b748979-sfmcm\" (UID: \"253fe340-d11c-4a1e-a36f-5defa59a27dd\") " pod="ai-tenant-e2e-shared-a-05b2bc/auth-test-model-05b2bc-kserve-79b748979-sfmcm" Aug 04 15:55:22.228684 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:22.228668 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/253fe340-d11c-4a1e-a36f-5defa59a27dd-model-cache\") pod \"auth-test-model-05b2bc-kserve-79b748979-sfmcm\" (UID: \"253fe340-d11c-4a1e-a36f-5defa59a27dd\") " pod="ai-tenant-e2e-shared-a-05b2bc/auth-test-model-05b2bc-kserve-79b748979-sfmcm" Aug 04 15:55:22.230381 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:22.230355 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/253fe340-d11c-4a1e-a36f-5defa59a27dd-dshm\") pod \"auth-test-model-05b2bc-kserve-79b748979-sfmcm\" (UID: \"253fe340-d11c-4a1e-a36f-5defa59a27dd\") " pod="ai-tenant-e2e-shared-a-05b2bc/auth-test-model-05b2bc-kserve-79b748979-sfmcm" Aug 04 15:55:22.230629 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:22.230614 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/253fe340-d11c-4a1e-a36f-5defa59a27dd-tls-certs\") pod \"auth-test-model-05b2bc-kserve-79b748979-sfmcm\" (UID: \"253fe340-d11c-4a1e-a36f-5defa59a27dd\") " pod="ai-tenant-e2e-shared-a-05b2bc/auth-test-model-05b2bc-kserve-79b748979-sfmcm" Aug 04 15:55:22.237796 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:22.237772 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9zbh\" (UniqueName: \"kubernetes.io/projected/253fe340-d11c-4a1e-a36f-5defa59a27dd-kube-api-access-h9zbh\") pod \"auth-test-model-05b2bc-kserve-79b748979-sfmcm\" (UID: \"253fe340-d11c-4a1e-a36f-5defa59a27dd\") " pod="ai-tenant-e2e-shared-a-05b2bc/auth-test-model-05b2bc-kserve-79b748979-sfmcm" Aug 04 15:55:22.310767 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:22.310699 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-05b2bc/auth-test-model-05b2bc-kserve-79b748979-sfmcm" Aug 04 15:55:22.448263 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:22.448238 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-05b2bc/auth-test-model-05b2bc-kserve-79b748979-sfmcm"] Aug 04 15:55:22.450010 ip-10-0-132-155 kubenswrapper[2574]: W0804 15:55:22.449978 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod253fe340_d11c_4a1e_a36f_5defa59a27dd.slice/crio-171bd27601bff4a3b6e7d2bc8800fa1f92d6f5d3a92e7bc997e52ca0455badd5 WatchSource:0}: Error finding container 171bd27601bff4a3b6e7d2bc8800fa1f92d6f5d3a92e7bc997e52ca0455badd5: Status 404 returned error can't find the container with id 171bd27601bff4a3b6e7d2bc8800fa1f92d6f5d3a92e7bc997e52ca0455badd5 Aug 04 15:55:22.994989 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:22.994947 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-05b2bc/auth-test-model-05b2bc-kserve-79b748979-sfmcm" event={"ID":"253fe340-d11c-4a1e-a36f-5defa59a27dd","Type":"ContainerStarted","Data":"cd14af39c2a458efe9130e93cda7ae486bd0fe99e874e5d992c08086f86247d2"} Aug 04 15:55:22.995161 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:22.994998 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-05b2bc/auth-test-model-05b2bc-kserve-79b748979-sfmcm" event={"ID":"253fe340-d11c-4a1e-a36f-5defa59a27dd","Type":"ContainerStarted","Data":"171bd27601bff4a3b6e7d2bc8800fa1f92d6f5d3a92e7bc997e52ca0455badd5"} Aug 04 15:55:23.014780 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:23.014732 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-05b2bc/auth-test-model-05b2bc-kserve-79b748979-sfmcm" podStartSLOduration=2.014718957 podStartE2EDuration="2.014718957s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 15:55:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 15:55:23.013367565 +0000 UTC m=+2946.473239913" watchObservedRunningTime="2026-08-04 15:55:23.014718957 +0000 UTC m=+2946.474591305" Aug 04 15:55:25.413672 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:25.413636 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:55:32.311464 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:32.311429 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-05b2bc/auth-test-model-05b2bc-kserve-79b748979-sfmcm" Aug 04 15:55:32.311880 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:32.311686 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-05b2bc/auth-test-model-05b2bc-kserve-79b748979-sfmcm" Aug 04 15:55:32.324130 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:32.324108 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-05b2bc/auth-test-model-05b2bc-kserve-79b748979-sfmcm" Aug 04 15:55:33.064205 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:33.064177 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-05b2bc/auth-test-model-05b2bc-kserve-79b748979-sfmcm" Aug 04 15:55:46.535294 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:55:46.535203 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:56:17.301014 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:56:17.300983 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-r8wkp_0580fdab-0381-4d8e-be5f-bde5eb44e0e6/console-operator/1.log" Aug 04 15:56:17.304934 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:56:17.304910 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8h6k_9c62e3b1-c2dc-4b2a-a61d-7d80049f355f/ovn-acl-logging/0.log" Aug 04 15:56:17.309850 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:56:17.309831 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-r8wkp_0580fdab-0381-4d8e-be5f-bde5eb44e0e6/console-operator/1.log" Aug 04 15:56:17.320187 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:56:17.320167 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8h6k_9c62e3b1-c2dc-4b2a-a61d-7d80049f355f/ovn-acl-logging/0.log" Aug 04 15:56:21.224447 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:56:21.224412 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:56:47.425245 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:56:47.425211 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:57:04.948965 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:57:04.948928 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:57:31.718693 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:57:31.718655 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:57:48.515657 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:57:48.515622 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:58:01.220029 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:58:01.219994 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:58:27.828466 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:58:27.828430 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:58:46.113606 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:58:46.113525 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:59:12.135149 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:59:12.135111 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:59:29.022039 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:59:29.022001 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 15:59:54.318060 ip-10-0-132-155 kubenswrapper[2574]: I0804 15:59:54.318023 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 16:00:13.211116 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:00:13.211079 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 16:00:41.116987 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:00:41.116950 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 16:00:43.982893 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:00:43.982851 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-05b2bc/auth-test-model-05b2bc-kserve-79b748979-sfmcm"] Aug 04 16:00:56.818992 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:00:56.818955 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 16:00:59.002557 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:00:59.002495 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-05b2bc/auth-test-model-05b2bc-kserve-79b748979-sfmcm" podUID="253fe340-d11c-4a1e-a36f-5defa59a27dd" containerName="main" containerID="cri-o://cd14af39c2a458efe9130e93cda7ae486bd0fe99e874e5d992c08086f86247d2" gracePeriod=45 Aug 04 16:00:59.265858 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:00:59.265797 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-05b2bc/auth-test-model-05b2bc-kserve-79b748979-sfmcm" Aug 04 16:00:59.371258 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:00:59.371215 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/253fe340-d11c-4a1e-a36f-5defa59a27dd-tls-certs\") pod \"253fe340-d11c-4a1e-a36f-5defa59a27dd\" (UID: \"253fe340-d11c-4a1e-a36f-5defa59a27dd\") " Aug 04 16:00:59.371258 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:00:59.371261 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/253fe340-d11c-4a1e-a36f-5defa59a27dd-dshm\") pod \"253fe340-d11c-4a1e-a36f-5defa59a27dd\" (UID: \"253fe340-d11c-4a1e-a36f-5defa59a27dd\") " Aug 04 16:00:59.371556 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:00:59.371342 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/253fe340-d11c-4a1e-a36f-5defa59a27dd-tmp-dir\") pod \"253fe340-d11c-4a1e-a36f-5defa59a27dd\" (UID: \"253fe340-d11c-4a1e-a36f-5defa59a27dd\") " Aug 04 16:00:59.371556 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:00:59.371376 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h9zbh\" (UniqueName: \"kubernetes.io/projected/253fe340-d11c-4a1e-a36f-5defa59a27dd-kube-api-access-h9zbh\") pod \"253fe340-d11c-4a1e-a36f-5defa59a27dd\" (UID: \"253fe340-d11c-4a1e-a36f-5defa59a27dd\") " Aug 04 16:00:59.371556 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:00:59.371440 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/253fe340-d11c-4a1e-a36f-5defa59a27dd-model-cache\") pod \"253fe340-d11c-4a1e-a36f-5defa59a27dd\" (UID: \"253fe340-d11c-4a1e-a36f-5defa59a27dd\") " Aug 04 16:00:59.371556 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:00:59.371498 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/253fe340-d11c-4a1e-a36f-5defa59a27dd-home\") pod \"253fe340-d11c-4a1e-a36f-5defa59a27dd\" (UID: \"253fe340-d11c-4a1e-a36f-5defa59a27dd\") " Aug 04 16:00:59.371760 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:00:59.371678 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/253fe340-d11c-4a1e-a36f-5defa59a27dd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "253fe340-d11c-4a1e-a36f-5defa59a27dd" (UID: "253fe340-d11c-4a1e-a36f-5defa59a27dd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:00:59.371808 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:00:59.371766 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/253fe340-d11c-4a1e-a36f-5defa59a27dd-model-cache" (OuterVolumeSpecName: "model-cache") pod "253fe340-d11c-4a1e-a36f-5defa59a27dd" (UID: "253fe340-d11c-4a1e-a36f-5defa59a27dd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:00:59.371859 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:00:59.371804 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/253fe340-d11c-4a1e-a36f-5defa59a27dd-home" (OuterVolumeSpecName: "home") pod "253fe340-d11c-4a1e-a36f-5defa59a27dd" (UID: "253fe340-d11c-4a1e-a36f-5defa59a27dd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:00:59.371907 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:00:59.371878 2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/253fe340-d11c-4a1e-a36f-5defa59a27dd-tmp-dir\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:00:59.371907 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:00:59.371900 2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/253fe340-d11c-4a1e-a36f-5defa59a27dd-model-cache\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:00:59.373660 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:00:59.373635 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/253fe340-d11c-4a1e-a36f-5defa59a27dd-dshm" (OuterVolumeSpecName: "dshm") pod "253fe340-d11c-4a1e-a36f-5defa59a27dd" (UID: "253fe340-d11c-4a1e-a36f-5defa59a27dd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:00:59.373773 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:00:59.373672 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/253fe340-d11c-4a1e-a36f-5defa59a27dd-kube-api-access-h9zbh" (OuterVolumeSpecName: "kube-api-access-h9zbh") pod "253fe340-d11c-4a1e-a36f-5defa59a27dd" (UID: "253fe340-d11c-4a1e-a36f-5defa59a27dd"). InnerVolumeSpecName "kube-api-access-h9zbh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:00:59.373773 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:00:59.373691 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/253fe340-d11c-4a1e-a36f-5defa59a27dd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "253fe340-d11c-4a1e-a36f-5defa59a27dd" (UID: "253fe340-d11c-4a1e-a36f-5defa59a27dd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 16:00:59.472567 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:00:59.472536 2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/253fe340-d11c-4a1e-a36f-5defa59a27dd-tls-certs\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:00:59.472567 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:00:59.472562 2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/253fe340-d11c-4a1e-a36f-5defa59a27dd-dshm\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:00:59.472567 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:00:59.472572 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9zbh\" (UniqueName: \"kubernetes.io/projected/253fe340-d11c-4a1e-a36f-5defa59a27dd-kube-api-access-h9zbh\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:00:59.472796 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:00:59.472583 2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/253fe340-d11c-4a1e-a36f-5defa59a27dd-home\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:00:59.886777 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:00:59.886745 2574 generic.go:356] "Generic (PLEG): container finished" podID="253fe340-d11c-4a1e-a36f-5defa59a27dd" containerID="cd14af39c2a458efe9130e93cda7ae486bd0fe99e874e5d992c08086f86247d2" exitCode=0 Aug 04 16:00:59.886946 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:00:59.886790 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-05b2bc/auth-test-model-05b2bc-kserve-79b748979-sfmcm" Aug 04 16:00:59.886946 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:00:59.886816 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-05b2bc/auth-test-model-05b2bc-kserve-79b748979-sfmcm" event={"ID":"253fe340-d11c-4a1e-a36f-5defa59a27dd","Type":"ContainerDied","Data":"cd14af39c2a458efe9130e93cda7ae486bd0fe99e874e5d992c08086f86247d2"} Aug 04 16:00:59.886946 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:00:59.886847 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-05b2bc/auth-test-model-05b2bc-kserve-79b748979-sfmcm" event={"ID":"253fe340-d11c-4a1e-a36f-5defa59a27dd","Type":"ContainerDied","Data":"171bd27601bff4a3b6e7d2bc8800fa1f92d6f5d3a92e7bc997e52ca0455badd5"} Aug 04 16:00:59.886946 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:00:59.886864 2574 scope.go:117] "RemoveContainer" containerID="cd14af39c2a458efe9130e93cda7ae486bd0fe99e874e5d992c08086f86247d2" Aug 04 16:00:59.897337 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:00:59.897319 2574 scope.go:117] "RemoveContainer" containerID="cd14af39c2a458efe9130e93cda7ae486bd0fe99e874e5d992c08086f86247d2" Aug 04 16:00:59.897612 ip-10-0-132-155 kubenswrapper[2574]: E0804 16:00:59.897592 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd14af39c2a458efe9130e93cda7ae486bd0fe99e874e5d992c08086f86247d2\": container with ID starting with cd14af39c2a458efe9130e93cda7ae486bd0fe99e874e5d992c08086f86247d2 not found: ID does not exist" containerID="cd14af39c2a458efe9130e93cda7ae486bd0fe99e874e5d992c08086f86247d2" Aug 04 16:00:59.897668 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:00:59.897620 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd14af39c2a458efe9130e93cda7ae486bd0fe99e874e5d992c08086f86247d2"} err="failed to get container status \"cd14af39c2a458efe9130e93cda7ae486bd0fe99e874e5d992c08086f86247d2\": rpc error: code = NotFound desc = could not find container \"cd14af39c2a458efe9130e93cda7ae486bd0fe99e874e5d992c08086f86247d2\": container with ID starting with cd14af39c2a458efe9130e93cda7ae486bd0fe99e874e5d992c08086f86247d2 not found: ID does not exist" Aug 04 16:00:59.909085 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:00:59.909063 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-05b2bc/auth-test-model-05b2bc-kserve-79b748979-sfmcm"] Aug 04 16:00:59.913194 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:00:59.913173 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-05b2bc/auth-test-model-05b2bc-kserve-79b748979-sfmcm"] Aug 04 16:01:01.054208 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:01:01.054166 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="253fe340-d11c-4a1e-a36f-5defa59a27dd" path="/var/lib/kubelet/pods/253fe340-d11c-4a1e-a36f-5defa59a27dd/volumes" Aug 04 16:01:05.914350 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:01:05.914315 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 16:01:17.336950 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:01:17.336920 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-r8wkp_0580fdab-0381-4d8e-be5f-bde5eb44e0e6/console-operator/1.log" Aug 04 16:01:17.341065 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:01:17.341042 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8h6k_9c62e3b1-c2dc-4b2a-a61d-7d80049f355f/ovn-acl-logging/0.log" Aug 04 16:01:17.353567 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:01:17.353551 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-r8wkp_0580fdab-0381-4d8e-be5f-bde5eb44e0e6/console-operator/1.log" Aug 04 16:01:17.357265 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:01:17.357248 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8h6k_9c62e3b1-c2dc-4b2a-a61d-7d80049f355f/ovn-acl-logging/0.log" Aug 04 16:01:28.817462 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:01:28.817420 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 16:02:24.313948 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:02:24.313911 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 16:02:43.931145 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:02:43.931111 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 16:03:06.729735 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:03:06.729696 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 16:03:19.414449 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:03:19.414352 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-05b2bc/rate-test-model-05b2bc-kserve-544cd576c9-l7pzl"] Aug 04 16:03:19.414833 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:03:19.414762 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="253fe340-d11c-4a1e-a36f-5defa59a27dd" containerName="main" Aug 04 16:03:19.414833 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:03:19.414773 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="253fe340-d11c-4a1e-a36f-5defa59a27dd" containerName="main" Aug 04 16:03:19.414911 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:03:19.414848 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="253fe340-d11c-4a1e-a36f-5defa59a27dd" containerName="main" Aug 04 16:03:19.417928 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:03:19.417912 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-05b2bc/rate-test-model-05b2bc-kserve-544cd576c9-l7pzl" Aug 04 16:03:19.421928 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:03:19.421891 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-05b2bc\"/\"kube-root-ca.crt\"" Aug 04 16:03:19.421928 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:03:19.421910 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-05b2bc\"/\"openshift-service-ca.crt\"" Aug 04 16:03:19.421928 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:03:19.421916 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-05b2bc\"/\"default-dockercfg-4nsxr\"" Aug 04 16:03:19.422147 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:03:19.422010 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-05b2bc\"/\"rate-test-model-05b2bc-kserve-self-signed-certs\"" Aug 04 16:03:19.432654 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:03:19.432631 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-05b2bc/rate-test-model-05b2bc-kserve-544cd576c9-l7pzl"] Aug 04 16:03:19.537843 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:03:19.537807 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/886ec14d-0ca4-42a3-b81b-e0bc29560783-model-cache\") pod \"rate-test-model-05b2bc-kserve-544cd576c9-l7pzl\" (UID: \"886ec14d-0ca4-42a3-b81b-e0bc29560783\") " pod="ai-tenant-e2e-shared-a-05b2bc/rate-test-model-05b2bc-kserve-544cd576c9-l7pzl" Aug 04 16:03:19.537990 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:03:19.537876 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/886ec14d-0ca4-42a3-b81b-e0bc29560783-tmp-dir\") pod \"rate-test-model-05b2bc-kserve-544cd576c9-l7pzl\" (UID: \"886ec14d-0ca4-42a3-b81b-e0bc29560783\") " pod="ai-tenant-e2e-shared-a-05b2bc/rate-test-model-05b2bc-kserve-544cd576c9-l7pzl" Aug 04 16:03:19.537990 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:03:19.537905 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnsvq\" (UniqueName: \"kubernetes.io/projected/886ec14d-0ca4-42a3-b81b-e0bc29560783-kube-api-access-gnsvq\") pod \"rate-test-model-05b2bc-kserve-544cd576c9-l7pzl\" (UID: \"886ec14d-0ca4-42a3-b81b-e0bc29560783\") " pod="ai-tenant-e2e-shared-a-05b2bc/rate-test-model-05b2bc-kserve-544cd576c9-l7pzl" Aug 04 16:03:19.537990 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:03:19.537927 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/886ec14d-0ca4-42a3-b81b-e0bc29560783-home\") pod \"rate-test-model-05b2bc-kserve-544cd576c9-l7pzl\" (UID: \"886ec14d-0ca4-42a3-b81b-e0bc29560783\") " pod="ai-tenant-e2e-shared-a-05b2bc/rate-test-model-05b2bc-kserve-544cd576c9-l7pzl" Aug 04 16:03:19.537990 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:03:19.537951 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/886ec14d-0ca4-42a3-b81b-e0bc29560783-tls-certs\") pod \"rate-test-model-05b2bc-kserve-544cd576c9-l7pzl\" (UID: \"886ec14d-0ca4-42a3-b81b-e0bc29560783\") " pod="ai-tenant-e2e-shared-a-05b2bc/rate-test-model-05b2bc-kserve-544cd576c9-l7pzl" Aug 04 16:03:19.538183 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:03:19.538010 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/886ec14d-0ca4-42a3-b81b-e0bc29560783-dshm\") pod \"rate-test-model-05b2bc-kserve-544cd576c9-l7pzl\" (UID: \"886ec14d-0ca4-42a3-b81b-e0bc29560783\") " pod="ai-tenant-e2e-shared-a-05b2bc/rate-test-model-05b2bc-kserve-544cd576c9-l7pzl" Aug 04 16:03:19.638561 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:03:19.638524 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/886ec14d-0ca4-42a3-b81b-e0bc29560783-tmp-dir\") pod \"rate-test-model-05b2bc-kserve-544cd576c9-l7pzl\" (UID: \"886ec14d-0ca4-42a3-b81b-e0bc29560783\") " pod="ai-tenant-e2e-shared-a-05b2bc/rate-test-model-05b2bc-kserve-544cd576c9-l7pzl" Aug 04 16:03:19.638758 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:03:19.638577 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnsvq\" (UniqueName: \"kubernetes.io/projected/886ec14d-0ca4-42a3-b81b-e0bc29560783-kube-api-access-gnsvq\") pod \"rate-test-model-05b2bc-kserve-544cd576c9-l7pzl\" (UID: \"886ec14d-0ca4-42a3-b81b-e0bc29560783\") " pod="ai-tenant-e2e-shared-a-05b2bc/rate-test-model-05b2bc-kserve-544cd576c9-l7pzl" Aug 04 16:03:19.638758 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:03:19.638609 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/886ec14d-0ca4-42a3-b81b-e0bc29560783-home\") pod \"rate-test-model-05b2bc-kserve-544cd576c9-l7pzl\" (UID: \"886ec14d-0ca4-42a3-b81b-e0bc29560783\") " pod="ai-tenant-e2e-shared-a-05b2bc/rate-test-model-05b2bc-kserve-544cd576c9-l7pzl" Aug 04 16:03:19.638758 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:03:19.638645 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/886ec14d-0ca4-42a3-b81b-e0bc29560783-tls-certs\") pod \"rate-test-model-05b2bc-kserve-544cd576c9-l7pzl\" (UID: \"886ec14d-0ca4-42a3-b81b-e0bc29560783\") " pod="ai-tenant-e2e-shared-a-05b2bc/rate-test-model-05b2bc-kserve-544cd576c9-l7pzl" Aug 04 16:03:19.638924 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:03:19.638760 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/886ec14d-0ca4-42a3-b81b-e0bc29560783-dshm\") pod \"rate-test-model-05b2bc-kserve-544cd576c9-l7pzl\" (UID: \"886ec14d-0ca4-42a3-b81b-e0bc29560783\") " pod="ai-tenant-e2e-shared-a-05b2bc/rate-test-model-05b2bc-kserve-544cd576c9-l7pzl" Aug 04 16:03:19.638924 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:03:19.638849 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/886ec14d-0ca4-42a3-b81b-e0bc29560783-model-cache\") pod \"rate-test-model-05b2bc-kserve-544cd576c9-l7pzl\" (UID: \"886ec14d-0ca4-42a3-b81b-e0bc29560783\") " pod="ai-tenant-e2e-shared-a-05b2bc/rate-test-model-05b2bc-kserve-544cd576c9-l7pzl" Aug 04 16:03:19.639036 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:03:19.638998 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/886ec14d-0ca4-42a3-b81b-e0bc29560783-tmp-dir\") pod \"rate-test-model-05b2bc-kserve-544cd576c9-l7pzl\" (UID: \"886ec14d-0ca4-42a3-b81b-e0bc29560783\") " pod="ai-tenant-e2e-shared-a-05b2bc/rate-test-model-05b2bc-kserve-544cd576c9-l7pzl" Aug 04 16:03:19.639097 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:03:19.639038 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/886ec14d-0ca4-42a3-b81b-e0bc29560783-home\") pod \"rate-test-model-05b2bc-kserve-544cd576c9-l7pzl\" (UID: \"886ec14d-0ca4-42a3-b81b-e0bc29560783\") " pod="ai-tenant-e2e-shared-a-05b2bc/rate-test-model-05b2bc-kserve-544cd576c9-l7pzl" Aug 04 16:03:19.639230 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:03:19.639210 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/886ec14d-0ca4-42a3-b81b-e0bc29560783-model-cache\") pod \"rate-test-model-05b2bc-kserve-544cd576c9-l7pzl\" (UID: \"886ec14d-0ca4-42a3-b81b-e0bc29560783\") " pod="ai-tenant-e2e-shared-a-05b2bc/rate-test-model-05b2bc-kserve-544cd576c9-l7pzl" Aug 04 16:03:19.641164 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:03:19.641138 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/886ec14d-0ca4-42a3-b81b-e0bc29560783-dshm\") pod \"rate-test-model-05b2bc-kserve-544cd576c9-l7pzl\" (UID: \"886ec14d-0ca4-42a3-b81b-e0bc29560783\") " pod="ai-tenant-e2e-shared-a-05b2bc/rate-test-model-05b2bc-kserve-544cd576c9-l7pzl" Aug 04 16:03:19.641383 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:03:19.641363 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/886ec14d-0ca4-42a3-b81b-e0bc29560783-tls-certs\") pod \"rate-test-model-05b2bc-kserve-544cd576c9-l7pzl\" (UID: \"886ec14d-0ca4-42a3-b81b-e0bc29560783\") " pod="ai-tenant-e2e-shared-a-05b2bc/rate-test-model-05b2bc-kserve-544cd576c9-l7pzl" Aug 04 16:03:19.649070 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:03:19.649050 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnsvq\" (UniqueName: \"kubernetes.io/projected/886ec14d-0ca4-42a3-b81b-e0bc29560783-kube-api-access-gnsvq\") pod \"rate-test-model-05b2bc-kserve-544cd576c9-l7pzl\" (UID: \"886ec14d-0ca4-42a3-b81b-e0bc29560783\") " pod="ai-tenant-e2e-shared-a-05b2bc/rate-test-model-05b2bc-kserve-544cd576c9-l7pzl" Aug 04 16:03:19.728559 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:03:19.728473 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-05b2bc/rate-test-model-05b2bc-kserve-544cd576c9-l7pzl" Aug 04 16:03:19.884319 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:03:19.884292 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-05b2bc/rate-test-model-05b2bc-kserve-544cd576c9-l7pzl"] Aug 04 16:03:19.885444 ip-10-0-132-155 kubenswrapper[2574]: W0804 16:03:19.885412 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod886ec14d_0ca4_42a3_b81b_e0bc29560783.slice/crio-9cc48d4689c584b331bbed4bbb462c1c8066fb2cd08ca926358da9f1edca27da WatchSource:0}: Error finding container 9cc48d4689c584b331bbed4bbb462c1c8066fb2cd08ca926358da9f1edca27da: Status 404 returned error can't find the container with id 9cc48d4689c584b331bbed4bbb462c1c8066fb2cd08ca926358da9f1edca27da Aug 04 16:03:20.670577 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:03:20.670540 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-05b2bc/rate-test-model-05b2bc-kserve-544cd576c9-l7pzl" event={"ID":"886ec14d-0ca4-42a3-b81b-e0bc29560783","Type":"ContainerStarted","Data":"9cedf27b47b8a3cf737e8406a59fe9df7531478d1203ea6ff44479fa1f3c0062"} Aug 04 16:03:20.671037 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:03:20.670585 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-05b2bc/rate-test-model-05b2bc-kserve-544cd576c9-l7pzl" event={"ID":"886ec14d-0ca4-42a3-b81b-e0bc29560783","Type":"ContainerStarted","Data":"9cc48d4689c584b331bbed4bbb462c1c8066fb2cd08ca926358da9f1edca27da"} Aug 04 16:03:20.696293 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:03:20.696245 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-05b2bc/rate-test-model-05b2bc-kserve-544cd576c9-l7pzl" podStartSLOduration=1.696230979 podStartE2EDuration="1.696230979s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:03:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:03:20.69400008 +0000 UTC m=+3424.153872432" watchObservedRunningTime="2026-08-04 16:03:20.696230979 +0000 UTC m=+3424.156103324" Aug 04 16:03:23.542046 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:03:23.542005 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 16:03:29.729190 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:03:29.729136 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-05b2bc/rate-test-model-05b2bc-kserve-544cd576c9-l7pzl" Aug 04 16:03:29.729632 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:03:29.729202 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-05b2bc/rate-test-model-05b2bc-kserve-544cd576c9-l7pzl" Aug 04 16:03:29.741746 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:03:29.741713 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-05b2bc/rate-test-model-05b2bc-kserve-544cd576c9-l7pzl" Aug 04 16:03:30.739854 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:03:30.739824 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-05b2bc/rate-test-model-05b2bc-kserve-544cd576c9-l7pzl" Aug 04 16:03:44.720094 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:03:44.720059 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 16:04:41.225172 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:04:41.225093 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 16:04:55.114995 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:04:55.114955 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 16:05:23.624216 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:23.624181 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 16:05:29.977115 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:29.977073 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-05b2bc/rate-test-model-05b2bc-kserve-544cd576c9-l7pzl"] Aug 04 16:05:37.527635 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:37.527602 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 16:05:44.996059 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:44.995999 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-05b2bc/rate-test-model-05b2bc-kserve-544cd576c9-l7pzl" podUID="886ec14d-0ca4-42a3-b81b-e0bc29560783" containerName="main" containerID="cri-o://9cedf27b47b8a3cf737e8406a59fe9df7531478d1203ea6ff44479fa1f3c0062" gracePeriod=45 Aug 04 16:05:45.250486 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:45.250430 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-05b2bc/rate-test-model-05b2bc-kserve-544cd576c9-l7pzl" Aug 04 16:05:45.390532 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:45.390498 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/886ec14d-0ca4-42a3-b81b-e0bc29560783-model-cache\") pod \"886ec14d-0ca4-42a3-b81b-e0bc29560783\" (UID: \"886ec14d-0ca4-42a3-b81b-e0bc29560783\") " Aug 04 16:05:45.390751 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:45.390599 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/886ec14d-0ca4-42a3-b81b-e0bc29560783-home\") pod \"886ec14d-0ca4-42a3-b81b-e0bc29560783\" (UID: \"886ec14d-0ca4-42a3-b81b-e0bc29560783\") " Aug 04 16:05:45.390751 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:45.390647 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/886ec14d-0ca4-42a3-b81b-e0bc29560783-tmp-dir\") pod \"886ec14d-0ca4-42a3-b81b-e0bc29560783\" (UID: \"886ec14d-0ca4-42a3-b81b-e0bc29560783\") " Aug 04 16:05:45.390751 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:45.390669 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/886ec14d-0ca4-42a3-b81b-e0bc29560783-dshm\") pod \"886ec14d-0ca4-42a3-b81b-e0bc29560783\" (UID: \"886ec14d-0ca4-42a3-b81b-e0bc29560783\") " Aug 04 16:05:45.390751 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:45.390691 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/886ec14d-0ca4-42a3-b81b-e0bc29560783-tls-certs\") pod \"886ec14d-0ca4-42a3-b81b-e0bc29560783\" (UID: \"886ec14d-0ca4-42a3-b81b-e0bc29560783\") " Aug 04 16:05:45.390751 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:45.390722 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gnsvq\" (UniqueName: \"kubernetes.io/projected/886ec14d-0ca4-42a3-b81b-e0bc29560783-kube-api-access-gnsvq\") pod \"886ec14d-0ca4-42a3-b81b-e0bc29560783\" (UID: \"886ec14d-0ca4-42a3-b81b-e0bc29560783\") " Aug 04 16:05:45.391013 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:45.390811 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/886ec14d-0ca4-42a3-b81b-e0bc29560783-model-cache" (OuterVolumeSpecName: "model-cache") pod "886ec14d-0ca4-42a3-b81b-e0bc29560783" (UID: "886ec14d-0ca4-42a3-b81b-e0bc29560783"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:05:45.391013 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:45.390919 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/886ec14d-0ca4-42a3-b81b-e0bc29560783-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "886ec14d-0ca4-42a3-b81b-e0bc29560783" (UID: "886ec14d-0ca4-42a3-b81b-e0bc29560783"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:05:45.391013 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:45.390932 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/886ec14d-0ca4-42a3-b81b-e0bc29560783-home" (OuterVolumeSpecName: "home") pod "886ec14d-0ca4-42a3-b81b-e0bc29560783" (UID: "886ec14d-0ca4-42a3-b81b-e0bc29560783"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:05:45.391181 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:45.391105 2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/886ec14d-0ca4-42a3-b81b-e0bc29560783-home\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:05:45.391181 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:45.391119 2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/886ec14d-0ca4-42a3-b81b-e0bc29560783-tmp-dir\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:05:45.391181 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:45.391129 2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/886ec14d-0ca4-42a3-b81b-e0bc29560783-model-cache\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:05:45.392902 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:45.392879 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/886ec14d-0ca4-42a3-b81b-e0bc29560783-dshm" (OuterVolumeSpecName: "dshm") pod "886ec14d-0ca4-42a3-b81b-e0bc29560783" (UID: "886ec14d-0ca4-42a3-b81b-e0bc29560783"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:05:45.393018 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:45.392939 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/886ec14d-0ca4-42a3-b81b-e0bc29560783-kube-api-access-gnsvq" (OuterVolumeSpecName: "kube-api-access-gnsvq") pod "886ec14d-0ca4-42a3-b81b-e0bc29560783" (UID: "886ec14d-0ca4-42a3-b81b-e0bc29560783"). InnerVolumeSpecName "kube-api-access-gnsvq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:05:45.393018 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:45.393001 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/886ec14d-0ca4-42a3-b81b-e0bc29560783-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "886ec14d-0ca4-42a3-b81b-e0bc29560783" (UID: "886ec14d-0ca4-42a3-b81b-e0bc29560783"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 16:05:45.491795 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:45.491765 2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/886ec14d-0ca4-42a3-b81b-e0bc29560783-dshm\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:05:45.491795 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:45.491791 2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/886ec14d-0ca4-42a3-b81b-e0bc29560783-tls-certs\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:05:45.491795 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:45.491802 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gnsvq\" (UniqueName: \"kubernetes.io/projected/886ec14d-0ca4-42a3-b81b-e0bc29560783-kube-api-access-gnsvq\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:05:45.494569 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:45.494544 2574 generic.go:356] "Generic (PLEG): container finished" podID="886ec14d-0ca4-42a3-b81b-e0bc29560783" containerID="9cedf27b47b8a3cf737e8406a59fe9df7531478d1203ea6ff44479fa1f3c0062" exitCode=0 Aug 04 16:05:45.494684 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:45.494590 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-05b2bc/rate-test-model-05b2bc-kserve-544cd576c9-l7pzl" Aug 04 16:05:45.494684 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:45.494629 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-05b2bc/rate-test-model-05b2bc-kserve-544cd576c9-l7pzl" event={"ID":"886ec14d-0ca4-42a3-b81b-e0bc29560783","Type":"ContainerDied","Data":"9cedf27b47b8a3cf737e8406a59fe9df7531478d1203ea6ff44479fa1f3c0062"} Aug 04 16:05:45.494684 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:45.494670 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-05b2bc/rate-test-model-05b2bc-kserve-544cd576c9-l7pzl" event={"ID":"886ec14d-0ca4-42a3-b81b-e0bc29560783","Type":"ContainerDied","Data":"9cc48d4689c584b331bbed4bbb462c1c8066fb2cd08ca926358da9f1edca27da"} Aug 04 16:05:45.494817 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:45.494688 2574 scope.go:117] "RemoveContainer" containerID="9cedf27b47b8a3cf737e8406a59fe9df7531478d1203ea6ff44479fa1f3c0062" Aug 04 16:05:45.504729 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:45.504712 2574 scope.go:117] "RemoveContainer" containerID="9cedf27b47b8a3cf737e8406a59fe9df7531478d1203ea6ff44479fa1f3c0062" Aug 04 16:05:45.505002 ip-10-0-132-155 kubenswrapper[2574]: E0804 16:05:45.504985 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cedf27b47b8a3cf737e8406a59fe9df7531478d1203ea6ff44479fa1f3c0062\": container with ID starting with 9cedf27b47b8a3cf737e8406a59fe9df7531478d1203ea6ff44479fa1f3c0062 not found: ID does not exist" containerID="9cedf27b47b8a3cf737e8406a59fe9df7531478d1203ea6ff44479fa1f3c0062" Aug 04 16:05:45.505055 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:45.505009 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cedf27b47b8a3cf737e8406a59fe9df7531478d1203ea6ff44479fa1f3c0062"} err="failed to get container status \"9cedf27b47b8a3cf737e8406a59fe9df7531478d1203ea6ff44479fa1f3c0062\": rpc error: code = NotFound desc = could not find container \"9cedf27b47b8a3cf737e8406a59fe9df7531478d1203ea6ff44479fa1f3c0062\": container with ID starting with 9cedf27b47b8a3cf737e8406a59fe9df7531478d1203ea6ff44479fa1f3c0062 not found: ID does not exist" Aug 04 16:05:45.518279 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:45.518255 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-05b2bc/rate-test-model-05b2bc-kserve-544cd576c9-l7pzl"] Aug 04 16:05:45.522795 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:45.522772 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-05b2bc/rate-test-model-05b2bc-kserve-544cd576c9-l7pzl"] Aug 04 16:05:46.475996 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:46.475960 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7386-140234341404720"] Aug 04 16:05:46.476382 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:46.476370 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="886ec14d-0ca4-42a3-b81b-e0bc29560783" containerName="main" Aug 04 16:05:46.476382 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:46.476383 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="886ec14d-0ca4-42a3-b81b-e0bc29560783" containerName="main" Aug 04 16:05:46.476488 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:46.476479 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="886ec14d-0ca4-42a3-b81b-e0bc29560783" containerName="main" Aug 04 16:05:46.481059 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:46.481039 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7386-140234341404720" Aug 04 16:05:46.484162 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:46.484133 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-nnz2r\"" Aug 04 16:05:46.486620 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:46.486598 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7386-140234341404720"] Aug 04 16:05:46.602079 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:46.602046 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fckj2\" (UniqueName: \"kubernetes.io/projected/a23de562-2757-45fe-808f-238b3d3b2a3f-kube-api-access-fckj2\") pod \"test-curl-7386-140234341404720\" (UID: \"a23de562-2757-45fe-808f-238b3d3b2a3f\") " pod="opendatahub/test-curl-7386-140234341404720" Aug 04 16:05:46.703305 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:46.703270 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fckj2\" (UniqueName: \"kubernetes.io/projected/a23de562-2757-45fe-808f-238b3d3b2a3f-kube-api-access-fckj2\") pod \"test-curl-7386-140234341404720\" (UID: \"a23de562-2757-45fe-808f-238b3d3b2a3f\") " pod="opendatahub/test-curl-7386-140234341404720" Aug 04 16:05:46.714734 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:46.714708 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fckj2\" (UniqueName: \"kubernetes.io/projected/a23de562-2757-45fe-808f-238b3d3b2a3f-kube-api-access-fckj2\") pod \"test-curl-7386-140234341404720\" (UID: \"a23de562-2757-45fe-808f-238b3d3b2a3f\") " pod="opendatahub/test-curl-7386-140234341404720" Aug 04 16:05:46.791319 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:46.791294 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7386-140234341404720" Aug 04 16:05:47.054081 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:47.054003 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="886ec14d-0ca4-42a3-b81b-e0bc29560783" path="/var/lib/kubelet/pods/886ec14d-0ca4-42a3-b81b-e0bc29560783/volumes" Aug 04 16:05:47.130777 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:47.130749 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7386-140234341404720"] Aug 04 16:05:47.132313 ip-10-0-132-155 kubenswrapper[2574]: W0804 16:05:47.132286 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda23de562_2757_45fe_808f_238b3d3b2a3f.slice/crio-afb1f0b61bd1fa7064a8bd27c4fc7e82bf040fcae906ba885d164b59059c17fa WatchSource:0}: Error finding container afb1f0b61bd1fa7064a8bd27c4fc7e82bf040fcae906ba885d164b59059c17fa: Status 404 returned error can't find the container with id afb1f0b61bd1fa7064a8bd27c4fc7e82bf040fcae906ba885d164b59059c17fa Aug 04 16:05:47.133635 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:47.133616 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 16:05:47.513202 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:47.513126 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7386-140234341404720" event={"ID":"a23de562-2757-45fe-808f-238b3d3b2a3f","Type":"ContainerStarted","Data":"afb1f0b61bd1fa7064a8bd27c4fc7e82bf040fcae906ba885d164b59059c17fa"} Aug 04 16:05:48.519528 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:48.519490 2574 generic.go:356] "Generic (PLEG): container finished" podID="a23de562-2757-45fe-808f-238b3d3b2a3f" containerID="741425f71a58ceb79a1f2726bbf97ae41abc392d98453dce74a22454581f82cd" exitCode=0 Aug 04 16:05:48.519902 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:48.519567 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7386-140234341404720" event={"ID":"a23de562-2757-45fe-808f-238b3d3b2a3f","Type":"ContainerDied","Data":"741425f71a58ceb79a1f2726bbf97ae41abc392d98453dce74a22454581f82cd"} Aug 04 16:05:49.663879 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:49.663853 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7386-140234341404720" Aug 04 16:05:49.708967 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:49.708937 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7386-140234341404720_a23de562-2757-45fe-808f-238b3d3b2a3f/test-curl-7386-140234341404720/0.log" Aug 04 16:05:49.735993 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:49.735962 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7386-140234341404720"] Aug 04 16:05:49.738273 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:49.738252 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7386-140234341404720"] Aug 04 16:05:49.833654 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:49.833563 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fckj2\" (UniqueName: \"kubernetes.io/projected/a23de562-2757-45fe-808f-238b3d3b2a3f-kube-api-access-fckj2\") pod \"a23de562-2757-45fe-808f-238b3d3b2a3f\" (UID: \"a23de562-2757-45fe-808f-238b3d3b2a3f\") " Aug 04 16:05:49.835823 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:49.835797 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a23de562-2757-45fe-808f-238b3d3b2a3f-kube-api-access-fckj2" (OuterVolumeSpecName: "kube-api-access-fckj2") pod "a23de562-2757-45fe-808f-238b3d3b2a3f" (UID: "a23de562-2757-45fe-808f-238b3d3b2a3f"). InnerVolumeSpecName "kube-api-access-fckj2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:05:49.934903 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:49.934870 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fckj2\" (UniqueName: \"kubernetes.io/projected/a23de562-2757-45fe-808f-238b3d3b2a3f-kube-api-access-fckj2\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:05:50.532549 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:50.532517 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="afb1f0b61bd1fa7064a8bd27c4fc7e82bf040fcae906ba885d164b59059c17fa" Aug 04 16:05:50.532549 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:50.532524 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7386-140234341404720" Aug 04 16:05:50.543100 ip-10-0-132-155 kubenswrapper[2574]: E0804 16:05:50.543069 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7386-140234341404720\" is forbidden: User \"system:node:ip-10-0-132-155.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-155.ec2.internal' and this object" podUID="a23de562-2757-45fe-808f-238b3d3b2a3f" pod="opendatahub/test-curl-7386-140234341404720" Aug 04 16:05:50.572367 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:50.572334 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z"] Aug 04 16:05:50.572843 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:50.572819 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a23de562-2757-45fe-808f-238b3d3b2a3f" containerName="test-curl-7386-140234341404720" Aug 04 16:05:50.572843 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:50.572838 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a23de562-2757-45fe-808f-238b3d3b2a3f" containerName="test-curl-7386-140234341404720" Aug 04 16:05:50.573036 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:50.573002 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a23de562-2757-45fe-808f-238b3d3b2a3f" containerName="test-curl-7386-140234341404720" Aug 04 16:05:50.576167 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:50.576149 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" Aug 04 16:05:50.579555 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:50.579538 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-e9e4ac-openshift-default-dockercfg-9mtq6\"" Aug 04 16:05:50.589864 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:50.589842 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z"] Aug 04 16:05:50.615337 ip-10-0-132-155 kubenswrapper[2574]: E0804 16:05:50.615293 2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7386-140234341404720\" is forbidden: User \"system:node:ip-10-0-132-155.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-155.ec2.internal' and this object" podUID="a23de562-2757-45fe-808f-238b3d3b2a3f" pod="opendatahub/test-curl-7386-140234341404720" Aug 04 16:05:50.741816 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:50.741778 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ee94230-2660-44c5-bb49-4e4d6b580ab5-workload-socket\") pod \"e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z\" (UID: \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\") " pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" Aug 04 16:05:50.742264 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:50.741841 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ee94230-2660-44c5-bb49-4e4d6b580ab5-istio-data\") pod \"e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z\" (UID: \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\") " pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" Aug 04 16:05:50.742264 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:50.741883 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktcgt\" (UniqueName: \"kubernetes.io/projected/8ee94230-2660-44c5-bb49-4e4d6b580ab5-kube-api-access-ktcgt\") pod \"e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z\" (UID: \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\") " pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" Aug 04 16:05:50.742264 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:50.742018 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ee94230-2660-44c5-bb49-4e4d6b580ab5-istio-podinfo\") pod \"e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z\" (UID: \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\") " pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" Aug 04 16:05:50.742264 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:50.742053 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ee94230-2660-44c5-bb49-4e4d6b580ab5-istio-token\") pod \"e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z\" (UID: \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\") " pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" Aug 04 16:05:50.742264 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:50.742091 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ee94230-2660-44c5-bb49-4e4d6b580ab5-istio-envoy\") pod \"e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z\" (UID: \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\") " pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" Aug 04 16:05:50.742264 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:50.742135 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ee94230-2660-44c5-bb49-4e4d6b580ab5-workload-certs\") pod \"e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z\" (UID: \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\") " pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" Aug 04 16:05:50.742264 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:50.742202 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ee94230-2660-44c5-bb49-4e4d6b580ab5-istiod-ca-cert\") pod \"e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z\" (UID: \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\") " pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" Aug 04 16:05:50.742264 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:50.742233 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ee94230-2660-44c5-bb49-4e4d6b580ab5-credential-socket\") pod \"e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z\" (UID: \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\") " pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" Aug 04 16:05:50.843303 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:50.843230 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ee94230-2660-44c5-bb49-4e4d6b580ab5-istiod-ca-cert\") pod \"e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z\" (UID: \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\") " pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" Aug 04 16:05:50.843303 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:50.843265 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ee94230-2660-44c5-bb49-4e4d6b580ab5-credential-socket\") pod \"e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z\" (UID: \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\") " pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" Aug 04 16:05:50.843303 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:50.843293 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ee94230-2660-44c5-bb49-4e4d6b580ab5-workload-socket\") pod \"e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z\" (UID: \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\") " pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" Aug 04 16:05:50.843582 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:50.843343 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ee94230-2660-44c5-bb49-4e4d6b580ab5-istio-data\") pod \"e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z\" (UID: \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\") " pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" Aug 04 16:05:50.843582 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:50.843366 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktcgt\" (UniqueName: \"kubernetes.io/projected/8ee94230-2660-44c5-bb49-4e4d6b580ab5-kube-api-access-ktcgt\") pod \"e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z\" (UID: \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\") " pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" Aug 04 16:05:50.843582 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:50.843462 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ee94230-2660-44c5-bb49-4e4d6b580ab5-istio-podinfo\") pod \"e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z\" (UID: \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\") " pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" Aug 04 16:05:50.843582 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:50.843496 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ee94230-2660-44c5-bb49-4e4d6b580ab5-istio-token\") pod \"e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z\" (UID: \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\") " pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" Aug 04 16:05:50.843582 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:50.843529 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ee94230-2660-44c5-bb49-4e4d6b580ab5-istio-envoy\") pod \"e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z\" (UID: \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\") " pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" Aug 04 16:05:50.843582 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:50.843568 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ee94230-2660-44c5-bb49-4e4d6b580ab5-workload-certs\") pod \"e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z\" (UID: \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\") " pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" Aug 04 16:05:50.843877 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:50.843679 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ee94230-2660-44c5-bb49-4e4d6b580ab5-credential-socket\") pod \"e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z\" (UID: \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\") " pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" Aug 04 16:05:50.843877 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:50.843710 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ee94230-2660-44c5-bb49-4e4d6b580ab5-istio-data\") pod \"e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z\" (UID: \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\") " pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" Aug 04 16:05:50.843877 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:50.843770 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ee94230-2660-44c5-bb49-4e4d6b580ab5-workload-socket\") pod \"e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z\" (UID: \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\") " pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" Aug 04 16:05:50.844035 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:50.844018 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ee94230-2660-44c5-bb49-4e4d6b580ab5-workload-certs\") pod \"e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z\" (UID: \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\") " pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" Aug 04 16:05:50.844242 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:50.844219 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ee94230-2660-44c5-bb49-4e4d6b580ab5-istiod-ca-cert\") pod \"e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z\" (UID: \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\") " pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" Aug 04 16:05:50.845830 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:50.845807 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ee94230-2660-44c5-bb49-4e4d6b580ab5-istio-envoy\") pod \"e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z\" (UID: \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\") " pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" Aug 04 16:05:50.846084 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:50.846065 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ee94230-2660-44c5-bb49-4e4d6b580ab5-istio-podinfo\") pod \"e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z\" (UID: \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\") " pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" Aug 04 16:05:50.852577 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:50.852556 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ee94230-2660-44c5-bb49-4e4d6b580ab5-istio-token\") pod \"e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z\" (UID: \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\") " pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" Aug 04 16:05:50.853042 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:50.853025 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktcgt\" (UniqueName: \"kubernetes.io/projected/8ee94230-2660-44c5-bb49-4e4d6b580ab5-kube-api-access-ktcgt\") pod \"e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z\" (UID: \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\") " pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" Aug 04 16:05:50.886352 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:50.886319 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" Aug 04 16:05:51.038744 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:51.038720 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z"] Aug 04 16:05:51.040843 ip-10-0-132-155 kubenswrapper[2574]: W0804 16:05:51.040813 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ee94230_2660_44c5_bb49_4e4d6b580ab5.slice/crio-ca49bd85e680ac706ce1f4467d57486651cd057e0a2cf6daf192364e2ebf7b89 WatchSource:0}: Error finding container ca49bd85e680ac706ce1f4467d57486651cd057e0a2cf6daf192364e2ebf7b89: Status 404 returned error can't find the container with id ca49bd85e680ac706ce1f4467d57486651cd057e0a2cf6daf192364e2ebf7b89 Aug 04 16:05:51.043138 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:51.043105 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 16:05:51.043215 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:51.043175 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 16:05:51.043215 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:51.043206 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 16:05:51.062938 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:51.062914 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a23de562-2757-45fe-808f-238b3d3b2a3f" path="/var/lib/kubelet/pods/a23de562-2757-45fe-808f-238b3d3b2a3f/volumes" Aug 04 16:05:51.539164 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:51.539124 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" event={"ID":"8ee94230-2660-44c5-bb49-4e4d6b580ab5","Type":"ContainerStarted","Data":"c2d1f9411995d12acb894dd50812b05e5e970b6631a39d6ca53aa8e70873b720"} Aug 04 16:05:51.539305 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:51.539179 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" event={"ID":"8ee94230-2660-44c5-bb49-4e4d6b580ab5","Type":"ContainerStarted","Data":"ca49bd85e680ac706ce1f4467d57486651cd057e0a2cf6daf192364e2ebf7b89"} Aug 04 16:05:51.567141 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:51.567087 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" podStartSLOduration=1.567071887 podStartE2EDuration="1.567071887s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:05:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:05:51.564870589 +0000 UTC m=+3575.024742937" watchObservedRunningTime="2026-08-04 16:05:51.567071887 +0000 UTC m=+3575.026944236" Aug 04 16:05:51.886505 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:51.886430 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" Aug 04 16:05:52.892141 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:52.892108 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" Aug 04 16:05:53.551044 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:53.551011 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" Aug 04 16:05:53.551932 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:05:53.551909 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" Aug 04 16:06:02.455251 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:06:02.455217 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 16:06:12.671887 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:06:12.671850 2574 generic.go:356] "Generic (PLEG): container finished" podID="8ee94230-2660-44c5-bb49-4e4d6b580ab5" containerID="c2d1f9411995d12acb894dd50812b05e5e970b6631a39d6ca53aa8e70873b720" exitCode=0 Aug 04 16:06:12.672300 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:06:12.671934 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" event={"ID":"8ee94230-2660-44c5-bb49-4e4d6b580ab5","Type":"ContainerDied","Data":"c2d1f9411995d12acb894dd50812b05e5e970b6631a39d6ca53aa8e70873b720"} Aug 04 16:06:12.672446 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:06:12.672431 2574 scope.go:117] "RemoveContainer" containerID="c2d1f9411995d12acb894dd50812b05e5e970b6631a39d6ca53aa8e70873b720" Aug 04 16:06:12.673556 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:06:12.673523 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 16:06:12.673699 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:06:12.673611 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 16:06:12.673699 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:06:12.673659 2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Aug 04 16:06:13.679338 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:06:13.679297 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" event={"ID":"8ee94230-2660-44c5-bb49-4e4d6b580ab5","Type":"ContainerStarted","Data":"8779bb13012348bba5968124338c75b50b725f745273e9595f0648d3732ccbe3"} Aug 04 16:06:13.887329 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:06:13.887287 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" Aug 04 16:06:13.887566 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:06:13.887541 2574 patch_prober.go:28] interesting pod/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Aug 04 16:06:13.887665 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:06:13.887611 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" podUID="8ee94230-2660-44c5-bb49-4e4d6b580ab5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Aug 04 16:06:14.887468 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:06:14.887435 2574 patch_prober.go:28] interesting pod/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Aug 04 16:06:14.887856 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:06:14.887490 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" podUID="8ee94230-2660-44c5-bb49-4e4d6b580ab5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Aug 04 16:06:15.887075 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:06:15.887043 2574 patch_prober.go:28] interesting pod/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Aug 04 16:06:15.887342 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:06:15.887106 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" podUID="8ee94230-2660-44c5-bb49-4e4d6b580ab5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Aug 04 16:06:16.887379 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:06:16.887338 2574 patch_prober.go:28] interesting pod/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Aug 04 16:06:16.887813 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:06:16.887424 2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" podUID="8ee94230-2660-44c5-bb49-4e4d6b580ab5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Aug 04 16:06:17.380105 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:06:17.380074 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-r8wkp_0580fdab-0381-4d8e-be5f-bde5eb44e0e6/console-operator/1.log" Aug 04 16:06:17.384176 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:06:17.384151 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8h6k_9c62e3b1-c2dc-4b2a-a61d-7d80049f355f/ovn-acl-logging/0.log" Aug 04 16:06:17.392509 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:06:17.392490 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-r8wkp_0580fdab-0381-4d8e-be5f-bde5eb44e0e6/console-operator/1.log" Aug 04 16:06:17.396339 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:06:17.396320 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8h6k_9c62e3b1-c2dc-4b2a-a61d-7d80049f355f/ovn-acl-logging/0.log" Aug 04 16:06:17.890955 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:06:17.890927 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" Aug 04 16:06:17.891356 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:06:17.891335 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" Aug 04 16:06:17.892043 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:06:17.892022 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" Aug 04 16:06:36.330086 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:06:36.330042 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 16:07:06.687278 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:07:06.687241 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-e9e4ac/ipp-route-e9e4ac-kserve-5c9c665f99-9t98s"] Aug 04 16:07:06.690837 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:07:06.690806 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-e9e4ac/ipp-route-e9e4ac-kserve-5c9c665f99-9t98s" Aug 04 16:07:06.694114 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:07:06.694089 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-e9e4ac\"/\"kube-root-ca.crt\"" Aug 04 16:07:06.695364 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:07:06.695335 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-e9e4ac\"/\"ipp-route-e9e4ac-kserve-self-signed-certs\"" Aug 04 16:07:06.695482 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:07:06.695363 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-e9e4ac\"/\"openshift-service-ca.crt\"" Aug 04 16:07:06.695482 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:07:06.695441 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-e9e4ac\"/\"default-dockercfg-fmcln\"" Aug 04 16:07:06.705487 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:07:06.705465 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-e9e4ac/ipp-route-e9e4ac-kserve-5c9c665f99-9t98s"] Aug 04 16:07:06.762289 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:07:06.762261 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfaf1ca8-4369-4a6d-aa9f-28e9390aab63-tls-certs\") pod \"ipp-route-e9e4ac-kserve-5c9c665f99-9t98s\" (UID: \"cfaf1ca8-4369-4a6d-aa9f-28e9390aab63\") " pod="ai-tenant-e2e-ipp-a-e9e4ac/ipp-route-e9e4ac-kserve-5c9c665f99-9t98s" Aug 04 16:07:06.762468 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:07:06.762303 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pwvf\" (UniqueName: \"kubernetes.io/projected/cfaf1ca8-4369-4a6d-aa9f-28e9390aab63-kube-api-access-4pwvf\") pod \"ipp-route-e9e4ac-kserve-5c9c665f99-9t98s\" (UID: \"cfaf1ca8-4369-4a6d-aa9f-28e9390aab63\") " pod="ai-tenant-e2e-ipp-a-e9e4ac/ipp-route-e9e4ac-kserve-5c9c665f99-9t98s" Aug 04 16:07:06.762468 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:07:06.762358 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfaf1ca8-4369-4a6d-aa9f-28e9390aab63-home\") pod \"ipp-route-e9e4ac-kserve-5c9c665f99-9t98s\" (UID: \"cfaf1ca8-4369-4a6d-aa9f-28e9390aab63\") " pod="ai-tenant-e2e-ipp-a-e9e4ac/ipp-route-e9e4ac-kserve-5c9c665f99-9t98s" Aug 04 16:07:06.762551 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:07:06.762484 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfaf1ca8-4369-4a6d-aa9f-28e9390aab63-dshm\") pod \"ipp-route-e9e4ac-kserve-5c9c665f99-9t98s\" (UID: \"cfaf1ca8-4369-4a6d-aa9f-28e9390aab63\") " pod="ai-tenant-e2e-ipp-a-e9e4ac/ipp-route-e9e4ac-kserve-5c9c665f99-9t98s" Aug 04 16:07:06.762551 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:07:06.762528 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfaf1ca8-4369-4a6d-aa9f-28e9390aab63-model-cache\") pod \"ipp-route-e9e4ac-kserve-5c9c665f99-9t98s\" (UID: \"cfaf1ca8-4369-4a6d-aa9f-28e9390aab63\") " pod="ai-tenant-e2e-ipp-a-e9e4ac/ipp-route-e9e4ac-kserve-5c9c665f99-9t98s" Aug 04 16:07:06.762619 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:07:06.762584 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfaf1ca8-4369-4a6d-aa9f-28e9390aab63-tmp-dir\") pod \"ipp-route-e9e4ac-kserve-5c9c665f99-9t98s\" (UID: \"cfaf1ca8-4369-4a6d-aa9f-28e9390aab63\") " pod="ai-tenant-e2e-ipp-a-e9e4ac/ipp-route-e9e4ac-kserve-5c9c665f99-9t98s" Aug 04 16:07:06.863766 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:07:06.863731 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfaf1ca8-4369-4a6d-aa9f-28e9390aab63-tls-certs\") pod \"ipp-route-e9e4ac-kserve-5c9c665f99-9t98s\" (UID: \"cfaf1ca8-4369-4a6d-aa9f-28e9390aab63\") " pod="ai-tenant-e2e-ipp-a-e9e4ac/ipp-route-e9e4ac-kserve-5c9c665f99-9t98s" Aug 04 16:07:06.863929 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:07:06.863775 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pwvf\" (UniqueName: \"kubernetes.io/projected/cfaf1ca8-4369-4a6d-aa9f-28e9390aab63-kube-api-access-4pwvf\") pod \"ipp-route-e9e4ac-kserve-5c9c665f99-9t98s\" (UID: \"cfaf1ca8-4369-4a6d-aa9f-28e9390aab63\") " pod="ai-tenant-e2e-ipp-a-e9e4ac/ipp-route-e9e4ac-kserve-5c9c665f99-9t98s" Aug 04 16:07:06.863929 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:07:06.863891 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfaf1ca8-4369-4a6d-aa9f-28e9390aab63-home\") pod \"ipp-route-e9e4ac-kserve-5c9c665f99-9t98s\" (UID: \"cfaf1ca8-4369-4a6d-aa9f-28e9390aab63\") " pod="ai-tenant-e2e-ipp-a-e9e4ac/ipp-route-e9e4ac-kserve-5c9c665f99-9t98s" Aug 04 16:07:06.864054 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:07:06.863965 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfaf1ca8-4369-4a6d-aa9f-28e9390aab63-dshm\") pod \"ipp-route-e9e4ac-kserve-5c9c665f99-9t98s\" (UID: \"cfaf1ca8-4369-4a6d-aa9f-28e9390aab63\") " pod="ai-tenant-e2e-ipp-a-e9e4ac/ipp-route-e9e4ac-kserve-5c9c665f99-9t98s" Aug 04 16:07:06.864054 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:07:06.864015 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfaf1ca8-4369-4a6d-aa9f-28e9390aab63-model-cache\") pod \"ipp-route-e9e4ac-kserve-5c9c665f99-9t98s\" (UID: \"cfaf1ca8-4369-4a6d-aa9f-28e9390aab63\") " pod="ai-tenant-e2e-ipp-a-e9e4ac/ipp-route-e9e4ac-kserve-5c9c665f99-9t98s" Aug 04 16:07:06.864163 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:07:06.864138 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfaf1ca8-4369-4a6d-aa9f-28e9390aab63-tmp-dir\") pod \"ipp-route-e9e4ac-kserve-5c9c665f99-9t98s\" (UID: \"cfaf1ca8-4369-4a6d-aa9f-28e9390aab63\") " pod="ai-tenant-e2e-ipp-a-e9e4ac/ipp-route-e9e4ac-kserve-5c9c665f99-9t98s" Aug 04 16:07:06.864315 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:07:06.864296 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfaf1ca8-4369-4a6d-aa9f-28e9390aab63-home\") pod \"ipp-route-e9e4ac-kserve-5c9c665f99-9t98s\" (UID: \"cfaf1ca8-4369-4a6d-aa9f-28e9390aab63\") " pod="ai-tenant-e2e-ipp-a-e9e4ac/ipp-route-e9e4ac-kserve-5c9c665f99-9t98s" Aug 04 16:07:06.864453 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:07:06.864431 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfaf1ca8-4369-4a6d-aa9f-28e9390aab63-model-cache\") pod \"ipp-route-e9e4ac-kserve-5c9c665f99-9t98s\" (UID: \"cfaf1ca8-4369-4a6d-aa9f-28e9390aab63\") " pod="ai-tenant-e2e-ipp-a-e9e4ac/ipp-route-e9e4ac-kserve-5c9c665f99-9t98s" Aug 04 16:07:06.864506 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:07:06.864457 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfaf1ca8-4369-4a6d-aa9f-28e9390aab63-tmp-dir\") pod \"ipp-route-e9e4ac-kserve-5c9c665f99-9t98s\" (UID: \"cfaf1ca8-4369-4a6d-aa9f-28e9390aab63\") " pod="ai-tenant-e2e-ipp-a-e9e4ac/ipp-route-e9e4ac-kserve-5c9c665f99-9t98s" Aug 04 16:07:06.866142 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:07:06.866118 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfaf1ca8-4369-4a6d-aa9f-28e9390aab63-dshm\") pod \"ipp-route-e9e4ac-kserve-5c9c665f99-9t98s\" (UID: \"cfaf1ca8-4369-4a6d-aa9f-28e9390aab63\") " pod="ai-tenant-e2e-ipp-a-e9e4ac/ipp-route-e9e4ac-kserve-5c9c665f99-9t98s" Aug 04 16:07:06.866413 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:07:06.866378 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfaf1ca8-4369-4a6d-aa9f-28e9390aab63-tls-certs\") pod \"ipp-route-e9e4ac-kserve-5c9c665f99-9t98s\" (UID: \"cfaf1ca8-4369-4a6d-aa9f-28e9390aab63\") " pod="ai-tenant-e2e-ipp-a-e9e4ac/ipp-route-e9e4ac-kserve-5c9c665f99-9t98s" Aug 04 16:07:06.875903 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:07:06.875875 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pwvf\" (UniqueName: \"kubernetes.io/projected/cfaf1ca8-4369-4a6d-aa9f-28e9390aab63-kube-api-access-4pwvf\") pod \"ipp-route-e9e4ac-kserve-5c9c665f99-9t98s\" (UID: \"cfaf1ca8-4369-4a6d-aa9f-28e9390aab63\") " pod="ai-tenant-e2e-ipp-a-e9e4ac/ipp-route-e9e4ac-kserve-5c9c665f99-9t98s" Aug 04 16:07:06.999884 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:07:06.999846 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-e9e4ac/ipp-route-e9e4ac-kserve-5c9c665f99-9t98s" Aug 04 16:07:07.148911 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:07:07.148879 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-e9e4ac/ipp-route-e9e4ac-kserve-5c9c665f99-9t98s"] Aug 04 16:07:07.150024 ip-10-0-132-155 kubenswrapper[2574]: W0804 16:07:07.149998 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcfaf1ca8_4369_4a6d_aa9f_28e9390aab63.slice/crio-730326914b7c294edd5eb70cda0f9b102f70d084f081e8cf8ae2b95ad94b83cf WatchSource:0}: Error finding container 730326914b7c294edd5eb70cda0f9b102f70d084f081e8cf8ae2b95ad94b83cf: Status 404 returned error can't find the container with id 730326914b7c294edd5eb70cda0f9b102f70d084f081e8cf8ae2b95ad94b83cf Aug 04 16:07:08.002188 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:07:08.002156 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-e9e4ac/ipp-route-e9e4ac-kserve-5c9c665f99-9t98s" event={"ID":"cfaf1ca8-4369-4a6d-aa9f-28e9390aab63","Type":"ContainerStarted","Data":"760cd2ff16f5a30a4a34b32106f0fc2be05e5624aac370d1f226a828f3aa876a"} Aug 04 16:07:08.002613 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:07:08.002197 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-e9e4ac/ipp-route-e9e4ac-kserve-5c9c665f99-9t98s" event={"ID":"cfaf1ca8-4369-4a6d-aa9f-28e9390aab63","Type":"ContainerStarted","Data":"730326914b7c294edd5eb70cda0f9b102f70d084f081e8cf8ae2b95ad94b83cf"} Aug 04 16:07:08.027334 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:07:08.027289 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-e9e4ac/ipp-route-e9e4ac-kserve-5c9c665f99-9t98s" podStartSLOduration=2.027275651 podStartE2EDuration="2.027275651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:07:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:07:08.023800421 +0000 UTC m=+3651.483672770" watchObservedRunningTime="2026-08-04 16:07:08.027275651 +0000 UTC m=+3651.487147999" Aug 04 16:07:10.419286 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:07:10.419246 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 16:07:17.000194 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:07:17.000165 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-e9e4ac/ipp-route-e9e4ac-kserve-5c9c665f99-9t98s" Aug 04 16:07:17.000683 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:07:17.000205 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-e9e4ac/ipp-route-e9e4ac-kserve-5c9c665f99-9t98s" Aug 04 16:07:17.012882 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:07:17.012855 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-e9e4ac/ipp-route-e9e4ac-kserve-5c9c665f99-9t98s" Aug 04 16:07:17.063833 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:07:17.063808 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-e9e4ac/ipp-route-e9e4ac-kserve-5c9c665f99-9t98s" Aug 04 16:07:44.423769 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:07:44.423734 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 16:08:04.121735 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:08:04.121696 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 16:08:34.936198 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:08:34.936162 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 16:08:58.156921 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:08:58.156878 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 16:09:03.122903 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:03.122866 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-e9e4ac/ipp-route-e9e4ac-kserve-5c9c665f99-9t98s"] Aug 04 16:09:08.531786 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:08.531755 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 16:09:18.142221 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:18.142129 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-e9e4ac/ipp-route-e9e4ac-kserve-5c9c665f99-9t98s" podUID="cfaf1ca8-4369-4a6d-aa9f-28e9390aab63" containerName="main" containerID="cri-o://760cd2ff16f5a30a4a34b32106f0fc2be05e5624aac370d1f226a828f3aa876a" gracePeriod=45 Aug 04 16:09:18.402123 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:18.402064 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-e9e4ac/ipp-route-e9e4ac-kserve-5c9c665f99-9t98s" Aug 04 16:09:18.488941 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:18.488907 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4pwvf\" (UniqueName: \"kubernetes.io/projected/cfaf1ca8-4369-4a6d-aa9f-28e9390aab63-kube-api-access-4pwvf\") pod \"cfaf1ca8-4369-4a6d-aa9f-28e9390aab63\" (UID: \"cfaf1ca8-4369-4a6d-aa9f-28e9390aab63\") " Aug 04 16:09:18.489129 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:18.488957 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfaf1ca8-4369-4a6d-aa9f-28e9390aab63-dshm\") pod \"cfaf1ca8-4369-4a6d-aa9f-28e9390aab63\" (UID: \"cfaf1ca8-4369-4a6d-aa9f-28e9390aab63\") " Aug 04 16:09:18.489129 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:18.488978 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfaf1ca8-4369-4a6d-aa9f-28e9390aab63-model-cache\") pod \"cfaf1ca8-4369-4a6d-aa9f-28e9390aab63\" (UID: \"cfaf1ca8-4369-4a6d-aa9f-28e9390aab63\") " Aug 04 16:09:18.489129 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:18.489008 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfaf1ca8-4369-4a6d-aa9f-28e9390aab63-home\") pod \"cfaf1ca8-4369-4a6d-aa9f-28e9390aab63\" (UID: \"cfaf1ca8-4369-4a6d-aa9f-28e9390aab63\") " Aug 04 16:09:18.489129 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:18.489028 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfaf1ca8-4369-4a6d-aa9f-28e9390aab63-tmp-dir\") pod \"cfaf1ca8-4369-4a6d-aa9f-28e9390aab63\" (UID: \"cfaf1ca8-4369-4a6d-aa9f-28e9390aab63\") " Aug 04 16:09:18.489129 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:18.489088 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfaf1ca8-4369-4a6d-aa9f-28e9390aab63-tls-certs\") pod \"cfaf1ca8-4369-4a6d-aa9f-28e9390aab63\" (UID: \"cfaf1ca8-4369-4a6d-aa9f-28e9390aab63\") " Aug 04 16:09:18.489423 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:18.489306 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfaf1ca8-4369-4a6d-aa9f-28e9390aab63-model-cache" (OuterVolumeSpecName: "model-cache") pod "cfaf1ca8-4369-4a6d-aa9f-28e9390aab63" (UID: "cfaf1ca8-4369-4a6d-aa9f-28e9390aab63"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:09:18.489423 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:18.489317 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfaf1ca8-4369-4a6d-aa9f-28e9390aab63-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cfaf1ca8-4369-4a6d-aa9f-28e9390aab63" (UID: "cfaf1ca8-4369-4a6d-aa9f-28e9390aab63"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:09:18.489423 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:18.489337 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfaf1ca8-4369-4a6d-aa9f-28e9390aab63-home" (OuterVolumeSpecName: "home") pod "cfaf1ca8-4369-4a6d-aa9f-28e9390aab63" (UID: "cfaf1ca8-4369-4a6d-aa9f-28e9390aab63"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:09:18.491086 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:18.491053 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfaf1ca8-4369-4a6d-aa9f-28e9390aab63-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cfaf1ca8-4369-4a6d-aa9f-28e9390aab63" (UID: "cfaf1ca8-4369-4a6d-aa9f-28e9390aab63"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Aug 04 16:09:18.491211 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:18.491097 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfaf1ca8-4369-4a6d-aa9f-28e9390aab63-dshm" (OuterVolumeSpecName: "dshm") pod "cfaf1ca8-4369-4a6d-aa9f-28e9390aab63" (UID: "cfaf1ca8-4369-4a6d-aa9f-28e9390aab63"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:09:18.491211 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:18.491150 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfaf1ca8-4369-4a6d-aa9f-28e9390aab63-kube-api-access-4pwvf" (OuterVolumeSpecName: "kube-api-access-4pwvf") pod "cfaf1ca8-4369-4a6d-aa9f-28e9390aab63" (UID: "cfaf1ca8-4369-4a6d-aa9f-28e9390aab63"). InnerVolumeSpecName "kube-api-access-4pwvf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:09:18.590197 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:18.590167 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4pwvf\" (UniqueName: \"kubernetes.io/projected/cfaf1ca8-4369-4a6d-aa9f-28e9390aab63-kube-api-access-4pwvf\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:09:18.590197 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:18.590193 2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfaf1ca8-4369-4a6d-aa9f-28e9390aab63-dshm\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:09:18.590197 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:18.590203 2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfaf1ca8-4369-4a6d-aa9f-28e9390aab63-model-cache\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:09:18.590430 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:18.590212 2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfaf1ca8-4369-4a6d-aa9f-28e9390aab63-home\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:09:18.590430 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:18.590220 2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfaf1ca8-4369-4a6d-aa9f-28e9390aab63-tmp-dir\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:09:18.590430 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:18.590227 2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfaf1ca8-4369-4a6d-aa9f-28e9390aab63-tls-certs\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:09:18.758737 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:18.758700 2574 generic.go:356] "Generic (PLEG): container finished" podID="cfaf1ca8-4369-4a6d-aa9f-28e9390aab63" containerID="760cd2ff16f5a30a4a34b32106f0fc2be05e5624aac370d1f226a828f3aa876a" exitCode=0 Aug 04 16:09:18.758905 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:18.758760 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-e9e4ac/ipp-route-e9e4ac-kserve-5c9c665f99-9t98s" Aug 04 16:09:18.758905 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:18.758780 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-e9e4ac/ipp-route-e9e4ac-kserve-5c9c665f99-9t98s" event={"ID":"cfaf1ca8-4369-4a6d-aa9f-28e9390aab63","Type":"ContainerDied","Data":"760cd2ff16f5a30a4a34b32106f0fc2be05e5624aac370d1f226a828f3aa876a"} Aug 04 16:09:18.758905 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:18.758822 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-e9e4ac/ipp-route-e9e4ac-kserve-5c9c665f99-9t98s" event={"ID":"cfaf1ca8-4369-4a6d-aa9f-28e9390aab63","Type":"ContainerDied","Data":"730326914b7c294edd5eb70cda0f9b102f70d084f081e8cf8ae2b95ad94b83cf"} Aug 04 16:09:18.758905 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:18.758839 2574 scope.go:117] "RemoveContainer" containerID="760cd2ff16f5a30a4a34b32106f0fc2be05e5624aac370d1f226a828f3aa876a" Aug 04 16:09:18.769200 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:18.769187 2574 scope.go:117] "RemoveContainer" containerID="760cd2ff16f5a30a4a34b32106f0fc2be05e5624aac370d1f226a828f3aa876a" Aug 04 16:09:18.769486 ip-10-0-132-155 kubenswrapper[2574]: E0804 16:09:18.769465 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"760cd2ff16f5a30a4a34b32106f0fc2be05e5624aac370d1f226a828f3aa876a\": container with ID starting with 760cd2ff16f5a30a4a34b32106f0fc2be05e5624aac370d1f226a828f3aa876a not found: ID does not exist" containerID="760cd2ff16f5a30a4a34b32106f0fc2be05e5624aac370d1f226a828f3aa876a" Aug 04 16:09:18.769563 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:18.769493 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"760cd2ff16f5a30a4a34b32106f0fc2be05e5624aac370d1f226a828f3aa876a"} err="failed to get container status \"760cd2ff16f5a30a4a34b32106f0fc2be05e5624aac370d1f226a828f3aa876a\": rpc error: code = NotFound desc = could not find container \"760cd2ff16f5a30a4a34b32106f0fc2be05e5624aac370d1f226a828f3aa876a\": container with ID starting with 760cd2ff16f5a30a4a34b32106f0fc2be05e5624aac370d1f226a828f3aa876a not found: ID does not exist" Aug 04 16:09:18.783605 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:18.783582 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-e9e4ac/ipp-route-e9e4ac-kserve-5c9c665f99-9t98s"] Aug 04 16:09:18.786149 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:18.786130 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-e9e4ac/ipp-route-e9e4ac-kserve-5c9c665f99-9t98s"] Aug 04 16:09:19.053485 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:19.053372 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfaf1ca8-4369-4a6d-aa9f-28e9390aab63" path="/var/lib/kubelet/pods/cfaf1ca8-4369-4a6d-aa9f-28e9390aab63/volumes" Aug 04 16:09:47.534179 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:47.534143 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z"] Aug 04 16:09:47.534655 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:47.534346 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" podUID="8ee94230-2660-44c5-bb49-4e4d6b580ab5" containerName="istio-proxy" containerID="cri-o://8779bb13012348bba5968124338c75b50b725f745273e9595f0648d3732ccbe3" gracePeriod=30 Aug 04 16:09:47.893651 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:47.893570 2574 patch_prober.go:28] interesting pod/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Aug 04 16:09:47.893790 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:47.893663 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" podUID="8ee94230-2660-44c5-bb49-4e4d6b580ab5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Aug 04 16:09:52.796736 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:52.796673 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" Aug 04 16:09:52.818286 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:52.818257 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ee94230-2660-44c5-bb49-4e4d6b580ab5-credential-socket\") pod \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\" (UID: \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\") " Aug 04 16:09:52.818437 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:52.818293 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ee94230-2660-44c5-bb49-4e4d6b580ab5-istio-envoy\") pod \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\" (UID: \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\") " Aug 04 16:09:52.818437 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:52.818322 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ktcgt\" (UniqueName: \"kubernetes.io/projected/8ee94230-2660-44c5-bb49-4e4d6b580ab5-kube-api-access-ktcgt\") pod \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\" (UID: \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\") " Aug 04 16:09:52.818437 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:52.818369 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ee94230-2660-44c5-bb49-4e4d6b580ab5-istiod-ca-cert\") pod \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\" (UID: \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\") " Aug 04 16:09:52.818566 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:52.818537 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ee94230-2660-44c5-bb49-4e4d6b580ab5-workload-certs\") pod \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\" (UID: \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\") " Aug 04 16:09:52.818566 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:52.818535 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ee94230-2660-44c5-bb49-4e4d6b580ab5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8ee94230-2660-44c5-bb49-4e4d6b580ab5" (UID: "8ee94230-2660-44c5-bb49-4e4d6b580ab5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:09:52.818675 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:52.818614 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ee94230-2660-44c5-bb49-4e4d6b580ab5-workload-socket\") pod \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\" (UID: \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\") " Aug 04 16:09:52.818734 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:52.818679 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ee94230-2660-44c5-bb49-4e4d6b580ab5-istio-token\") pod \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\" (UID: \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\") " Aug 04 16:09:52.818790 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:52.818738 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ee94230-2660-44c5-bb49-4e4d6b580ab5-istio-podinfo\") pod \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\" (UID: \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\") " Aug 04 16:09:52.818846 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:52.818783 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ee94230-2660-44c5-bb49-4e4d6b580ab5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8ee94230-2660-44c5-bb49-4e4d6b580ab5" (UID: "8ee94230-2660-44c5-bb49-4e4d6b580ab5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:09:52.818846 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:52.818797 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ee94230-2660-44c5-bb49-4e4d6b580ab5-istio-data\") pod \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\" (UID: \"8ee94230-2660-44c5-bb49-4e4d6b580ab5\") " Aug 04 16:09:52.818955 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:52.818850 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ee94230-2660-44c5-bb49-4e4d6b580ab5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8ee94230-2660-44c5-bb49-4e4d6b580ab5" (UID: "8ee94230-2660-44c5-bb49-4e4d6b580ab5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:09:52.818955 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:52.818857 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ee94230-2660-44c5-bb49-4e4d6b580ab5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8ee94230-2660-44c5-bb49-4e4d6b580ab5" (UID: "8ee94230-2660-44c5-bb49-4e4d6b580ab5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:09:52.819120 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:52.819097 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ee94230-2660-44c5-bb49-4e4d6b580ab5-credential-socket\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:09:52.819192 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:52.819126 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8ee94230-2660-44c5-bb49-4e4d6b580ab5-istiod-ca-cert\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:09:52.819192 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:52.819143 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8ee94230-2660-44c5-bb49-4e4d6b580ab5-workload-certs\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:09:52.819192 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:52.819154 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8ee94230-2660-44c5-bb49-4e4d6b580ab5-workload-socket\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:09:52.819576 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:52.819560 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ee94230-2660-44c5-bb49-4e4d6b580ab5-istio-data" (OuterVolumeSpecName: "istio-data") pod "8ee94230-2660-44c5-bb49-4e4d6b580ab5" (UID: "8ee94230-2660-44c5-bb49-4e4d6b580ab5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:09:52.821018 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:52.820989 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ee94230-2660-44c5-bb49-4e4d6b580ab5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8ee94230-2660-44c5-bb49-4e4d6b580ab5" (UID: "8ee94230-2660-44c5-bb49-4e4d6b580ab5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:09:52.821174 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:52.821050 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ee94230-2660-44c5-bb49-4e4d6b580ab5-kube-api-access-ktcgt" (OuterVolumeSpecName: "kube-api-access-ktcgt") pod "8ee94230-2660-44c5-bb49-4e4d6b580ab5" (UID: "8ee94230-2660-44c5-bb49-4e4d6b580ab5"). InnerVolumeSpecName "kube-api-access-ktcgt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:09:52.821174 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:52.821093 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ee94230-2660-44c5-bb49-4e4d6b580ab5-istio-token" (OuterVolumeSpecName: "istio-token") pod "8ee94230-2660-44c5-bb49-4e4d6b580ab5" (UID: "8ee94230-2660-44c5-bb49-4e4d6b580ab5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:09:52.821174 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:52.821158 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8ee94230-2660-44c5-bb49-4e4d6b580ab5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8ee94230-2660-44c5-bb49-4e4d6b580ab5" (UID: "8ee94230-2660-44c5-bb49-4e4d6b580ab5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 16:09:52.919728 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:52.919699 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8ee94230-2660-44c5-bb49-4e4d6b580ab5-istio-data\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:09:52.919728 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:52.919725 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8ee94230-2660-44c5-bb49-4e4d6b580ab5-istio-envoy\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:09:52.919728 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:52.919735 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ktcgt\" (UniqueName: \"kubernetes.io/projected/8ee94230-2660-44c5-bb49-4e4d6b580ab5-kube-api-access-ktcgt\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:09:52.919956 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:52.919743 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8ee94230-2660-44c5-bb49-4e4d6b580ab5-istio-token\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:09:52.919956 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:52.919752 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8ee94230-2660-44c5-bb49-4e4d6b580ab5-istio-podinfo\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:09:52.954799 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:52.954768 2574 generic.go:356] "Generic (PLEG): container finished" podID="8ee94230-2660-44c5-bb49-4e4d6b580ab5" containerID="8779bb13012348bba5968124338c75b50b725f745273e9595f0648d3732ccbe3" exitCode=0 Aug 04 16:09:52.954943 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:52.954816 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" Aug 04 16:09:52.954943 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:52.954850 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" event={"ID":"8ee94230-2660-44c5-bb49-4e4d6b580ab5","Type":"ContainerDied","Data":"8779bb13012348bba5968124338c75b50b725f745273e9595f0648d3732ccbe3"} Aug 04 16:09:52.954943 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:52.954902 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z" event={"ID":"8ee94230-2660-44c5-bb49-4e4d6b580ab5","Type":"ContainerDied","Data":"ca49bd85e680ac706ce1f4467d57486651cd057e0a2cf6daf192364e2ebf7b89"} Aug 04 16:09:52.954943 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:52.954925 2574 scope.go:117] "RemoveContainer" containerID="8779bb13012348bba5968124338c75b50b725f745273e9595f0648d3732ccbe3" Aug 04 16:09:52.965125 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:52.965105 2574 scope.go:117] "RemoveContainer" containerID="c2d1f9411995d12acb894dd50812b05e5e970b6631a39d6ca53aa8e70873b720" Aug 04 16:09:52.973328 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:52.973307 2574 scope.go:117] "RemoveContainer" containerID="8779bb13012348bba5968124338c75b50b725f745273e9595f0648d3732ccbe3" Aug 04 16:09:52.973598 ip-10-0-132-155 kubenswrapper[2574]: E0804 16:09:52.973578 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8779bb13012348bba5968124338c75b50b725f745273e9595f0648d3732ccbe3\": container with ID starting with 8779bb13012348bba5968124338c75b50b725f745273e9595f0648d3732ccbe3 not found: ID does not exist" containerID="8779bb13012348bba5968124338c75b50b725f745273e9595f0648d3732ccbe3" Aug 04 16:09:52.973668 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:52.973604 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8779bb13012348bba5968124338c75b50b725f745273e9595f0648d3732ccbe3"} err="failed to get container status \"8779bb13012348bba5968124338c75b50b725f745273e9595f0648d3732ccbe3\": rpc error: code = NotFound desc = could not find container \"8779bb13012348bba5968124338c75b50b725f745273e9595f0648d3732ccbe3\": container with ID starting with 8779bb13012348bba5968124338c75b50b725f745273e9595f0648d3732ccbe3 not found: ID does not exist" Aug 04 16:09:52.973668 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:52.973620 2574 scope.go:117] "RemoveContainer" containerID="c2d1f9411995d12acb894dd50812b05e5e970b6631a39d6ca53aa8e70873b720" Aug 04 16:09:52.973852 ip-10-0-132-155 kubenswrapper[2574]: E0804 16:09:52.973834 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2d1f9411995d12acb894dd50812b05e5e970b6631a39d6ca53aa8e70873b720\": container with ID starting with c2d1f9411995d12acb894dd50812b05e5e970b6631a39d6ca53aa8e70873b720 not found: ID does not exist" containerID="c2d1f9411995d12acb894dd50812b05e5e970b6631a39d6ca53aa8e70873b720" Aug 04 16:09:52.973902 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:52.973855 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2d1f9411995d12acb894dd50812b05e5e970b6631a39d6ca53aa8e70873b720"} err="failed to get container status \"c2d1f9411995d12acb894dd50812b05e5e970b6631a39d6ca53aa8e70873b720\": rpc error: code = NotFound desc = could not find container \"c2d1f9411995d12acb894dd50812b05e5e970b6631a39d6ca53aa8e70873b720\": container with ID starting with c2d1f9411995d12acb894dd50812b05e5e970b6631a39d6ca53aa8e70873b720 not found: ID does not exist" Aug 04 16:09:52.986926 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:52.986899 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z"] Aug 04 16:09:52.994498 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:52.994478 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-e9e4ac-openshift-default-77f5f8446d-6lh9z"] Aug 04 16:09:53.054225 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:09:53.054157 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ee94230-2660-44c5-bb49-4e4d6b580ab5" path="/var/lib/kubelet/pods/8ee94230-2660-44c5-bb49-4e4d6b580ab5/volumes" Aug 04 16:10:01.625538 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:01.625504 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 16:10:16.421572 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:16.417450 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 16:10:19.000953 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:19.000917 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg"] Aug 04 16:10:19.001447 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:19.001135 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" podUID="3c977f26-4e15-4788-ac7e-b0933b030ac2" containerName="istio-proxy" containerID="cri-o://d0d92338fcbed7591ead42b2a8c8f672a0149b2c576b22ebb9862e57b412ae72" gracePeriod=30 Aug 04 16:10:24.139723 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:24.139687 2574 generic.go:356] "Generic (PLEG): container finished" podID="3c977f26-4e15-4788-ac7e-b0933b030ac2" containerID="d0d92338fcbed7591ead42b2a8c8f672a0149b2c576b22ebb9862e57b412ae72" exitCode=0 Aug 04 16:10:24.140238 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:24.139760 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" event={"ID":"3c977f26-4e15-4788-ac7e-b0933b030ac2","Type":"ContainerDied","Data":"d0d92338fcbed7591ead42b2a8c8f672a0149b2c576b22ebb9862e57b412ae72"} Aug 04 16:10:24.140238 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:24.139823 2574 scope.go:117] "RemoveContainer" containerID="91f819113cef6db375d6af01f1ce282b7078fc5dbff2e612fb150ed1b55f3461" Aug 04 16:10:24.268468 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:24.268414 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" Aug 04 16:10:24.405317 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:24.405285 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c977f26-4e15-4788-ac7e-b0933b030ac2-istio-token\") pod \"3c977f26-4e15-4788-ac7e-b0933b030ac2\" (UID: \"3c977f26-4e15-4788-ac7e-b0933b030ac2\") " Aug 04 16:10:24.405511 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:24.405335 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c977f26-4e15-4788-ac7e-b0933b030ac2-istiod-ca-cert\") pod \"3c977f26-4e15-4788-ac7e-b0933b030ac2\" (UID: \"3c977f26-4e15-4788-ac7e-b0933b030ac2\") " Aug 04 16:10:24.405511 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:24.405430 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c977f26-4e15-4788-ac7e-b0933b030ac2-credential-socket\") pod \"3c977f26-4e15-4788-ac7e-b0933b030ac2\" (UID: \"3c977f26-4e15-4788-ac7e-b0933b030ac2\") " Aug 04 16:10:24.405588 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:24.405538 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zgdvf\" (UniqueName: \"kubernetes.io/projected/3c977f26-4e15-4788-ac7e-b0933b030ac2-kube-api-access-zgdvf\") pod \"3c977f26-4e15-4788-ac7e-b0933b030ac2\" (UID: \"3c977f26-4e15-4788-ac7e-b0933b030ac2\") " Aug 04 16:10:24.405623 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:24.405589 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c977f26-4e15-4788-ac7e-b0933b030ac2-workload-socket\") pod \"3c977f26-4e15-4788-ac7e-b0933b030ac2\" (UID: \"3c977f26-4e15-4788-ac7e-b0933b030ac2\") " Aug 04 16:10:24.405661 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:24.405618 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c977f26-4e15-4788-ac7e-b0933b030ac2-istio-data\") pod \"3c977f26-4e15-4788-ac7e-b0933b030ac2\" (UID: \"3c977f26-4e15-4788-ac7e-b0933b030ac2\") " Aug 04 16:10:24.405661 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:24.405650 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c977f26-4e15-4788-ac7e-b0933b030ac2-istio-podinfo\") pod \"3c977f26-4e15-4788-ac7e-b0933b030ac2\" (UID: \"3c977f26-4e15-4788-ac7e-b0933b030ac2\") " Aug 04 16:10:24.405764 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:24.405696 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c977f26-4e15-4788-ac7e-b0933b030ac2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3c977f26-4e15-4788-ac7e-b0933b030ac2" (UID: "3c977f26-4e15-4788-ac7e-b0933b030ac2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:10:24.405764 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:24.405705 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c977f26-4e15-4788-ac7e-b0933b030ac2-istio-envoy\") pod \"3c977f26-4e15-4788-ac7e-b0933b030ac2\" (UID: \"3c977f26-4e15-4788-ac7e-b0933b030ac2\") " Aug 04 16:10:24.405764 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:24.405757 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c977f26-4e15-4788-ac7e-b0933b030ac2-workload-certs\") pod \"3c977f26-4e15-4788-ac7e-b0933b030ac2\" (UID: \"3c977f26-4e15-4788-ac7e-b0933b030ac2\") " Aug 04 16:10:24.405917 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:24.405769 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c977f26-4e15-4788-ac7e-b0933b030ac2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3c977f26-4e15-4788-ac7e-b0933b030ac2" (UID: "3c977f26-4e15-4788-ac7e-b0933b030ac2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:10:24.405970 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:24.405918 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c977f26-4e15-4788-ac7e-b0933b030ac2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3c977f26-4e15-4788-ac7e-b0933b030ac2" (UID: "3c977f26-4e15-4788-ac7e-b0933b030ac2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:10:24.406112 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:24.406091 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c977f26-4e15-4788-ac7e-b0933b030ac2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3c977f26-4e15-4788-ac7e-b0933b030ac2" (UID: "3c977f26-4e15-4788-ac7e-b0933b030ac2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:10:24.406181 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:24.406152 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c977f26-4e15-4788-ac7e-b0933b030ac2-workload-socket\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:10:24.406181 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:24.406175 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c977f26-4e15-4788-ac7e-b0933b030ac2-istiod-ca-cert\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:10:24.406285 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:24.406191 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c977f26-4e15-4788-ac7e-b0933b030ac2-credential-socket\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:10:24.406574 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:24.406549 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c977f26-4e15-4788-ac7e-b0933b030ac2-istio-data" (OuterVolumeSpecName: "istio-data") pod "3c977f26-4e15-4788-ac7e-b0933b030ac2" (UID: "3c977f26-4e15-4788-ac7e-b0933b030ac2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:10:24.407723 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:24.407693 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c977f26-4e15-4788-ac7e-b0933b030ac2-istio-token" (OuterVolumeSpecName: "istio-token") pod "3c977f26-4e15-4788-ac7e-b0933b030ac2" (UID: "3c977f26-4e15-4788-ac7e-b0933b030ac2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:10:24.407877 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:24.407860 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3c977f26-4e15-4788-ac7e-b0933b030ac2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3c977f26-4e15-4788-ac7e-b0933b030ac2" (UID: "3c977f26-4e15-4788-ac7e-b0933b030ac2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 16:10:24.408142 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:24.408112 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c977f26-4e15-4788-ac7e-b0933b030ac2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3c977f26-4e15-4788-ac7e-b0933b030ac2" (UID: "3c977f26-4e15-4788-ac7e-b0933b030ac2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:10:24.408142 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:24.408128 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c977f26-4e15-4788-ac7e-b0933b030ac2-kube-api-access-zgdvf" (OuterVolumeSpecName: "kube-api-access-zgdvf") pod "3c977f26-4e15-4788-ac7e-b0933b030ac2" (UID: "3c977f26-4e15-4788-ac7e-b0933b030ac2"). InnerVolumeSpecName "kube-api-access-zgdvf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:10:24.507353 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:24.507316 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zgdvf\" (UniqueName: \"kubernetes.io/projected/3c977f26-4e15-4788-ac7e-b0933b030ac2-kube-api-access-zgdvf\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:10:24.507353 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:24.507347 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c977f26-4e15-4788-ac7e-b0933b030ac2-istio-data\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:10:24.507353 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:24.507357 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c977f26-4e15-4788-ac7e-b0933b030ac2-istio-podinfo\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:10:24.507612 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:24.507365 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c977f26-4e15-4788-ac7e-b0933b030ac2-istio-envoy\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:10:24.507612 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:24.507373 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c977f26-4e15-4788-ac7e-b0933b030ac2-workload-certs\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:10:24.507612 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:24.507380 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c977f26-4e15-4788-ac7e-b0933b030ac2-istio-token\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:10:25.150419 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:25.150378 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" Aug 04 16:10:25.150853 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:25.150425 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg" event={"ID":"3c977f26-4e15-4788-ac7e-b0933b030ac2","Type":"ContainerDied","Data":"88d8b5f1b7649e28b1830634d649161fcc8e3e300ddd90b1f456509a9963df4a"} Aug 04 16:10:25.150853 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:25.150480 2574 scope.go:117] "RemoveContainer" containerID="d0d92338fcbed7591ead42b2a8c8f672a0149b2c576b22ebb9862e57b412ae72" Aug 04 16:10:25.170921 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:25.170893 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg"] Aug 04 16:10:25.178201 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:25.178171 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-05b2bc-openshift-default-779f768b4d-z7drg"] Aug 04 16:10:27.054049 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:27.054015 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c977f26-4e15-4788-ac7e-b0933b030ac2" path="/var/lib/kubelet/pods/3c977f26-4e15-4788-ac7e-b0933b030ac2/volumes" Aug 04 16:10:32.729382 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:32.729343 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-gpqx7"] Aug 04 16:10:37.150489 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:37.150456 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb"] Aug 04 16:10:37.150906 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:37.150678 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" podUID="d5f28101-5e3f-445e-ab77-dd5d594e6b12" containerName="istio-proxy" containerID="cri-o://38e167a78eefab81081111f130bfdb4a848f1243e15efe37a84f542ef1dcad05" gracePeriod=30 Aug 04 16:10:42.412006 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:42.411977 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" Aug 04 16:10:42.466328 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:42.466247 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5f28101-5e3f-445e-ab77-dd5d594e6b12-istiod-ca-cert\") pod \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\" (UID: \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\") " Aug 04 16:10:42.466328 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:42.466298 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5f28101-5e3f-445e-ab77-dd5d594e6b12-istio-envoy\") pod \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\" (UID: \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\") " Aug 04 16:10:42.466576 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:42.466344 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5f28101-5e3f-445e-ab77-dd5d594e6b12-istio-podinfo\") pod \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\" (UID: \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\") " Aug 04 16:10:42.466576 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:42.466363 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5f28101-5e3f-445e-ab77-dd5d594e6b12-credential-socket\") pod \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\" (UID: \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\") " Aug 04 16:10:42.466576 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:42.466414 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5f28101-5e3f-445e-ab77-dd5d594e6b12-workload-socket\") pod \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\" (UID: \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\") " Aug 04 16:10:42.466576 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:42.466444 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k9ck5\" (UniqueName: \"kubernetes.io/projected/d5f28101-5e3f-445e-ab77-dd5d594e6b12-kube-api-access-k9ck5\") pod \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\" (UID: \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\") " Aug 04 16:10:42.466576 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:42.466460 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5f28101-5e3f-445e-ab77-dd5d594e6b12-istio-token\") pod \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\" (UID: \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\") " Aug 04 16:10:42.466576 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:42.466486 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5f28101-5e3f-445e-ab77-dd5d594e6b12-workload-certs\") pod \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\" (UID: \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\") " Aug 04 16:10:42.466576 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:42.466516 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5f28101-5e3f-445e-ab77-dd5d594e6b12-istio-data\") pod \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\" (UID: \"d5f28101-5e3f-445e-ab77-dd5d594e6b12\") " Aug 04 16:10:42.466922 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:42.466709 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5f28101-5e3f-445e-ab77-dd5d594e6b12-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d5f28101-5e3f-445e-ab77-dd5d594e6b12" (UID: "d5f28101-5e3f-445e-ab77-dd5d594e6b12"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:10:42.466922 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:42.466723 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5f28101-5e3f-445e-ab77-dd5d594e6b12-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d5f28101-5e3f-445e-ab77-dd5d594e6b12" (UID: "d5f28101-5e3f-445e-ab77-dd5d594e6b12"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:10:42.466922 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:42.466772 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5f28101-5e3f-445e-ab77-dd5d594e6b12-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d5f28101-5e3f-445e-ab77-dd5d594e6b12" (UID: "d5f28101-5e3f-445e-ab77-dd5d594e6b12"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Aug 04 16:10:42.466922 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:42.466867 2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5f28101-5e3f-445e-ab77-dd5d594e6b12-credential-socket\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:10:42.466922 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:42.466886 2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d5f28101-5e3f-445e-ab77-dd5d594e6b12-workload-socket\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:10:42.466922 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:42.466900 2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d5f28101-5e3f-445e-ab77-dd5d594e6b12-istiod-ca-cert\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:10:42.467237 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:42.467002 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5f28101-5e3f-445e-ab77-dd5d594e6b12-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d5f28101-5e3f-445e-ab77-dd5d594e6b12" (UID: "d5f28101-5e3f-445e-ab77-dd5d594e6b12"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:10:42.467543 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:42.467523 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5f28101-5e3f-445e-ab77-dd5d594e6b12-istio-data" (OuterVolumeSpecName: "istio-data") pod "d5f28101-5e3f-445e-ab77-dd5d594e6b12" (UID: "d5f28101-5e3f-445e-ab77-dd5d594e6b12"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:10:42.468669 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:42.468647 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5f28101-5e3f-445e-ab77-dd5d594e6b12-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d5f28101-5e3f-445e-ab77-dd5d594e6b12" (UID: "d5f28101-5e3f-445e-ab77-dd5d594e6b12"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Aug 04 16:10:42.468774 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:42.468684 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5f28101-5e3f-445e-ab77-dd5d594e6b12-kube-api-access-k9ck5" (OuterVolumeSpecName: "kube-api-access-k9ck5") pod "d5f28101-5e3f-445e-ab77-dd5d594e6b12" (UID: "d5f28101-5e3f-445e-ab77-dd5d594e6b12"). InnerVolumeSpecName "kube-api-access-k9ck5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:10:42.469044 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:42.469022 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d5f28101-5e3f-445e-ab77-dd5d594e6b12-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d5f28101-5e3f-445e-ab77-dd5d594e6b12" (UID: "d5f28101-5e3f-445e-ab77-dd5d594e6b12"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Aug 04 16:10:42.469139 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:42.469041 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5f28101-5e3f-445e-ab77-dd5d594e6b12-istio-token" (OuterVolumeSpecName: "istio-token") pod "d5f28101-5e3f-445e-ab77-dd5d594e6b12" (UID: "d5f28101-5e3f-445e-ab77-dd5d594e6b12"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Aug 04 16:10:42.567880 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:42.567837 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k9ck5\" (UniqueName: \"kubernetes.io/projected/d5f28101-5e3f-445e-ab77-dd5d594e6b12-kube-api-access-k9ck5\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:10:42.567880 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:42.567878 2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d5f28101-5e3f-445e-ab77-dd5d594e6b12-istio-token\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:10:42.567880 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:42.567888 2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d5f28101-5e3f-445e-ab77-dd5d594e6b12-workload-certs\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:10:42.567880 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:42.567896 2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d5f28101-5e3f-445e-ab77-dd5d594e6b12-istio-data\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:10:42.568138 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:42.567903 2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d5f28101-5e3f-445e-ab77-dd5d594e6b12-istio-envoy\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:10:42.568138 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:42.567910 2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d5f28101-5e3f-445e-ab77-dd5d594e6b12-istio-podinfo\") on node \"ip-10-0-132-155.ec2.internal\" DevicePath \"\"" Aug 04 16:10:43.257233 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:43.257199 2574 generic.go:356] "Generic (PLEG): container finished" podID="d5f28101-5e3f-445e-ab77-dd5d594e6b12" containerID="38e167a78eefab81081111f130bfdb4a848f1243e15efe37a84f542ef1dcad05" exitCode=0 Aug 04 16:10:43.257455 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:43.257245 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" Aug 04 16:10:43.257455 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:43.257266 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" event={"ID":"d5f28101-5e3f-445e-ab77-dd5d594e6b12","Type":"ContainerDied","Data":"38e167a78eefab81081111f130bfdb4a848f1243e15efe37a84f542ef1dcad05"} Aug 04 16:10:43.257455 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:43.257318 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb" event={"ID":"d5f28101-5e3f-445e-ab77-dd5d594e6b12","Type":"ContainerDied","Data":"3462b6a3e5a35b754334014bbcb3febafccab975acb27dcc1e5499ffdddbffee"} Aug 04 16:10:43.257455 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:43.257349 2574 scope.go:117] "RemoveContainer" containerID="38e167a78eefab81081111f130bfdb4a848f1243e15efe37a84f542ef1dcad05" Aug 04 16:10:43.266740 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:43.266721 2574 scope.go:117] "RemoveContainer" containerID="f9d4332625914b88904dfb19ae75a79187a09810b212603545413708c36a77af" Aug 04 16:10:43.275001 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:43.274805 2574 scope.go:117] "RemoveContainer" containerID="38e167a78eefab81081111f130bfdb4a848f1243e15efe37a84f542ef1dcad05" Aug 04 16:10:43.275072 ip-10-0-132-155 kubenswrapper[2574]: E0804 16:10:43.275054 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38e167a78eefab81081111f130bfdb4a848f1243e15efe37a84f542ef1dcad05\": container with ID starting with 38e167a78eefab81081111f130bfdb4a848f1243e15efe37a84f542ef1dcad05 not found: ID does not exist" containerID="38e167a78eefab81081111f130bfdb4a848f1243e15efe37a84f542ef1dcad05" Aug 04 16:10:43.275116 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:43.275080 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38e167a78eefab81081111f130bfdb4a848f1243e15efe37a84f542ef1dcad05"} err="failed to get container status \"38e167a78eefab81081111f130bfdb4a848f1243e15efe37a84f542ef1dcad05\": rpc error: code = NotFound desc = could not find container \"38e167a78eefab81081111f130bfdb4a848f1243e15efe37a84f542ef1dcad05\": container with ID starting with 38e167a78eefab81081111f130bfdb4a848f1243e15efe37a84f542ef1dcad05 not found: ID does not exist" Aug 04 16:10:43.275116 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:43.275096 2574 scope.go:117] "RemoveContainer" containerID="f9d4332625914b88904dfb19ae75a79187a09810b212603545413708c36a77af" Aug 04 16:10:43.275381 ip-10-0-132-155 kubenswrapper[2574]: E0804 16:10:43.275355 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9d4332625914b88904dfb19ae75a79187a09810b212603545413708c36a77af\": container with ID starting with f9d4332625914b88904dfb19ae75a79187a09810b212603545413708c36a77af not found: ID does not exist" containerID="f9d4332625914b88904dfb19ae75a79187a09810b212603545413708c36a77af" Aug 04 16:10:43.275505 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:43.275400 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9d4332625914b88904dfb19ae75a79187a09810b212603545413708c36a77af"} err="failed to get container status \"f9d4332625914b88904dfb19ae75a79187a09810b212603545413708c36a77af\": rpc error: code = NotFound desc = could not find container \"f9d4332625914b88904dfb19ae75a79187a09810b212603545413708c36a77af\": container with ID starting with f9d4332625914b88904dfb19ae75a79187a09810b212603545413708c36a77af not found: ID does not exist" Aug 04 16:10:43.278341 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:43.278319 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb"] Aug 04 16:10:43.282498 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:43.282479 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-7b41f3-openshift-default-599b6cbdc6-m2mvb"] Aug 04 16:10:43.490845 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:43.490818 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-w9wtz_632ef889-6c6f-437f-9af4-ba9e3b00bf2a/manager/0.log" Aug 04 16:10:43.617955 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:43.617928 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-tkrnj_b7992fa0-a315-4350-ba4c-0704718db979/manager/0.log" Aug 04 16:10:43.865417 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:43.865331 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-dnbrf_dbb8d592-c4e9-4263-bb54-7656658e165b/server/0.log" Aug 04 16:10:43.988832 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:43.988803 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-q4t7p_7aaa449d-38a3-4003-af61-e36cbab919c5/manager/0.log" Aug 04 16:10:44.414005 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:44.413973 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-764fcd8dc-mtplw_6d4760e1-d114-4df8-b496-5c91481b1c24/manager/0.log" Aug 04 16:10:44.421380 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:44.421361 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-764fcd8dc-mtplw_6d4760e1-d114-4df8-b496-5c91481b1c24/copy-manifests/0.log" Aug 04 16:10:45.053982 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:45.053950 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5f28101-5e3f-445e-ab77-dd5d594e6b12" path="/var/lib/kubelet/pods/d5f28101-5e3f-445e-ab77-dd5d594e6b12/volumes" Aug 04 16:10:46.100535 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:46.100510 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lxrwj_53d8340a-04de-4a94-9865-a6259ed37066/util/0.log" Aug 04 16:10:46.107380 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:46.107359 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lxrwj_53d8340a-04de-4a94-9865-a6259ed37066/pull/0.log" Aug 04 16:10:46.113683 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:46.113666 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lxrwj_53d8340a-04de-4a94-9865-a6259ed37066/extract/0.log" Aug 04 16:10:46.230331 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:46.230303 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06lzhh_04d06a27-7273-48eb-bc1b-fbe7ac3ff51e/util/0.log" Aug 04 16:10:46.240770 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:46.240744 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06lzhh_04d06a27-7273-48eb-bc1b-fbe7ac3ff51e/pull/0.log" Aug 04 16:10:46.247846 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:46.247827 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06lzhh_04d06a27-7273-48eb-bc1b-fbe7ac3ff51e/extract/0.log" Aug 04 16:10:46.366759 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:46.366685 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738s8t6_a6e10417-aaa7-426d-9d7a-8f946e766d35/util/0.log" Aug 04 16:10:46.376170 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:46.376149 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738s8t6_a6e10417-aaa7-426d-9d7a-8f946e766d35/pull/0.log" Aug 04 16:10:46.383205 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:46.383164 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738s8t6_a6e10417-aaa7-426d-9d7a-8f946e766d35/extract/0.log" Aug 04 16:10:46.505222 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:46.505189 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hsrn7_034b04b9-45c8-4ee1-b3cf-98754b6c93d0/util/0.log" Aug 04 16:10:46.512651 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:46.512624 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hsrn7_034b04b9-45c8-4ee1-b3cf-98754b6c93d0/pull/0.log" Aug 04 16:10:46.520124 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:46.520103 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hsrn7_034b04b9-45c8-4ee1-b3cf-98754b6c93d0/extract/0.log" Aug 04 16:10:46.766886 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:46.766849 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-qvxnw_00bbf14a-0e93-4b87-b93f-a894364f6ede/manager/0.log" Aug 04 16:10:46.883959 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:46.883906 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-vczk2_5d0b639a-d4fe-4409-aa4a-5a4a8db4e889/manager/0.log" Aug 04 16:10:46.997626 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:46.997602 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-sp4x9_bdacdb71-39a3-4cb7-9c61-8c5844764a09/kuadrant-console-plugin/0.log" Aug 04 16:10:47.115034 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:47.114929 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-2vwn8_07a74b29-3084-41c0-b79f-5848579687a8/registry-server/0.log" Aug 04 16:10:47.353975 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:47.353931 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-gpqx7_2ce38caa-ed33-46bf-854c-ee78469016c1/limitador/0.log" Aug 04 16:10:47.474571 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:47.474501 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-p7g8l_f9fc1877-be08-44d4-b5fc-5b5349718121/manager/0.log" Aug 04 16:10:47.866337 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:47.866308 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9_5c6a911a-c21d-4ac0-beeb-c491710bfb72/istio-proxy/0.log" Aug 04 16:10:48.109560 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:48.109519 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c7cf99455-6rvm7_6f18365b-4250-4cd6-8217-9f5efdc88129/kube-auth-proxy/0.log" Aug 04 16:10:48.354894 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:48.354846 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-27rnn_6dd49fc0-1d34-4825-a841-249a773b23ae/istio-proxy/1.log" Aug 04 16:10:48.732722 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:48.732623 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7fc97bc64b-45b4c_d201490f-504d-4d2a-8b4f-c4eea2023469/router/0.log" Aug 04 16:10:49.582830 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:49.582787 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-ztjrs_800feb8e-32f3-4d4d-bb6d-262d6a1bafe5/main/0.log" Aug 04 16:10:49.698216 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:49.698192 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-tscq2_a6a6ac91-159b-46ae-8b44-c8b6d490a3c3/main/0.log" Aug 04 16:10:55.714237 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:55.714197 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-sz4ht/must-gather-mdmrj"] Aug 04 16:10:55.714663 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:55.714577 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfaf1ca8-4369-4a6d-aa9f-28e9390aab63" containerName="main" Aug 04 16:10:55.714663 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:55.714588 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfaf1ca8-4369-4a6d-aa9f-28e9390aab63" containerName="main" Aug 04 16:10:55.714663 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:55.714604 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5f28101-5e3f-445e-ab77-dd5d594e6b12" containerName="istio-proxy" Aug 04 16:10:55.714663 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:55.714609 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5f28101-5e3f-445e-ab77-dd5d594e6b12" containerName="istio-proxy" Aug 04 16:10:55.714663 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:55.714617 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c977f26-4e15-4788-ac7e-b0933b030ac2" containerName="istio-proxy" Aug 04 16:10:55.714663 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:55.714623 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c977f26-4e15-4788-ac7e-b0933b030ac2" containerName="istio-proxy" Aug 04 16:10:55.714663 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:55.714629 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ee94230-2660-44c5-bb49-4e4d6b580ab5" containerName="istio-proxy" Aug 04 16:10:55.714663 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:55.714634 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ee94230-2660-44c5-bb49-4e4d6b580ab5" containerName="istio-proxy" Aug 04 16:10:55.714663 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:55.714645 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ee94230-2660-44c5-bb49-4e4d6b580ab5" containerName="istio-proxy" Aug 04 16:10:55.714663 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:55.714650 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ee94230-2660-44c5-bb49-4e4d6b580ab5" containerName="istio-proxy" Aug 04 16:10:55.714663 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:55.714657 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c977f26-4e15-4788-ac7e-b0933b030ac2" containerName="istio-proxy" Aug 04 16:10:55.714663 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:55.714664 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c977f26-4e15-4788-ac7e-b0933b030ac2" containerName="istio-proxy" Aug 04 16:10:55.715078 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:55.714673 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5f28101-5e3f-445e-ab77-dd5d594e6b12" containerName="istio-proxy" Aug 04 16:10:55.715078 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:55.714679 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5f28101-5e3f-445e-ab77-dd5d594e6b12" containerName="istio-proxy" Aug 04 16:10:55.715078 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:55.714738 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c977f26-4e15-4788-ac7e-b0933b030ac2" containerName="istio-proxy" Aug 04 16:10:55.715078 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:55.714746 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5f28101-5e3f-445e-ab77-dd5d594e6b12" containerName="istio-proxy" Aug 04 16:10:55.715078 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:55.714754 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ee94230-2660-44c5-bb49-4e4d6b580ab5" containerName="istio-proxy" Aug 04 16:10:55.715078 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:55.714759 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfaf1ca8-4369-4a6d-aa9f-28e9390aab63" containerName="main" Aug 04 16:10:55.715078 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:55.714765 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ee94230-2660-44c5-bb49-4e4d6b580ab5" containerName="istio-proxy" Aug 04 16:10:55.715078 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:55.714898 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c977f26-4e15-4788-ac7e-b0933b030ac2" containerName="istio-proxy" Aug 04 16:10:55.715078 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:55.714904 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5f28101-5e3f-445e-ab77-dd5d594e6b12" containerName="istio-proxy" Aug 04 16:10:55.719515 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:55.719499 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sz4ht/must-gather-mdmrj" Aug 04 16:10:55.722723 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:55.722702 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sz4ht\"/\"openshift-service-ca.crt\"" Aug 04 16:10:55.722950 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:55.722934 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-sz4ht\"/\"default-dockercfg-twtxh\"" Aug 04 16:10:55.723996 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:55.723982 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sz4ht\"/\"kube-root-ca.crt\"" Aug 04 16:10:55.731494 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:55.731468 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sz4ht/must-gather-mdmrj"] Aug 04 16:10:55.786315 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:55.786285 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/fccd5614-a077-4c3a-b47c-ded3318eddf3-must-gather-output\") pod \"must-gather-mdmrj\" (UID: \"fccd5614-a077-4c3a-b47c-ded3318eddf3\") " pod="openshift-must-gather-sz4ht/must-gather-mdmrj" Aug 04 16:10:55.786462 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:55.786349 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bt4rz\" (UniqueName: \"kubernetes.io/projected/fccd5614-a077-4c3a-b47c-ded3318eddf3-kube-api-access-bt4rz\") pod \"must-gather-mdmrj\" (UID: \"fccd5614-a077-4c3a-b47c-ded3318eddf3\") " pod="openshift-must-gather-sz4ht/must-gather-mdmrj" Aug 04 16:10:55.887216 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:55.887184 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bt4rz\" (UniqueName: \"kubernetes.io/projected/fccd5614-a077-4c3a-b47c-ded3318eddf3-kube-api-access-bt4rz\") pod \"must-gather-mdmrj\" (UID: \"fccd5614-a077-4c3a-b47c-ded3318eddf3\") " pod="openshift-must-gather-sz4ht/must-gather-mdmrj" Aug 04 16:10:55.887376 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:55.887240 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/fccd5614-a077-4c3a-b47c-ded3318eddf3-must-gather-output\") pod \"must-gather-mdmrj\" (UID: \"fccd5614-a077-4c3a-b47c-ded3318eddf3\") " pod="openshift-must-gather-sz4ht/must-gather-mdmrj" Aug 04 16:10:55.887584 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:55.887568 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/fccd5614-a077-4c3a-b47c-ded3318eddf3-must-gather-output\") pod \"must-gather-mdmrj\" (UID: \"fccd5614-a077-4c3a-b47c-ded3318eddf3\") " pod="openshift-must-gather-sz4ht/must-gather-mdmrj" Aug 04 16:10:55.897539 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:55.897516 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bt4rz\" (UniqueName: \"kubernetes.io/projected/fccd5614-a077-4c3a-b47c-ded3318eddf3-kube-api-access-bt4rz\") pod \"must-gather-mdmrj\" (UID: \"fccd5614-a077-4c3a-b47c-ded3318eddf3\") " pod="openshift-must-gather-sz4ht/must-gather-mdmrj" Aug 04 16:10:56.028893 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:56.028862 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sz4ht/must-gather-mdmrj" Aug 04 16:10:56.166961 ip-10-0-132-155 kubenswrapper[2574]: W0804 16:10:56.166922 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfccd5614_a077_4c3a_b47c_ded3318eddf3.slice/crio-2c06d4b756611251314ebca075a230ff5642edd0161c067f7d635361cfcb0244 WatchSource:0}: Error finding container 2c06d4b756611251314ebca075a230ff5642edd0161c067f7d635361cfcb0244: Status 404 returned error can't find the container with id 2c06d4b756611251314ebca075a230ff5642edd0161c067f7d635361cfcb0244 Aug 04 16:10:56.168177 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:56.168153 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sz4ht/must-gather-mdmrj"] Aug 04 16:10:56.168924 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:56.168904 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Aug 04 16:10:56.328922 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:56.328838 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sz4ht/must-gather-mdmrj" event={"ID":"fccd5614-a077-4c3a-b47c-ded3318eddf3","Type":"ContainerStarted","Data":"2c06d4b756611251314ebca075a230ff5642edd0161c067f7d635361cfcb0244"} Aug 04 16:10:57.339277 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:57.339241 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sz4ht/must-gather-mdmrj" event={"ID":"fccd5614-a077-4c3a-b47c-ded3318eddf3","Type":"ContainerStarted","Data":"238534e825e0ea7045616569548ec4e5ed4ab0585abbdd347341252863d87cc9"} Aug 04 16:10:57.339698 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:57.339289 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sz4ht/must-gather-mdmrj" event={"ID":"fccd5614-a077-4c3a-b47c-ded3318eddf3","Type":"ContainerStarted","Data":"671205ca7fe8794b14dffa15a3e3e1da81b631efc43b0473d09752df688cfedb"} Aug 04 16:10:57.370220 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:57.370154 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-sz4ht/must-gather-mdmrj" podStartSLOduration=1.646047427 podStartE2EDuration="2.370110115s" totalImagesPullingTime="724.062688ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-04 16:10:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:10:57.368039786 +0000 UTC m=+3880.827912141" watchObservedRunningTime="2026-08-04 16:10:57.370110115 +0000 UTC m=+3880.829982512" Aug 04 16:10:58.788524 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:58.788491 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xxmsv_46bd5ac9-b189-4e6c-b2c6-df0ed33641e2/global-pull-secret-syncer/0.log" Aug 04 16:10:58.903369 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:58.903338 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-xmh7f_214ec271-9cb1-4e3b-9b10-b7d15c10333f/konnectivity-agent/0.log" Aug 04 16:10:58.968288 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:10:58.968254 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-155.ec2.internal_b47c00155e76867b44c5cd48f4a5b502/haproxy/0.log" Aug 04 16:11:02.572097 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:02.572063 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lxrwj_53d8340a-04de-4a94-9865-a6259ed37066/extract/0.log" Aug 04 16:11:02.594927 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:02.594835 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lxrwj_53d8340a-04de-4a94-9865-a6259ed37066/util/0.log" Aug 04 16:11:02.629050 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:02.629023 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759lxrwj_53d8340a-04de-4a94-9865-a6259ed37066/pull/0.log" Aug 04 16:11:02.692237 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:02.692208 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06lzhh_04d06a27-7273-48eb-bc1b-fbe7ac3ff51e/extract/0.log" Aug 04 16:11:02.718762 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:02.718733 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06lzhh_04d06a27-7273-48eb-bc1b-fbe7ac3ff51e/util/0.log" Aug 04 16:11:02.753461 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:02.753424 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06lzhh_04d06a27-7273-48eb-bc1b-fbe7ac3ff51e/pull/0.log" Aug 04 16:11:02.882316 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:02.882240 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738s8t6_a6e10417-aaa7-426d-9d7a-8f946e766d35/extract/0.log" Aug 04 16:11:02.922932 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:02.922905 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738s8t6_a6e10417-aaa7-426d-9d7a-8f946e766d35/util/0.log" Aug 04 16:11:02.962592 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:02.962135 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738s8t6_a6e10417-aaa7-426d-9d7a-8f946e766d35/pull/0.log" Aug 04 16:11:02.999232 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:02.999201 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hsrn7_034b04b9-45c8-4ee1-b3cf-98754b6c93d0/extract/0.log" Aug 04 16:11:03.038376 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:03.038338 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hsrn7_034b04b9-45c8-4ee1-b3cf-98754b6c93d0/util/0.log" Aug 04 16:11:03.076825 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:03.076356 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hsrn7_034b04b9-45c8-4ee1-b3cf-98754b6c93d0/pull/0.log" Aug 04 16:11:03.846450 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:03.846419 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-qvxnw_00bbf14a-0e93-4b87-b93f-a894364f6ede/manager/0.log" Aug 04 16:11:03.888783 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:03.888720 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-vczk2_5d0b639a-d4fe-4409-aa4a-5a4a8db4e889/manager/0.log" Aug 04 16:11:03.934996 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:03.934958 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-sp4x9_bdacdb71-39a3-4cb7-9c61-8c5844764a09/kuadrant-console-plugin/0.log" Aug 04 16:11:04.024170 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:04.024080 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-2vwn8_07a74b29-3084-41c0-b79f-5848579687a8/registry-server/0.log" Aug 04 16:11:04.170091 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:04.169977 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-gpqx7_2ce38caa-ed33-46bf-854c-ee78469016c1/limitador/0.log" Aug 04 16:11:04.297820 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:04.297789 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-p7g8l_f9fc1877-be08-44d4-b5fc-5b5349718121/manager/0.log" Aug 04 16:11:06.295150 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:06.295116 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6f55c4f58d-6qdqp_e46dbe2d-eb92-47cd-9bea-47228f399ba3/cluster-monitoring-operator/0.log" Aug 04 16:11:06.496588 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:06.496536 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mgmxj_c1225731-e66d-44a6-82db-3f00d98225fe/node-exporter/0.log" Aug 04 16:11:06.528238 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:06.528210 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mgmxj_c1225731-e66d-44a6-82db-3f00d98225fe/kube-rbac-proxy/0.log" Aug 04 16:11:06.558270 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:06.558197 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mgmxj_c1225731-e66d-44a6-82db-3f00d98225fe/init-textfile/0.log" Aug 04 16:11:08.626722 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:08.626684 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-sz4ht/perf-node-gather-daemonset-q2sln"] Aug 04 16:11:08.632037 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:08.632007 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sz4ht/perf-node-gather-daemonset-q2sln" Aug 04 16:11:08.649579 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:08.649548 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sz4ht/perf-node-gather-daemonset-q2sln"] Aug 04 16:11:08.734491 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:08.734445 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4z7fh\" (UniqueName: \"kubernetes.io/projected/80ab6b01-ffce-4932-9f59-8fcf584e5aab-kube-api-access-4z7fh\") pod \"perf-node-gather-daemonset-q2sln\" (UID: \"80ab6b01-ffce-4932-9f59-8fcf584e5aab\") " pod="openshift-must-gather-sz4ht/perf-node-gather-daemonset-q2sln" Aug 04 16:11:08.734671 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:08.734495 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/80ab6b01-ffce-4932-9f59-8fcf584e5aab-podres\") pod \"perf-node-gather-daemonset-q2sln\" (UID: \"80ab6b01-ffce-4932-9f59-8fcf584e5aab\") " pod="openshift-must-gather-sz4ht/perf-node-gather-daemonset-q2sln" Aug 04 16:11:08.734671 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:08.734596 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/80ab6b01-ffce-4932-9f59-8fcf584e5aab-lib-modules\") pod \"perf-node-gather-daemonset-q2sln\" (UID: \"80ab6b01-ffce-4932-9f59-8fcf584e5aab\") " pod="openshift-must-gather-sz4ht/perf-node-gather-daemonset-q2sln" Aug 04 16:11:08.734798 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:08.734691 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/80ab6b01-ffce-4932-9f59-8fcf584e5aab-proc\") pod \"perf-node-gather-daemonset-q2sln\" (UID: \"80ab6b01-ffce-4932-9f59-8fcf584e5aab\") " pod="openshift-must-gather-sz4ht/perf-node-gather-daemonset-q2sln" Aug 04 16:11:08.734798 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:08.734763 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/80ab6b01-ffce-4932-9f59-8fcf584e5aab-sys\") pod \"perf-node-gather-daemonset-q2sln\" (UID: \"80ab6b01-ffce-4932-9f59-8fcf584e5aab\") " pod="openshift-must-gather-sz4ht/perf-node-gather-daemonset-q2sln" Aug 04 16:11:08.746501 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:08.746468 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-f77f48c6d-48zkn_e011eeba-5d48-483b-905a-a1f7b7e0ec1f/networking-console-plugin/0.log" Aug 04 16:11:08.835965 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:08.835924 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z7fh\" (UniqueName: \"kubernetes.io/projected/80ab6b01-ffce-4932-9f59-8fcf584e5aab-kube-api-access-4z7fh\") pod \"perf-node-gather-daemonset-q2sln\" (UID: \"80ab6b01-ffce-4932-9f59-8fcf584e5aab\") " pod="openshift-must-gather-sz4ht/perf-node-gather-daemonset-q2sln" Aug 04 16:11:08.836971 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:08.836948 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/80ab6b01-ffce-4932-9f59-8fcf584e5aab-podres\") pod \"perf-node-gather-daemonset-q2sln\" (UID: \"80ab6b01-ffce-4932-9f59-8fcf584e5aab\") " pod="openshift-must-gather-sz4ht/perf-node-gather-daemonset-q2sln" Aug 04 16:11:08.837257 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:08.837241 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/80ab6b01-ffce-4932-9f59-8fcf584e5aab-lib-modules\") pod \"perf-node-gather-daemonset-q2sln\" (UID: \"80ab6b01-ffce-4932-9f59-8fcf584e5aab\") " pod="openshift-must-gather-sz4ht/perf-node-gather-daemonset-q2sln" Aug 04 16:11:08.837542 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:08.837527 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/80ab6b01-ffce-4932-9f59-8fcf584e5aab-proc\") pod \"perf-node-gather-daemonset-q2sln\" (UID: \"80ab6b01-ffce-4932-9f59-8fcf584e5aab\") " pod="openshift-must-gather-sz4ht/perf-node-gather-daemonset-q2sln" Aug 04 16:11:08.837716 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:08.837700 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/80ab6b01-ffce-4932-9f59-8fcf584e5aab-sys\") pod \"perf-node-gather-daemonset-q2sln\" (UID: \"80ab6b01-ffce-4932-9f59-8fcf584e5aab\") " pod="openshift-must-gather-sz4ht/perf-node-gather-daemonset-q2sln" Aug 04 16:11:08.837891 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:08.837864 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/80ab6b01-ffce-4932-9f59-8fcf584e5aab-sys\") pod \"perf-node-gather-daemonset-q2sln\" (UID: \"80ab6b01-ffce-4932-9f59-8fcf584e5aab\") " pod="openshift-must-gather-sz4ht/perf-node-gather-daemonset-q2sln" Aug 04 16:11:08.837969 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:08.837443 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/80ab6b01-ffce-4932-9f59-8fcf584e5aab-lib-modules\") pod \"perf-node-gather-daemonset-q2sln\" (UID: \"80ab6b01-ffce-4932-9f59-8fcf584e5aab\") " pod="openshift-must-gather-sz4ht/perf-node-gather-daemonset-q2sln" Aug 04 16:11:08.837969 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:08.837199 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/80ab6b01-ffce-4932-9f59-8fcf584e5aab-podres\") pod \"perf-node-gather-daemonset-q2sln\" (UID: \"80ab6b01-ffce-4932-9f59-8fcf584e5aab\") " pod="openshift-must-gather-sz4ht/perf-node-gather-daemonset-q2sln" Aug 04 16:11:08.838134 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:08.837827 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/80ab6b01-ffce-4932-9f59-8fcf584e5aab-proc\") pod \"perf-node-gather-daemonset-q2sln\" (UID: \"80ab6b01-ffce-4932-9f59-8fcf584e5aab\") " pod="openshift-must-gather-sz4ht/perf-node-gather-daemonset-q2sln" Aug 04 16:11:08.848619 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:08.848589 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4z7fh\" (UniqueName: \"kubernetes.io/projected/80ab6b01-ffce-4932-9f59-8fcf584e5aab-kube-api-access-4z7fh\") pod \"perf-node-gather-daemonset-q2sln\" (UID: \"80ab6b01-ffce-4932-9f59-8fcf584e5aab\") " pod="openshift-must-gather-sz4ht/perf-node-gather-daemonset-q2sln" Aug 04 16:11:08.951306 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:08.951227 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sz4ht/perf-node-gather-daemonset-q2sln" Aug 04 16:11:09.135892 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:09.135859 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sz4ht/perf-node-gather-daemonset-q2sln"] Aug 04 16:11:09.138139 ip-10-0-132-155 kubenswrapper[2574]: W0804 16:11:09.138096 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod80ab6b01_ffce_4932_9f59_8fcf584e5aab.slice/crio-ed6797226fa621ea803e1cdc03fd38ca97d03c21bcb96845268542f9d5863c4d WatchSource:0}: Error finding container ed6797226fa621ea803e1cdc03fd38ca97d03c21bcb96845268542f9d5863c4d: Status 404 returned error can't find the container with id ed6797226fa621ea803e1cdc03fd38ca97d03c21bcb96845268542f9d5863c4d Aug 04 16:11:09.448225 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:09.448182 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sz4ht/perf-node-gather-daemonset-q2sln" event={"ID":"80ab6b01-ffce-4932-9f59-8fcf584e5aab","Type":"ContainerStarted","Data":"15e935375f77f9a4521c344f54313d1553339536c675a56f8b791e969955b413"} Aug 04 16:11:09.448435 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:09.448259 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sz4ht/perf-node-gather-daemonset-q2sln" event={"ID":"80ab6b01-ffce-4932-9f59-8fcf584e5aab","Type":"ContainerStarted","Data":"ed6797226fa621ea803e1cdc03fd38ca97d03c21bcb96845268542f9d5863c4d"} Aug 04 16:11:09.448435 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:09.448301 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-sz4ht/perf-node-gather-daemonset-q2sln" Aug 04 16:11:09.469723 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:09.469636 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-r8wkp_0580fdab-0381-4d8e-be5f-bde5eb44e0e6/console-operator/1.log" Aug 04 16:11:09.470777 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:09.470709 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-sz4ht/perf-node-gather-daemonset-q2sln" podStartSLOduration=1.470663467 podStartE2EDuration="1.470663467s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-04 16:11:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-04 16:11:09.468440467 +0000 UTC m=+3892.928312817" watchObservedRunningTime="2026-08-04 16:11:09.470663467 +0000 UTC m=+3892.930535815" Aug 04 16:11:09.476346 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:09.476321 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-r8wkp_0580fdab-0381-4d8e-be5f-bde5eb44e0e6/console-operator/2.log" Aug 04 16:11:10.678698 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:10.678668 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5b8776669-r6lsj_f1b0f182-034b-4512-9bae-213071366d67/volume-data-source-validator/0.log" Aug 04 16:11:11.899593 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:11.899556 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hqwr6_ded4fb33-8f70-4475-9a73-e8fff5c7e638/dns/0.log" Aug 04 16:11:11.926001 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:11.925970 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hqwr6_ded4fb33-8f70-4475-9a73-e8fff5c7e638/kube-rbac-proxy/0.log" Aug 04 16:11:12.149887 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:12.149799 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xphfh_7ec6b008-bb9e-42e2-a233-a57bc7449804/dns-node-resolver/0.log" Aug 04 16:11:12.828639 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:12.828610 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ql74b_1e62b3d6-309b-4cf1-bb50-c1d15a0672b1/node-ca/0.log" Aug 04 16:11:13.956919 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:13.956877 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9_5c6a911a-c21d-4ac0-beeb-c491710bfb72/istio-proxy/0.log" Aug 04 16:11:14.123455 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:14.123422 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c7cf99455-6rvm7_6f18365b-4250-4cd6-8217-9f5efdc88129/kube-auth-proxy/0.log" Aug 04 16:11:14.194178 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:14.194150 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-27rnn_6dd49fc0-1d34-4825-a841-249a773b23ae/istio-proxy/0.log" Aug 04 16:11:14.230652 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:14.230479 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-27rnn_6dd49fc0-1d34-4825-a841-249a773b23ae/istio-proxy/1.log" Aug 04 16:11:14.343049 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:14.343020 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7fc97bc64b-45b4c_d201490f-504d-4d2a-8b4f-c4eea2023469/router/0.log" Aug 04 16:11:15.281189 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:15.281158 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rcqrj_43e78d75-11c3-457d-aec1-5d4379ab2470/serve-healthcheck-canary/0.log" Aug 04 16:11:15.477582 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:15.477553 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-sz4ht/perf-node-gather-daemonset-q2sln" Aug 04 16:11:15.828906 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:15.828869 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-77696685b-kpmh2_91c9aec0-0301-40bd-8bbd-ebd389fb4faf/insights-operator/1.log" Aug 04 16:11:15.834050 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:15.834016 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-77696685b-kpmh2_91c9aec0-0301-40bd-8bbd-ebd389fb4faf/insights-operator/0.log" Aug 04 16:11:15.864319 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:15.864282 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kjjf8_dcee583d-d05a-4b0a-bb25-da3cfe507e2b/kube-rbac-proxy/0.log" Aug 04 16:11:15.891225 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:15.891198 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kjjf8_dcee583d-d05a-4b0a-bb25-da3cfe507e2b/exporter/0.log" Aug 04 16:11:15.918961 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:15.918924 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kjjf8_dcee583d-d05a-4b0a-bb25-da3cfe507e2b/extractor/0.log" Aug 04 16:11:17.433432 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:17.433370 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-r8wkp_0580fdab-0381-4d8e-be5f-bde5eb44e0e6/console-operator/1.log" Aug 04 16:11:17.440916 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:17.440890 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8h6k_9c62e3b1-c2dc-4b2a-a61d-7d80049f355f/ovn-acl-logging/0.log" Aug 04 16:11:17.445088 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:17.445059 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-5ccd6b445f-r8wkp_0580fdab-0381-4d8e-be5f-bde5eb44e0e6/console-operator/1.log" Aug 04 16:11:17.449581 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:17.449562 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z8h6k_9c62e3b1-c2dc-4b2a-a61d-7d80049f355f/ovn-acl-logging/0.log" Aug 04 16:11:18.610713 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:18.610685 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-w9wtz_632ef889-6c6f-437f-9af4-ba9e3b00bf2a/manager/0.log" Aug 04 16:11:18.927722 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:18.927687 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-tkrnj_b7992fa0-a315-4350-ba4c-0704718db979/manager/0.log" Aug 04 16:11:19.205119 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:19.205085 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-dnbrf_dbb8d592-c4e9-4263-bb54-7656658e165b/server/0.log" Aug 04 16:11:19.307174 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:19.307137 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-q4t7p_7aaa449d-38a3-4003-af61-e36cbab919c5/manager/0.log" Aug 04 16:11:19.365939 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:19.365908 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9_5c6a911a-c21d-4ac0-beeb-c491710bfb72/istio-proxy/0.log" Aug 04 16:11:19.566745 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:19.566712 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c7cf99455-6rvm7_6f18365b-4250-4cd6-8217-9f5efdc88129/kube-auth-proxy/0.log" Aug 04 16:11:19.571601 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:19.571576 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-764fcd8dc-mtplw_6d4760e1-d114-4df8-b496-5c91481b1c24/manager/0.log" Aug 04 16:11:19.600669 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:19.600638 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-764fcd8dc-mtplw_6d4760e1-d114-4df8-b496-5c91481b1c24/copy-manifests/0.log" Aug 04 16:11:19.611528 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:19.611494 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-27rnn_6dd49fc0-1d34-4825-a841-249a773b23ae/istio-proxy/0.log" Aug 04 16:11:19.653299 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:19.653269 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-27rnn_6dd49fc0-1d34-4825-a841-249a773b23ae/istio-proxy/1.log" Aug 04 16:11:19.733283 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:19.733254 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7fc97bc64b-45b4c_d201490f-504d-4d2a-8b4f-c4eea2023469/router/0.log" Aug 04 16:11:21.226497 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:21.226409 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-l5kp9_5c6a911a-c21d-4ac0-beeb-c491710bfb72/istio-proxy/0.log" Aug 04 16:11:21.394843 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:21.394770 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c7cf99455-6rvm7_6f18365b-4250-4cd6-8217-9f5efdc88129/kube-auth-proxy/0.log" Aug 04 16:11:21.443472 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:21.443445 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-27rnn_6dd49fc0-1d34-4825-a841-249a773b23ae/istio-proxy/0.log" Aug 04 16:11:21.482865 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:21.482839 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-27rnn_6dd49fc0-1d34-4825-a841-249a773b23ae/istio-proxy/1.log" Aug 04 16:11:21.568552 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:21.568524 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7fc97bc64b-45b4c_d201490f-504d-4d2a-8b4f-c4eea2023469/router/0.log" Aug 04 16:11:26.449696 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:26.449662 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7878f4b88c-tqzqn_1a2285d4-f4ce-47be-80d4-7b4be7f930b1/kube-storage-version-migrator-operator/1.log" Aug 04 16:11:26.450859 ip-10-0-132-155 kubenswrapper[2574]: I0804 16:11:26.450817 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7878f4b88c-tqzqn_1a2285d4-f4ce-47be-80d4-7b4be7f930b1/kube-storage-version-migrator-operator/0.log"