Jul 21 13:42:15.159265 ip-10-0-132-245 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 21 13:42:15.159276 ip-10-0-132-245 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 21 13:42:15.159284 ip-10-0-132-245 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 21 13:42:15.159548 ip-10-0-132-245 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 21 13:42:24.529129 ip-10-0-132-245 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found.
Jul 21 13:42:24.529143 ip-10-0-132-245 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot cca1af0fa33e4df6a8bee6507af94fa4 --
Jul 21 13:44:11.767721 ip-10-0-132-245 systemd[1]: Starting Kubernetes Kubelet...
Jul 21 13:44:12.199974 ip-10-0-132-245 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 21 13:44:12.199974 ip-10-0-132-245 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 21 13:44:12.199974 ip-10-0-132-245 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 21 13:44:12.199974 ip-10-0-132-245 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 21 13:44:12.199974 ip-10-0-132-245 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 21 13:44:12.199974 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.097870    2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 21 13:44:12.199974 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103162    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 13:44:12.199974 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103175    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 13:44:12.199974 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103179    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 13:44:12.199974 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103182    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 13:44:12.199974 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103185    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 13:44:12.199974 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103187    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 13:44:12.199974 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103190    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 13:44:12.199974 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103193    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 13:44:12.169743 ip-10-0-132-245 systemd[1]: Started Kubernetes Kubelet.
Jul 21 13:44:12.201178 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103195    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 13:44:12.201178 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103198    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 13:44:12.201178 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103202    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 13:44:12.201178 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103205    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 13:44:12.201178 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103207    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 13:44:12.201178 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103210    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 13:44:12.201178 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103213    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 13:44:12.201178 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103216    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 13:44:12.201178 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103219    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 13:44:12.201178 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103228    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 13:44:12.201178 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103231    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 13:44:12.201178 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103234    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 13:44:12.201178 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103237    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 13:44:12.201178 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103240    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 13:44:12.201178 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103242    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 13:44:12.201178 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103245    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 13:44:12.201178 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103248    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 13:44:12.201178 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103250    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 13:44:12.201178 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103253    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 13:44:12.201178 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103256    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 13:44:12.202011 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103258    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 13:44:12.202011 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103262    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 13:44:12.202011 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103265    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 13:44:12.202011 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103268    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 13:44:12.202011 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103271    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 13:44:12.202011 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103273    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 13:44:12.202011 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103276    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 13:44:12.202011 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103279    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 13:44:12.202011 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103281    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 13:44:12.202011 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103283    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 13:44:12.202011 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103286    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 13:44:12.202011 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103288    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 13:44:12.202011 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103291    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 13:44:12.202011 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103293    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 13:44:12.202011 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103296    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 13:44:12.202011 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103298    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 13:44:12.202011 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103301    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 13:44:12.202011 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103303    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 13:44:12.202011 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103306    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 13:44:12.202011 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103309    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 13:44:12.202866 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103312    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 13:44:12.202866 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103314    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 13:44:12.202866 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103317    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 13:44:12.202866 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103320    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 13:44:12.202866 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103323    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 13:44:12.202866 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103326    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 13:44:12.202866 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103329    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 13:44:12.202866 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103332    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 13:44:12.202866 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103334    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 13:44:12.202866 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103337    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 13:44:12.202866 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103339    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 13:44:12.202866 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103342    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 13:44:12.202866 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103345    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 13:44:12.202866 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103347    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 13:44:12.202866 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103351    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 13:44:12.202866 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103355    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 13:44:12.202866 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103358    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 13:44:12.202866 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103361    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 13:44:12.202866 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103363    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 13:44:12.202866 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103366    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 13:44:12.203623 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103369    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 13:44:12.203623 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103372    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 13:44:12.203623 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103375    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 13:44:12.203623 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103377    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 13:44:12.203623 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103379    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 13:44:12.203623 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103383    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 13:44:12.203623 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103386    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 13:44:12.203623 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103388    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 13:44:12.203623 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103393    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 13:44:12.203623 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103396    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 13:44:12.203623 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103400    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 13:44:12.203623 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103404    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 13:44:12.203623 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103407    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 13:44:12.203623 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103410    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 13:44:12.203623 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103429    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 13:44:12.203623 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103433    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 13:44:12.203623 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103436    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 13:44:12.203623 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103439    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 13:44:12.203623 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103442    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 13:44:12.204288 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103445    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 13:44:12.204288 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103448    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 13:44:12.204288 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103451    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 13:44:12.204288 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103454    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 13:44:12.204288 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103457    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 13:44:12.204288 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103460    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 13:44:12.204288 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103462    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 13:44:12.204288 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103465    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 13:44:12.204288 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103468    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 13:44:12.204288 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103470    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 13:44:12.204288 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103473    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 13:44:12.204288 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.103476    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 13:44:12.204288 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104190    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 13:44:12.204288 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104196    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 13:44:12.204288 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104200    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 13:44:12.204288 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104203    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 13:44:12.204288 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104206    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 13:44:12.204288 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104209    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 13:44:12.204288 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104211    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 13:44:12.204288 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104214    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 13:44:12.205017 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104217    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 13:44:12.205017 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104219    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 13:44:12.205017 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104222    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 13:44:12.205017 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104225    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 13:44:12.205017 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104228    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 13:44:12.205017 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104232    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 13:44:12.205017 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104235    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 13:44:12.205017 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104238    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 13:44:12.205017 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104240    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 13:44:12.205017 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104243    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 13:44:12.205017 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104245    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 13:44:12.205017 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104248    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 13:44:12.205017 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104251    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 13:44:12.205017 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104253    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 13:44:12.205017 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104256    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 13:44:12.205017 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104259    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 13:44:12.205017 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104262    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 13:44:12.205017 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104264    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 13:44:12.205017 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104267    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 13:44:12.205017 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104270    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 13:44:12.205686 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104273    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 13:44:12.205686 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104275    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 13:44:12.205686 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104278    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 13:44:12.205686 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104281    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 13:44:12.205686 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104283    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 13:44:12.205686 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104286    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 13:44:12.205686 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104288    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 13:44:12.205686 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104290    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 13:44:12.205686 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104294    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 13:44:12.205686 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104299    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 13:44:12.205686 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104302    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 13:44:12.205686 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104304    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 13:44:12.205686 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104307    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 13:44:12.205686 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104309    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 13:44:12.205686 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104311    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 13:44:12.205686 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104314    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 13:44:12.205686 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104316    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 13:44:12.205686 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104319    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 13:44:12.205686 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104322    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 13:44:12.206338 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104325    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 13:44:12.206338 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104327    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 13:44:12.206338 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104330    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 13:44:12.206338 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104332    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 13:44:12.206338 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104335    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 13:44:12.206338 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104337    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 13:44:12.206338 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104340    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 13:44:12.206338 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104343    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 13:44:12.206338 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104346    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 13:44:12.206338 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104348    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 13:44:12.206338 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104351    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 13:44:12.206338 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104355    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 13:44:12.206338 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104358    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 13:44:12.206338 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104361    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 13:44:12.206338 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104364    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 13:44:12.206338 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104367    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 13:44:12.206338 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104370    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 13:44:12.206338 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104373    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 13:44:12.206338 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104376    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 13:44:12.206338 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104378    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 13:44:12.206977 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104381    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 13:44:12.206977 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104383    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 13:44:12.206977 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104386    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 13:44:12.206977 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104389    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 13:44:12.206977 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104392    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 13:44:12.206977 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104395    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 13:44:12.206977 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104398    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 13:44:12.206977 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104400    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 13:44:12.206977 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104403    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 13:44:12.206977 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104405    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 13:44:12.206977 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104408    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 13:44:12.206977 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104426    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 13:44:12.206977 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104430    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 13:44:12.206977 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104434    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 13:44:12.206977 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104438    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 13:44:12.206977 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104441    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 13:44:12.206977 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104444    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 13:44:12.206977 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104447    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 13:44:12.206977 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104450    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 13:44:12.207579 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104453    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 13:44:12.207579 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104455    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 13:44:12.207579 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104458    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 13:44:12.207579 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104460    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 13:44:12.207579 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104463    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 13:44:12.207579 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104465    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 13:44:12.207579 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104467    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 13:44:12.207579 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104470    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 13:44:12.207579 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104473    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 13:44:12.207579 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104475    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 13:44:12.207579 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104478    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 13:44:12.207579 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104481    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 13:44:12.207579 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104483    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 13:44:12.207579 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104846    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 13:44:12.207579 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104851    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 13:44:12.207579 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104854    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 13:44:12.207579 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104857    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 13:44:12.207579 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104860    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 13:44:12.207579 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104864    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 13:44:12.208282 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104868    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 13:44:12.208282 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104871    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 13:44:12.208282 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104874    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 13:44:12.208282 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104877    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 13:44:12.208282 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104880    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 13:44:12.208282 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104883    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 13:44:12.208282 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104892    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 13:44:12.208282 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104894    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 13:44:12.208282 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104897    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 13:44:12.208282 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104899    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 13:44:12.208282 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104902    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 13:44:12.208282 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104905    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 13:44:12.208282 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104907    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 13:44:12.208282 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104910    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 13:44:12.208282 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104913    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 13:44:12.208282 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104915    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 13:44:12.208282 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104918    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 13:44:12.208282 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104920    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 13:44:12.208282 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104923    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 13:44:12.208282 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104925    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 13:44:12.209046 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104928    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 13:44:12.209046 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104935    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 13:44:12.209046 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104937    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 13:44:12.209046 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104940    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 13:44:12.209046 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104943    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 13:44:12.209046 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104946    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 13:44:12.209046 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104948    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 13:44:12.209046 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104951    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 13:44:12.209046 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104954    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 13:44:12.209046 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104957    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 13:44:12.209046 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104959    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 13:44:12.209046 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104962    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 13:44:12.209046 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104965    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 13:44:12.209046 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104967    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 13:44:12.209046 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104970    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 13:44:12.209046 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104973    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 13:44:12.209046 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104975    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 13:44:12.209046 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104978    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 13:44:12.209046 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104980    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 13:44:12.209046 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104986    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 13:44:12.209724 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104989    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 13:44:12.209724 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104993    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 13:44:12.209724 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.104996    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 13:44:12.209724 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105001    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 13:44:12.209724 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105005    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 13:44:12.209724 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105008    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 13:44:12.209724 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105011    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 13:44:12.209724 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105014    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 13:44:12.209724 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105017    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 13:44:12.209724 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105020    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 13:44:12.209724 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105023    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 13:44:12.209724 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105025    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 13:44:12.209724 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105028    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 13:44:12.209724 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105032    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 13:44:12.209724 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105034    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 13:44:12.209724 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105037    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 13:44:12.209724 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105040    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 13:44:12.209724 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105042    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 13:44:12.209724 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105045    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 13:44:12.210257 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105047    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 13:44:12.210257 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105050    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 13:44:12.210257 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105052    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 13:44:12.210257 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105055    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 13:44:12.210257 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105058    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 13:44:12.210257 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105060    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 13:44:12.210257 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105063    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 13:44:12.210257 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105065    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 13:44:12.210257 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105068    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 13:44:12.210257 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105070    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 13:44:12.210257 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105073    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 13:44:12.210257 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105076    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 13:44:12.210257 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105079    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 13:44:12.210257 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105082    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 13:44:12.210257 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105085    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 13:44:12.210257 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105087    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 13:44:12.210257 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105092    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 13:44:12.210257 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105095    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 13:44:12.210257 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105097    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 13:44:12.210257 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105100    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 13:44:12.210876 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105102    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 13:44:12.210876 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105105    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 13:44:12.210876 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105107    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 13:44:12.210876 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105110    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 13:44:12.210876 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105112    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 13:44:12.210876 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105115    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 13:44:12.210876 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105119    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 13:44:12.210876 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105122    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 13:44:12.210876 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105124    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 13:44:12.210876 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105127    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 13:44:12.210876 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105130    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 13:44:12.210876 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105132    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 13:44:12.210876 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105135    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 13:44:12.210876 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105138    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 13:44:12.210876 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105216    2578 flags.go:64] FLAG: --address="0.0.0.0"
Jul 21 13:44:12.210876 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105223    2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 21 13:44:12.210876 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105230    2578 flags.go:64] FLAG: --anonymous-auth="true"
Jul 21 13:44:12.210876 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105234    2578 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 21 13:44:12.210876 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105238    2578 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 21 13:44:12.210876 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105241    2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 21 13:44:12.210876 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105246    2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 21 13:44:12.211554 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105250    2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 21 13:44:12.211554 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105254    2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 21 13:44:12.211554 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105257    2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 21 13:44:12.211554 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105260    2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 21 13:44:12.211554 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105263    2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 21 13:44:12.211554 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105267    2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 21 13:44:12.211554 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105271    2578 flags.go:64] FLAG: --cgroup-root=""
Jul 21 13:44:12.211554 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105276    2578 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 21 13:44:12.211554 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105279    2578 flags.go:64] FLAG: --client-ca-file=""
Jul 21 13:44:12.211554 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105282    2578 flags.go:64] FLAG: --cloud-provider="external"
Jul 21 13:44:12.211554 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105284    2578 flags.go:64] FLAG: --cluster-dns="[]"
Jul 21 13:44:12.211554 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105289    2578 flags.go:64] FLAG: --cluster-domain=""
Jul 21 13:44:12.211554 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105292    2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 21 13:44:12.211554 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105295    2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 21 13:44:12.211554 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105298    2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 21 13:44:12.211554 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105301    2578 flags.go:64] FLAG: --container-log-max-files="5"
Jul 21 13:44:12.211554 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105305    2578 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 21 13:44:12.211554 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105309    2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 21 13:44:12.211554 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105312    2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 21 13:44:12.211554 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105316    2578 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 21 13:44:12.211554 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105319    2578 flags.go:64] FLAG: --contention-profiling="false"
Jul 21 13:44:12.211554 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105322    2578 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 21 13:44:12.211554 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105325    2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 21 13:44:12.211554 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105328    2578 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 21 13:44:12.212318 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105330    2578 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 21 13:44:12.212318 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105334    2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 21 13:44:12.212318 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105337    2578 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 21 13:44:12.212318 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105340    2578 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 21 13:44:12.212318 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105343    2578 flags.go:64] FLAG: --enable-load-reader="false"
Jul 21 13:44:12.212318 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105346    2578 flags.go:64] FLAG: --enable-server="true"
Jul 21 13:44:12.212318 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105349    2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 21 13:44:12.212318 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105353    2578 flags.go:64] FLAG: --event-burst="100"
Jul 21 13:44:12.212318 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105357    2578 flags.go:64] FLAG: --event-qps="50"
Jul 21 13:44:12.212318 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105359    2578 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 21 13:44:12.212318 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105362    2578 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 21 13:44:12.212318 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105365    2578 flags.go:64] FLAG: --eviction-hard=""
Jul 21 13:44:12.212318 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105369    2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 21 13:44:12.212318 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105372    2578 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 21 13:44:12.212318 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105375    2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 21 13:44:12.212318 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105380    2578 flags.go:64] FLAG: --eviction-soft=""
Jul 21 13:44:12.212318 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105383    2578 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 21 13:44:12.212318 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105386    2578 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 21 13:44:12.212318 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105389    2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 21 13:44:12.212318 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105392    2578 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 21 13:44:12.212318 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105395    2578 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 21 13:44:12.212318 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105398    2578 flags.go:64] FLAG: --fail-swap-on="true"
Jul 21 13:44:12.212318 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105400    2578 flags.go:64] FLAG: --feature-gates=""
Jul 21 13:44:12.212318 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105404    2578 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 21 13:44:12.212318 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105407    2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 21 13:44:12.213058 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105424    2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 21 13:44:12.213058 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105428    2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 21 13:44:12.213058 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105431    2578 flags.go:64] FLAG: --healthz-port="10248"
Jul 21 13:44:12.213058 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105434    2578 flags.go:64] FLAG: --help="false"
Jul 21 13:44:12.213058 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105437    2578 flags.go:64] FLAG: --hostname-override="ip-10-0-132-245.ec2.internal"
Jul 21 13:44:12.213058 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105440    2578 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 21 13:44:12.213058 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105443    2578 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 21 13:44:12.213058 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105446    2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 21 13:44:12.213058 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105450    2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 21 13:44:12.213058 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105453    2578 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 21 13:44:12.213058 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105456    2578 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 21 13:44:12.213058 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105459    2578 flags.go:64] FLAG: --image-service-endpoint=""
Jul 21 13:44:12.213058 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105462    2578 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 21 13:44:12.213058 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105465    2578 flags.go:64] FLAG: --kube-api-burst="100"
Jul 21 13:44:12.213058 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105468    2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 21 13:44:12.213058 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105471    2578 flags.go:64] FLAG: --kube-api-qps="50"
Jul 21 13:44:12.213058 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105474    2578 flags.go:64] FLAG: --kube-reserved=""
Jul 21 13:44:12.213058 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105477    2578 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 21 13:44:12.213058 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105479    2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 21 13:44:12.213058 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105483    2578 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 21 13:44:12.213058 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105485    2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 21 13:44:12.213058 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105488    2578 flags.go:64] FLAG: --lock-file=""
Jul 21 13:44:12.213058 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105493    2578 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 21 13:44:12.213058 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105496    2578 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 21 13:44:12.213775 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105499    2578 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 21 13:44:12.213775 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105504    2578 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 21 13:44:12.213775 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105507    2578 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 21 13:44:12.213775 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105510    2578 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 21 13:44:12.213775 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105513    2578 flags.go:64] FLAG: --logging-format="text"
Jul 21 13:44:12.213775 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105516    2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 21 13:44:12.213775 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105520    2578 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 21 13:44:12.213775 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105522    2578 flags.go:64] FLAG: --manifest-url=""
Jul 21 13:44:12.213775 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105527    2578 flags.go:64] FLAG: --manifest-url-header=""
Jul 21 13:44:12.213775 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105532    2578 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 21 13:44:12.213775 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105535    2578 flags.go:64] FLAG: --max-open-files="1000000"
Jul 21 13:44:12.213775 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105544    2578 flags.go:64] FLAG: --max-pods="110"
Jul 21 13:44:12.213775 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105547    2578 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 21 13:44:12.213775 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105550    2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 21 13:44:12.213775 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105553    2578 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 21 13:44:12.213775 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105556    2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 21 13:44:12.213775 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105559    2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 21 13:44:12.213775 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105562    2578 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 21 13:44:12.213775 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105565    2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 21 13:44:12.213775 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105572    2578 flags.go:64] FLAG: --node-status-max-images="50"
Jul 21 13:44:12.213775 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105575    2578 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 21 13:44:12.213775 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105578    2578 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 21 13:44:12.213775 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105582    2578 flags.go:64] FLAG: --pod-cidr=""
Jul 21 13:44:12.216623 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105584    2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 21 13:44:12.216623 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105590    2578 flags.go:64] FLAG: --pod-manifest-path=""
Jul 21 13:44:12.216623 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105593    2578 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 21 13:44:12.216623 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105596    2578 flags.go:64] FLAG: --pods-per-core="0"
Jul 21 13:44:12.216623 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105599    2578 flags.go:64] FLAG: --port="10250"
Jul 21 13:44:12.216623 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105602    2578 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 21 13:44:12.216623 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105605    2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0440b2d8864dfc60b"
Jul 21 13:44:12.216623 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105610    2578 flags.go:64] FLAG: --qos-reserved=""
Jul 21 13:44:12.216623 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105613    2578 flags.go:64] FLAG: --read-only-port="10255"
Jul 21 13:44:12.216623 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105616    2578 flags.go:64] FLAG: --register-node="true"
Jul 21 13:44:12.216623 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105619    2578 flags.go:64] FLAG: --register-with-taints=""
Jul 21 13:44:12.216623 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105623    2578 flags.go:64] FLAG: --registry-burst="10"
Jul 21 13:44:12.216623 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105625    2578 flags.go:64] FLAG: --registry-qps="5"
Jul 21 13:44:12.216623 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105628    2578 flags.go:64] FLAG: --reserved-cpus=""
Jul 21 13:44:12.216623 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105631    2578 flags.go:64] FLAG: --reserved-memory=""
Jul 21 13:44:12.216623 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105635    2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 21 13:44:12.216623 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105638    2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 21 13:44:12.216623 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105641    2578 flags.go:64] FLAG: --rotate-certificates="false"
Jul 21 13:44:12.216623 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105645    2578 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 21 13:44:12.216623 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105648    2578 flags.go:64] FLAG: --runonce="false"
Jul 21 13:44:12.216623 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105651    2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 21 13:44:12.216623 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105654    2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 21 13:44:12.216623 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105657    2578 flags.go:64] FLAG: --seccomp-default="false"
Jul 21 13:44:12.216623 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105660    2578 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 21 13:44:12.216623 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105663    2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 21 13:44:12.217406 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105666    2578 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 21 13:44:12.217406 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105669    2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 21 13:44:12.217406 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105673    2578 flags.go:64] FLAG: --storage-driver-password="root"
Jul 21 13:44:12.217406 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105676    2578 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 21 13:44:12.217406 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105679    2578 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 21 13:44:12.217406 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105681    2578 flags.go:64] FLAG: --storage-driver-user="root"
Jul 21 13:44:12.217406 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105684    2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 21 13:44:12.217406 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105687    2578 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 21 13:44:12.217406 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105690    2578 flags.go:64] FLAG: --system-cgroups=""
Jul 21 13:44:12.217406 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105693    2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 21 13:44:12.217406 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105698    2578 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 21 13:44:12.217406 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105701    2578 flags.go:64] FLAG: --tls-cert-file=""
Jul 21 13:44:12.217406 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105704    2578 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 21 13:44:12.217406 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105707    2578 flags.go:64] FLAG: --tls-min-version=""
Jul 21 13:44:12.217406 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105711    2578 flags.go:64] FLAG: --tls-private-key-file=""
Jul 21 13:44:12.217406 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105714    2578 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 21 13:44:12.217406 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105718    2578 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 21 13:44:12.217406 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105721    2578 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 21 13:44:12.217406 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105724    2578 flags.go:64] FLAG: --v="2"
Jul 21 13:44:12.217406 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105728    2578 flags.go:64] FLAG: --version="false"
Jul 21 13:44:12.217406 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105732    2578 flags.go:64] FLAG: --vmodule=""
Jul 21 13:44:12.217406 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105735    2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 21 13:44:12.217406 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.105738    2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 21 13:44:12.217406 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105854    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 13:44:12.217406 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105857    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 13:44:12.218224 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105862    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 13:44:12.218224 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105866    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 13:44:12.218224 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105870    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 13:44:12.218224 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105874    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 13:44:12.218224 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105877    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 13:44:12.218224 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105879    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 13:44:12.218224 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105882    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 13:44:12.218224 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105885    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 13:44:12.218224 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105887    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 13:44:12.218224 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105890    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 13:44:12.218224 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105893    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 13:44:12.218224 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105896    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 13:44:12.218224 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105898    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 13:44:12.218224 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105901    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 13:44:12.218224 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105903    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 13:44:12.218224 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105906    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 13:44:12.218224 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105908    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 13:44:12.218224 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105911    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 13:44:12.218224 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105913    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 13:44:12.218870 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105917    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 13:44:12.218870 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105919    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 13:44:12.218870 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105923    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 13:44:12.218870 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105927    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 13:44:12.218870 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105929    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 13:44:12.218870 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105938    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 13:44:12.218870 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105940    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 13:44:12.218870 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105943    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 13:44:12.218870 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105945    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 13:44:12.218870 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105948    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 13:44:12.218870 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105951    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 13:44:12.218870 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105954    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 13:44:12.218870 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105956    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 13:44:12.218870 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105960    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 13:44:12.218870 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105962    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 13:44:12.218870 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105965    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 13:44:12.218870 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105968    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 13:44:12.218870 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105970    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 13:44:12.218870 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105973    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 13:44:12.218870 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105975    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 13:44:12.219555 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105978    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 13:44:12.219555 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105980    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 13:44:12.219555 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105983    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 13:44:12.219555 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105985    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 13:44:12.219555 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105988    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 13:44:12.219555 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105990    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 13:44:12.219555 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105993    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 13:44:12.219555 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105996    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 13:44:12.219555 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.105998    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 13:44:12.219555 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.106001    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 13:44:12.219555 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.106003    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 13:44:12.219555 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.106006    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 13:44:12.219555 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.106008    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 13:44:12.219555 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.106011    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 13:44:12.219555 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.106014    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 13:44:12.219555 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.106017    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 13:44:12.219555 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.106020    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 13:44:12.219555 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.106022    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 13:44:12.219555 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.106025    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 13:44:12.219555 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.106028    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 13:44:12.220122 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.106030    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 13:44:12.220122 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.106035    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 13:44:12.220122 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.106038    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 13:44:12.220122 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.106041    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 13:44:12.220122 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.106044    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 13:44:12.220122 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.106048    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 13:44:12.220122 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.106050    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 13:44:12.220122 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.106053    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 13:44:12.220122 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.106056    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 13:44:12.220122 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.106058    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 13:44:12.220122 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.106061    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 13:44:12.220122 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.106064    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 13:44:12.220122 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.106102    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 13:44:12.220122 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.106106    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 13:44:12.220122 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.106125    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 13:44:12.220122 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.106128    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 13:44:12.220122 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.106323    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 13:44:12.220122 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.106326    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 13:44:12.220122 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.106329    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 13:44:12.220122 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.106332    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 13:44:12.220751 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.106334    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 13:44:12.220751 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.106737    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 13:44:12.220751 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.106741    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 13:44:12.220751 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.106744    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 13:44:12.220751 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.106747    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 13:44:12.220751 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.106941    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 13:44:12.220751 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.106946    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 13:44:12.220751 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.106949    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 13:44:12.220751 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.106952    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 13:44:12.220751 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.107149    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 13:44:12.220751 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.107152    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 13:44:12.220751 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.107155    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 13:44:12.220751 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.107158    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 13:44:12.220751 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.107357    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 13:44:12.220751 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.107359    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 13:44:12.220751 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.107362    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 13:44:12.220751 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.107567    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 13:44:12.220751 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.107764    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 13:44:13.586585 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.108838    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 21 13:44:13.586585 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.115715    2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 21 13:44:13.586585 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.115729    2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 21 13:44:13.586585 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.115746    2578 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 21 13:44:13.586585 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.115755    2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 21 13:44:13.586585 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115821    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 13:44:13.586585 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115826    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 13:44:13.586585 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115829    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 13:44:13.586585 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115832    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 13:44:13.586585 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115835    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 13:44:13.586585 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115837    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 13:44:13.586585 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115840    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 13:44:13.586585 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115843    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 13:44:13.586585 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115845    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 13:44:13.586585 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115849    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 13:44:13.611451 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115851    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 13:44:13.611451 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115854    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 13:44:13.611451 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115856    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 13:44:13.611451 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115859    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 13:44:13.611451 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115861    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 13:44:13.611451 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115866    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 13:44:13.611451 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115869    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 13:44:13.611451 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115872    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 13:44:13.611451 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115874    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 13:44:13.611451 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115877    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 13:44:13.611451 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115881    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 13:44:13.611451 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115883    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 13:44:13.611451 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115887    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 13:44:13.611451 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115890    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 13:44:13.611451 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115893    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 13:44:13.611451 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115896    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 13:44:13.611451 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115899    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 13:44:13.611451 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115904    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 13:44:13.611451 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115907    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 13:44:13.612250 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115910    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 13:44:13.612250 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115913    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 13:44:13.612250 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115916    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 13:44:13.612250 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115919    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 13:44:13.612250 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115922    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 13:44:13.612250 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115925    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 13:44:13.612250 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115928    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 13:44:13.612250 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115931    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 13:44:13.612250 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115934    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 13:44:13.612250 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115937    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 13:44:13.612250 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115939    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 13:44:13.612250 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115942    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 13:44:13.612250 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115944    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 13:44:13.612250 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115947    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 13:44:13.612250 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115949    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 13:44:13.612250 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115952    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 13:44:13.612250 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115955    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 13:44:13.612250 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115957    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 13:44:13.612250 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115961    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 13:44:13.612250 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115964    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 13:44:13.613326 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115966    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 13:44:13.613326 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115969    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 13:44:13.613326 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115972    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 13:44:13.613326 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115975    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 13:44:13.613326 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115978    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 13:44:13.613326 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115981    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 13:44:13.613326 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115983    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 13:44:13.613326 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115986    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 13:44:13.613326 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115988    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 13:44:13.613326 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115991    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 13:44:13.613326 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115995    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 13:44:13.613326 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.115998    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 13:44:13.613326 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116000    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 13:44:13.613326 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116003    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 13:44:13.613326 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116005    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 13:44:13.613326 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116008    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 13:44:13.613326 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116010    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 13:44:13.613326 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116013    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 13:44:13.613326 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116015    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 13:44:13.613326 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116018    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 13:44:13.614115 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116020    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 13:44:13.614115 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116023    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 13:44:13.614115 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116027    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 13:44:13.614115 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116030    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 13:44:13.614115 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116033    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 13:44:13.614115 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116036    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 13:44:13.614115 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116039    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 13:44:13.614115 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116042    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 13:44:13.614115 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116044    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 13:44:13.614115 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116046    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 13:44:13.614115 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116050    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 13:44:13.614115 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116054    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 13:44:13.614115 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116056    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 13:44:13.614115 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116059    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 13:44:13.614115 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116061    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 13:44:13.614115 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116064    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 13:44:13.614115 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116067    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 13:44:13.614115 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116070    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 13:44:13.614115 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116073    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 13:44:13.614115 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116075    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 13:44:13.615321 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116078    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 13:44:13.615321 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116081    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 13:44:13.615321 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116084    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 13:44:13.615321 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116087    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 13:44:13.615321 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116090    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 13:44:13.615321 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116092    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 13:44:13.615321 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116095    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 13:44:13.615321 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116097    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 13:44:13.615321 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116100    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 13:44:13.615321 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116102    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 13:44:13.615321 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.116107    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 21 13:44:13.615321 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116223    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 13:44:13.615321 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116228    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 13:44:13.615321 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116231    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 13:44:13.615321 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116234    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 13:44:13.615800 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116237    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 13:44:13.615800 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116240    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 13:44:13.615800 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116243    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 13:44:13.615800 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116246    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 13:44:13.615800 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116249    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 13:44:13.615800 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116252    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 13:44:13.615800 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116255    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 13:44:13.615800 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116259    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 13:44:13.615800 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116262    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 13:44:13.615800 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116264    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 13:44:13.615800 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116267    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 13:44:13.615800 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116269    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 13:44:13.615800 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116272    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 13:44:13.615800 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116275    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 13:44:13.615800 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116277    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 13:44:13.615800 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116280    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 13:44:13.615800 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116283    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 13:44:13.615800 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116285    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 13:44:13.615800 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116288    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 13:44:13.616387 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116291    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 13:44:13.616387 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116294    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 13:44:13.616387 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116297    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 13:44:13.616387 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116300    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 13:44:13.616387 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116302    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 13:44:13.616387 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116305    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 13:44:13.616387 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116308    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 13:44:13.616387 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116310    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 13:44:13.616387 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116313    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 13:44:13.616387 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116315    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 13:44:13.616387 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116318    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 13:44:13.616387 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116321    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 13:44:13.616387 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116323    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 13:44:13.616387 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116326    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 13:44:13.616387 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116329    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 13:44:13.616387 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116333    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 13:44:13.616387 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116336    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 13:44:13.616387 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116339    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 13:44:13.616387 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116342    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 13:44:13.616387 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116345    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 13:44:13.616996 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116350    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 13:44:13.616996 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116353    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 13:44:13.616996 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116356    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 13:44:13.616996 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116359    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 13:44:13.616996 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116362    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 13:44:13.616996 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116365    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 13:44:13.616996 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116368    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 13:44:13.616996 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116371    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 13:44:13.616996 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116373    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 13:44:13.616996 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116377    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 13:44:13.616996 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116380    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 13:44:13.616996 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116382    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 13:44:13.616996 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116385    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 13:44:13.616996 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116390    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 13:44:13.616996 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116393    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 13:44:13.616996 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116395    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 13:44:13.616996 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116398    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 13:44:13.616996 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116400    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 13:44:13.616996 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116403    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 13:44:13.616996 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116406    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 13:44:13.617578 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116408    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 13:44:13.617578 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116426    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 13:44:13.617578 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116429    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 13:44:13.617578 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116432    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 13:44:13.617578 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116434    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 13:44:13.617578 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116437    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 13:44:13.617578 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116439    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 13:44:13.617578 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116442    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 13:44:13.617578 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116445    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 13:44:13.617578 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116448    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 13:44:13.617578 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116450    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 13:44:13.617578 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116453    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 13:44:13.617578 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116457    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 13:44:13.617578 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116460    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 13:44:13.617578 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116462    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 13:44:13.617578 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116465    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 13:44:13.617578 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116468    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 13:44:13.617578 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116472    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 13:44:13.617578 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116476    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 13:44:13.618172 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116479    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 13:44:13.618172 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116483    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 13:44:13.618172 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116486    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 13:44:13.618172 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116489    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 13:44:13.618172 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116492    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 13:44:13.618172 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116495    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 13:44:13.618172 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116500    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 13:44:13.618172 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116502    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 13:44:13.618172 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116505    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 13:44:13.618172 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116508    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 13:44:13.618172 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116511    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 13:44:13.618172 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116513    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 13:44:13.618172 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116516    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 13:44:13.618172 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116518    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 13:44:13.618172 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116521    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 13:44:13.618172 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116523    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 13:44:13.618172 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:12.116526    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 13:44:13.618749 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.116531    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 21 13:44:13.618749 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.116634    2578 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 21 13:44:13.618749 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.119326    2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 21 13:44:13.618749 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.120489    2578 server.go:1019] "Starting client certificate rotation"
Jul 21 13:44:13.618749 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.120592    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 21 13:44:13.618749 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.120631    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 21 13:44:13.618749 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.122925    2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 21 13:44:13.618749 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.124442    2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 21 13:44:13.618749 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.135778    2578 log.go:25] "Validated CRI v1 runtime API"
Jul 21 13:44:13.618749 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.137736    2578 log.go:25] "Validated CRI v1 image API"
Jul 21 13:44:13.618749 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.138587    2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 21 13:44:13.618749 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.142957    2578 fs.go:135] Filesystem UUIDs: map[5d5dbf11-0898-4732-87bb-05f33c53274c:/dev/nvme0n1p4 630e9951-e52f-4b8e-98b8-502aa56bd4c2:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Jul 21 13:44:13.619167 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.142972    2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 21 13:44:13.619386 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.148480    2578 manager.go:216] Machine: {Timestamp:2026-07-21 13:44:12.147310889 +0000 UTC m=+0.278061345 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099738 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec223bf40b9409446929f1970cf68858 SystemUUID:ec223bf4-0b94-0944-6929-f1970cf68858 BootID:cca1af0f-a33e-4df6-a8be-e6507af94fa4 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:7d:9c:9b:55:13 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:7d:9c:9b:55:13 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2e:33:58:ed:2c:7f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 21 13:44:13.619386 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.148636    2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 21 13:44:13.619386 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.148774    2578 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 21 13:44:13.619386 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.150582    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 21 13:44:13.619386 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.151081    2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 21 13:44:13.619386 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.151110    2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-245.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 21 13:44:13.619386 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.152388    2578 topology_manager.go:138] "Creating topology manager with none policy"
Jul 21 13:44:13.619676 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.152398    2578 container_manager_linux.go:309] "Creating device plugin manager"
Jul 21 13:44:13.619676 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.152406    2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 21 13:44:13.619676 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.153016    2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 21 13:44:13.619676 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.154292    2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 21 13:44:13.619676 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.158142    2578 state_mem.go:36] "Initialized new in-memory state store"
Jul 21 13:44:13.619676 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.158258    2578 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 21 13:44:13.619676 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.158288    2578 kubelet.go:486] "Attempting to sync node with API server"
Jul 21 13:44:13.619676 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.158297    2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 21 13:44:13.619676 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.158312    2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 21 13:44:13.619676 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.158323    2578 kubelet.go:398] "Adding apiserver pod source"
Jul 21 13:44:13.619676 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.158332    2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 21 13:44:13.619676 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.158671    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 21 13:44:13.619676 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.158688    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 21 13:44:13.619676 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.160425    2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 21 13:44:13.619676 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.162390    2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 21 13:44:13.619676 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.162433    2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 21 13:44:13.619676 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.163656    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 21 13:44:13.619676 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.163671    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 21 13:44:13.619676 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.163677    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 21 13:44:13.619676 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.163683    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 21 13:44:13.619676 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.163691    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 21 13:44:13.619676 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.163696    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 21 13:44:13.619676 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.163704    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 21 13:44:13.619676 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.163710    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 21 13:44:13.619676 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.163717    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 21 13:44:13.619676 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.163723    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 21 13:44:13.619676 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.163735    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 21 13:44:13.619676 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.163740    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 21 13:44:13.619676 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.163764    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 21 13:44:13.619676 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.163772    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 21 13:44:13.619676 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.167105    2578 server.go:1268] "Started kubelet"
Jul 21 13:44:13.619676 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.167187    2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 21 13:44:13.619676 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.167326    2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 21 13:44:13.619676 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.167385    2578 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 21 13:44:13.619676 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.169064    2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-245.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 21 13:44:13.619676 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:12.169573    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-245.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 21 13:44:13.619676 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:12.169573    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 21 13:44:13.619676 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.169811    2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 21 13:44:13.619676 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.171350    2578 server.go:310] "Adding debug handlers to kubelet server"
Jul 21 13:44:13.620858 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:12.175000    2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 21 13:44:13.620858 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.177513    2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 21 13:44:13.620858 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.177612    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 21 13:44:13.620858 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.177720    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ch4wp"
Jul 21 13:44:13.620858 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:12.177768    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-245.ec2.internal\" not found"
Jul 21 13:44:13.620858 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.178171    2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 21 13:44:13.620858 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.178174    2578 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 21 13:44:13.620858 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.178208    2578 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 21 13:44:13.620858 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.178289    2578 factory.go:55] Registering systemd factory
Jul 21 13:44:13.620858 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.178302    2578 factory.go:223] Registration of the systemd container factory successfully
Jul 21 13:44:13.620858 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.178337    2578 reconstruct.go:97] "Volume reconstruction finished"
Jul 21 13:44:13.620858 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.178344    2578 reconciler.go:29] "Reconciler: start to sync state"
Jul 21 13:44:13.620858 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.178583    2578 factory.go:153] Registering CRI-O factory
Jul 21 13:44:13.620858 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.178600    2578 factory.go:223] Registration of the crio container factory successfully
Jul 21 13:44:13.620858 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.178688    2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 21 13:44:13.620858 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.178712    2578 factory.go:103] Registering Raw factory
Jul 21 13:44:13.620858 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.178726    2578 manager.go:1196] Started watching for new ooms in manager
Jul 21 13:44:13.620858 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.179080    2578 manager.go:349] Starting recovery of all containers
Jul 21 13:44:13.620858 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:12.179430    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-245.ec2.internal.18c451f1ffdb1e6f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-245.ec2.internal,UID:ip-10-0-132-245.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-245.ec2.internal,},FirstTimestamp:2026-07-21 13:44:12.167077487 +0000 UTC m=+0.297827943,LastTimestamp:2026-07-21 13:44:12.167077487 +0000 UTC m=+0.297827943,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-245.ec2.internal,}"
Jul 21 13:44:13.620858 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.187013    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ch4wp"
Jul 21 13:44:13.620858 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.190041    2578 manager.go:354] Recovery completed
Jul 21 13:44:13.620858 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:12.193010    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 21 13:44:13.620858 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:12.193022    2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-245.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 21 13:44:13.620858 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.196400    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 13:44:13.620858 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.202243    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 13:44:13.620858 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.202275    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 13:44:13.620858 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.202289    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasSufficientPID"
Jul 21 13:44:13.620858 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.202883    2578 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 21 13:44:13.620858 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.202893    2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 21 13:44:13.620858 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.202907    2578 state_mem.go:36] "Initialized new in-memory state store"
Jul 21 13:44:13.620858 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.202920    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 13:44:13.620858 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.203637    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 13:44:13.621817 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.203667    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 13:44:13.621817 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.203679    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasSufficientPID"
Jul 21 13:44:13.621817 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.204981    2578 policy_none.go:49] "None policy: Start"
Jul 21 13:44:13.621817 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.204995    2578 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 21 13:44:13.621817 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.205007    2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 21 13:44:13.621817 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.207069    2578 policy_none.go:47] "Start"
Jul 21 13:44:13.621817 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.251053    2578 manager.go:337] "Starting Device Plugin manager"
Jul 21 13:44:13.621817 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:12.251291    2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 21 13:44:13.621817 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.251304    2578 server.go:87] "Starting device plugin registration server"
Jul 21 13:44:13.621817 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.251601    2578 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 21 13:44:13.621817 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.251613    2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 21 13:44:13.621817 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.251717    2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 21 13:44:13.621817 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.251790    2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 21 13:44:13.621817 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:12.251798    2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 21 13:44:13.621817 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:12.252383    2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 21 13:44:13.621817 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:12.252447    2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-245.ec2.internal\" not found"
Jul 21 13:44:13.621817 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:13.495890    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 21 13:44:13.621817 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:13.497256    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 21 13:44:13.621817 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:13.497307    2578 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 21 13:44:13.621817 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:13.497340    2578 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 21 13:44:13.621817 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:13.497384    2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 21 13:44:13.621817 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:13.499492    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 13:44:13.621817 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:13.523085    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 13:44:13.621817 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:13.524125    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 13:44:13.621817 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:13.524155    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 13:44:13.621817 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:13.524165    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasSufficientPID"
Jul 21 13:44:13.621817 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:13.524189    2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-245.ec2.internal"
Jul 21 13:44:13.621817 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:13.532894    2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-245.ec2.internal"
Jul 21 13:44:13.621817 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:13.532918    2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-245.ec2.internal\": node \"ip-10-0-132-245.ec2.internal\" not found"
Jul 21 13:44:13.621817 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:13.547789    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-245.ec2.internal\" not found"
Jul 21 13:44:13.621817 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:13.597795    2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-132-245.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal"]
Jul 21 13:44:13.621817 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:13.597861    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 13:44:13.621817 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:13.598932    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 13:44:13.621817 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:13.598967    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 13:44:13.621817 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:13.598982    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasSufficientPID"
Jul 21 13:44:13.621817 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:13.601986    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 13:44:13.621817 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:13.602044    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-245.ec2.internal"
Jul 21 13:44:13.621817 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:13.602072    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 13:44:13.622868 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:13.602779    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 13:44:13.622868 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:13.602779    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 13:44:13.622868 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:13.602809    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 13:44:13.622868 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:13.602820    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasSufficientPID"
Jul 21 13:44:13.622868 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:13.602829    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 13:44:13.622868 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:13.602855    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasSufficientPID"
Jul 21 13:44:13.622868 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:13.605783    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal"
Jul 21 13:44:13.622868 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:13.605807    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 13:44:13.622868 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:13.606587    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 13:44:13.622868 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:13.606610    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 13:44:13.622868 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:13.606620    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeHasSufficientPID"
Jul 21 13:44:13.623827 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:13.623590    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-245.ec2.internal\" not found" node="ip-10-0-132-245.ec2.internal"
Jul 21 13:44:13.788689 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:13.627926    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-245.ec2.internal\" not found" node="ip-10-0-132-245.ec2.internal"
Jul 21 13:44:13.788689 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:13.648106    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-245.ec2.internal\" not found"
Jul 21 13:44:13.788689 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:13.650316    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/de55c8b128a47fa32f279f57a92fe691-config\") pod \"kube-apiserver-proxy-ip-10-0-132-245.ec2.internal\" (UID: \"de55c8b128a47fa32f279f57a92fe691\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-245.ec2.internal"
Jul 21 13:44:13.788689 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:13.650339    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a28dba002f2c85352dc44bcb9751469c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal\" (UID: \"a28dba002f2c85352dc44bcb9751469c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal"
Jul 21 13:44:13.788689 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:13.650358    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a28dba002f2c85352dc44bcb9751469c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal\" (UID: \"a28dba002f2c85352dc44bcb9751469c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal"
Jul 21 13:44:13.788689 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:13.748387    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-245.ec2.internal\" not found"
Jul 21 13:44:13.788689 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:13.750581    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a28dba002f2c85352dc44bcb9751469c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal\" (UID: \"a28dba002f2c85352dc44bcb9751469c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal"
Jul 21 13:44:13.788689 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:13.750610    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/de55c8b128a47fa32f279f57a92fe691-config\") pod \"kube-apiserver-proxy-ip-10-0-132-245.ec2.internal\" (UID: \"de55c8b128a47fa32f279f57a92fe691\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-245.ec2.internal"
Jul 21 13:44:13.788689 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:13.750645    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a28dba002f2c85352dc44bcb9751469c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal\" (UID: \"a28dba002f2c85352dc44bcb9751469c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal"
Jul 21 13:44:13.788689 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:13.750688    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a28dba002f2c85352dc44bcb9751469c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal\" (UID: \"a28dba002f2c85352dc44bcb9751469c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal"
Jul 21 13:44:13.788689 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:13.750695    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/de55c8b128a47fa32f279f57a92fe691-config\") pod \"kube-apiserver-proxy-ip-10-0-132-245.ec2.internal\" (UID: \"de55c8b128a47fa32f279f57a92fe691\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-245.ec2.internal"
Jul 21 13:44:13.788689 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:13.750695    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a28dba002f2c85352dc44bcb9751469c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal\" (UID: \"a28dba002f2c85352dc44bcb9751469c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal"
Jul 21 13:44:13.893790 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:13.848987    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-245.ec2.internal\" not found"
Jul 21 13:44:14.016700 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:13.928381    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-245.ec2.internal"
Jul 21 13:44:14.016700 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:13.930816    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal"
Jul 21 13:44:14.016700 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:13.949722    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-245.ec2.internal\" not found"
Jul 21 13:44:14.137681 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:14.050096    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-245.ec2.internal\" not found"
Jul 21 13:44:14.246283 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:14.150562    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-245.ec2.internal\" not found"
Jul 21 13:44:14.252473 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:14.251113    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-245.ec2.internal\" not found"
Jul 21 13:44:14.290434 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:14.290405    2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 21 13:44:14.290576 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:14.290561    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 21 13:44:14.332033 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:14.332012    2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 13:44:14.352228 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:14.352195    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-245.ec2.internal\" not found"
Jul 21 13:44:14.363022 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:14.362980    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-20 13:39:13 +0000 UTC" deadline="2028-04-25 17:31:19.519217472 +0000 UTC"
Jul 21 13:44:14.363022 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:14.363010    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15459h47m5.156210859s"
Jul 21 13:44:14.435051 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:14.435018    2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 13:44:14.449095 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:14.449067    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-245.ec2.internal"
Jul 21 13:44:14.459055 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:14.459033    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 21 13:44:14.460565 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:14.460551    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal"
Jul 21 13:44:14.472771 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:14.472745    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 21 13:44:14.537520 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:14.537435    2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 13:44:14.971707 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:14.971677    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda28dba002f2c85352dc44bcb9751469c.slice/crio-0de95ab37f141b5e8cfa3dac0b474105f20321e82e9cc9d04398705a02e66d5a WatchSource:0}: Error finding container 0de95ab37f141b5e8cfa3dac0b474105f20321e82e9cc9d04398705a02e66d5a: Status 404 returned error can't find the container with id 0de95ab37f141b5e8cfa3dac0b474105f20321e82e9cc9d04398705a02e66d5a
Jul 21 13:44:14.978966 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:14.978944    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 13:44:14.985982 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:14.985954    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde55c8b128a47fa32f279f57a92fe691.slice/crio-e45c8c8aca524f0c0512654890ddeec21f911ccee551bbfb1839d4c9481112c0 WatchSource:0}: Error finding container e45c8c8aca524f0c0512654890ddeec21f911ccee551bbfb1839d4c9481112c0: Status 404 returned error can't find the container with id e45c8c8aca524f0c0512654890ddeec21f911ccee551bbfb1839d4c9481112c0
Jul 21 13:44:15.329575 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.329495    2578 apiserver.go:52] "Watching apiserver"
Jul 21 13:44:15.338748 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.338725    2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 21 13:44:15.339127 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.339106    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-6w5sm","openshift-ovn-kubernetes/ovnkube-node-zcs5b","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5d6xt","openshift-cluster-node-tuning-operator/tuned-q5mrj","openshift-dns/node-resolver-hprzk","openshift-image-registry/node-ca-swmgt","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal","openshift-multus/multus-additional-cni-plugins-6kwdx","openshift-multus/multus-kk2pk","openshift-multus/network-metrics-daemon-lskgb","kube-system/global-pull-secret-syncer-cg5rh","kube-system/konnectivity-agent-mnstv","kube-system/kube-apiserver-proxy-ip-10-0-132-245.ec2.internal","openshift-network-diagnostics/network-check-target-8f7c2"]
Jul 21 13:44:15.341926 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.341908    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lskgb"
Jul 21 13:44:15.342001 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:15.341984    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lskgb" podUID="9780c38e-e96c-47a2-bd63-283bac4f8f59"
Jul 21 13:44:15.343902 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.343887    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cg5rh"
Jul 21 13:44:15.343949 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:15.343936    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cg5rh" podUID="f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913"
Jul 21 13:44:15.347439 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.347396    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5d6xt"
Jul 21 13:44:15.348574 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.348553    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.349085 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.349069    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 21 13:44:15.349729 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.349714    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 21 13:44:15.349785 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.349738    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 21 13:44:15.349990 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.349974    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 21 13:44:15.350069 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.349980    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-xh5b9\""
Jul 21 13:44:15.350765 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.350752    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hprzk"
Jul 21 13:44:15.350915 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.350896    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 21 13:44:15.350990 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.350903    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 21 13:44:15.350990 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.350964    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-h8zq8\""
Jul 21 13:44:15.352975 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.352954    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-swmgt"
Jul 21 13:44:15.353063 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.353001    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-gjlmz\""
Jul 21 13:44:15.353063 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.353001    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 21 13:44:15.353063 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.353046    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 21 13:44:15.355133 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.355116    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 21 13:44:15.355241 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.355230    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.355633 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.355618    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 21 13:44:15.355716 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.355670    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 21 13:44:15.355796 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.355778    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-77mnp\""
Jul 21 13:44:15.357354 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.357339    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 21 13:44:15.357446 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.357409    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 21 13:44:15.357522 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.357480    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 21 13:44:15.357631 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.357584    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 21 13:44:15.357695 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.357630    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8f7c2"
Jul 21 13:44:15.357753 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:15.357687    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8f7c2" podUID="18e5c194-5162-4442-98b5-61bfaf0b8f66"
Jul 21 13:44:15.357809 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.357753    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-l7495\""
Jul 21 13:44:15.359132 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.359117    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bbaf7683-e99d-4fe9-9ebf-07504bb66729-etc-sysconfig\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.359203 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.359139    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bbaf7683-e99d-4fe9-9ebf-07504bb66729-etc-systemd\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.359203 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.359156    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bbaf7683-e99d-4fe9-9ebf-07504bb66729-run\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.359203 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.359170    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/86868b9f-3bee-44f9-b662-ed05ab2b323a-hosts-file\") pod \"node-resolver-hprzk\" (UID: \"86868b9f-3bee-44f9-b662-ed05ab2b323a\") " pod="openshift-dns/node-resolver-hprzk"
Jul 21 13:44:15.359203 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.359186    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2fjf\" (UniqueName: \"kubernetes.io/projected/9780c38e-e96c-47a2-bd63-283bac4f8f59-kube-api-access-f2fjf\") pod \"network-metrics-daemon-lskgb\" (UID: \"9780c38e-e96c-47a2-bd63-283bac4f8f59\") " pod="openshift-multus/network-metrics-daemon-lskgb"
Jul 21 13:44:15.359400 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.359208    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ad6d2b51-331e-464a-8ec5-f518a487bf24-socket-dir\") pod \"aws-ebs-csi-driver-node-5d6xt\" (UID: \"ad6d2b51-331e-464a-8ec5-f518a487bf24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5d6xt"
Jul 21 13:44:15.359400 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.359231    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ad6d2b51-331e-464a-8ec5-f518a487bf24-registration-dir\") pod \"aws-ebs-csi-driver-node-5d6xt\" (UID: \"ad6d2b51-331e-464a-8ec5-f518a487bf24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5d6xt"
Jul 21 13:44:15.359400 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.359249    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ad6d2b51-331e-464a-8ec5-f518a487bf24-sys-fs\") pod \"aws-ebs-csi-driver-node-5d6xt\" (UID: \"ad6d2b51-331e-464a-8ec5-f518a487bf24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5d6xt"
Jul 21 13:44:15.359400 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.359266    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bbaf7683-e99d-4fe9-9ebf-07504bb66729-etc-kubernetes\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.359400 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.359283    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bbaf7683-e99d-4fe9-9ebf-07504bb66729-sys\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.359400 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.359304    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bbaf7683-e99d-4fe9-9ebf-07504bb66729-var-lib-kubelet\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.359400 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.359327    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86868b9f-3bee-44f9-b662-ed05ab2b323a-tmp-dir\") pod \"node-resolver-hprzk\" (UID: \"86868b9f-3bee-44f9-b662-ed05ab2b323a\") " pod="openshift-dns/node-resolver-hprzk"
Jul 21 13:44:15.359400 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.359382    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bbaf7683-e99d-4fe9-9ebf-07504bb66729-lib-modules\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.359764 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.359405    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cm26v\" (UniqueName: \"kubernetes.io/projected/bbaf7683-e99d-4fe9-9ebf-07504bb66729-kube-api-access-cm26v\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.359764 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.359422    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bbaf7683-e99d-4fe9-9ebf-07504bb66729-host\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.359764 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.359453    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ad6d2b51-331e-464a-8ec5-f518a487bf24-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5d6xt\" (UID: \"ad6d2b51-331e-464a-8ec5-f518a487bf24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5d6xt"
Jul 21 13:44:15.359764 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.359469    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bbaf7683-e99d-4fe9-9ebf-07504bb66729-etc-modprobe-d\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.359764 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.359483    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bbaf7683-e99d-4fe9-9ebf-07504bb66729-etc-sysctl-d\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.359764 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.359518    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bbaf7683-e99d-4fe9-9ebf-07504bb66729-etc-sysctl-conf\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.359764 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.359553    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfr7n\" (UniqueName: \"kubernetes.io/projected/86868b9f-3bee-44f9-b662-ed05ab2b323a-kube-api-access-vfr7n\") pod \"node-resolver-hprzk\" (UID: \"86868b9f-3bee-44f9-b662-ed05ab2b323a\") " pod="openshift-dns/node-resolver-hprzk"
Jul 21 13:44:15.359764 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.359589    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9780c38e-e96c-47a2-bd63-283bac4f8f59-metrics-certs\") pod \"network-metrics-daemon-lskgb\" (UID: \"9780c38e-e96c-47a2-bd63-283bac4f8f59\") " pod="openshift-multus/network-metrics-daemon-lskgb"
Jul 21 13:44:15.359764 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.359610    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913-kubelet-config\") pod \"global-pull-secret-syncer-cg5rh\" (UID: \"f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913\") " pod="kube-system/global-pull-secret-syncer-cg5rh"
Jul 21 13:44:15.359764 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.359635    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913-dbus\") pod \"global-pull-secret-syncer-cg5rh\" (UID: \"f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913\") " pod="kube-system/global-pull-secret-syncer-cg5rh"
Jul 21 13:44:15.359764 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.359670    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hnwl\" (UniqueName: \"kubernetes.io/projected/ad6d2b51-331e-464a-8ec5-f518a487bf24-kube-api-access-2hnwl\") pod \"aws-ebs-csi-driver-node-5d6xt\" (UID: \"ad6d2b51-331e-464a-8ec5-f518a487bf24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5d6xt"
Jul 21 13:44:15.359764 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.359707    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bbaf7683-e99d-4fe9-9ebf-07504bb66729-etc-tuned\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.359764 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.359734    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bbaf7683-e99d-4fe9-9ebf-07504bb66729-tmp\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.360336 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.359783    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913-original-pull-secret\") pod \"global-pull-secret-syncer-cg5rh\" (UID: \"f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913\") " pod="kube-system/global-pull-secret-syncer-cg5rh"
Jul 21 13:44:15.360336 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.359819    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ad6d2b51-331e-464a-8ec5-f518a487bf24-device-dir\") pod \"aws-ebs-csi-driver-node-5d6xt\" (UID: \"ad6d2b51-331e-464a-8ec5-f518a487bf24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5d6xt"
Jul 21 13:44:15.360336 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.359842    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ad6d2b51-331e-464a-8ec5-f518a487bf24-etc-selinux\") pod \"aws-ebs-csi-driver-node-5d6xt\" (UID: \"ad6d2b51-331e-464a-8ec5-f518a487bf24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5d6xt"
Jul 21 13:44:15.360336 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.359974    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.361619 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.361603    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 21 13:44:15.361843 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.361831    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 21 13:44:15.362370 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.362352    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6w5sm"
Jul 21 13:44:15.362612 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.362593    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 21 13:44:15.363064 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.363041    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 21 13:44:15.363160 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.363090    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 21 13:44:15.363160 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.363098    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xkh97\""
Jul 21 13:44:15.363160 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.363109    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 21 13:44:15.363160 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.363151    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 21 13:44:15.364709 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.364690    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 21 13:44:15.364798 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.364762    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 21 13:44:15.364798 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.364786    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 21 13:44:15.364949 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.364903    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6kwdx"
Jul 21 13:44:15.365033 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.365018    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-wlqmd\""
Jul 21 13:44:15.367038 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.367021    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 21 13:44:15.367341 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.367315    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 21 13:44:15.367412 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.367320    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mnstv"
Jul 21 13:44:15.367412 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.367384    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-255sv\""
Jul 21 13:44:15.369649 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.369632    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-hxvbv\""
Jul 21 13:44:15.369806 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.369792    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 21 13:44:15.370169 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.370151    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 21 13:44:15.386658 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.386632    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-p6cfd"
Jul 21 13:44:15.396050 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.396027    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-p6cfd"
Jul 21 13:44:15.449805 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.449768    2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 21 13:44:15.460979 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.460947    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bbaf7683-e99d-4fe9-9ebf-07504bb66729-etc-systemd\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.460979 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.460989    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bbaf7683-e99d-4fe9-9ebf-07504bb66729-run\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.461213 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461016    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ad6d2b51-331e-464a-8ec5-f518a487bf24-socket-dir\") pod \"aws-ebs-csi-driver-node-5d6xt\" (UID: \"ad6d2b51-331e-464a-8ec5-f518a487bf24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5d6xt"
Jul 21 13:44:15.461213 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461041    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ad6d2b51-331e-464a-8ec5-f518a487bf24-sys-fs\") pod \"aws-ebs-csi-driver-node-5d6xt\" (UID: \"ad6d2b51-331e-464a-8ec5-f518a487bf24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5d6xt"
Jul 21 13:44:15.461213 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461070    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6kwdx\" (UID: \"3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9\") " pod="openshift-multus/multus-additional-cni-plugins-6kwdx"
Jul 21 13:44:15.461213 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461079    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bbaf7683-e99d-4fe9-9ebf-07504bb66729-etc-systemd\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.461213 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461095    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-host-run-multus-certs\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.461213 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461137    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-run-ovn\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.461213 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461152    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ad6d2b51-331e-464a-8ec5-f518a487bf24-sys-fs\") pod \"aws-ebs-csi-driver-node-5d6xt\" (UID: \"ad6d2b51-331e-464a-8ec5-f518a487bf24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5d6xt"
Jul 21 13:44:15.461213 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461167    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/629e5be3-662a-4713-82fb-3c7892da2245-ovn-node-metrics-cert\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.461213 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461180    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bbaf7683-e99d-4fe9-9ebf-07504bb66729-run\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.461213 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461187    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ad6d2b51-331e-464a-8ec5-f518a487bf24-socket-dir\") pod \"aws-ebs-csi-driver-node-5d6xt\" (UID: \"ad6d2b51-331e-464a-8ec5-f518a487bf24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5d6xt"
Jul 21 13:44:15.461213 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461198    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bbaf7683-e99d-4fe9-9ebf-07504bb66729-etc-kubernetes\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.461213 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461218    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bbaf7683-e99d-4fe9-9ebf-07504bb66729-var-lib-kubelet\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.461773 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461245    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dmpl\" (UniqueName: \"kubernetes.io/projected/30032785-7cc9-421c-b65a-674a329a954f-kube-api-access-8dmpl\") pod \"node-ca-swmgt\" (UID: \"30032785-7cc9-421c-b65a-674a329a954f\") " pod="openshift-image-registry/node-ca-swmgt"
Jul 21 13:44:15.461773 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461263    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bbaf7683-e99d-4fe9-9ebf-07504bb66729-etc-kubernetes\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.461773 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461271    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7905842a-33f8-4cfb-941a-371ced12fead-cni-binary-copy\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.461773 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461280    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bbaf7683-e99d-4fe9-9ebf-07504bb66729-var-lib-kubelet\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.461773 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461295    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-multus-socket-dir-parent\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.461773 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461318    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqq4t\" (UniqueName: \"kubernetes.io/projected/18e5c194-5162-4442-98b5-61bfaf0b8f66-kube-api-access-rqq4t\") pod \"network-check-target-8f7c2\" (UID: \"18e5c194-5162-4442-98b5-61bfaf0b8f66\") " pod="openshift-network-diagnostics/network-check-target-8f7c2"
Jul 21 13:44:15.461773 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461335    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-host-kubelet\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.461773 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461350    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-host-cni-netd\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.461773 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461368    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bbaf7683-e99d-4fe9-9ebf-07504bb66729-lib-modules\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.461773 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461393    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cm26v\" (UniqueName: \"kubernetes.io/projected/bbaf7683-e99d-4fe9-9ebf-07504bb66729-kube-api-access-cm26v\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.461773 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461416    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ls72x\" (UniqueName: \"kubernetes.io/projected/f8473ef3-0b52-479b-b042-cecf34a517c7-kube-api-access-ls72x\") pod \"iptables-alerter-6w5sm\" (UID: \"f8473ef3-0b52-479b-b042-cecf34a517c7\") " pod="openshift-network-operator/iptables-alerter-6w5sm"
Jul 21 13:44:15.461773 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461452    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v8w7\" (UniqueName: \"kubernetes.io/projected/3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9-kube-api-access-6v8w7\") pod \"multus-additional-cni-plugins-6kwdx\" (UID: \"3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9\") " pod="openshift-multus/multus-additional-cni-plugins-6kwdx"
Jul 21 13:44:15.461773 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461476    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-host-run-netns\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.461773 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461502    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bbaf7683-e99d-4fe9-9ebf-07504bb66729-lib-modules\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.462440 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461497    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-log-socket\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.462440 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461535    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/629e5be3-662a-4713-82fb-3c7892da2245-ovnkube-config\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.462440 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461565    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/30032785-7cc9-421c-b65a-674a329a954f-host\") pod \"node-ca-swmgt\" (UID: \"30032785-7cc9-421c-b65a-674a329a954f\") " pod="openshift-image-registry/node-ca-swmgt"
Jul 21 13:44:15.462440 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461592    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6kwdx\" (UID: \"3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9\") " pod="openshift-multus/multus-additional-cni-plugins-6kwdx"
Jul 21 13:44:15.462440 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461609    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b63df369-72fd-4d11-805c-5204b768db57-konnectivity-ca\") pod \"konnectivity-agent-mnstv\" (UID: \"b63df369-72fd-4d11-805c-5204b768db57\") " pod="kube-system/konnectivity-agent-mnstv"
Jul 21 13:44:15.462440 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461632    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-host-run-k8s-cni-cncf-io\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.462440 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461658    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrv84\" (UniqueName: \"kubernetes.io/projected/629e5be3-662a-4713-82fb-3c7892da2245-kube-api-access-hrv84\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.462440 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461680    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f8473ef3-0b52-479b-b042-cecf34a517c7-host-slash\") pod \"iptables-alerter-6w5sm\" (UID: \"f8473ef3-0b52-479b-b042-cecf34a517c7\") " pod="openshift-network-operator/iptables-alerter-6w5sm"
Jul 21 13:44:15.462440 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461709    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9-os-release\") pod \"multus-additional-cni-plugins-6kwdx\" (UID: \"3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9\") " pod="openshift-multus/multus-additional-cni-plugins-6kwdx"
Jul 21 13:44:15.462440 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461727    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-cnibin\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.462440 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461748    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bbaf7683-e99d-4fe9-9ebf-07504bb66729-etc-sysctl-conf\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.462440 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461768    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9780c38e-e96c-47a2-bd63-283bac4f8f59-metrics-certs\") pod \"network-metrics-daemon-lskgb\" (UID: \"9780c38e-e96c-47a2-bd63-283bac4f8f59\") " pod="openshift-multus/network-metrics-daemon-lskgb"
Jul 21 13:44:15.462440 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461794    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913-kubelet-config\") pod \"global-pull-secret-syncer-cg5rh\" (UID: \"f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913\") " pod="kube-system/global-pull-secret-syncer-cg5rh"
Jul 21 13:44:15.462440 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461824    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-etc-openvswitch\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.463339 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461848    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-run-openvswitch\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.463339 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461851    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913-kubelet-config\") pod \"global-pull-secret-syncer-cg5rh\" (UID: \"f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913\") " pod="kube-system/global-pull-secret-syncer-cg5rh"
Jul 21 13:44:15.463339 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461870    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-host-run-netns\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.463339 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:15.461864    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 13:44:15.463339 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461898    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bbaf7683-e99d-4fe9-9ebf-07504bb66729-etc-sysctl-conf\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.463339 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.461916    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-run-systemd\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.463339 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:15.462012    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9780c38e-e96c-47a2-bd63-283bac4f8f59-metrics-certs podName:9780c38e-e96c-47a2-bd63-283bac4f8f59 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:15.961975087 +0000 UTC m=+2.922263623 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9780c38e-e96c-47a2-bd63-283bac4f8f59-metrics-certs") pod "network-metrics-daemon-lskgb" (UID: "9780c38e-e96c-47a2-bd63-283bac4f8f59") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 13:44:15.463339 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.462030    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8bz6\" (UniqueName: \"kubernetes.io/projected/7905842a-33f8-4cfb-941a-371ced12fead-kube-api-access-f8bz6\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.463339 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.462052    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bbaf7683-e99d-4fe9-9ebf-07504bb66729-etc-sysconfig\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.463339 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.462077    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/86868b9f-3bee-44f9-b662-ed05ab2b323a-hosts-file\") pod \"node-resolver-hprzk\" (UID: \"86868b9f-3bee-44f9-b662-ed05ab2b323a\") " pod="openshift-dns/node-resolver-hprzk"
Jul 21 13:44:15.463339 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.462102    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2fjf\" (UniqueName: \"kubernetes.io/projected/9780c38e-e96c-47a2-bd63-283bac4f8f59-kube-api-access-f2fjf\") pod \"network-metrics-daemon-lskgb\" (UID: \"9780c38e-e96c-47a2-bd63-283bac4f8f59\") " pod="openshift-multus/network-metrics-daemon-lskgb"
Jul 21 13:44:15.463339 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.462120    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ad6d2b51-331e-464a-8ec5-f518a487bf24-registration-dir\") pod \"aws-ebs-csi-driver-node-5d6xt\" (UID: \"ad6d2b51-331e-464a-8ec5-f518a487bf24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5d6xt"
Jul 21 13:44:15.463339 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.462136    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bbaf7683-e99d-4fe9-9ebf-07504bb66729-etc-sysconfig\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.463339 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.462183    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ad6d2b51-331e-464a-8ec5-f518a487bf24-registration-dir\") pod \"aws-ebs-csi-driver-node-5d6xt\" (UID: \"ad6d2b51-331e-464a-8ec5-f518a487bf24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5d6xt"
Jul 21 13:44:15.464002 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.462183    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/86868b9f-3bee-44f9-b662-ed05ab2b323a-hosts-file\") pod \"node-resolver-hprzk\" (UID: \"86868b9f-3bee-44f9-b662-ed05ab2b323a\") " pod="openshift-dns/node-resolver-hprzk"
Jul 21 13:44:15.464002 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.462226    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-multus-cni-dir\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.464002 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.462246    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-systemd-units\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.464002 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.462261    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/629e5be3-662a-4713-82fb-3c7892da2245-env-overrides\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.464002 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.462287    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bbaf7683-e99d-4fe9-9ebf-07504bb66729-sys\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.464002 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.462321    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86868b9f-3bee-44f9-b662-ed05ab2b323a-tmp-dir\") pod \"node-resolver-hprzk\" (UID: \"86868b9f-3bee-44f9-b662-ed05ab2b323a\") " pod="openshift-dns/node-resolver-hprzk"
Jul 21 13:44:15.464002 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.462338    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6kwdx\" (UID: \"3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9\") " pod="openshift-multus/multus-additional-cni-plugins-6kwdx"
Jul 21 13:44:15.464002 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.462376    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b63df369-72fd-4d11-805c-5204b768db57-agent-certs\") pod \"konnectivity-agent-mnstv\" (UID: \"b63df369-72fd-4d11-805c-5204b768db57\") " pod="kube-system/konnectivity-agent-mnstv"
Jul 21 13:44:15.464002 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.462399    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bbaf7683-e99d-4fe9-9ebf-07504bb66729-sys\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.464002 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.462402    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-host-var-lib-cni-bin\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.464002 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.462445    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-host-cni-bin\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.464002 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.462488    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9-system-cni-dir\") pod \"multus-additional-cni-plugins-6kwdx\" (UID: \"3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9\") " pod="openshift-multus/multus-additional-cni-plugins-6kwdx"
Jul 21 13:44:15.464002 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.462516    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-node-log\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.464002 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.462542    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bbaf7683-e99d-4fe9-9ebf-07504bb66729-host\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.464665 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.462567    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ad6d2b51-331e-464a-8ec5-f518a487bf24-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5d6xt\" (UID: \"ad6d2b51-331e-464a-8ec5-f518a487bf24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5d6xt"
Jul 21 13:44:15.464665 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.462593    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/30032785-7cc9-421c-b65a-674a329a954f-serviceca\") pod \"node-ca-swmgt\" (UID: \"30032785-7cc9-421c-b65a-674a329a954f\") " pod="openshift-image-registry/node-ca-swmgt"
Jul 21 13:44:15.464665 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.462605    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bbaf7683-e99d-4fe9-9ebf-07504bb66729-host\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.464665 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.462619    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-system-cni-dir\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.464665 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.462643    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-host-var-lib-kubelet\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.464665 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.462646    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/86868b9f-3bee-44f9-b662-ed05ab2b323a-tmp-dir\") pod \"node-resolver-hprzk\" (UID: \"86868b9f-3bee-44f9-b662-ed05ab2b323a\") " pod="openshift-dns/node-resolver-hprzk"
Jul 21 13:44:15.464665 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.462655    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ad6d2b51-331e-464a-8ec5-f518a487bf24-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5d6xt\" (UID: \"ad6d2b51-331e-464a-8ec5-f518a487bf24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5d6xt"
Jul 21 13:44:15.464665 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.462666    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-multus-conf-dir\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.464665 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.462691    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.464665 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.462720    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bbaf7683-e99d-4fe9-9ebf-07504bb66729-etc-modprobe-d\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.464665 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.462745    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bbaf7683-e99d-4fe9-9ebf-07504bb66729-etc-sysctl-d\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.464665 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.462776    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfr7n\" (UniqueName: \"kubernetes.io/projected/86868b9f-3bee-44f9-b662-ed05ab2b323a-kube-api-access-vfr7n\") pod \"node-resolver-hprzk\" (UID: \"86868b9f-3bee-44f9-b662-ed05ab2b323a\") " pod="openshift-dns/node-resolver-hprzk"
Jul 21 13:44:15.464665 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.462871    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bbaf7683-e99d-4fe9-9ebf-07504bb66729-etc-modprobe-d\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.464665 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.462910    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bbaf7683-e99d-4fe9-9ebf-07504bb66729-etc-sysctl-d\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.465371 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.462915    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913-dbus\") pod \"global-pull-secret-syncer-cg5rh\" (UID: \"f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913\") " pod="kube-system/global-pull-secret-syncer-cg5rh"
Jul 21 13:44:15.465371 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.462958    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hnwl\" (UniqueName: \"kubernetes.io/projected/ad6d2b51-331e-464a-8ec5-f518a487bf24-kube-api-access-2hnwl\") pod \"aws-ebs-csi-driver-node-5d6xt\" (UID: \"ad6d2b51-331e-464a-8ec5-f518a487bf24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5d6xt"
Jul 21 13:44:15.465371 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.462976    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-os-release\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.465371 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.462998    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-hostroot\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.465371 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.463025    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913-dbus\") pod \"global-pull-secret-syncer-cg5rh\" (UID: \"f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913\") " pod="kube-system/global-pull-secret-syncer-cg5rh"
Jul 21 13:44:15.465371 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.463022    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-host-run-ovn-kubernetes\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.465371 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.463066    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bbaf7683-e99d-4fe9-9ebf-07504bb66729-etc-tuned\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.465371 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.463140    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bbaf7683-e99d-4fe9-9ebf-07504bb66729-tmp\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.465371 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.463250    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913-original-pull-secret\") pod \"global-pull-secret-syncer-cg5rh\" (UID: \"f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913\") " pod="kube-system/global-pull-secret-syncer-cg5rh"
Jul 21 13:44:15.465371 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.463277    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f8473ef3-0b52-479b-b042-cecf34a517c7-iptables-alerter-script\") pod \"iptables-alerter-6w5sm\" (UID: \"f8473ef3-0b52-479b-b042-cecf34a517c7\") " pod="openshift-network-operator/iptables-alerter-6w5sm"
Jul 21 13:44:15.465371 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.463314    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9-cnibin\") pod \"multus-additional-cni-plugins-6kwdx\" (UID: \"3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9\") " pod="openshift-multus/multus-additional-cni-plugins-6kwdx"
Jul 21 13:44:15.465371 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.463338    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-host-var-lib-cni-multus\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.465371 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:15.463353    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 13:44:15.465371 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.463363    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-etc-kubernetes\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.465371 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.463396    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-host-slash\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.466087 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:15.463413    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913-original-pull-secret podName:f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:15.963396515 +0000 UTC m=+2.923685053 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913-original-pull-secret") pod "global-pull-secret-syncer-cg5rh" (UID: "f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913") : object "kube-system"/"original-pull-secret" not registered
Jul 21 13:44:15.466087 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.463453    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ad6d2b51-331e-464a-8ec5-f518a487bf24-device-dir\") pod \"aws-ebs-csi-driver-node-5d6xt\" (UID: \"ad6d2b51-331e-464a-8ec5-f518a487bf24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5d6xt"
Jul 21 13:44:15.466087 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.463480    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ad6d2b51-331e-464a-8ec5-f518a487bf24-etc-selinux\") pod \"aws-ebs-csi-driver-node-5d6xt\" (UID: \"ad6d2b51-331e-464a-8ec5-f518a487bf24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5d6xt"
Jul 21 13:44:15.466087 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.463487    2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 21 13:44:15.466087 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.463504    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9-cni-binary-copy\") pod \"multus-additional-cni-plugins-6kwdx\" (UID: \"3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9\") " pod="openshift-multus/multus-additional-cni-plugins-6kwdx"
Jul 21 13:44:15.466087 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.463527    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7905842a-33f8-4cfb-941a-371ced12fead-multus-daemon-config\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.466087 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.463545    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-var-lib-openvswitch\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.466087 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.463551    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ad6d2b51-331e-464a-8ec5-f518a487bf24-device-dir\") pod \"aws-ebs-csi-driver-node-5d6xt\" (UID: \"ad6d2b51-331e-464a-8ec5-f518a487bf24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5d6xt"
Jul 21 13:44:15.466087 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.463565    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/629e5be3-662a-4713-82fb-3c7892da2245-ovnkube-script-lib\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.466087 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.463605    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ad6d2b51-331e-464a-8ec5-f518a487bf24-etc-selinux\") pod \"aws-ebs-csi-driver-node-5d6xt\" (UID: \"ad6d2b51-331e-464a-8ec5-f518a487bf24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5d6xt"
Jul 21 13:44:15.466735 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.466530    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bbaf7683-e99d-4fe9-9ebf-07504bb66729-tmp\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.466735 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.466568    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bbaf7683-e99d-4fe9-9ebf-07504bb66729-etc-tuned\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.472324 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.472294    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cm26v\" (UniqueName: \"kubernetes.io/projected/bbaf7683-e99d-4fe9-9ebf-07504bb66729-kube-api-access-cm26v\") pod \"tuned-q5mrj\" (UID: \"bbaf7683-e99d-4fe9-9ebf-07504bb66729\") " pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.472693 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.472665    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2fjf\" (UniqueName: \"kubernetes.io/projected/9780c38e-e96c-47a2-bd63-283bac4f8f59-kube-api-access-f2fjf\") pod \"network-metrics-daemon-lskgb\" (UID: \"9780c38e-e96c-47a2-bd63-283bac4f8f59\") " pod="openshift-multus/network-metrics-daemon-lskgb"
Jul 21 13:44:15.473694 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.473671    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hnwl\" (UniqueName: \"kubernetes.io/projected/ad6d2b51-331e-464a-8ec5-f518a487bf24-kube-api-access-2hnwl\") pod \"aws-ebs-csi-driver-node-5d6xt\" (UID: \"ad6d2b51-331e-464a-8ec5-f518a487bf24\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5d6xt"
Jul 21 13:44:15.474101 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.474080    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfr7n\" (UniqueName: \"kubernetes.io/projected/86868b9f-3bee-44f9-b662-ed05ab2b323a-kube-api-access-vfr7n\") pod \"node-resolver-hprzk\" (UID: \"86868b9f-3bee-44f9-b662-ed05ab2b323a\") " pod="openshift-dns/node-resolver-hprzk"
Jul 21 13:44:15.502047 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.501991    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-245.ec2.internal" event={"ID":"de55c8b128a47fa32f279f57a92fe691","Type":"ContainerStarted","Data":"e45c8c8aca524f0c0512654890ddeec21f911ccee551bbfb1839d4c9481112c0"}
Jul 21 13:44:15.503152 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.503117    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal" event={"ID":"a28dba002f2c85352dc44bcb9751469c","Type":"ContainerStarted","Data":"0de95ab37f141b5e8cfa3dac0b474105f20321e82e9cc9d04398705a02e66d5a"}
Jul 21 13:44:15.564473 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.564437    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-etc-kubernetes\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.564666 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.564479    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-host-slash\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.564666 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.564553    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-host-slash\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.564666 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.564557    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-etc-kubernetes\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.564666 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.564587    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9-cni-binary-copy\") pod \"multus-additional-cni-plugins-6kwdx\" (UID: \"3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9\") " pod="openshift-multus/multus-additional-cni-plugins-6kwdx"
Jul 21 13:44:15.564666 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.564614    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7905842a-33f8-4cfb-941a-371ced12fead-multus-daemon-config\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.564666 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.564636    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-var-lib-openvswitch\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.564666 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.564657    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/629e5be3-662a-4713-82fb-3c7892da2245-ovnkube-script-lib\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.565060 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.564713    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-var-lib-openvswitch\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.565060 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.564758    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6kwdx\" (UID: \"3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9\") " pod="openshift-multus/multus-additional-cni-plugins-6kwdx"
Jul 21 13:44:15.565060 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.564788    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-host-run-multus-certs\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.565060 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.564840    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-run-ovn\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.565060 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.564865    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/629e5be3-662a-4713-82fb-3c7892da2245-ovn-node-metrics-cert\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.565060 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.564916    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dmpl\" (UniqueName: \"kubernetes.io/projected/30032785-7cc9-421c-b65a-674a329a954f-kube-api-access-8dmpl\") pod \"node-ca-swmgt\" (UID: \"30032785-7cc9-421c-b65a-674a329a954f\") " pod="openshift-image-registry/node-ca-swmgt"
Jul 21 13:44:15.565060 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.565005    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7905842a-33f8-4cfb-941a-371ced12fead-cni-binary-copy\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.565060 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.565022    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-multus-socket-dir-parent\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.565060 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.565037    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqq4t\" (UniqueName: \"kubernetes.io/projected/18e5c194-5162-4442-98b5-61bfaf0b8f66-kube-api-access-rqq4t\") pod \"network-check-target-8f7c2\" (UID: \"18e5c194-5162-4442-98b5-61bfaf0b8f66\") " pod="openshift-network-diagnostics/network-check-target-8f7c2"
Jul 21 13:44:15.565060 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.565051    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-host-kubelet\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.565524 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.565066    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-host-cni-netd\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.565524 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.565090    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ls72x\" (UniqueName: \"kubernetes.io/projected/f8473ef3-0b52-479b-b042-cecf34a517c7-kube-api-access-ls72x\") pod \"iptables-alerter-6w5sm\" (UID: \"f8473ef3-0b52-479b-b042-cecf34a517c7\") " pod="openshift-network-operator/iptables-alerter-6w5sm"
Jul 21 13:44:15.565524 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.565104    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v8w7\" (UniqueName: \"kubernetes.io/projected/3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9-kube-api-access-6v8w7\") pod \"multus-additional-cni-plugins-6kwdx\" (UID: \"3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9\") " pod="openshift-multus/multus-additional-cni-plugins-6kwdx"
Jul 21 13:44:15.565524 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.565157    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-host-run-netns\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.565524 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.565179    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-log-socket\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.565524 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.565193    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/629e5be3-662a-4713-82fb-3c7892da2245-ovnkube-config\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.565524 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.565207    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/30032785-7cc9-421c-b65a-674a329a954f-host\") pod \"node-ca-swmgt\" (UID: \"30032785-7cc9-421c-b65a-674a329a954f\") " pod="openshift-image-registry/node-ca-swmgt"
Jul 21 13:44:15.565524 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.565231    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6kwdx\" (UID: \"3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9\") " pod="openshift-multus/multus-additional-cni-plugins-6kwdx"
Jul 21 13:44:15.565524 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.565253    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b63df369-72fd-4d11-805c-5204b768db57-konnectivity-ca\") pod \"konnectivity-agent-mnstv\" (UID: \"b63df369-72fd-4d11-805c-5204b768db57\") " pod="kube-system/konnectivity-agent-mnstv"
Jul 21 13:44:15.565524 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.565269    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-host-run-k8s-cni-cncf-io\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.565524 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.565292    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrv84\" (UniqueName: \"kubernetes.io/projected/629e5be3-662a-4713-82fb-3c7892da2245-kube-api-access-hrv84\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.565524 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.565329    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f8473ef3-0b52-479b-b042-cecf34a517c7-host-slash\") pod \"iptables-alerter-6w5sm\" (UID: \"f8473ef3-0b52-479b-b042-cecf34a517c7\") " pod="openshift-network-operator/iptables-alerter-6w5sm"
Jul 21 13:44:15.565524 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.565344    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9-os-release\") pod \"multus-additional-cni-plugins-6kwdx\" (UID: \"3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9\") " pod="openshift-multus/multus-additional-cni-plugins-6kwdx"
Jul 21 13:44:15.565524 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.565358    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-cnibin\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.566132 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.565383    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-etc-openvswitch\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.566132 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.565379    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9-cni-binary-copy\") pod \"multus-additional-cni-plugins-6kwdx\" (UID: \"3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9\") " pod="openshift-multus/multus-additional-cni-plugins-6kwdx"
Jul 21 13:44:15.566132 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.565414    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-run-openvswitch\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.566132 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.565430    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-host-run-netns\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.566132 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.565463    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/629e5be3-662a-4713-82fb-3c7892da2245-ovnkube-script-lib\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.566132 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.565471    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-run-systemd\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.566132 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.565498    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-run-systemd\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.566132 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.565500    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-run-ovn\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.566132 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.565530    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8bz6\" (UniqueName: \"kubernetes.io/projected/7905842a-33f8-4cfb-941a-371ced12fead-kube-api-access-f8bz6\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.566132 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.565549    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-log-socket\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.566132 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.565567    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-multus-cni-dir\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.566132 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.565557    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-host-run-netns\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.566132 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.565599    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-systemd-units\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.566132 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.565663    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/629e5be3-662a-4713-82fb-3c7892da2245-env-overrides\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.566132 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.565698    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6kwdx\" (UID: \"3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9\") " pod="openshift-multus/multus-additional-cni-plugins-6kwdx"
Jul 21 13:44:15.566776 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.566100    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b63df369-72fd-4d11-805c-5204b768db57-agent-certs\") pod \"konnectivity-agent-mnstv\" (UID: \"b63df369-72fd-4d11-805c-5204b768db57\") " pod="kube-system/konnectivity-agent-mnstv"
Jul 21 13:44:15.566776 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.566145    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-host-var-lib-cni-bin\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.566776 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.566238    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-host-cni-bin\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.566776 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.566275    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9-system-cni-dir\") pod \"multus-additional-cni-plugins-6kwdx\" (UID: \"3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9\") " pod="openshift-multus/multus-additional-cni-plugins-6kwdx"
Jul 21 13:44:15.566776 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.566315    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-node-log\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.566776 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.566345    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/30032785-7cc9-421c-b65a-674a329a954f-serviceca\") pod \"node-ca-swmgt\" (UID: \"30032785-7cc9-421c-b65a-674a329a954f\") " pod="openshift-image-registry/node-ca-swmgt"
Jul 21 13:44:15.566776 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.566362    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-system-cni-dir\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.566776 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.566398    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-host-var-lib-kubelet\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.566776 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.566504    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9-os-release\") pod \"multus-additional-cni-plugins-6kwdx\" (UID: \"3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9\") " pod="openshift-multus/multus-additional-cni-plugins-6kwdx"
Jul 21 13:44:15.566776 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.566608    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-multus-conf-dir\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.566776 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.566674    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-multus-socket-dir-parent\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.566776 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.566710    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/629e5be3-662a-4713-82fb-3c7892da2245-ovnkube-config\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.567317 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.566863    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-multus-cni-dir\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.567317 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.567116    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-host-cni-bin\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.567317 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.567187    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-host-kubelet\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.567317 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.567243    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7905842a-33f8-4cfb-941a-371ced12fead-cni-binary-copy\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.567511 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.567322    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9-system-cni-dir\") pod \"multus-additional-cni-plugins-6kwdx\" (UID: \"3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9\") " pod="openshift-multus/multus-additional-cni-plugins-6kwdx"
Jul 21 13:44:15.567511 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.567317    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-host-var-lib-cni-bin\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.567511 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.567406    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-host-cni-netd\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.567511 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.567460    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-systemd-units\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.567680 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.567522    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-node-log\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.567732 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.567704    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-system-cni-dir\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.569153 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.567945    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-host-var-lib-kubelet\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.569153 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.568016    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/30032785-7cc9-421c-b65a-674a329a954f-serviceca\") pod \"node-ca-swmgt\" (UID: \"30032785-7cc9-421c-b65a-674a329a954f\") " pod="openshift-image-registry/node-ca-swmgt"
Jul 21 13:44:15.569153 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.568205    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-cnibin\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.569153 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.568260    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/30032785-7cc9-421c-b65a-674a329a954f-host\") pod \"node-ca-swmgt\" (UID: \"30032785-7cc9-421c-b65a-674a329a954f\") " pod="openshift-image-registry/node-ca-swmgt"
Jul 21 13:44:15.569153 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.568277    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-etc-openvswitch\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.569153 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.568280    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6kwdx\" (UID: \"3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9\") " pod="openshift-multus/multus-additional-cni-plugins-6kwdx"
Jul 21 13:44:15.569153 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.568409    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6kwdx\" (UID: \"3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9\") " pod="openshift-multus/multus-additional-cni-plugins-6kwdx"
Jul 21 13:44:15.569153 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.568468    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f8473ef3-0b52-479b-b042-cecf34a517c7-host-slash\") pod \"iptables-alerter-6w5sm\" (UID: \"f8473ef3-0b52-479b-b042-cecf34a517c7\") " pod="openshift-network-operator/iptables-alerter-6w5sm"
Jul 21 13:44:15.569153 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.565379    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7905842a-33f8-4cfb-941a-371ced12fead-multus-daemon-config\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.569153 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.565442    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-host-run-multus-certs\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.569153 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.568692    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-run-openvswitch\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.569153 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.568760    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/629e5be3-662a-4713-82fb-3c7892da2245-env-overrides\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.569153 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.568847    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-host-run-netns\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.569153 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.568891    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-host-run-k8s-cni-cncf-io\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.569153 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.568938    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-multus-conf-dir\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.569969 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.568943    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6kwdx\" (UID: \"3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9\") " pod="openshift-multus/multus-additional-cni-plugins-6kwdx"
Jul 21 13:44:15.569969 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.568981    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.569969 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.569022    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.569969 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.569071    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-os-release\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.569969 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.569107    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-hostroot\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.569969 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.569154    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-host-run-ovn-kubernetes\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.569969 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.569180    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-os-release\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.569969 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.569197    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f8473ef3-0b52-479b-b042-cecf34a517c7-iptables-alerter-script\") pod \"iptables-alerter-6w5sm\" (UID: \"f8473ef3-0b52-479b-b042-cecf34a517c7\") " pod="openshift-network-operator/iptables-alerter-6w5sm"
Jul 21 13:44:15.569969 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.569231    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9-cnibin\") pod \"multus-additional-cni-plugins-6kwdx\" (UID: \"3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9\") " pod="openshift-multus/multus-additional-cni-plugins-6kwdx"
Jul 21 13:44:15.569969 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.569243    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-hostroot\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.569969 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.569263    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-host-var-lib-cni-multus\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.569969 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.569340    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7905842a-33f8-4cfb-941a-371ced12fead-host-var-lib-cni-multus\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.569969 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.569387    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9-cnibin\") pod \"multus-additional-cni-plugins-6kwdx\" (UID: \"3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9\") " pod="openshift-multus/multus-additional-cni-plugins-6kwdx"
Jul 21 13:44:15.569969 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.569437    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/629e5be3-662a-4713-82fb-3c7892da2245-host-run-ovn-kubernetes\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.570595 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.569576    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b63df369-72fd-4d11-805c-5204b768db57-konnectivity-ca\") pod \"konnectivity-agent-mnstv\" (UID: \"b63df369-72fd-4d11-805c-5204b768db57\") " pod="kube-system/konnectivity-agent-mnstv"
Jul 21 13:44:15.570595 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.569800    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f8473ef3-0b52-479b-b042-cecf34a517c7-iptables-alerter-script\") pod \"iptables-alerter-6w5sm\" (UID: \"f8473ef3-0b52-479b-b042-cecf34a517c7\") " pod="openshift-network-operator/iptables-alerter-6w5sm"
Jul 21 13:44:15.570915 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.570849    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b63df369-72fd-4d11-805c-5204b768db57-agent-certs\") pod \"konnectivity-agent-mnstv\" (UID: \"b63df369-72fd-4d11-805c-5204b768db57\") " pod="kube-system/konnectivity-agent-mnstv"
Jul 21 13:44:15.571176 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.571157    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/629e5be3-662a-4713-82fb-3c7892da2245-ovn-node-metrics-cert\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.572297 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:15.572203    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 13:44:15.572297 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:15.572233    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 13:44:15.572297 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:15.572250    2578 projected.go:196] Error preparing data for projected volume kube-api-access-rqq4t for pod openshift-network-diagnostics/network-check-target-8f7c2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 13:44:15.572599 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:15.572320    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/18e5c194-5162-4442-98b5-61bfaf0b8f66-kube-api-access-rqq4t podName:18e5c194-5162-4442-98b5-61bfaf0b8f66 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:16.072302293 +0000 UTC m=+3.032590848 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rqq4t" (UniqueName: "kubernetes.io/projected/18e5c194-5162-4442-98b5-61bfaf0b8f66-kube-api-access-rqq4t") pod "network-check-target-8f7c2" (UID: "18e5c194-5162-4442-98b5-61bfaf0b8f66") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 13:44:15.574890 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.574847    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dmpl\" (UniqueName: \"kubernetes.io/projected/30032785-7cc9-421c-b65a-674a329a954f-kube-api-access-8dmpl\") pod \"node-ca-swmgt\" (UID: \"30032785-7cc9-421c-b65a-674a329a954f\") " pod="openshift-image-registry/node-ca-swmgt"
Jul 21 13:44:15.575030 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.574996    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8bz6\" (UniqueName: \"kubernetes.io/projected/7905842a-33f8-4cfb-941a-371ced12fead-kube-api-access-f8bz6\") pod \"multus-kk2pk\" (UID: \"7905842a-33f8-4cfb-941a-371ced12fead\") " pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.575030 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.575009    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ls72x\" (UniqueName: \"kubernetes.io/projected/f8473ef3-0b52-479b-b042-cecf34a517c7-kube-api-access-ls72x\") pod \"iptables-alerter-6w5sm\" (UID: \"f8473ef3-0b52-479b-b042-cecf34a517c7\") " pod="openshift-network-operator/iptables-alerter-6w5sm"
Jul 21 13:44:15.575394 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.575374    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v8w7\" (UniqueName: \"kubernetes.io/projected/3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9-kube-api-access-6v8w7\") pod \"multus-additional-cni-plugins-6kwdx\" (UID: \"3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9\") " pod="openshift-multus/multus-additional-cni-plugins-6kwdx"
Jul 21 13:44:15.576761 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.576744    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrv84\" (UniqueName: \"kubernetes.io/projected/629e5be3-662a-4713-82fb-3c7892da2245-kube-api-access-hrv84\") pod \"ovnkube-node-zcs5b\" (UID: \"629e5be3-662a-4713-82fb-3c7892da2245\") " pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.647849 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.647756    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 13:44:15.658051 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.658017    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5d6xt"
Jul 21 13:44:15.663828 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.663805    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-q5mrj"
Jul 21 13:44:15.671654 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.671633    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hprzk"
Jul 21 13:44:15.672242 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:15.672216    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad6d2b51_331e_464a_8ec5_f518a487bf24.slice/crio-772001c147a84105977fcf624607e6cd5bda7159c1a4011dd286e01ff789f6b4 WatchSource:0}: Error finding container 772001c147a84105977fcf624607e6cd5bda7159c1a4011dd286e01ff789f6b4: Status 404 returned error can't find the container with id 772001c147a84105977fcf624607e6cd5bda7159c1a4011dd286e01ff789f6b4
Jul 21 13:44:15.674962 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:15.674938    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbaf7683_e99d_4fe9_9ebf_07504bb66729.slice/crio-ba1aad1d3326847feca52e8d3fad5944251531ca465f77910992d67ef7e57451 WatchSource:0}: Error finding container ba1aad1d3326847feca52e8d3fad5944251531ca465f77910992d67ef7e57451: Status 404 returned error can't find the container with id ba1aad1d3326847feca52e8d3fad5944251531ca465f77910992d67ef7e57451
Jul 21 13:44:15.679683 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.679659    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-swmgt"
Jul 21 13:44:15.681216 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.681199    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kk2pk"
Jul 21 13:44:15.685029 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:15.685007    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86868b9f_3bee_44f9_b662_ed05ab2b323a.slice/crio-09881486a22389f64c8199605db1e12bc692f07d0c23463d8663a0ea2280d91d WatchSource:0}: Error finding container 09881486a22389f64c8199605db1e12bc692f07d0c23463d8663a0ea2280d91d: Status 404 returned error can't find the container with id 09881486a22389f64c8199605db1e12bc692f07d0c23463d8663a0ea2280d91d
Jul 21 13:44:15.686299 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.686281    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:15.694656 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.694634    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6w5sm"
Jul 21 13:44:15.698909 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.698861    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6kwdx"
Jul 21 13:44:15.701930 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:15.701758    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30032785_7cc9_421c_b65a_674a329a954f.slice/crio-a80e9de62ab5744b8187582deabb61abdf71d612bb8b7f94e47cfed0339e7f4f WatchSource:0}: Error finding container a80e9de62ab5744b8187582deabb61abdf71d612bb8b7f94e47cfed0339e7f4f: Status 404 returned error can't find the container with id a80e9de62ab5744b8187582deabb61abdf71d612bb8b7f94e47cfed0339e7f4f
Jul 21 13:44:15.703086 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.703065    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mnstv"
Jul 21 13:44:15.703470 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:15.703431    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7905842a_33f8_4cfb_941a_371ced12fead.slice/crio-e7af8cc8cf4c7441c8e64a9f8b34645b7a8048ace086412d4515321e0fcc4b52 WatchSource:0}: Error finding container e7af8cc8cf4c7441c8e64a9f8b34645b7a8048ace086412d4515321e0fcc4b52: Status 404 returned error can't find the container with id e7af8cc8cf4c7441c8e64a9f8b34645b7a8048ace086412d4515321e0fcc4b52
Jul 21 13:44:15.711147 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:15.711124    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod629e5be3_662a_4713_82fb_3c7892da2245.slice/crio-4d9c24d4463afc9eb4a1e2f5876da4f853c454a515ad3ee6755c4b6b61f02719 WatchSource:0}: Error finding container 4d9c24d4463afc9eb4a1e2f5876da4f853c454a515ad3ee6755c4b6b61f02719: Status 404 returned error can't find the container with id 4d9c24d4463afc9eb4a1e2f5876da4f853c454a515ad3ee6755c4b6b61f02719
Jul 21 13:44:15.725132 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:15.725109    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8473ef3_0b52_479b_b042_cecf34a517c7.slice/crio-3e22d112c0d8f3804ba3404a1c9d5f1a43740b7d6df0807df468457dcbb1d53a WatchSource:0}: Error finding container 3e22d112c0d8f3804ba3404a1c9d5f1a43740b7d6df0807df468457dcbb1d53a: Status 404 returned error can't find the container with id 3e22d112c0d8f3804ba3404a1c9d5f1a43740b7d6df0807df468457dcbb1d53a
Jul 21 13:44:15.973288 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.973205    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9780c38e-e96c-47a2-bd63-283bac4f8f59-metrics-certs\") pod \"network-metrics-daemon-lskgb\" (UID: \"9780c38e-e96c-47a2-bd63-283bac4f8f59\") " pod="openshift-multus/network-metrics-daemon-lskgb"
Jul 21 13:44:15.973288 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:15.973271    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913-original-pull-secret\") pod \"global-pull-secret-syncer-cg5rh\" (UID: \"f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913\") " pod="kube-system/global-pull-secret-syncer-cg5rh"
Jul 21 13:44:15.973525 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:15.973382    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 13:44:15.973525 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:15.973396    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 13:44:15.973525 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:15.973480    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9780c38e-e96c-47a2-bd63-283bac4f8f59-metrics-certs podName:9780c38e-e96c-47a2-bd63-283bac4f8f59 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:16.973448102 +0000 UTC m=+3.933736660 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9780c38e-e96c-47a2-bd63-283bac4f8f59-metrics-certs") pod "network-metrics-daemon-lskgb" (UID: "9780c38e-e96c-47a2-bd63-283bac4f8f59") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 13:44:15.973525 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:15.973504    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913-original-pull-secret podName:f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:16.973492947 +0000 UTC m=+3.933781488 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913-original-pull-secret") pod "global-pull-secret-syncer-cg5rh" (UID: "f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913") : object "kube-system"/"original-pull-secret" not registered
Jul 21 13:44:16.074554 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:16.074522    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqq4t\" (UniqueName: \"kubernetes.io/projected/18e5c194-5162-4442-98b5-61bfaf0b8f66-kube-api-access-rqq4t\") pod \"network-check-target-8f7c2\" (UID: \"18e5c194-5162-4442-98b5-61bfaf0b8f66\") " pod="openshift-network-diagnostics/network-check-target-8f7c2"
Jul 21 13:44:16.074741 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:16.074691    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 13:44:16.074741 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:16.074720    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 13:44:16.074741 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:16.074733    2578 projected.go:196] Error preparing data for projected volume kube-api-access-rqq4t for pod openshift-network-diagnostics/network-check-target-8f7c2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 13:44:16.074909 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:16.074789    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/18e5c194-5162-4442-98b5-61bfaf0b8f66-kube-api-access-rqq4t podName:18e5c194-5162-4442-98b5-61bfaf0b8f66 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:17.074775401 +0000 UTC m=+4.035063937 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rqq4t" (UniqueName: "kubernetes.io/projected/18e5c194-5162-4442-98b5-61bfaf0b8f66-kube-api-access-rqq4t") pod "network-check-target-8f7c2" (UID: "18e5c194-5162-4442-98b5-61bfaf0b8f66") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 13:44:16.399639 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:16.399495    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 13:39:15 +0000 UTC" deadline="2028-01-08 00:57:32.348666958 +0000 UTC"
Jul 21 13:44:16.399639 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:16.399533    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12851h13m15.949138481s"
Jul 21 13:44:16.498805 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:16.498390    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cg5rh"
Jul 21 13:44:16.498805 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:16.498389    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8f7c2"
Jul 21 13:44:16.498805 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:16.498507    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cg5rh" podUID="f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913"
Jul 21 13:44:16.498805 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:16.498573    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8f7c2" podUID="18e5c194-5162-4442-98b5-61bfaf0b8f66"
Jul 21 13:44:16.506650 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:16.506573    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mnstv" event={"ID":"b63df369-72fd-4d11-805c-5204b768db57","Type":"ContainerStarted","Data":"6f3a19e51004bd1129e9eefd714b39304c185ac6d24ec7f17d7a4a7a6e565e51"}
Jul 21 13:44:16.509510 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:16.509424    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6w5sm" event={"ID":"f8473ef3-0b52-479b-b042-cecf34a517c7","Type":"ContainerStarted","Data":"3e22d112c0d8f3804ba3404a1c9d5f1a43740b7d6df0807df468457dcbb1d53a"}
Jul 21 13:44:16.511651 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:16.511574    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kk2pk" event={"ID":"7905842a-33f8-4cfb-941a-371ced12fead","Type":"ContainerStarted","Data":"e7af8cc8cf4c7441c8e64a9f8b34645b7a8048ace086412d4515321e0fcc4b52"}
Jul 21 13:44:16.514116 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:16.514024    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5d6xt" event={"ID":"ad6d2b51-331e-464a-8ec5-f518a487bf24","Type":"ContainerStarted","Data":"772001c147a84105977fcf624607e6cd5bda7159c1a4011dd286e01ff789f6b4"}
Jul 21 13:44:16.516002 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:16.515972    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6kwdx" event={"ID":"3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9","Type":"ContainerStarted","Data":"b290703d2dd5888ba167dc94ac6f2aafdd03f601d94af596def3ad4f7d13efbc"}
Jul 21 13:44:16.517454 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:16.517427    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b" event={"ID":"629e5be3-662a-4713-82fb-3c7892da2245","Type":"ContainerStarted","Data":"4d9c24d4463afc9eb4a1e2f5876da4f853c454a515ad3ee6755c4b6b61f02719"}
Jul 21 13:44:16.519311 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:16.519285    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-swmgt" event={"ID":"30032785-7cc9-421c-b65a-674a329a954f","Type":"ContainerStarted","Data":"a80e9de62ab5744b8187582deabb61abdf71d612bb8b7f94e47cfed0339e7f4f"}
Jul 21 13:44:16.522334 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:16.521960    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hprzk" event={"ID":"86868b9f-3bee-44f9-b662-ed05ab2b323a","Type":"ContainerStarted","Data":"09881486a22389f64c8199605db1e12bc692f07d0c23463d8663a0ea2280d91d"}
Jul 21 13:44:16.523997 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:16.523971    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-q5mrj" event={"ID":"bbaf7683-e99d-4fe9-9ebf-07504bb66729","Type":"ContainerStarted","Data":"ba1aad1d3326847feca52e8d3fad5944251531ca465f77910992d67ef7e57451"}
Jul 21 13:44:16.981859 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:16.981807    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913-original-pull-secret\") pod \"global-pull-secret-syncer-cg5rh\" (UID: \"f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913\") " pod="kube-system/global-pull-secret-syncer-cg5rh"
Jul 21 13:44:16.982044 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:16.981898    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9780c38e-e96c-47a2-bd63-283bac4f8f59-metrics-certs\") pod \"network-metrics-daemon-lskgb\" (UID: \"9780c38e-e96c-47a2-bd63-283bac4f8f59\") " pod="openshift-multus/network-metrics-daemon-lskgb"
Jul 21 13:44:16.982044 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:16.982023    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 13:44:16.982163 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:16.982083    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9780c38e-e96c-47a2-bd63-283bac4f8f59-metrics-certs podName:9780c38e-e96c-47a2-bd63-283bac4f8f59 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:18.982066251 +0000 UTC m=+5.942354786 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9780c38e-e96c-47a2-bd63-283bac4f8f59-metrics-certs") pod "network-metrics-daemon-lskgb" (UID: "9780c38e-e96c-47a2-bd63-283bac4f8f59") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 13:44:16.982163 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:16.982141    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 13:44:16.982275 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:16.982168    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913-original-pull-secret podName:f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:18.982159602 +0000 UTC m=+5.942448149 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913-original-pull-secret") pod "global-pull-secret-syncer-cg5rh" (UID: "f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913") : object "kube-system"/"original-pull-secret" not registered
Jul 21 13:44:17.082827 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:17.082623    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqq4t\" (UniqueName: \"kubernetes.io/projected/18e5c194-5162-4442-98b5-61bfaf0b8f66-kube-api-access-rqq4t\") pod \"network-check-target-8f7c2\" (UID: \"18e5c194-5162-4442-98b5-61bfaf0b8f66\") " pod="openshift-network-diagnostics/network-check-target-8f7c2"
Jul 21 13:44:17.082827 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:17.082793    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 13:44:17.082827 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:17.082817    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 13:44:17.082827 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:17.082830    2578 projected.go:196] Error preparing data for projected volume kube-api-access-rqq4t for pod openshift-network-diagnostics/network-check-target-8f7c2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 13:44:17.083182 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:17.082912    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/18e5c194-5162-4442-98b5-61bfaf0b8f66-kube-api-access-rqq4t podName:18e5c194-5162-4442-98b5-61bfaf0b8f66 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:19.082893479 +0000 UTC m=+6.043182028 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rqq4t" (UniqueName: "kubernetes.io/projected/18e5c194-5162-4442-98b5-61bfaf0b8f66-kube-api-access-rqq4t") pod "network-check-target-8f7c2" (UID: "18e5c194-5162-4442-98b5-61bfaf0b8f66") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 13:44:17.400381 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:17.400294    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 13:39:15 +0000 UTC" deadline="2028-05-04 18:21:15.743367908 +0000 UTC"
Jul 21 13:44:17.400381 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:17.400337    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15676h36m58.343035386s"
Jul 21 13:44:17.500192 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:17.500101    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lskgb"
Jul 21 13:44:17.500702 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:17.500238    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lskgb" podUID="9780c38e-e96c-47a2-bd63-283bac4f8f59"
Jul 21 13:44:18.498652 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:18.498548    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cg5rh"
Jul 21 13:44:18.498852 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:18.498681    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cg5rh" podUID="f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913"
Jul 21 13:44:18.499306 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:18.498946    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8f7c2"
Jul 21 13:44:18.499306 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:18.499043    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8f7c2" podUID="18e5c194-5162-4442-98b5-61bfaf0b8f66"
Jul 21 13:44:19.001147 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:19.000963    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9780c38e-e96c-47a2-bd63-283bac4f8f59-metrics-certs\") pod \"network-metrics-daemon-lskgb\" (UID: \"9780c38e-e96c-47a2-bd63-283bac4f8f59\") " pod="openshift-multus/network-metrics-daemon-lskgb"
Jul 21 13:44:19.001147 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:19.001061    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913-original-pull-secret\") pod \"global-pull-secret-syncer-cg5rh\" (UID: \"f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913\") " pod="kube-system/global-pull-secret-syncer-cg5rh"
Jul 21 13:44:19.001147 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:19.001134    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 13:44:19.001703 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:19.001205    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9780c38e-e96c-47a2-bd63-283bac4f8f59-metrics-certs podName:9780c38e-e96c-47a2-bd63-283bac4f8f59 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:23.001185614 +0000 UTC m=+9.961474163 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9780c38e-e96c-47a2-bd63-283bac4f8f59-metrics-certs") pod "network-metrics-daemon-lskgb" (UID: "9780c38e-e96c-47a2-bd63-283bac4f8f59") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 13:44:19.001703 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:19.001211    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 13:44:19.001703 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:19.001273    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913-original-pull-secret podName:f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:23.001254507 +0000 UTC m=+9.961543044 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913-original-pull-secret") pod "global-pull-secret-syncer-cg5rh" (UID: "f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913") : object "kube-system"/"original-pull-secret" not registered
Jul 21 13:44:19.102144 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:19.102106    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqq4t\" (UniqueName: \"kubernetes.io/projected/18e5c194-5162-4442-98b5-61bfaf0b8f66-kube-api-access-rqq4t\") pod \"network-check-target-8f7c2\" (UID: \"18e5c194-5162-4442-98b5-61bfaf0b8f66\") " pod="openshift-network-diagnostics/network-check-target-8f7c2"
Jul 21 13:44:19.102332 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:19.102269    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 13:44:19.102332 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:19.102302    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 13:44:19.102332 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:19.102318    2578 projected.go:196] Error preparing data for projected volume kube-api-access-rqq4t for pod openshift-network-diagnostics/network-check-target-8f7c2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 13:44:19.102490 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:19.102395    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/18e5c194-5162-4442-98b5-61bfaf0b8f66-kube-api-access-rqq4t podName:18e5c194-5162-4442-98b5-61bfaf0b8f66 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:23.102367237 +0000 UTC m=+10.062655787 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rqq4t" (UniqueName: "kubernetes.io/projected/18e5c194-5162-4442-98b5-61bfaf0b8f66-kube-api-access-rqq4t") pod "network-check-target-8f7c2" (UID: "18e5c194-5162-4442-98b5-61bfaf0b8f66") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 13:44:19.498678 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:19.498574    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lskgb"
Jul 21 13:44:19.498844 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:19.498721    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lskgb" podUID="9780c38e-e96c-47a2-bd63-283bac4f8f59"
Jul 21 13:44:20.498755 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:20.498063    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8f7c2"
Jul 21 13:44:20.498755 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:20.498188    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8f7c2" podUID="18e5c194-5162-4442-98b5-61bfaf0b8f66"
Jul 21 13:44:20.498755 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:20.498608    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cg5rh"
Jul 21 13:44:20.498755 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:20.498703    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cg5rh" podUID="f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913"
Jul 21 13:44:21.498600 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:21.498565    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lskgb"
Jul 21 13:44:21.498781 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:21.498702    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lskgb" podUID="9780c38e-e96c-47a2-bd63-283bac4f8f59"
Jul 21 13:44:22.498328 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:22.498290    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8f7c2"
Jul 21 13:44:22.498473 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:22.498421    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8f7c2" podUID="18e5c194-5162-4442-98b5-61bfaf0b8f66"
Jul 21 13:44:22.498509 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:22.498478    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cg5rh"
Jul 21 13:44:22.498574 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:22.498547    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cg5rh" podUID="f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913"
Jul 21 13:44:23.035457 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:23.035428    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913-original-pull-secret\") pod \"global-pull-secret-syncer-cg5rh\" (UID: \"f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913\") " pod="kube-system/global-pull-secret-syncer-cg5rh"
Jul 21 13:44:23.035917 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:23.035485    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9780c38e-e96c-47a2-bd63-283bac4f8f59-metrics-certs\") pod \"network-metrics-daemon-lskgb\" (UID: \"9780c38e-e96c-47a2-bd63-283bac4f8f59\") " pod="openshift-multus/network-metrics-daemon-lskgb"
Jul 21 13:44:23.035917 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:23.035585    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 13:44:23.035917 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:23.035628    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9780c38e-e96c-47a2-bd63-283bac4f8f59-metrics-certs podName:9780c38e-e96c-47a2-bd63-283bac4f8f59 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:31.03561328 +0000 UTC m=+17.995901815 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9780c38e-e96c-47a2-bd63-283bac4f8f59-metrics-certs") pod "network-metrics-daemon-lskgb" (UID: "9780c38e-e96c-47a2-bd63-283bac4f8f59") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 13:44:23.035917 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:23.035688    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 13:44:23.035917 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:23.035750    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913-original-pull-secret podName:f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:31.035730256 +0000 UTC m=+17.996018791 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913-original-pull-secret") pod "global-pull-secret-syncer-cg5rh" (UID: "f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913") : object "kube-system"/"original-pull-secret" not registered
Jul 21 13:44:23.136083 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:23.136052    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqq4t\" (UniqueName: \"kubernetes.io/projected/18e5c194-5162-4442-98b5-61bfaf0b8f66-kube-api-access-rqq4t\") pod \"network-check-target-8f7c2\" (UID: \"18e5c194-5162-4442-98b5-61bfaf0b8f66\") " pod="openshift-network-diagnostics/network-check-target-8f7c2"
Jul 21 13:44:23.136253 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:23.136204    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 13:44:23.136253 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:23.136222    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 13:44:23.136253 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:23.136235    2578 projected.go:196] Error preparing data for projected volume kube-api-access-rqq4t for pod openshift-network-diagnostics/network-check-target-8f7c2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 13:44:23.136375 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:23.136298    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/18e5c194-5162-4442-98b5-61bfaf0b8f66-kube-api-access-rqq4t podName:18e5c194-5162-4442-98b5-61bfaf0b8f66 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:31.136280043 +0000 UTC m=+18.096568598 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rqq4t" (UniqueName: "kubernetes.io/projected/18e5c194-5162-4442-98b5-61bfaf0b8f66-kube-api-access-rqq4t") pod "network-check-target-8f7c2" (UID: "18e5c194-5162-4442-98b5-61bfaf0b8f66") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 13:44:23.501102 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:23.500573    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lskgb"
Jul 21 13:44:23.501102 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:23.500697    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lskgb" podUID="9780c38e-e96c-47a2-bd63-283bac4f8f59"
Jul 21 13:44:24.498564 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:24.498525    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8f7c2"
Jul 21 13:44:24.499127 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:24.498663    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8f7c2" podUID="18e5c194-5162-4442-98b5-61bfaf0b8f66"
Jul 21 13:44:24.499127 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:24.498784    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cg5rh"
Jul 21 13:44:24.499127 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:24.498899    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cg5rh" podUID="f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913"
Jul 21 13:44:25.498085 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:25.498006    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lskgb"
Jul 21 13:44:25.498276 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:25.498158    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lskgb" podUID="9780c38e-e96c-47a2-bd63-283bac4f8f59"
Jul 21 13:44:26.498309 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:26.498280    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8f7c2"
Jul 21 13:44:26.498798 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:26.498381    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8f7c2" podUID="18e5c194-5162-4442-98b5-61bfaf0b8f66"
Jul 21 13:44:26.498798 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:26.498447    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cg5rh"
Jul 21 13:44:26.498798 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:26.498545    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cg5rh" podUID="f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913"
Jul 21 13:44:27.497643 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:27.497610    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lskgb"
Jul 21 13:44:27.497813 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:27.497750    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lskgb" podUID="9780c38e-e96c-47a2-bd63-283bac4f8f59"
Jul 21 13:44:28.497796 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:28.497582    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cg5rh"
Jul 21 13:44:28.498287 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:28.497582    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8f7c2"
Jul 21 13:44:28.498287 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:28.497922    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cg5rh" podUID="f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913"
Jul 21 13:44:28.498287 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:28.497979    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8f7c2" podUID="18e5c194-5162-4442-98b5-61bfaf0b8f66"
Jul 21 13:44:29.498075 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:29.498042    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lskgb"
Jul 21 13:44:29.498550 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:29.498160    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lskgb" podUID="9780c38e-e96c-47a2-bd63-283bac4f8f59"
Jul 21 13:44:30.497965 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:30.497885    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8f7c2"
Jul 21 13:44:30.498171 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:30.497887    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cg5rh"
Jul 21 13:44:30.498171 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:30.498006    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8f7c2" podUID="18e5c194-5162-4442-98b5-61bfaf0b8f66"
Jul 21 13:44:30.498171 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:30.498072    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cg5rh" podUID="f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913"
Jul 21 13:44:31.095821 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:31.095782    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913-original-pull-secret\") pod \"global-pull-secret-syncer-cg5rh\" (UID: \"f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913\") " pod="kube-system/global-pull-secret-syncer-cg5rh"
Jul 21 13:44:31.096003 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:31.095857    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9780c38e-e96c-47a2-bd63-283bac4f8f59-metrics-certs\") pod \"network-metrics-daemon-lskgb\" (UID: \"9780c38e-e96c-47a2-bd63-283bac4f8f59\") " pod="openshift-multus/network-metrics-daemon-lskgb"
Jul 21 13:44:31.096003 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:31.095962    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 13:44:31.096003 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:31.095973    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 13:44:31.096137 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:31.096036    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9780c38e-e96c-47a2-bd63-283bac4f8f59-metrics-certs podName:9780c38e-e96c-47a2-bd63-283bac4f8f59 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:47.096019547 +0000 UTC m=+34.056308099 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9780c38e-e96c-47a2-bd63-283bac4f8f59-metrics-certs") pod "network-metrics-daemon-lskgb" (UID: "9780c38e-e96c-47a2-bd63-283bac4f8f59") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 13:44:31.096137 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:31.096055    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913-original-pull-secret podName:f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:47.096045105 +0000 UTC m=+34.056333652 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913-original-pull-secret") pod "global-pull-secret-syncer-cg5rh" (UID: "f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913") : object "kube-system"/"original-pull-secret" not registered
Jul 21 13:44:31.196619 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:31.196576    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqq4t\" (UniqueName: \"kubernetes.io/projected/18e5c194-5162-4442-98b5-61bfaf0b8f66-kube-api-access-rqq4t\") pod \"network-check-target-8f7c2\" (UID: \"18e5c194-5162-4442-98b5-61bfaf0b8f66\") " pod="openshift-network-diagnostics/network-check-target-8f7c2"
Jul 21 13:44:31.196793 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:31.196754    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 13:44:31.196793 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:31.196779    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 13:44:31.196793 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:31.196791    2578 projected.go:196] Error preparing data for projected volume kube-api-access-rqq4t for pod openshift-network-diagnostics/network-check-target-8f7c2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 13:44:31.196964 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:31.196850    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/18e5c194-5162-4442-98b5-61bfaf0b8f66-kube-api-access-rqq4t podName:18e5c194-5162-4442-98b5-61bfaf0b8f66 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:47.196837075 +0000 UTC m=+34.157125611 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rqq4t" (UniqueName: "kubernetes.io/projected/18e5c194-5162-4442-98b5-61bfaf0b8f66-kube-api-access-rqq4t") pod "network-check-target-8f7c2" (UID: "18e5c194-5162-4442-98b5-61bfaf0b8f66") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 13:44:31.497695 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:31.497621    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lskgb"
Jul 21 13:44:31.497838 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:31.497775    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lskgb" podUID="9780c38e-e96c-47a2-bd63-283bac4f8f59"
Jul 21 13:44:32.498076 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:32.498043    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8f7c2"
Jul 21 13:44:32.498547 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:32.498163    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8f7c2" podUID="18e5c194-5162-4442-98b5-61bfaf0b8f66"
Jul 21 13:44:32.498547 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:32.498229    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cg5rh"
Jul 21 13:44:32.498547 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:32.498337    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cg5rh" podUID="f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913"
Jul 21 13:44:33.499388 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:33.499357    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lskgb"
Jul 21 13:44:33.499896 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:33.499478    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lskgb" podUID="9780c38e-e96c-47a2-bd63-283bac4f8f59"
Jul 21 13:44:34.497811 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:34.497786    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8f7c2"
Jul 21 13:44:34.497943 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:34.497900    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cg5rh"
Jul 21 13:44:34.497943 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:34.497904    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8f7c2" podUID="18e5c194-5162-4442-98b5-61bfaf0b8f66"
Jul 21 13:44:34.498022 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:34.497984    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cg5rh" podUID="f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913"
Jul 21 13:44:34.562844 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:34.562762    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-245.ec2.internal" event={"ID":"de55c8b128a47fa32f279f57a92fe691","Type":"ContainerStarted","Data":"479bb2b056d0dd01278acb78793afb8c505ab01c3babdece4f035e111b0f0b56"}
Jul 21 13:44:34.579095 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:34.579038    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-245.ec2.internal" podStartSLOduration=20.579021852 podStartE2EDuration="20.579021852s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 13:44:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:44:34.578472857 +0000 UTC m=+21.538761413" watchObservedRunningTime="2026-07-21 13:44:34.579021852 +0000 UTC m=+21.539310428"
Jul 21 13:44:35.498702 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:35.498533    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lskgb"
Jul 21 13:44:35.498815 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:35.498784    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lskgb" podUID="9780c38e-e96c-47a2-bd63-283bac4f8f59"
Jul 21 13:44:35.565954 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:35.565918    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hprzk" event={"ID":"86868b9f-3bee-44f9-b662-ed05ab2b323a","Type":"ContainerStarted","Data":"b4e0a7ba8c69da8d18b3b58ae918cabf349ab318e2cb71d7d2bfc4756dae2dc9"}
Jul 21 13:44:35.567246 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:35.567225    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-q5mrj" event={"ID":"bbaf7683-e99d-4fe9-9ebf-07504bb66729","Type":"ContainerStarted","Data":"136ef44e859c4bf80722334b3ef300ac558d13bad5d5bdaefd03a96afcc5e9a3"}
Jul 21 13:44:35.568510 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:35.568478    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mnstv" event={"ID":"b63df369-72fd-4d11-805c-5204b768db57","Type":"ContainerStarted","Data":"86efeb479b2b58271897c467175b67704f80ea6b1b2164f852db7ed0fee3b978"}
Jul 21 13:44:35.569689 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:35.569671    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kk2pk" event={"ID":"7905842a-33f8-4cfb-941a-371ced12fead","Type":"ContainerStarted","Data":"6e6b7697b0a712612dc6358b6e1a214e392317a00b9839dd35f721d3d3d5eee0"}
Jul 21 13:44:35.570987 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:35.570969    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5d6xt" event={"ID":"ad6d2b51-331e-464a-8ec5-f518a487bf24","Type":"ContainerStarted","Data":"f56aea8d8fe42b5fc41cfb226e5396f11ba501a7f73b48637e0bd01631fb60fb"}
Jul 21 13:44:35.572151 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:35.572127    2578 generic.go:356] "Generic (PLEG): container finished" podID="a28dba002f2c85352dc44bcb9751469c" containerID="1f46029f3a1b77f1ce1ef16e641e57670199c38d0ccd82d3368c9a5e5cb8828b" exitCode=0
Jul 21 13:44:35.572220 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:35.572184    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal" event={"ID":"a28dba002f2c85352dc44bcb9751469c","Type":"ContainerDied","Data":"1f46029f3a1b77f1ce1ef16e641e57670199c38d0ccd82d3368c9a5e5cb8828b"}
Jul 21 13:44:35.573824 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:35.573804    2578 generic.go:356] "Generic (PLEG): container finished" podID="3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9" containerID="a765950b6ba7fd8b6fb3faa6d391270866a355ebd9de0c26e2bc8ef436b01635" exitCode=0
Jul 21 13:44:35.573937 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:35.573866    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6kwdx" event={"ID":"3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9","Type":"ContainerDied","Data":"a765950b6ba7fd8b6fb3faa6d391270866a355ebd9de0c26e2bc8ef436b01635"}
Jul 21 13:44:35.576345 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:35.576324    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b" event={"ID":"629e5be3-662a-4713-82fb-3c7892da2245","Type":"ContainerStarted","Data":"d71fd8a125238add0d34c30d701b062345a2d09fb444914ec7c0537cb9134e45"}
Jul 21 13:44:35.576434 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:35.576399    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b" event={"ID":"629e5be3-662a-4713-82fb-3c7892da2245","Type":"ContainerStarted","Data":"dbb228b821358736dcff49b061c09275112fe2d1014c36a71bb5bb5d07e3b1e1"}
Jul 21 13:44:35.576551 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:35.576450    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b" event={"ID":"629e5be3-662a-4713-82fb-3c7892da2245","Type":"ContainerStarted","Data":"d1ca386ec5f939cd8135561a626dad1f51297e3c292976c3f3d364b529d46627"}
Jul 21 13:44:35.576551 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:35.576501    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b" event={"ID":"629e5be3-662a-4713-82fb-3c7892da2245","Type":"ContainerStarted","Data":"ea99f650a71693478482a102823b82f0604466216611866f0749e85eb0eef5b0"}
Jul 21 13:44:35.577638 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:35.577619    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-swmgt" event={"ID":"30032785-7cc9-421c-b65a-674a329a954f","Type":"ContainerStarted","Data":"fddd03665d9b19b6268f72ed0bfa6665f321f715d191390115b58e1b574cd532"}
Jul 21 13:44:35.603910 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:35.603855    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-hprzk" podStartSLOduration=12.361445727 podStartE2EDuration="22.603841216s" totalImagesPullingTime="10.242395489s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 13:44:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:44:35.585837064 +0000 UTC m=+22.546125633" watchObservedRunningTime="2026-07-21 13:44:35.603841216 +0000 UTC m=+22.564129773"
Jul 21 13:44:35.632398 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:35.632342    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-mnstv" podStartSLOduration=12.42511658 podStartE2EDuration="22.632321196s" totalImagesPullingTime="10.207204616s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:44:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:44:35.60369608 +0000 UTC m=+22.563984636" watchObservedRunningTime="2026-07-21 13:44:35.632321196 +0000 UTC m=+22.592609754"
Jul 21 13:44:35.680001 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:35.679961    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-q5mrj" podStartSLOduration=4.053982188 podStartE2EDuration="22.679944281s" totalImagesPullingTime="18.625962093s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 13:44:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:44:35.652827228 +0000 UTC m=+22.613115784" watchObservedRunningTime="2026-07-21 13:44:35.679944281 +0000 UTC m=+22.640232816"
Jul 21 13:44:35.697017 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:35.696971    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-swmgt" podStartSLOduration=12.470941168 podStartE2EDuration="22.696950728s" totalImagesPullingTime="10.22600956s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 13:44:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:44:35.696361929 +0000 UTC m=+22.656650514" watchObservedRunningTime="2026-07-21 13:44:35.696950728 +0000 UTC m=+22.657239285"
Jul 21 13:44:35.717301 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:35.717248    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-kk2pk" podStartSLOduration=4.081959365 podStartE2EDuration="22.717233563s" totalImagesPullingTime="18.635274198s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 13:44:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:44:35.716781533 +0000 UTC m=+22.677070089" watchObservedRunningTime="2026-07-21 13:44:35.717233563 +0000 UTC m=+22.677522119"
Jul 21 13:44:36.497854 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:36.497821    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cg5rh"
Jul 21 13:44:36.498026 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:36.497967    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cg5rh" podUID="f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913"
Jul 21 13:44:36.498412 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:36.498347    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8f7c2"
Jul 21 13:44:36.498502 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:36.498434    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8f7c2" podUID="18e5c194-5162-4442-98b5-61bfaf0b8f66"
Jul 21 13:44:36.566386 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:36.566360    2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 21 13:44:36.581293 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:36.581246    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6w5sm" event={"ID":"f8473ef3-0b52-479b-b042-cecf34a517c7","Type":"ContainerStarted","Data":"6bff302bf0fe0056a6d94533c837a17efc039f46eee5cb0ab68966dc5e8a0874"}
Jul 21 13:44:36.583369 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:36.583339    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5d6xt" event={"ID":"ad6d2b51-331e-464a-8ec5-f518a487bf24","Type":"ContainerStarted","Data":"04c48e61874c4b4932e84852a52d3efa50dffb5cfc9cfe29fff89a4732951ca2"}
Jul 21 13:44:36.586038 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:36.586011    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal" event={"ID":"a28dba002f2c85352dc44bcb9751469c","Type":"ContainerStarted","Data":"86b0a3ffb673cbef8490d9241aa45086001016f49e023d6a1983152e3d0fe144"}
Jul 21 13:44:36.589326 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:36.589301    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b" event={"ID":"629e5be3-662a-4713-82fb-3c7892da2245","Type":"ContainerStarted","Data":"55a20d831dc0d97f69d0e4f9d0d0bc78e92bb4b5adaee59433c28f9b2c1adbe5"}
Jul 21 13:44:36.589429 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:36.589378    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b" event={"ID":"629e5be3-662a-4713-82fb-3c7892da2245","Type":"ContainerStarted","Data":"d80214003b0d9fec650a8d037314bf2453ff0e70aa74deb3de1628e16b12ae50"}
Jul 21 13:44:36.595703 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:36.595662    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-6w5sm" podStartSLOduration=5.040011909 podStartE2EDuration="23.595647729s" totalImagesPullingTime="18.55563582s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 13:44:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:44:36.595412376 +0000 UTC m=+23.555700933" watchObservedRunningTime="2026-07-21 13:44:36.595647729 +0000 UTC m=+23.555936289"
Jul 21 13:44:36.856433 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:36.856349    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-mnstv"
Jul 21 13:44:36.857068 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:36.857038    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-mnstv"
Jul 21 13:44:36.876561 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:36.876485    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-245.ec2.internal" podStartSLOduration=22.876465007 podStartE2EDuration="22.876465007s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 13:44:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:44:36.61112423 +0000 UTC m=+23.571412790" watchObservedRunningTime="2026-07-21 13:44:36.876465007 +0000 UTC m=+23.836753565"
Jul 21 13:44:37.436217 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:37.436075    2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-21T13:44:36.566381262Z","UUID":"d0a4cec8-3f92-4b42-a1c8-b684d525a66b","Handler":null,"Name":"","Endpoint":""}
Jul 21 13:44:37.437854 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:37.437817    2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 21 13:44:37.437854 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:37.437859    2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 21 13:44:37.498205 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:37.498168    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lskgb"
Jul 21 13:44:37.498383 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:37.498309    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lskgb" podUID="9780c38e-e96c-47a2-bd63-283bac4f8f59"
Jul 21 13:44:37.592454 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:37.592429    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-mnstv"
Jul 21 13:44:37.593169 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:37.593136    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-mnstv"
Jul 21 13:44:38.497834 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:38.497801    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cg5rh"
Jul 21 13:44:38.498030 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:38.497949    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cg5rh" podUID="f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913"
Jul 21 13:44:38.498030 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:38.498011    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8f7c2"
Jul 21 13:44:38.498146 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:38.498119    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8f7c2" podUID="18e5c194-5162-4442-98b5-61bfaf0b8f66"
Jul 21 13:44:38.596530 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:38.596487    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5d6xt" event={"ID":"ad6d2b51-331e-464a-8ec5-f518a487bf24","Type":"ContainerStarted","Data":"6247a10338664b690bf7eba8f7e95ac174bf5fa077c9fc7f15d212634351d7ac"}
Jul 21 13:44:38.600293 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:38.600253    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b" event={"ID":"629e5be3-662a-4713-82fb-3c7892da2245","Type":"ContainerStarted","Data":"0dc09fc30b86d8852d90e03260f0ec94b3ac34fe151c457eb3f71fadd37b6c6e"}
Jul 21 13:44:39.498375 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:39.498340    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lskgb"
Jul 21 13:44:39.498572 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:39.498472    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lskgb" podUID="9780c38e-e96c-47a2-bd63-283bac4f8f59"
Jul 21 13:44:40.498509 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:40.498329    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8f7c2"
Jul 21 13:44:40.499134 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:40.498381    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cg5rh"
Jul 21 13:44:40.499134 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:40.498579    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8f7c2" podUID="18e5c194-5162-4442-98b5-61bfaf0b8f66"
Jul 21 13:44:40.499134 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:40.498658    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cg5rh" podUID="f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913"
Jul 21 13:44:40.606725 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:40.606688    2578 generic.go:356] "Generic (PLEG): container finished" podID="3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9" containerID="c63e48c3ab83ad2803e03a6a5a28d4f3cf997e44ed25f93f6fc994de1d3b4127" exitCode=0
Jul 21 13:44:40.606898 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:40.606765    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6kwdx" event={"ID":"3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9","Type":"ContainerDied","Data":"c63e48c3ab83ad2803e03a6a5a28d4f3cf997e44ed25f93f6fc994de1d3b4127"}
Jul 21 13:44:40.610514 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:40.610492    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b" event={"ID":"629e5be3-662a-4713-82fb-3c7892da2245","Type":"ContainerStarted","Data":"a6ef1f8981175d6a00fbb2580b84a02fb0515ebb14912b29be0e56b59d441956"}
Jul 21 13:44:40.610808 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:40.610789    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:40.610900 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:40.610864    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:40.610959 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:40.610927    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:40.625812 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:40.625790    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:40.625943 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:40.625908    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:44:40.642504 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:40.642458    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5d6xt" podStartSLOduration=11.344348758 podStartE2EDuration="27.642444038s" totalImagesPullingTime="16.29809528s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 13:44:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:44:38.620277393 +0000 UTC m=+25.580565950" watchObservedRunningTime="2026-07-21 13:44:40.642444038 +0000 UTC m=+27.602732594"
Jul 21 13:44:40.674936 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:40.674867    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b" podStartSLOduration=8.628156390000001 podStartE2EDuration="27.674852584s" totalImagesPullingTime="19.046696194s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 13:44:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:44:40.674689694 +0000 UTC m=+27.634978279" watchObservedRunningTime="2026-07-21 13:44:40.674852584 +0000 UTC m=+27.635141140"
Jul 21 13:44:41.498097 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:41.497855    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lskgb"
Jul 21 13:44:41.498269 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:41.498146    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lskgb" podUID="9780c38e-e96c-47a2-bd63-283bac4f8f59"
Jul 21 13:44:42.419314 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:42.419228    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cg5rh"]
Jul 21 13:44:42.419683 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:42.419363    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cg5rh"
Jul 21 13:44:42.419683 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:42.419446    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cg5rh" podUID="f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913"
Jul 21 13:44:42.422689 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:42.422489    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lskgb"]
Jul 21 13:44:42.422689 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:42.422600    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lskgb"
Jul 21 13:44:42.422948 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:42.422696    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lskgb" podUID="9780c38e-e96c-47a2-bd63-283bac4f8f59"
Jul 21 13:44:42.423294 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:42.423272    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8f7c2"]
Jul 21 13:44:42.423404 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:42.423371    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8f7c2"
Jul 21 13:44:42.423491 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:42.423459    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8f7c2" podUID="18e5c194-5162-4442-98b5-61bfaf0b8f66"
Jul 21 13:44:42.617382 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:42.617350    2578 generic.go:356] "Generic (PLEG): container finished" podID="3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9" containerID="9d0c6bf2505b174ab2c41019307e27f970c89c7a80c8b41c46c654f5f4c4e8a1" exitCode=0
Jul 21 13:44:42.617562 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:42.617434    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6kwdx" event={"ID":"3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9","Type":"ContainerDied","Data":"9d0c6bf2505b174ab2c41019307e27f970c89c7a80c8b41c46c654f5f4c4e8a1"}
Jul 21 13:44:43.622448 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:43.622368    2578 generic.go:356] "Generic (PLEG): container finished" podID="3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9" containerID="6808a40504e2bc6ae01600f7b44d77fdc75172237d1e70b13cea99feefa6a831" exitCode=0
Jul 21 13:44:43.622448 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:43.622433    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6kwdx" event={"ID":"3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9","Type":"ContainerDied","Data":"6808a40504e2bc6ae01600f7b44d77fdc75172237d1e70b13cea99feefa6a831"}
Jul 21 13:44:44.497763 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:44.497728    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8f7c2"
Jul 21 13:44:44.497963 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:44.497848    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8f7c2" podUID="18e5c194-5162-4442-98b5-61bfaf0b8f66"
Jul 21 13:44:44.497963 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:44.497871    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cg5rh"
Jul 21 13:44:44.498044 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:44.497993    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cg5rh" podUID="f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913"
Jul 21 13:44:44.498044 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:44.498019    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lskgb"
Jul 21 13:44:44.498105 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:44.498087    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lskgb" podUID="9780c38e-e96c-47a2-bd63-283bac4f8f59"
Jul 21 13:44:46.498006 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:46.497923    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lskgb"
Jul 21 13:44:46.498467 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:46.497924    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8f7c2"
Jul 21 13:44:46.498467 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:46.498044    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lskgb" podUID="9780c38e-e96c-47a2-bd63-283bac4f8f59"
Jul 21 13:44:46.498467 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:46.498127    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8f7c2" podUID="18e5c194-5162-4442-98b5-61bfaf0b8f66"
Jul 21 13:44:46.498467 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:46.497924    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cg5rh"
Jul 21 13:44:46.498467 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:46.498233    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cg5rh" podUID="f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913"
Jul 21 13:44:47.116170 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:47.115982    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913-original-pull-secret\") pod \"global-pull-secret-syncer-cg5rh\" (UID: \"f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913\") " pod="kube-system/global-pull-secret-syncer-cg5rh"
Jul 21 13:44:47.116170 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:47.116157    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 13:44:47.116408 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:47.116223    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9780c38e-e96c-47a2-bd63-283bac4f8f59-metrics-certs\") pod \"network-metrics-daemon-lskgb\" (UID: \"9780c38e-e96c-47a2-bd63-283bac4f8f59\") " pod="openshift-multus/network-metrics-daemon-lskgb"
Jul 21 13:44:47.116408 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:47.116241    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913-original-pull-secret podName:f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913 nodeName:}" failed. No retries permitted until 2026-07-21 13:45:19.116218506 +0000 UTC m=+66.076507054 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913-original-pull-secret") pod "global-pull-secret-syncer-cg5rh" (UID: "f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913") : object "kube-system"/"original-pull-secret" not registered
Jul 21 13:44:47.116408 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:47.116308    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 13:44:47.116408 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:47.116359    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9780c38e-e96c-47a2-bd63-283bac4f8f59-metrics-certs podName:9780c38e-e96c-47a2-bd63-283bac4f8f59 nodeName:}" failed. No retries permitted until 2026-07-21 13:45:19.116343435 +0000 UTC m=+66.076631984 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9780c38e-e96c-47a2-bd63-283bac4f8f59-metrics-certs") pod "network-metrics-daemon-lskgb" (UID: "9780c38e-e96c-47a2-bd63-283bac4f8f59") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 13:44:47.217063 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:47.217014    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqq4t\" (UniqueName: \"kubernetes.io/projected/18e5c194-5162-4442-98b5-61bfaf0b8f66-kube-api-access-rqq4t\") pod \"network-check-target-8f7c2\" (UID: \"18e5c194-5162-4442-98b5-61bfaf0b8f66\") " pod="openshift-network-diagnostics/network-check-target-8f7c2"
Jul 21 13:44:47.217229 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:47.217198    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 13:44:47.217229 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:47.217215    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 13:44:47.217229 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:47.217223    2578 projected.go:196] Error preparing data for projected volume kube-api-access-rqq4t for pod openshift-network-diagnostics/network-check-target-8f7c2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 13:44:47.217359 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:47.217271    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/18e5c194-5162-4442-98b5-61bfaf0b8f66-kube-api-access-rqq4t podName:18e5c194-5162-4442-98b5-61bfaf0b8f66 nodeName:}" failed. No retries permitted until 2026-07-21 13:45:19.217258709 +0000 UTC m=+66.177547258 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-rqq4t" (UniqueName: "kubernetes.io/projected/18e5c194-5162-4442-98b5-61bfaf0b8f66-kube-api-access-rqq4t") pod "network-check-target-8f7c2" (UID: "18e5c194-5162-4442-98b5-61bfaf0b8f66") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 13:44:47.291038 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:47.291012    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hprzk_86868b9f-3bee-44f9-b662-ed05ab2b323a/dns-node-resolver/0.log"
Jul 21 13:44:48.254760 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.254736    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-swmgt_30032785-7cc9-421c-b65a-674a329a954f/node-ca/0.log"
Jul 21 13:44:48.352096 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.352070    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-245.ec2.internal" event="NodeReady"
Jul 21 13:44:48.352238 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.352227    2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 21 13:44:48.391468 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.391442    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5f7fcdffb9-tfpcs"]
Jul 21 13:44:48.434485 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.434452    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-gxpzz"]
Jul 21 13:44:48.434593 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.434572    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f7fcdffb9-tfpcs"
Jul 21 13:44:48.438039 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.438001    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 21 13:44:48.438168 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.438047    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 21 13:44:48.438168 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.438095    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-mm57m\""
Jul 21 13:44:48.438339 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.438318    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 21 13:44:48.442557 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.442527    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 21 13:44:48.454687 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.454649    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-s2n8n"]
Jul 21 13:44:48.454806 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.454782    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gxpzz"
Jul 21 13:44:48.457645 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.457618    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 21 13:44:48.457946 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.457923    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 21 13:44:48.458062 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.457956    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 21 13:44:48.458304 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.458288    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-dghlb\""
Jul 21 13:44:48.478855 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.478821    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5f7fcdffb9-tfpcs"]
Jul 21 13:44:48.478855 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.478853    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gxpzz"]
Jul 21 13:44:48.479037 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.478867    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-s2n8n"]
Jul 21 13:44:48.479037 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.478905    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vvrf8"]
Jul 21 13:44:48.479096 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.479036    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-s2n8n"
Jul 21 13:44:48.481558 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.481539    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 21 13:44:48.481678 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.481664    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 21 13:44:48.481754 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.481741    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 21 13:44:48.481833 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.481813    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-s5m7b\""
Jul 21 13:44:48.481975 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.481960    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 21 13:44:48.498936 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.498804    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vvrf8"]
Jul 21 13:44:48.498936 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.498904    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lskgb"
Jul 21 13:44:48.498936 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.498907    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cg5rh"
Jul 21 13:44:48.498936 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.498908    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8f7c2"
Jul 21 13:44:48.498936 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.498908    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vvrf8"
Jul 21 13:44:48.507663 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.507641    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 21 13:44:48.507663 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.507655    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 21 13:44:48.507916 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.507898    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 21 13:44:48.507987 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.507956    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 21 13:44:48.508177 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.508159    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pt8sb\""
Jul 21 13:44:48.508294 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.508219    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 21 13:44:48.508294 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.508254    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-29gkd\""
Jul 21 13:44:48.508442 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.508329    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 21 13:44:48.508829 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.508813    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2pd27\""
Jul 21 13:44:48.530003 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.529980    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/51aea150-df52-44c2-a4f2-1f03c268f070-ca-trust-extracted\") pod \"image-registry-5f7fcdffb9-tfpcs\" (UID: \"51aea150-df52-44c2-a4f2-1f03c268f070\") " pod="openshift-image-registry/image-registry-5f7fcdffb9-tfpcs"
Jul 21 13:44:48.530100 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.530021    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/51aea150-df52-44c2-a4f2-1f03c268f070-trusted-ca\") pod \"image-registry-5f7fcdffb9-tfpcs\" (UID: \"51aea150-df52-44c2-a4f2-1f03c268f070\") " pod="openshift-image-registry/image-registry-5f7fcdffb9-tfpcs"
Jul 21 13:44:48.530100 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.530050    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/51aea150-df52-44c2-a4f2-1f03c268f070-image-registry-private-configuration\") pod \"image-registry-5f7fcdffb9-tfpcs\" (UID: \"51aea150-df52-44c2-a4f2-1f03c268f070\") " pod="openshift-image-registry/image-registry-5f7fcdffb9-tfpcs"
Jul 21 13:44:48.530100 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.530080    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/51aea150-df52-44c2-a4f2-1f03c268f070-installation-pull-secrets\") pod \"image-registry-5f7fcdffb9-tfpcs\" (UID: \"51aea150-df52-44c2-a4f2-1f03c268f070\") " pod="openshift-image-registry/image-registry-5f7fcdffb9-tfpcs"
Jul 21 13:44:48.530217 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.530107    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51aea150-df52-44c2-a4f2-1f03c268f070-registry-tls\") pod \"image-registry-5f7fcdffb9-tfpcs\" (UID: \"51aea150-df52-44c2-a4f2-1f03c268f070\") " pod="openshift-image-registry/image-registry-5f7fcdffb9-tfpcs"
Jul 21 13:44:48.530217 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.530135    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkzhr\" (UniqueName: \"kubernetes.io/projected/51aea150-df52-44c2-a4f2-1f03c268f070-kube-api-access-dkzhr\") pod \"image-registry-5f7fcdffb9-tfpcs\" (UID: \"51aea150-df52-44c2-a4f2-1f03c268f070\") " pod="openshift-image-registry/image-registry-5f7fcdffb9-tfpcs"
Jul 21 13:44:48.530289 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.530240    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/51aea150-df52-44c2-a4f2-1f03c268f070-registry-certificates\") pod \"image-registry-5f7fcdffb9-tfpcs\" (UID: \"51aea150-df52-44c2-a4f2-1f03c268f070\") " pod="openshift-image-registry/image-registry-5f7fcdffb9-tfpcs"
Jul 21 13:44:48.530289 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.530268    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51aea150-df52-44c2-a4f2-1f03c268f070-bound-sa-token\") pod \"image-registry-5f7fcdffb9-tfpcs\" (UID: \"51aea150-df52-44c2-a4f2-1f03c268f070\") " pod="openshift-image-registry/image-registry-5f7fcdffb9-tfpcs"
Jul 21 13:44:48.630672 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.630630    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a700e703-22e3-4a59-b70c-c2a489a2a828-data-volume\") pod \"insights-runtime-extractor-s2n8n\" (UID: \"a700e703-22e3-4a59-b70c-c2a489a2a828\") " pod="openshift-insights/insights-runtime-extractor-s2n8n"
Jul 21 13:44:48.630905 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.630683    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6wtg\" (UniqueName: \"kubernetes.io/projected/a700e703-22e3-4a59-b70c-c2a489a2a828-kube-api-access-v6wtg\") pod \"insights-runtime-extractor-s2n8n\" (UID: \"a700e703-22e3-4a59-b70c-c2a489a2a828\") " pod="openshift-insights/insights-runtime-extractor-s2n8n"
Jul 21 13:44:48.630905 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.630714    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/51aea150-df52-44c2-a4f2-1f03c268f070-registry-certificates\") pod \"image-registry-5f7fcdffb9-tfpcs\" (UID: \"51aea150-df52-44c2-a4f2-1f03c268f070\") " pod="openshift-image-registry/image-registry-5f7fcdffb9-tfpcs"
Jul 21 13:44:48.630905 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.630730    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51aea150-df52-44c2-a4f2-1f03c268f070-bound-sa-token\") pod \"image-registry-5f7fcdffb9-tfpcs\" (UID: \"51aea150-df52-44c2-a4f2-1f03c268f070\") " pod="openshift-image-registry/image-registry-5f7fcdffb9-tfpcs"
Jul 21 13:44:48.630905 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.630750    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2b88\" (UniqueName: \"kubernetes.io/projected/79957af5-da1d-4787-9965-1b536cb01bca-kube-api-access-g2b88\") pod \"dns-default-vvrf8\" (UID: \"79957af5-da1d-4787-9965-1b536cb01bca\") " pod="openshift-dns/dns-default-vvrf8"
Jul 21 13:44:48.630905 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.630766    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/79957af5-da1d-4787-9965-1b536cb01bca-metrics-tls\") pod \"dns-default-vvrf8\" (UID: \"79957af5-da1d-4787-9965-1b536cb01bca\") " pod="openshift-dns/dns-default-vvrf8"
Jul 21 13:44:48.630905 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.630789    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkgr5\" (UniqueName: \"kubernetes.io/projected/6dfb88f6-dd03-45f2-96cf-3adc22acbd81-kube-api-access-kkgr5\") pod \"ingress-canary-gxpzz\" (UID: \"6dfb88f6-dd03-45f2-96cf-3adc22acbd81\") " pod="openshift-ingress-canary/ingress-canary-gxpzz"
Jul 21 13:44:48.630905 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.630859    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a700e703-22e3-4a59-b70c-c2a489a2a828-crio-socket\") pod \"insights-runtime-extractor-s2n8n\" (UID: \"a700e703-22e3-4a59-b70c-c2a489a2a828\") " pod="openshift-insights/insights-runtime-extractor-s2n8n"
Jul 21 13:44:48.631259 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.630916    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/79957af5-da1d-4787-9965-1b536cb01bca-config-volume\") pod \"dns-default-vvrf8\" (UID: \"79957af5-da1d-4787-9965-1b536cb01bca\") " pod="openshift-dns/dns-default-vvrf8"
Jul 21 13:44:48.631259 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.630937    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6dfb88f6-dd03-45f2-96cf-3adc22acbd81-cert\") pod \"ingress-canary-gxpzz\" (UID: \"6dfb88f6-dd03-45f2-96cf-3adc22acbd81\") " pod="openshift-ingress-canary/ingress-canary-gxpzz"
Jul 21 13:44:48.631259 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.630959    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/51aea150-df52-44c2-a4f2-1f03c268f070-ca-trust-extracted\") pod \"image-registry-5f7fcdffb9-tfpcs\" (UID: \"51aea150-df52-44c2-a4f2-1f03c268f070\") " pod="openshift-image-registry/image-registry-5f7fcdffb9-tfpcs"
Jul 21 13:44:48.631259 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.630974    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79957af5-da1d-4787-9965-1b536cb01bca-tmp-dir\") pod \"dns-default-vvrf8\" (UID: \"79957af5-da1d-4787-9965-1b536cb01bca\") " pod="openshift-dns/dns-default-vvrf8"
Jul 21 13:44:48.631259 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.631037    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/51aea150-df52-44c2-a4f2-1f03c268f070-trusted-ca\") pod \"image-registry-5f7fcdffb9-tfpcs\" (UID: \"51aea150-df52-44c2-a4f2-1f03c268f070\") " pod="openshift-image-registry/image-registry-5f7fcdffb9-tfpcs"
Jul 21 13:44:48.631259 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.631076    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/51aea150-df52-44c2-a4f2-1f03c268f070-image-registry-private-configuration\") pod \"image-registry-5f7fcdffb9-tfpcs\" (UID: \"51aea150-df52-44c2-a4f2-1f03c268f070\") " pod="openshift-image-registry/image-registry-5f7fcdffb9-tfpcs"
Jul 21 13:44:48.631259 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.631103    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/51aea150-df52-44c2-a4f2-1f03c268f070-installation-pull-secrets\") pod \"image-registry-5f7fcdffb9-tfpcs\" (UID: \"51aea150-df52-44c2-a4f2-1f03c268f070\") " pod="openshift-image-registry/image-registry-5f7fcdffb9-tfpcs"
Jul 21 13:44:48.631259 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.631130    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a700e703-22e3-4a59-b70c-c2a489a2a828-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s2n8n\" (UID: \"a700e703-22e3-4a59-b70c-c2a489a2a828\") " pod="openshift-insights/insights-runtime-extractor-s2n8n"
Jul 21 13:44:48.631259 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.631160    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51aea150-df52-44c2-a4f2-1f03c268f070-registry-tls\") pod \"image-registry-5f7fcdffb9-tfpcs\" (UID: \"51aea150-df52-44c2-a4f2-1f03c268f070\") " pod="openshift-image-registry/image-registry-5f7fcdffb9-tfpcs"
Jul 21 13:44:48.631259 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.631181    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a700e703-22e3-4a59-b70c-c2a489a2a828-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s2n8n\" (UID: \"a700e703-22e3-4a59-b70c-c2a489a2a828\") " pod="openshift-insights/insights-runtime-extractor-s2n8n"
Jul 21 13:44:48.631259 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.631207    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkzhr\" (UniqueName: \"kubernetes.io/projected/51aea150-df52-44c2-a4f2-1f03c268f070-kube-api-access-dkzhr\") pod \"image-registry-5f7fcdffb9-tfpcs\" (UID: \"51aea150-df52-44c2-a4f2-1f03c268f070\") " pod="openshift-image-registry/image-registry-5f7fcdffb9-tfpcs"
Jul 21 13:44:48.631729 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.631362    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/51aea150-df52-44c2-a4f2-1f03c268f070-ca-trust-extracted\") pod \"image-registry-5f7fcdffb9-tfpcs\" (UID: \"51aea150-df52-44c2-a4f2-1f03c268f070\") " pod="openshift-image-registry/image-registry-5f7fcdffb9-tfpcs"
Jul 21 13:44:48.631729 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.631432    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/51aea150-df52-44c2-a4f2-1f03c268f070-registry-certificates\") pod \"image-registry-5f7fcdffb9-tfpcs\" (UID: \"51aea150-df52-44c2-a4f2-1f03c268f070\") " pod="openshift-image-registry/image-registry-5f7fcdffb9-tfpcs"
Jul 21 13:44:48.631729 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:48.631460    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 21 13:44:48.631729 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:48.631474    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f7fcdffb9-tfpcs: secret "image-registry-tls" not found
Jul 21 13:44:48.631729 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:48.631538    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51aea150-df52-44c2-a4f2-1f03c268f070-registry-tls podName:51aea150-df52-44c2-a4f2-1f03c268f070 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:49.131514345 +0000 UTC m=+36.091802885 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/51aea150-df52-44c2-a4f2-1f03c268f070-registry-tls") pod "image-registry-5f7fcdffb9-tfpcs" (UID: "51aea150-df52-44c2-a4f2-1f03c268f070") : secret "image-registry-tls" not found
Jul 21 13:44:48.631984 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.631965    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/51aea150-df52-44c2-a4f2-1f03c268f070-trusted-ca\") pod \"image-registry-5f7fcdffb9-tfpcs\" (UID: \"51aea150-df52-44c2-a4f2-1f03c268f070\") " pod="openshift-image-registry/image-registry-5f7fcdffb9-tfpcs"
Jul 21 13:44:48.635520 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.635493    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/51aea150-df52-44c2-a4f2-1f03c268f070-installation-pull-secrets\") pod \"image-registry-5f7fcdffb9-tfpcs\" (UID: \"51aea150-df52-44c2-a4f2-1f03c268f070\") " pod="openshift-image-registry/image-registry-5f7fcdffb9-tfpcs"
Jul 21 13:44:48.635638 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.635526    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/51aea150-df52-44c2-a4f2-1f03c268f070-image-registry-private-configuration\") pod \"image-registry-5f7fcdffb9-tfpcs\" (UID: \"51aea150-df52-44c2-a4f2-1f03c268f070\") " pod="openshift-image-registry/image-registry-5f7fcdffb9-tfpcs"
Jul 21 13:44:48.637167 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.637141    2578 generic.go:356] "Generic (PLEG): container finished" podID="3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9" containerID="49be3a6fdd62d8a84ca97bc6128e5699da6ac269c7e0ad9caec9c5b87caff68d" exitCode=0
Jul 21 13:44:48.637255 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.637196    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6kwdx" event={"ID":"3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9","Type":"ContainerDied","Data":"49be3a6fdd62d8a84ca97bc6128e5699da6ac269c7e0ad9caec9c5b87caff68d"}
Jul 21 13:44:48.641473 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.641455    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51aea150-df52-44c2-a4f2-1f03c268f070-bound-sa-token\") pod \"image-registry-5f7fcdffb9-tfpcs\" (UID: \"51aea150-df52-44c2-a4f2-1f03c268f070\") " pod="openshift-image-registry/image-registry-5f7fcdffb9-tfpcs"
Jul 21 13:44:48.641594 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.641578    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkzhr\" (UniqueName: \"kubernetes.io/projected/51aea150-df52-44c2-a4f2-1f03c268f070-kube-api-access-dkzhr\") pod \"image-registry-5f7fcdffb9-tfpcs\" (UID: \"51aea150-df52-44c2-a4f2-1f03c268f070\") " pod="openshift-image-registry/image-registry-5f7fcdffb9-tfpcs"
Jul 21 13:44:48.731677 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.731645    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a700e703-22e3-4a59-b70c-c2a489a2a828-crio-socket\") pod \"insights-runtime-extractor-s2n8n\" (UID: \"a700e703-22e3-4a59-b70c-c2a489a2a828\") " pod="openshift-insights/insights-runtime-extractor-s2n8n"
Jul 21 13:44:48.731677 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.731679    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/79957af5-da1d-4787-9965-1b536cb01bca-config-volume\") pod \"dns-default-vvrf8\" (UID: \"79957af5-da1d-4787-9965-1b536cb01bca\") " pod="openshift-dns/dns-default-vvrf8"
Jul 21 13:44:48.731901 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.731696    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6dfb88f6-dd03-45f2-96cf-3adc22acbd81-cert\") pod \"ingress-canary-gxpzz\" (UID: \"6dfb88f6-dd03-45f2-96cf-3adc22acbd81\") " pod="openshift-ingress-canary/ingress-canary-gxpzz"
Jul 21 13:44:48.731901 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.731719    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79957af5-da1d-4787-9965-1b536cb01bca-tmp-dir\") pod \"dns-default-vvrf8\" (UID: \"79957af5-da1d-4787-9965-1b536cb01bca\") " pod="openshift-dns/dns-default-vvrf8"
Jul 21 13:44:48.731901 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.731754    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a700e703-22e3-4a59-b70c-c2a489a2a828-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s2n8n\" (UID: \"a700e703-22e3-4a59-b70c-c2a489a2a828\") " pod="openshift-insights/insights-runtime-extractor-s2n8n"
Jul 21 13:44:48.731901 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.731793    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a700e703-22e3-4a59-b70c-c2a489a2a828-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s2n8n\" (UID: \"a700e703-22e3-4a59-b70c-c2a489a2a828\") " pod="openshift-insights/insights-runtime-extractor-s2n8n"
Jul 21 13:44:48.731901 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.731862    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a700e703-22e3-4a59-b70c-c2a489a2a828-data-volume\") pod \"insights-runtime-extractor-s2n8n\" (UID: \"a700e703-22e3-4a59-b70c-c2a489a2a828\") " pod="openshift-insights/insights-runtime-extractor-s2n8n"
Jul 21 13:44:48.732152 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.731905    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a700e703-22e3-4a59-b70c-c2a489a2a828-crio-socket\") pod \"insights-runtime-extractor-s2n8n\" (UID: \"a700e703-22e3-4a59-b70c-c2a489a2a828\") " pod="openshift-insights/insights-runtime-extractor-s2n8n"
Jul 21 13:44:48.732152 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.731962    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6wtg\" (UniqueName: \"kubernetes.io/projected/a700e703-22e3-4a59-b70c-c2a489a2a828-kube-api-access-v6wtg\") pod \"insights-runtime-extractor-s2n8n\" (UID: \"a700e703-22e3-4a59-b70c-c2a489a2a828\") " pod="openshift-insights/insights-runtime-extractor-s2n8n"
Jul 21 13:44:48.732152 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.731993    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2b88\" (UniqueName: \"kubernetes.io/projected/79957af5-da1d-4787-9965-1b536cb01bca-kube-api-access-g2b88\") pod \"dns-default-vvrf8\" (UID: \"79957af5-da1d-4787-9965-1b536cb01bca\") " pod="openshift-dns/dns-default-vvrf8"
Jul 21 13:44:48.732152 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.732017    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/79957af5-da1d-4787-9965-1b536cb01bca-metrics-tls\") pod \"dns-default-vvrf8\" (UID: \"79957af5-da1d-4787-9965-1b536cb01bca\") " pod="openshift-dns/dns-default-vvrf8"
Jul 21 13:44:48.732152 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.732043    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkgr5\" (UniqueName: \"kubernetes.io/projected/6dfb88f6-dd03-45f2-96cf-3adc22acbd81-kube-api-access-kkgr5\") pod \"ingress-canary-gxpzz\" (UID: \"6dfb88f6-dd03-45f2-96cf-3adc22acbd81\") " pod="openshift-ingress-canary/ingress-canary-gxpzz"
Jul 21 13:44:48.732152 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:48.732055    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 21 13:44:48.732152 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:48.732126    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a700e703-22e3-4a59-b70c-c2a489a2a828-kube-rbac-proxy-cm podName:a700e703-22e3-4a59-b70c-c2a489a2a828 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:49.232109028 +0000 UTC m=+36.192397577 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a700e703-22e3-4a59-b70c-c2a489a2a828-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s2n8n" (UID: "a700e703-22e3-4a59-b70c-c2a489a2a828") : configmap "kube-rbac-proxy" not found
Jul 21 13:44:48.732152 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.732119    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79957af5-da1d-4787-9965-1b536cb01bca-tmp-dir\") pod \"dns-default-vvrf8\" (UID: \"79957af5-da1d-4787-9965-1b536cb01bca\") " pod="openshift-dns/dns-default-vvrf8"
Jul 21 13:44:48.732775 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.732375    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/79957af5-da1d-4787-9965-1b536cb01bca-config-volume\") pod \"dns-default-vvrf8\" (UID: \"79957af5-da1d-4787-9965-1b536cb01bca\") " pod="openshift-dns/dns-default-vvrf8"
Jul 21 13:44:48.732775 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.732460    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a700e703-22e3-4a59-b70c-c2a489a2a828-data-volume\") pod \"insights-runtime-extractor-s2n8n\" (UID: \"a700e703-22e3-4a59-b70c-c2a489a2a828\") " pod="openshift-insights/insights-runtime-extractor-s2n8n"
Jul 21 13:44:48.732775 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:48.732470    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 13:44:48.732775 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:48.732535    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79957af5-da1d-4787-9965-1b536cb01bca-metrics-tls podName:79957af5-da1d-4787-9965-1b536cb01bca nodeName:}" failed. No retries permitted until 2026-07-21 13:44:49.23251944 +0000 UTC m=+36.192807976 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/79957af5-da1d-4787-9965-1b536cb01bca-metrics-tls") pod "dns-default-vvrf8" (UID: "79957af5-da1d-4787-9965-1b536cb01bca") : secret "dns-default-metrics-tls" not found
Jul 21 13:44:48.732775 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:48.732584    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 13:44:48.732775 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:48.732663    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a700e703-22e3-4a59-b70c-c2a489a2a828-insights-runtime-extractor-tls podName:a700e703-22e3-4a59-b70c-c2a489a2a828 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:49.232645342 +0000 UTC m=+36.192933886 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a700e703-22e3-4a59-b70c-c2a489a2a828-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s2n8n" (UID: "a700e703-22e3-4a59-b70c-c2a489a2a828") : secret "insights-runtime-extractor-tls" not found
Jul 21 13:44:48.734341 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.734322    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6dfb88f6-dd03-45f2-96cf-3adc22acbd81-cert\") pod \"ingress-canary-gxpzz\" (UID: \"6dfb88f6-dd03-45f2-96cf-3adc22acbd81\") " pod="openshift-ingress-canary/ingress-canary-gxpzz"
Jul 21 13:44:48.748004 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.747979    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkgr5\" (UniqueName: \"kubernetes.io/projected/6dfb88f6-dd03-45f2-96cf-3adc22acbd81-kube-api-access-kkgr5\") pod \"ingress-canary-gxpzz\" (UID: \"6dfb88f6-dd03-45f2-96cf-3adc22acbd81\") " pod="openshift-ingress-canary/ingress-canary-gxpzz"
Jul 21 13:44:48.749252 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.749231    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6wtg\" (UniqueName: \"kubernetes.io/projected/a700e703-22e3-4a59-b70c-c2a489a2a828-kube-api-access-v6wtg\") pod \"insights-runtime-extractor-s2n8n\" (UID: \"a700e703-22e3-4a59-b70c-c2a489a2a828\") " pod="openshift-insights/insights-runtime-extractor-s2n8n"
Jul 21 13:44:48.755156 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.755133    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2b88\" (UniqueName: \"kubernetes.io/projected/79957af5-da1d-4787-9965-1b536cb01bca-kube-api-access-g2b88\") pod \"dns-default-vvrf8\" (UID: \"79957af5-da1d-4787-9965-1b536cb01bca\") " pod="openshift-dns/dns-default-vvrf8"
Jul 21 13:44:48.763681 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.763660    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gxpzz"
Jul 21 13:44:48.960638 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:48.960376    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gxpzz"]
Jul 21 13:44:48.963909 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:48.963861    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dfb88f6_dd03_45f2_96cf_3adc22acbd81.slice/crio-977a2283b3c3c7009631b01e401b004b903133da1c0f90e0185a66d3d21a57a1 WatchSource:0}: Error finding container 977a2283b3c3c7009631b01e401b004b903133da1c0f90e0185a66d3d21a57a1: Status 404 returned error can't find the container with id 977a2283b3c3c7009631b01e401b004b903133da1c0f90e0185a66d3d21a57a1
Jul 21 13:44:49.135769 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:49.135726    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51aea150-df52-44c2-a4f2-1f03c268f070-registry-tls\") pod \"image-registry-5f7fcdffb9-tfpcs\" (UID: \"51aea150-df52-44c2-a4f2-1f03c268f070\") " pod="openshift-image-registry/image-registry-5f7fcdffb9-tfpcs"
Jul 21 13:44:49.135964 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:49.135844    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 21 13:44:49.135964 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:49.135855    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f7fcdffb9-tfpcs: secret "image-registry-tls" not found
Jul 21 13:44:49.135964 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:49.135928    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51aea150-df52-44c2-a4f2-1f03c268f070-registry-tls podName:51aea150-df52-44c2-a4f2-1f03c268f070 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:50.135913899 +0000 UTC m=+37.096202434 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/51aea150-df52-44c2-a4f2-1f03c268f070-registry-tls") pod "image-registry-5f7fcdffb9-tfpcs" (UID: "51aea150-df52-44c2-a4f2-1f03c268f070") : secret "image-registry-tls" not found
Jul 21 13:44:49.236920 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:49.236788    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/79957af5-da1d-4787-9965-1b536cb01bca-metrics-tls\") pod \"dns-default-vvrf8\" (UID: \"79957af5-da1d-4787-9965-1b536cb01bca\") " pod="openshift-dns/dns-default-vvrf8"
Jul 21 13:44:49.236920 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:49.236844    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a700e703-22e3-4a59-b70c-c2a489a2a828-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s2n8n\" (UID: \"a700e703-22e3-4a59-b70c-c2a489a2a828\") " pod="openshift-insights/insights-runtime-extractor-s2n8n"
Jul 21 13:44:49.237110 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:49.236941    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a700e703-22e3-4a59-b70c-c2a489a2a828-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s2n8n\" (UID: \"a700e703-22e3-4a59-b70c-c2a489a2a828\") " pod="openshift-insights/insights-runtime-extractor-s2n8n"
Jul 21 13:44:49.237110 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:49.237015    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 21 13:44:49.237110 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:49.237063    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a700e703-22e3-4a59-b70c-c2a489a2a828-kube-rbac-proxy-cm podName:a700e703-22e3-4a59-b70c-c2a489a2a828 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:50.237049896 +0000 UTC m=+37.197338430 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a700e703-22e3-4a59-b70c-c2a489a2a828-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s2n8n" (UID: "a700e703-22e3-4a59-b70c-c2a489a2a828") : configmap "kube-rbac-proxy" not found
Jul 21 13:44:49.237110 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:49.237080    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 13:44:49.237312 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:49.237121    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a700e703-22e3-4a59-b70c-c2a489a2a828-insights-runtime-extractor-tls podName:a700e703-22e3-4a59-b70c-c2a489a2a828 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:50.237110222 +0000 UTC m=+37.197398757 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a700e703-22e3-4a59-b70c-c2a489a2a828-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s2n8n" (UID: "a700e703-22e3-4a59-b70c-c2a489a2a828") : secret "insights-runtime-extractor-tls" not found
Jul 21 13:44:49.237312 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:49.237172    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 13:44:49.237312 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:49.237228    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79957af5-da1d-4787-9965-1b536cb01bca-metrics-tls podName:79957af5-da1d-4787-9965-1b536cb01bca nodeName:}" failed. No retries permitted until 2026-07-21 13:44:50.237213389 +0000 UTC m=+37.197501928 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/79957af5-da1d-4787-9965-1b536cb01bca-metrics-tls") pod "dns-default-vvrf8" (UID: "79957af5-da1d-4787-9965-1b536cb01bca") : secret "dns-default-metrics-tls" not found
Jul 21 13:44:49.640775 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:49.640737    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gxpzz" event={"ID":"6dfb88f6-dd03-45f2-96cf-3adc22acbd81","Type":"ContainerStarted","Data":"977a2283b3c3c7009631b01e401b004b903133da1c0f90e0185a66d3d21a57a1"}
Jul 21 13:44:49.643707 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:49.643679    2578 generic.go:356] "Generic (PLEG): container finished" podID="3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9" containerID="b2f7834a05cb341444debe37b2759cf6f6bb6af7899777c819e2d682e907e6ba" exitCode=0
Jul 21 13:44:49.643824 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:49.643732    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6kwdx" event={"ID":"3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9","Type":"ContainerDied","Data":"b2f7834a05cb341444debe37b2759cf6f6bb6af7899777c819e2d682e907e6ba"}
Jul 21 13:44:50.145187 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:50.145153    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51aea150-df52-44c2-a4f2-1f03c268f070-registry-tls\") pod \"image-registry-5f7fcdffb9-tfpcs\" (UID: \"51aea150-df52-44c2-a4f2-1f03c268f070\") " pod="openshift-image-registry/image-registry-5f7fcdffb9-tfpcs"
Jul 21 13:44:50.145368 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:50.145350    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 21 13:44:50.145429 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:50.145375    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f7fcdffb9-tfpcs: secret "image-registry-tls" not found
Jul 21 13:44:50.145472 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:50.145440    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51aea150-df52-44c2-a4f2-1f03c268f070-registry-tls podName:51aea150-df52-44c2-a4f2-1f03c268f070 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:52.145425164 +0000 UTC m=+39.105713700 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/51aea150-df52-44c2-a4f2-1f03c268f070-registry-tls") pod "image-registry-5f7fcdffb9-tfpcs" (UID: "51aea150-df52-44c2-a4f2-1f03c268f070") : secret "image-registry-tls" not found
Jul 21 13:44:50.245827 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:50.245789    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a700e703-22e3-4a59-b70c-c2a489a2a828-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s2n8n\" (UID: \"a700e703-22e3-4a59-b70c-c2a489a2a828\") " pod="openshift-insights/insights-runtime-extractor-s2n8n"
Jul 21 13:44:50.246015 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:50.245853    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a700e703-22e3-4a59-b70c-c2a489a2a828-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s2n8n\" (UID: \"a700e703-22e3-4a59-b70c-c2a489a2a828\") " pod="openshift-insights/insights-runtime-extractor-s2n8n"
Jul 21 13:44:50.246015 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:50.245947    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/79957af5-da1d-4787-9965-1b536cb01bca-metrics-tls\") pod \"dns-default-vvrf8\" (UID: \"79957af5-da1d-4787-9965-1b536cb01bca\") " pod="openshift-dns/dns-default-vvrf8"
Jul 21 13:44:50.246015 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:50.245990    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 21 13:44:50.246170 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:50.246053    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 13:44:50.246170 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:50.246053    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 13:44:50.246170 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:50.246063    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a700e703-22e3-4a59-b70c-c2a489a2a828-kube-rbac-proxy-cm podName:a700e703-22e3-4a59-b70c-c2a489a2a828 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:52.2460435 +0000 UTC m=+39.206332043 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a700e703-22e3-4a59-b70c-c2a489a2a828-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s2n8n" (UID: "a700e703-22e3-4a59-b70c-c2a489a2a828") : configmap "kube-rbac-proxy" not found
Jul 21 13:44:50.246170 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:50.246133    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79957af5-da1d-4787-9965-1b536cb01bca-metrics-tls podName:79957af5-da1d-4787-9965-1b536cb01bca nodeName:}" failed. No retries permitted until 2026-07-21 13:44:52.24611601 +0000 UTC m=+39.206404545 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/79957af5-da1d-4787-9965-1b536cb01bca-metrics-tls") pod "dns-default-vvrf8" (UID: "79957af5-da1d-4787-9965-1b536cb01bca") : secret "dns-default-metrics-tls" not found
Jul 21 13:44:50.246170 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:50.246150    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a700e703-22e3-4a59-b70c-c2a489a2a828-insights-runtime-extractor-tls podName:a700e703-22e3-4a59-b70c-c2a489a2a828 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:52.246139275 +0000 UTC m=+39.206427810 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a700e703-22e3-4a59-b70c-c2a489a2a828-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s2n8n" (UID: "a700e703-22e3-4a59-b70c-c2a489a2a828") : secret "insights-runtime-extractor-tls" not found
Jul 21 13:44:50.650500 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:50.650462    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6kwdx" event={"ID":"3c50f7dd-5f7d-46e1-84d8-c359e33ac5c9","Type":"ContainerStarted","Data":"f1e0b07453a7a28e014986c2e6a089a0cced9a3d07e20098399dc828514cbdbc"}
Jul 21 13:44:50.676557 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:50.676510    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-6kwdx" podStartSLOduration=7.503018025 podStartE2EDuration="37.676485456s" totalImagesPullingTime="30.173467431s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 13:44:13 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:44:50.674886117 +0000 UTC m=+37.635174669" watchObservedRunningTime="2026-07-21 13:44:50.676485456 +0000 UTC m=+37.636774014"
Jul 21 13:44:52.162529 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:52.162486    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51aea150-df52-44c2-a4f2-1f03c268f070-registry-tls\") pod \"image-registry-5f7fcdffb9-tfpcs\" (UID: \"51aea150-df52-44c2-a4f2-1f03c268f070\") " pod="openshift-image-registry/image-registry-5f7fcdffb9-tfpcs"
Jul 21 13:44:52.162936 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:52.162633    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 21 13:44:52.162936 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:52.162655    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f7fcdffb9-tfpcs: secret "image-registry-tls" not found
Jul 21 13:44:52.162936 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:52.162712    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51aea150-df52-44c2-a4f2-1f03c268f070-registry-tls podName:51aea150-df52-44c2-a4f2-1f03c268f070 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:56.162697662 +0000 UTC m=+43.122986210 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/51aea150-df52-44c2-a4f2-1f03c268f070-registry-tls") pod "image-registry-5f7fcdffb9-tfpcs" (UID: "51aea150-df52-44c2-a4f2-1f03c268f070") : secret "image-registry-tls" not found
Jul 21 13:44:52.263202 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:52.263158    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/79957af5-da1d-4787-9965-1b536cb01bca-metrics-tls\") pod \"dns-default-vvrf8\" (UID: \"79957af5-da1d-4787-9965-1b536cb01bca\") " pod="openshift-dns/dns-default-vvrf8"
Jul 21 13:44:52.263375 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:52.263219    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a700e703-22e3-4a59-b70c-c2a489a2a828-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s2n8n\" (UID: \"a700e703-22e3-4a59-b70c-c2a489a2a828\") " pod="openshift-insights/insights-runtime-extractor-s2n8n"
Jul 21 13:44:52.263375 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:52.263246    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a700e703-22e3-4a59-b70c-c2a489a2a828-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s2n8n\" (UID: \"a700e703-22e3-4a59-b70c-c2a489a2a828\") " pod="openshift-insights/insights-runtime-extractor-s2n8n"
Jul 21 13:44:52.263375 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:52.263329    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 13:44:52.263487 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:52.263403    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a700e703-22e3-4a59-b70c-c2a489a2a828-insights-runtime-extractor-tls podName:a700e703-22e3-4a59-b70c-c2a489a2a828 nodeName:}" failed. No retries permitted until 2026-07-21 13:44:56.263387709 +0000 UTC m=+43.223676263 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a700e703-22e3-4a59-b70c-c2a489a2a828-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s2n8n" (UID: "a700e703-22e3-4a59-b70c-c2a489a2a828") : secret "insights-runtime-extractor-tls" not found
Jul 21 13:44:52.263487 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:52.263342    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 13:44:52.263487 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:52.263469    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79957af5-da1d-4787-9965-1b536cb01bca-metrics-tls podName:79957af5-da1d-4787-9965-1b536cb01bca nodeName:}" failed. No retries permitted until 2026-07-21 13:44:56.263456016 +0000 UTC m=+43.223744566 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/79957af5-da1d-4787-9965-1b536cb01bca-metrics-tls") pod "dns-default-vvrf8" (UID: "79957af5-da1d-4787-9965-1b536cb01bca") : secret "dns-default-metrics-tls" not found
Jul 21 13:44:52.264159 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:52.264002    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a700e703-22e3-4a59-b70c-c2a489a2a828-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s2n8n\" (UID: \"a700e703-22e3-4a59-b70c-c2a489a2a828\") " pod="openshift-insights/insights-runtime-extractor-s2n8n"
Jul 21 13:44:52.656670 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:52.656634    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gxpzz" event={"ID":"6dfb88f6-dd03-45f2-96cf-3adc22acbd81","Type":"ContainerStarted","Data":"c2c6d4d034682256cd93e295064a456f54c66e836e03eb552c2a3c01d6c7ae81"}
Jul 21 13:44:52.675650 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:52.675600    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-gxpzz" podStartSLOduration=1.893992117 podStartE2EDuration="4.675585259s" totalImagesPullingTime="2.781593142s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:44:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:44:52.67523175 +0000 UTC m=+39.635520330" watchObservedRunningTime="2026-07-21 13:44:52.675585259 +0000 UTC m=+39.635873816"
Jul 21 13:44:56.196255 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:56.196214    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51aea150-df52-44c2-a4f2-1f03c268f070-registry-tls\") pod \"image-registry-5f7fcdffb9-tfpcs\" (UID: \"51aea150-df52-44c2-a4f2-1f03c268f070\") " pod="openshift-image-registry/image-registry-5f7fcdffb9-tfpcs"
Jul 21 13:44:56.200421 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:56.200391    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51aea150-df52-44c2-a4f2-1f03c268f070-registry-tls\") pod \"image-registry-5f7fcdffb9-tfpcs\" (UID: \"51aea150-df52-44c2-a4f2-1f03c268f070\") " pod="openshift-image-registry/image-registry-5f7fcdffb9-tfpcs"
Jul 21 13:44:56.246402 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:56.246366    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f7fcdffb9-tfpcs"
Jul 21 13:44:56.296825 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:56.296793    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/79957af5-da1d-4787-9965-1b536cb01bca-metrics-tls\") pod \"dns-default-vvrf8\" (UID: \"79957af5-da1d-4787-9965-1b536cb01bca\") " pod="openshift-dns/dns-default-vvrf8"
Jul 21 13:44:56.296998 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:56.296861    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a700e703-22e3-4a59-b70c-c2a489a2a828-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s2n8n\" (UID: \"a700e703-22e3-4a59-b70c-c2a489a2a828\") " pod="openshift-insights/insights-runtime-extractor-s2n8n"
Jul 21 13:44:56.297076 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:56.297048    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 13:44:56.297178 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:44:56.297134    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a700e703-22e3-4a59-b70c-c2a489a2a828-insights-runtime-extractor-tls podName:a700e703-22e3-4a59-b70c-c2a489a2a828 nodeName:}" failed. No retries permitted until 2026-07-21 13:45:04.297111079 +0000 UTC m=+51.257399623 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a700e703-22e3-4a59-b70c-c2a489a2a828-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s2n8n" (UID: "a700e703-22e3-4a59-b70c-c2a489a2a828") : secret "insights-runtime-extractor-tls" not found
Jul 21 13:44:56.299609 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:56.299588    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/79957af5-da1d-4787-9965-1b536cb01bca-metrics-tls\") pod \"dns-default-vvrf8\" (UID: \"79957af5-da1d-4787-9965-1b536cb01bca\") " pod="openshift-dns/dns-default-vvrf8"
Jul 21 13:44:56.329295 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:56.329249    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vvrf8"
Jul 21 13:44:56.410892 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:56.410837    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5f7fcdffb9-tfpcs"]
Jul 21 13:44:56.414014 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:56.413986    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51aea150_df52_44c2_a4f2_1f03c268f070.slice/crio-31e9baa0ec76285f4d2c6ca136778be60bf53047258bd2d75bb3379eb6f2de94 WatchSource:0}: Error finding container 31e9baa0ec76285f4d2c6ca136778be60bf53047258bd2d75bb3379eb6f2de94: Status 404 returned error can't find the container with id 31e9baa0ec76285f4d2c6ca136778be60bf53047258bd2d75bb3379eb6f2de94
Jul 21 13:44:56.494178 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:56.494148    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vvrf8"]
Jul 21 13:44:56.511979 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:44:56.511952    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79957af5_da1d_4787_9965_1b536cb01bca.slice/crio-5667c7cbb44b11094c7e076218c4852b8a28017f7b42f7c2a8451d5ba7aec007 WatchSource:0}: Error finding container 5667c7cbb44b11094c7e076218c4852b8a28017f7b42f7c2a8451d5ba7aec007: Status 404 returned error can't find the container with id 5667c7cbb44b11094c7e076218c4852b8a28017f7b42f7c2a8451d5ba7aec007
Jul 21 13:44:56.667507 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:56.667471    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f7fcdffb9-tfpcs" event={"ID":"51aea150-df52-44c2-a4f2-1f03c268f070","Type":"ContainerStarted","Data":"3b933687702e412811e102556eb32180b17b6fe3cafd8c13ec2a1ddc4c2bbdad"}
Jul 21 13:44:56.667694 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:56.667526    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f7fcdffb9-tfpcs" event={"ID":"51aea150-df52-44c2-a4f2-1f03c268f070","Type":"ContainerStarted","Data":"31e9baa0ec76285f4d2c6ca136778be60bf53047258bd2d75bb3379eb6f2de94"}
Jul 21 13:44:56.667694 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:56.667581    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5f7fcdffb9-tfpcs"
Jul 21 13:44:56.668628 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:56.668608    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vvrf8" event={"ID":"79957af5-da1d-4787-9965-1b536cb01bca","Type":"ContainerStarted","Data":"5667c7cbb44b11094c7e076218c4852b8a28017f7b42f7c2a8451d5ba7aec007"}
Jul 21 13:44:56.694964 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:44:56.694906    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5f7fcdffb9-tfpcs" podStartSLOduration=28.694890597 podStartE2EDuration="28.694890597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:44:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:44:56.69404492 +0000 UTC m=+43.654333477" watchObservedRunningTime="2026-07-21 13:44:56.694890597 +0000 UTC m=+43.655179150"
Jul 21 13:45:00.681946 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:00.681909    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vvrf8" event={"ID":"79957af5-da1d-4787-9965-1b536cb01bca","Type":"ContainerStarted","Data":"92dbb8f414149606271f998338061cb8498b6cf5ce7cbf901a4ae2bfaa686069"}
Jul 21 13:45:00.682417 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:00.681988    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vvrf8" event={"ID":"79957af5-da1d-4787-9965-1b536cb01bca","Type":"ContainerStarted","Data":"b182c387fb80fa877595368402cd815f82f8de4096ebe7d23cb25d512973cfab"}
Jul 21 13:45:00.682417 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:00.682027    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vvrf8"
Jul 21 13:45:00.705924 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:00.705847    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-vvrf8" podStartSLOduration=9.29678984 podStartE2EDuration="12.705832173s" totalImagesPullingTime="3.409042333s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:44:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:45:00.704088689 +0000 UTC m=+47.664377247" watchObservedRunningTime="2026-07-21 13:45:00.705832173 +0000 UTC m=+47.666120708"
Jul 21 13:45:04.356708 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:04.356669    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a700e703-22e3-4a59-b70c-c2a489a2a828-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s2n8n\" (UID: \"a700e703-22e3-4a59-b70c-c2a489a2a828\") " pod="openshift-insights/insights-runtime-extractor-s2n8n"
Jul 21 13:45:04.358988 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:04.358962    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a700e703-22e3-4a59-b70c-c2a489a2a828-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s2n8n\" (UID: \"a700e703-22e3-4a59-b70c-c2a489a2a828\") " pod="openshift-insights/insights-runtime-extractor-s2n8n"
Jul 21 13:45:04.386944 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:04.386914    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-s2n8n"
Jul 21 13:45:04.524828 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:04.524792    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-s2n8n"]
Jul 21 13:45:04.529101 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:45:04.529075    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda700e703_22e3_4a59_b70c_c2a489a2a828.slice/crio-3f5229ecd3a2c162aa3131eca671c479aa4eda9fcc216a5f205e1147aa278df4 WatchSource:0}: Error finding container 3f5229ecd3a2c162aa3131eca671c479aa4eda9fcc216a5f205e1147aa278df4: Status 404 returned error can't find the container with id 3f5229ecd3a2c162aa3131eca671c479aa4eda9fcc216a5f205e1147aa278df4
Jul 21 13:45:04.692660 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:04.692577    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s2n8n" event={"ID":"a700e703-22e3-4a59-b70c-c2a489a2a828","Type":"ContainerStarted","Data":"3f5229ecd3a2c162aa3131eca671c479aa4eda9fcc216a5f205e1147aa278df4"}
Jul 21 13:45:09.707079 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:09.707047    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s2n8n" event={"ID":"a700e703-22e3-4a59-b70c-c2a489a2a828","Type":"ContainerStarted","Data":"c8535389c8ce8a1edb7101007c20ad55f5c79836207fd185d7af96a29a39fd9d"}
Jul 21 13:45:10.164834 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.164803    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c65bb4b58-57mcd"]
Jul 21 13:45:10.186124 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.186086    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c65bb4b58-57mcd"]
Jul 21 13:45:10.186297 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.186179    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c65bb4b58-57mcd"
Jul 21 13:45:10.188765 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.188737    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6d8f8dcd-w78hr"]
Jul 21 13:45:10.192548 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.192495    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 21 13:45:10.192548 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.192521    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 21 13:45:10.192739 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.192530    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 21 13:45:10.192739 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.192603    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 21 13:45:10.202943 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.202923    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95f7fc78-8b7f-44ca-ae95-a9a3acc6edbb-tmp\") pod \"klusterlet-addon-workmgr-6c65bb4b58-57mcd\" (UID: \"95f7fc78-8b7f-44ca-ae95-a9a3acc6edbb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c65bb4b58-57mcd"
Jul 21 13:45:10.203044 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.202947    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/95f7fc78-8b7f-44ca-ae95-a9a3acc6edbb-klusterlet-config\") pod \"klusterlet-addon-workmgr-6c65bb4b58-57mcd\" (UID: \"95f7fc78-8b7f-44ca-ae95-a9a3acc6edbb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c65bb4b58-57mcd"
Jul 21 13:45:10.203044 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.202967    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrp9v\" (UniqueName: \"kubernetes.io/projected/95f7fc78-8b7f-44ca-ae95-a9a3acc6edbb-kube-api-access-rrp9v\") pod \"klusterlet-addon-workmgr-6c65bb4b58-57mcd\" (UID: \"95f7fc78-8b7f-44ca-ae95-a9a3acc6edbb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c65bb4b58-57mcd"
Jul 21 13:45:10.206793 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.206774    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc8bd98db-nfdhb"]
Jul 21 13:45:10.206931 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.206918    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6d8f8dcd-w78hr"
Jul 21 13:45:10.209580 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.209558    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 21 13:45:10.209848 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.209832    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 21 13:45:10.209960 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.209931    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 21 13:45:10.210417 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.210402    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 21 13:45:10.224974 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.224950    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6d8f8dcd-w78hr"]
Jul 21 13:45:10.225093 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.224980    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc8bd98db-nfdhb"]
Jul 21 13:45:10.225093 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.225047    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc8bd98db-nfdhb"
Jul 21 13:45:10.230531 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.230504    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 21 13:45:10.233747 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.233725    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5f7fcdffb9-tfpcs"]
Jul 21 13:45:10.238384 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.238359    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-6q54h\""
Jul 21 13:45:10.303361 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.303329    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrp9v\" (UniqueName: \"kubernetes.io/projected/95f7fc78-8b7f-44ca-ae95-a9a3acc6edbb-kube-api-access-rrp9v\") pod \"klusterlet-addon-workmgr-6c65bb4b58-57mcd\" (UID: \"95f7fc78-8b7f-44ca-ae95-a9a3acc6edbb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c65bb4b58-57mcd"
Jul 21 13:45:10.303361 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.303375    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9d1d867b-be67-4448-9f50-2ae1ea35100d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6bc8bd98db-nfdhb\" (UID: \"9d1d867b-be67-4448-9f50-2ae1ea35100d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc8bd98db-nfdhb"
Jul 21 13:45:10.303573 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.303399    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/115bee6d-6782-47b2-b262-7e6723c2c87b-hub\") pod \"cluster-proxy-proxy-agent-f6d8f8dcd-w78hr\" (UID: \"115bee6d-6782-47b2-b262-7e6723c2c87b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6d8f8dcd-w78hr"
Jul 21 13:45:10.303573 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.303416    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/115bee6d-6782-47b2-b262-7e6723c2c87b-ca\") pod \"cluster-proxy-proxy-agent-f6d8f8dcd-w78hr\" (UID: \"115bee6d-6782-47b2-b262-7e6723c2c87b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6d8f8dcd-w78hr"
Jul 21 13:45:10.303573 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.303470    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/115bee6d-6782-47b2-b262-7e6723c2c87b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-f6d8f8dcd-w78hr\" (UID: \"115bee6d-6782-47b2-b262-7e6723c2c87b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6d8f8dcd-w78hr"
Jul 21 13:45:10.303573 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.303537    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2q99\" (UniqueName: \"kubernetes.io/projected/115bee6d-6782-47b2-b262-7e6723c2c87b-kube-api-access-x2q99\") pod \"cluster-proxy-proxy-agent-f6d8f8dcd-w78hr\" (UID: \"115bee6d-6782-47b2-b262-7e6723c2c87b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6d8f8dcd-w78hr"
Jul 21 13:45:10.303573 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.303566    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvz68\" (UniqueName: \"kubernetes.io/projected/9d1d867b-be67-4448-9f50-2ae1ea35100d-kube-api-access-vvz68\") pod \"managed-serviceaccount-addon-agent-6bc8bd98db-nfdhb\" (UID: \"9d1d867b-be67-4448-9f50-2ae1ea35100d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc8bd98db-nfdhb"
Jul 21 13:45:10.303779 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.303597    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/95f7fc78-8b7f-44ca-ae95-a9a3acc6edbb-klusterlet-config\") pod \"klusterlet-addon-workmgr-6c65bb4b58-57mcd\" (UID: \"95f7fc78-8b7f-44ca-ae95-a9a3acc6edbb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c65bb4b58-57mcd"
Jul 21 13:45:10.303779 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.303657    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/115bee6d-6782-47b2-b262-7e6723c2c87b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-f6d8f8dcd-w78hr\" (UID: \"115bee6d-6782-47b2-b262-7e6723c2c87b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6d8f8dcd-w78hr"
Jul 21 13:45:10.303779 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.303684    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95f7fc78-8b7f-44ca-ae95-a9a3acc6edbb-tmp\") pod \"klusterlet-addon-workmgr-6c65bb4b58-57mcd\" (UID: \"95f7fc78-8b7f-44ca-ae95-a9a3acc6edbb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c65bb4b58-57mcd"
Jul 21 13:45:10.303779 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.303709    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/115bee6d-6782-47b2-b262-7e6723c2c87b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-f6d8f8dcd-w78hr\" (UID: \"115bee6d-6782-47b2-b262-7e6723c2c87b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6d8f8dcd-w78hr"
Jul 21 13:45:10.304080 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.304061    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95f7fc78-8b7f-44ca-ae95-a9a3acc6edbb-tmp\") pod \"klusterlet-addon-workmgr-6c65bb4b58-57mcd\" (UID: \"95f7fc78-8b7f-44ca-ae95-a9a3acc6edbb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c65bb4b58-57mcd"
Jul 21 13:45:10.306215 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.306195    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/95f7fc78-8b7f-44ca-ae95-a9a3acc6edbb-klusterlet-config\") pod \"klusterlet-addon-workmgr-6c65bb4b58-57mcd\" (UID: \"95f7fc78-8b7f-44ca-ae95-a9a3acc6edbb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c65bb4b58-57mcd"
Jul 21 13:45:10.332736 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.332706    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrp9v\" (UniqueName: \"kubernetes.io/projected/95f7fc78-8b7f-44ca-ae95-a9a3acc6edbb-kube-api-access-rrp9v\") pod \"klusterlet-addon-workmgr-6c65bb4b58-57mcd\" (UID: \"95f7fc78-8b7f-44ca-ae95-a9a3acc6edbb\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c65bb4b58-57mcd"
Jul 21 13:45:10.336176 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.336151    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5747f97549-lrm92"]
Jul 21 13:45:10.349050 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.349030    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5747f97549-lrm92"
Jul 21 13:45:10.368340 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.368306    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5747f97549-lrm92"]
Jul 21 13:45:10.404345 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.404315    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/115bee6d-6782-47b2-b262-7e6723c2c87b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-f6d8f8dcd-w78hr\" (UID: \"115bee6d-6782-47b2-b262-7e6723c2c87b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6d8f8dcd-w78hr"
Jul 21 13:45:10.404497 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.404360    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7b6a97cd-7c40-40f9-8280-18fdd3e5f38c-installation-pull-secrets\") pod \"image-registry-5747f97549-lrm92\" (UID: \"7b6a97cd-7c40-40f9-8280-18fdd3e5f38c\") " pod="openshift-image-registry/image-registry-5747f97549-lrm92"
Jul 21 13:45:10.404497 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.404388    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7b6a97cd-7c40-40f9-8280-18fdd3e5f38c-image-registry-private-configuration\") pod \"image-registry-5747f97549-lrm92\" (UID: \"7b6a97cd-7c40-40f9-8280-18fdd3e5f38c\") " pod="openshift-image-registry/image-registry-5747f97549-lrm92"
Jul 21 13:45:10.404497 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.404442    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9d1d867b-be67-4448-9f50-2ae1ea35100d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6bc8bd98db-nfdhb\" (UID: \"9d1d867b-be67-4448-9f50-2ae1ea35100d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc8bd98db-nfdhb"
Jul 21 13:45:10.404497 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.404491    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7b6a97cd-7c40-40f9-8280-18fdd3e5f38c-registry-tls\") pod \"image-registry-5747f97549-lrm92\" (UID: \"7b6a97cd-7c40-40f9-8280-18fdd3e5f38c\") " pod="openshift-image-registry/image-registry-5747f97549-lrm92"
Jul 21 13:45:10.404711 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.404525    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/115bee6d-6782-47b2-b262-7e6723c2c87b-hub\") pod \"cluster-proxy-proxy-agent-f6d8f8dcd-w78hr\" (UID: \"115bee6d-6782-47b2-b262-7e6723c2c87b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6d8f8dcd-w78hr"
Jul 21 13:45:10.404711 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.404550    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7b6a97cd-7c40-40f9-8280-18fdd3e5f38c-registry-certificates\") pod \"image-registry-5747f97549-lrm92\" (UID: \"7b6a97cd-7c40-40f9-8280-18fdd3e5f38c\") " pod="openshift-image-registry/image-registry-5747f97549-lrm92"
Jul 21 13:45:10.404711 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.404584    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/115bee6d-6782-47b2-b262-7e6723c2c87b-ca\") pod \"cluster-proxy-proxy-agent-f6d8f8dcd-w78hr\" (UID: \"115bee6d-6782-47b2-b262-7e6723c2c87b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6d8f8dcd-w78hr"
Jul 21 13:45:10.404711 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.404635    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/115bee6d-6782-47b2-b262-7e6723c2c87b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-f6d8f8dcd-w78hr\" (UID: \"115bee6d-6782-47b2-b262-7e6723c2c87b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6d8f8dcd-w78hr"
Jul 21 13:45:10.404711 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.404702    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7b6a97cd-7c40-40f9-8280-18fdd3e5f38c-trusted-ca\") pod \"image-registry-5747f97549-lrm92\" (UID: \"7b6a97cd-7c40-40f9-8280-18fdd3e5f38c\") " pod="openshift-image-registry/image-registry-5747f97549-lrm92"
Jul 21 13:45:10.405010 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.404731    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7b6a97cd-7c40-40f9-8280-18fdd3e5f38c-ca-trust-extracted\") pod \"image-registry-5747f97549-lrm92\" (UID: \"7b6a97cd-7c40-40f9-8280-18fdd3e5f38c\") " pod="openshift-image-registry/image-registry-5747f97549-lrm92"
Jul 21 13:45:10.405010 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.404772    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx77r\" (UniqueName: \"kubernetes.io/projected/7b6a97cd-7c40-40f9-8280-18fdd3e5f38c-kube-api-access-tx77r\") pod \"image-registry-5747f97549-lrm92\" (UID: \"7b6a97cd-7c40-40f9-8280-18fdd3e5f38c\") " pod="openshift-image-registry/image-registry-5747f97549-lrm92"
Jul 21 13:45:10.405010 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.404809    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2q99\" (UniqueName: \"kubernetes.io/projected/115bee6d-6782-47b2-b262-7e6723c2c87b-kube-api-access-x2q99\") pod \"cluster-proxy-proxy-agent-f6d8f8dcd-w78hr\" (UID: \"115bee6d-6782-47b2-b262-7e6723c2c87b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6d8f8dcd-w78hr"
Jul 21 13:45:10.405010 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.404833    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvz68\" (UniqueName: \"kubernetes.io/projected/9d1d867b-be67-4448-9f50-2ae1ea35100d-kube-api-access-vvz68\") pod \"managed-serviceaccount-addon-agent-6bc8bd98db-nfdhb\" (UID: \"9d1d867b-be67-4448-9f50-2ae1ea35100d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc8bd98db-nfdhb"
Jul 21 13:45:10.405010 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.404865    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7b6a97cd-7c40-40f9-8280-18fdd3e5f38c-bound-sa-token\") pod \"image-registry-5747f97549-lrm92\" (UID: \"7b6a97cd-7c40-40f9-8280-18fdd3e5f38c\") " pod="openshift-image-registry/image-registry-5747f97549-lrm92"
Jul 21 13:45:10.405010 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.404932    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/115bee6d-6782-47b2-b262-7e6723c2c87b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-f6d8f8dcd-w78hr\" (UID: \"115bee6d-6782-47b2-b262-7e6723c2c87b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6d8f8dcd-w78hr"
Jul 21 13:45:10.405701 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.405677    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/115bee6d-6782-47b2-b262-7e6723c2c87b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-f6d8f8dcd-w78hr\" (UID: \"115bee6d-6782-47b2-b262-7e6723c2c87b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6d8f8dcd-w78hr"
Jul 21 13:45:10.407286 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.407259    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/115bee6d-6782-47b2-b262-7e6723c2c87b-hub\") pod \"cluster-proxy-proxy-agent-f6d8f8dcd-w78hr\" (UID: \"115bee6d-6782-47b2-b262-7e6723c2c87b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6d8f8dcd-w78hr"
Jul 21 13:45:10.407443 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.407282    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9d1d867b-be67-4448-9f50-2ae1ea35100d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6bc8bd98db-nfdhb\" (UID: \"9d1d867b-be67-4448-9f50-2ae1ea35100d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc8bd98db-nfdhb"
Jul 21 13:45:10.407443 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.407318    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/115bee6d-6782-47b2-b262-7e6723c2c87b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-f6d8f8dcd-w78hr\" (UID: \"115bee6d-6782-47b2-b262-7e6723c2c87b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6d8f8dcd-w78hr"
Jul 21 13:45:10.407646 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.407627    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/115bee6d-6782-47b2-b262-7e6723c2c87b-ca\") pod \"cluster-proxy-proxy-agent-f6d8f8dcd-w78hr\" (UID: \"115bee6d-6782-47b2-b262-7e6723c2c87b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6d8f8dcd-w78hr"
Jul 21 13:45:10.408054 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.408034    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/115bee6d-6782-47b2-b262-7e6723c2c87b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-f6d8f8dcd-w78hr\" (UID: \"115bee6d-6782-47b2-b262-7e6723c2c87b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6d8f8dcd-w78hr"
Jul 21 13:45:10.423028 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.422979    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvz68\" (UniqueName: \"kubernetes.io/projected/9d1d867b-be67-4448-9f50-2ae1ea35100d-kube-api-access-vvz68\") pod \"managed-serviceaccount-addon-agent-6bc8bd98db-nfdhb\" (UID: \"9d1d867b-be67-4448-9f50-2ae1ea35100d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc8bd98db-nfdhb"
Jul 21 13:45:10.423351 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.423335    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2q99\" (UniqueName: \"kubernetes.io/projected/115bee6d-6782-47b2-b262-7e6723c2c87b-kube-api-access-x2q99\") pod \"cluster-proxy-proxy-agent-f6d8f8dcd-w78hr\" (UID: \"115bee6d-6782-47b2-b262-7e6723c2c87b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6d8f8dcd-w78hr"
Jul 21 13:45:10.496095 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.496066    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c65bb4b58-57mcd"
Jul 21 13:45:10.505633 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.505593    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7b6a97cd-7c40-40f9-8280-18fdd3e5f38c-installation-pull-secrets\") pod \"image-registry-5747f97549-lrm92\" (UID: \"7b6a97cd-7c40-40f9-8280-18fdd3e5f38c\") " pod="openshift-image-registry/image-registry-5747f97549-lrm92"
Jul 21 13:45:10.505800 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.505662    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7b6a97cd-7c40-40f9-8280-18fdd3e5f38c-image-registry-private-configuration\") pod \"image-registry-5747f97549-lrm92\" (UID: \"7b6a97cd-7c40-40f9-8280-18fdd3e5f38c\") " pod="openshift-image-registry/image-registry-5747f97549-lrm92"
Jul 21 13:45:10.505800 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.505697    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7b6a97cd-7c40-40f9-8280-18fdd3e5f38c-registry-tls\") pod \"image-registry-5747f97549-lrm92\" (UID: \"7b6a97cd-7c40-40f9-8280-18fdd3e5f38c\") " pod="openshift-image-registry/image-registry-5747f97549-lrm92"
Jul 21 13:45:10.505800 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.505731    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7b6a97cd-7c40-40f9-8280-18fdd3e5f38c-registry-certificates\") pod \"image-registry-5747f97549-lrm92\" (UID: \"7b6a97cd-7c40-40f9-8280-18fdd3e5f38c\") " pod="openshift-image-registry/image-registry-5747f97549-lrm92"
Jul 21 13:45:10.505800 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.505784    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7b6a97cd-7c40-40f9-8280-18fdd3e5f38c-trusted-ca\") pod \"image-registry-5747f97549-lrm92\" (UID: \"7b6a97cd-7c40-40f9-8280-18fdd3e5f38c\") " pod="openshift-image-registry/image-registry-5747f97549-lrm92"
Jul 21 13:45:10.506095 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.505810    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7b6a97cd-7c40-40f9-8280-18fdd3e5f38c-ca-trust-extracted\") pod \"image-registry-5747f97549-lrm92\" (UID: \"7b6a97cd-7c40-40f9-8280-18fdd3e5f38c\") " pod="openshift-image-registry/image-registry-5747f97549-lrm92"
Jul 21 13:45:10.506095 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.505834    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx77r\" (UniqueName: \"kubernetes.io/projected/7b6a97cd-7c40-40f9-8280-18fdd3e5f38c-kube-api-access-tx77r\") pod \"image-registry-5747f97549-lrm92\" (UID: \"7b6a97cd-7c40-40f9-8280-18fdd3e5f38c\") " pod="openshift-image-registry/image-registry-5747f97549-lrm92"
Jul 21 13:45:10.506095 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.505906    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7b6a97cd-7c40-40f9-8280-18fdd3e5f38c-bound-sa-token\") pod \"image-registry-5747f97549-lrm92\" (UID: \"7b6a97cd-7c40-40f9-8280-18fdd3e5f38c\") " pod="openshift-image-registry/image-registry-5747f97549-lrm92"
Jul 21 13:45:10.506705 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.506676    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7b6a97cd-7c40-40f9-8280-18fdd3e5f38c-ca-trust-extracted\") pod \"image-registry-5747f97549-lrm92\" (UID: \"7b6a97cd-7c40-40f9-8280-18fdd3e5f38c\") " pod="openshift-image-registry/image-registry-5747f97549-lrm92"
Jul 21 13:45:10.507279 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.507255    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7b6a97cd-7c40-40f9-8280-18fdd3e5f38c-registry-certificates\") pod \"image-registry-5747f97549-lrm92\" (UID: \"7b6a97cd-7c40-40f9-8280-18fdd3e5f38c\") " pod="openshift-image-registry/image-registry-5747f97549-lrm92"
Jul 21 13:45:10.507393 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.507368    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7b6a97cd-7c40-40f9-8280-18fdd3e5f38c-trusted-ca\") pod \"image-registry-5747f97549-lrm92\" (UID: \"7b6a97cd-7c40-40f9-8280-18fdd3e5f38c\") " pod="openshift-image-registry/image-registry-5747f97549-lrm92"
Jul 21 13:45:10.509157 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.509131    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7b6a97cd-7c40-40f9-8280-18fdd3e5f38c-registry-tls\") pod \"image-registry-5747f97549-lrm92\" (UID: \"7b6a97cd-7c40-40f9-8280-18fdd3e5f38c\") " pod="openshift-image-registry/image-registry-5747f97549-lrm92"
Jul 21 13:45:10.509565 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.509540    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7b6a97cd-7c40-40f9-8280-18fdd3e5f38c-installation-pull-secrets\") pod \"image-registry-5747f97549-lrm92\" (UID: \"7b6a97cd-7c40-40f9-8280-18fdd3e5f38c\") " pod="openshift-image-registry/image-registry-5747f97549-lrm92"
Jul 21 13:45:10.511560 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.511538    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7b6a97cd-7c40-40f9-8280-18fdd3e5f38c-image-registry-private-configuration\") pod \"image-registry-5747f97549-lrm92\" (UID: \"7b6a97cd-7c40-40f9-8280-18fdd3e5f38c\") " pod="openshift-image-registry/image-registry-5747f97549-lrm92"
Jul 21 13:45:10.523717 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.523687    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6d8f8dcd-w78hr"
Jul 21 13:45:10.525709 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.525686    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7b6a97cd-7c40-40f9-8280-18fdd3e5f38c-bound-sa-token\") pod \"image-registry-5747f97549-lrm92\" (UID: \"7b6a97cd-7c40-40f9-8280-18fdd3e5f38c\") " pod="openshift-image-registry/image-registry-5747f97549-lrm92"
Jul 21 13:45:10.526496 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.526450    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx77r\" (UniqueName: \"kubernetes.io/projected/7b6a97cd-7c40-40f9-8280-18fdd3e5f38c-kube-api-access-tx77r\") pod \"image-registry-5747f97549-lrm92\" (UID: \"7b6a97cd-7c40-40f9-8280-18fdd3e5f38c\") " pod="openshift-image-registry/image-registry-5747f97549-lrm92"
Jul 21 13:45:10.533141 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.533049    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc8bd98db-nfdhb"
Jul 21 13:45:10.673683 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.673137    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5747f97549-lrm92"
Jul 21 13:45:10.687196 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.686381    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vvrf8"
Jul 21 13:45:10.699750 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.697532    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c65bb4b58-57mcd"]
Jul 21 13:45:10.727739 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.727675    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c65bb4b58-57mcd" event={"ID":"95f7fc78-8b7f-44ca-ae95-a9a3acc6edbb","Type":"ContainerStarted","Data":"00e69b73375e8c93d5681cb4a464d5f08ff7226b01f0ebb0e49f0ae0aa73e662"}
Jul 21 13:45:10.740217 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.740157    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6d8f8dcd-w78hr"]
Jul 21 13:45:10.747396 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:45:10.746560    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod115bee6d_6782_47b2_b262_7e6723c2c87b.slice/crio-731ce48254586553ca1e306c002288edaa94074487b79ce59fe7db02d082d881 WatchSource:0}: Error finding container 731ce48254586553ca1e306c002288edaa94074487b79ce59fe7db02d082d881: Status 404 returned error can't find the container with id 731ce48254586553ca1e306c002288edaa94074487b79ce59fe7db02d082d881
Jul 21 13:45:10.768469 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.768428    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc8bd98db-nfdhb"]
Jul 21 13:45:10.774666 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:45:10.773891    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9d1d867b_be67_4448_9f50_2ae1ea35100d.slice/crio-bca1c712cdea0c527f475cd00d6e13790d200186e5fe498c700222ae2a8eba86 WatchSource:0}: Error finding container bca1c712cdea0c527f475cd00d6e13790d200186e5fe498c700222ae2a8eba86: Status 404 returned error can't find the container with id bca1c712cdea0c527f475cd00d6e13790d200186e5fe498c700222ae2a8eba86
Jul 21 13:45:10.868392 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:10.868345    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5747f97549-lrm92"]
Jul 21 13:45:10.871802 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:45:10.871771    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b6a97cd_7c40_40f9_8280_18fdd3e5f38c.slice/crio-e4b6c8a71bfa2e091f3e808bbd65052982b641dec66fdd837a5c1b295133cc56 WatchSource:0}: Error finding container e4b6c8a71bfa2e091f3e808bbd65052982b641dec66fdd837a5c1b295133cc56: Status 404 returned error can't find the container with id e4b6c8a71bfa2e091f3e808bbd65052982b641dec66fdd837a5c1b295133cc56
Jul 21 13:45:11.737370 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:11.737333    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6d8f8dcd-w78hr" event={"ID":"115bee6d-6782-47b2-b262-7e6723c2c87b","Type":"ContainerStarted","Data":"731ce48254586553ca1e306c002288edaa94074487b79ce59fe7db02d082d881"}
Jul 21 13:45:11.739044 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:11.739015    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc8bd98db-nfdhb" event={"ID":"9d1d867b-be67-4448-9f50-2ae1ea35100d","Type":"ContainerStarted","Data":"bca1c712cdea0c527f475cd00d6e13790d200186e5fe498c700222ae2a8eba86"}
Jul 21 13:45:11.742295 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:11.742268    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5747f97549-lrm92" event={"ID":"7b6a97cd-7c40-40f9-8280-18fdd3e5f38c","Type":"ContainerStarted","Data":"bb628cd0df68757923315583a9fbe802a7dd26b83bfcb60ccbd45449ea57f171"}
Jul 21 13:45:11.742389 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:11.742320    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5747f97549-lrm92" event={"ID":"7b6a97cd-7c40-40f9-8280-18fdd3e5f38c","Type":"ContainerStarted","Data":"e4b6c8a71bfa2e091f3e808bbd65052982b641dec66fdd837a5c1b295133cc56"}
Jul 21 13:45:11.747534 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:11.744115    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5747f97549-lrm92"
Jul 21 13:45:11.771172 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:11.770900    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5747f97549-lrm92" podStartSLOduration=1.770864985 podStartE2EDuration="1.770864985s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:45:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:45:11.768673496 +0000 UTC m=+58.728962059" watchObservedRunningTime="2026-07-21 13:45:11.770864985 +0000 UTC m=+58.731153523"
Jul 21 13:45:12.639111 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:12.638829    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zcs5b"
Jul 21 13:45:18.772861 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:18.772777    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s2n8n" event={"ID":"a700e703-22e3-4a59-b70c-c2a489a2a828","Type":"ContainerStarted","Data":"9c20142ee0b12325ef130eb249748523721ac8b2d56350fa1b527e9fb39d07e8"}
Jul 21 13:45:18.774542 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:18.774508    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c65bb4b58-57mcd" event={"ID":"95f7fc78-8b7f-44ca-ae95-a9a3acc6edbb","Type":"ContainerStarted","Data":"e0755a8f8cee019c9c45f359e452dad72c0a4ff42cf6e3373a6460e0225b65c7"}
Jul 21 13:45:18.774938 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:18.774914    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c65bb4b58-57mcd"
Jul 21 13:45:18.776309 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:18.776289    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c65bb4b58-57mcd"
Jul 21 13:45:18.778102 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:18.777865    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6d8f8dcd-w78hr" event={"ID":"115bee6d-6782-47b2-b262-7e6723c2c87b","Type":"ContainerStarted","Data":"e196a896cc6b3f370778e56688af31b196a9d49cebc81eb3f7978658fc5c5f15"}
Jul 21 13:45:18.778102 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:18.777928    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6d8f8dcd-w78hr" event={"ID":"115bee6d-6782-47b2-b262-7e6723c2c87b","Type":"ContainerStarted","Data":"c2e93deb58e18156af8149cd9fe87b4e32c87597e0ebd3e052d7f0669d624775"}
Jul 21 13:45:18.778102 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:18.777948    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6d8f8dcd-w78hr" event={"ID":"115bee6d-6782-47b2-b262-7e6723c2c87b","Type":"ContainerStarted","Data":"44c6f6a98e23e364577d50c312766a35d53fd57b22b353e09e81e14d9fd763ad"}
Jul 21 13:45:18.780755 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:18.780725    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc8bd98db-nfdhb" event={"ID":"9d1d867b-be67-4448-9f50-2ae1ea35100d","Type":"ContainerStarted","Data":"4edc01076344e38029c93eadb7be2fec99ca73e5d4add6c0ff8607685153e0eb"}
Jul 21 13:45:18.791868 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:18.791816    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c65bb4b58-57mcd" podStartSLOduration=1.272162608 podStartE2EDuration="8.791801914s" totalImagesPullingTime="7.519639306s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:45:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:45:18.791059156 +0000 UTC m=+65.751347714" watchObservedRunningTime="2026-07-21 13:45:18.791801914 +0000 UTC m=+65.752090471"
Jul 21 13:45:18.807798 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:18.807745    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6bc8bd98db-nfdhb" podStartSLOduration=1.343891456 podStartE2EDuration="8.807723858s" totalImagesPullingTime="7.463832402s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:45:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:45:18.80670359 +0000 UTC m=+65.766992148" watchObservedRunningTime="2026-07-21 13:45:18.807723858 +0000 UTC m=+65.768012415"
Jul 21 13:45:18.843148 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:18.843086    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-f6d8f8dcd-w78hr" podStartSLOduration=1.346717415 podStartE2EDuration="8.843071958s" totalImagesPullingTime="7.496354543s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:45:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:45:18.842062169 +0000 UTC m=+65.802350748" watchObservedRunningTime="2026-07-21 13:45:18.843071958 +0000 UTC m=+65.803360516"
Jul 21 13:45:19.188074 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:19.188035    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9780c38e-e96c-47a2-bd63-283bac4f8f59-metrics-certs\") pod \"network-metrics-daemon-lskgb\" (UID: \"9780c38e-e96c-47a2-bd63-283bac4f8f59\") " pod="openshift-multus/network-metrics-daemon-lskgb"
Jul 21 13:45:19.188276 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:19.188142    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913-original-pull-secret\") pod \"global-pull-secret-syncer-cg5rh\" (UID: \"f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913\") " pod="kube-system/global-pull-secret-syncer-cg5rh"
Jul 21 13:45:19.190858 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:19.190834    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 21 13:45:19.190979 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:19.190891    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 21 13:45:19.201552 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:19.201527    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913-original-pull-secret\") pod \"global-pull-secret-syncer-cg5rh\" (UID: \"f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913\") " pod="kube-system/global-pull-secret-syncer-cg5rh"
Jul 21 13:45:19.201679 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:19.201610    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9780c38e-e96c-47a2-bd63-283bac4f8f59-metrics-certs\") pod \"network-metrics-daemon-lskgb\" (UID: \"9780c38e-e96c-47a2-bd63-283bac4f8f59\") " pod="openshift-multus/network-metrics-daemon-lskgb"
Jul 21 13:45:19.289108 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:19.289066    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqq4t\" (UniqueName: \"kubernetes.io/projected/18e5c194-5162-4442-98b5-61bfaf0b8f66-kube-api-access-rqq4t\") pod \"network-check-target-8f7c2\" (UID: \"18e5c194-5162-4442-98b5-61bfaf0b8f66\") " pod="openshift-network-diagnostics/network-check-target-8f7c2"
Jul 21 13:45:19.291806 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:19.291776    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 21 13:45:19.302091 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:19.302062    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 21 13:45:19.312791 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:19.312765    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqq4t\" (UniqueName: \"kubernetes.io/projected/18e5c194-5162-4442-98b5-61bfaf0b8f66-kube-api-access-rqq4t\") pod \"network-check-target-8f7c2\" (UID: \"18e5c194-5162-4442-98b5-61bfaf0b8f66\") " pod="openshift-network-diagnostics/network-check-target-8f7c2"
Jul 21 13:45:19.411116 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:19.411084    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pt8sb\""
Jul 21 13:45:19.418471 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:19.418444    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cg5rh"
Jul 21 13:45:19.419519 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:19.419495    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lskgb"
Jul 21 13:45:19.427396 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:19.426991    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-29gkd\""
Jul 21 13:45:19.435127 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:19.435085    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8f7c2"
Jul 21 13:45:19.588306 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:19.588218    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cg5rh"]
Jul 21 13:45:19.597964 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:45:19.597930    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7e1e1d1_eab3_43fb_ad47_cb6b5fd5b913.slice/crio-3ad7a923d859781af2fb4f4ce2ff6880c77777ca233666fff566018953ac0855 WatchSource:0}: Error finding container 3ad7a923d859781af2fb4f4ce2ff6880c77777ca233666fff566018953ac0855: Status 404 returned error can't find the container with id 3ad7a923d859781af2fb4f4ce2ff6880c77777ca233666fff566018953ac0855
Jul 21 13:45:19.607470 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:19.607445    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lskgb"]
Jul 21 13:45:19.610926 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:45:19.610871    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9780c38e_e96c_47a2_bd63_283bac4f8f59.slice/crio-0466ecf900317b468df513705ae571238701504f32a8022a5ecbdbf2cbdbc891 WatchSource:0}: Error finding container 0466ecf900317b468df513705ae571238701504f32a8022a5ecbdbf2cbdbc891: Status 404 returned error can't find the container with id 0466ecf900317b468df513705ae571238701504f32a8022a5ecbdbf2cbdbc891
Jul 21 13:45:19.627319 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:19.627291    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8f7c2"]
Jul 21 13:45:19.630404 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:45:19.630376    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18e5c194_5162_4442_98b5_61bfaf0b8f66.slice/crio-6f8ec9671ce1f7968026249e7d5d2683649e35a7c40f5f1347afa4ad1bbd4d52 WatchSource:0}: Error finding container 6f8ec9671ce1f7968026249e7d5d2683649e35a7c40f5f1347afa4ad1bbd4d52: Status 404 returned error can't find the container with id 6f8ec9671ce1f7968026249e7d5d2683649e35a7c40f5f1347afa4ad1bbd4d52
Jul 21 13:45:19.785130 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:19.785042    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8f7c2" event={"ID":"18e5c194-5162-4442-98b5-61bfaf0b8f66","Type":"ContainerStarted","Data":"6f8ec9671ce1f7968026249e7d5d2683649e35a7c40f5f1347afa4ad1bbd4d52"}
Jul 21 13:45:19.786419 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:19.786389    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lskgb" event={"ID":"9780c38e-e96c-47a2-bd63-283bac4f8f59","Type":"ContainerStarted","Data":"0466ecf900317b468df513705ae571238701504f32a8022a5ecbdbf2cbdbc891"}
Jul 21 13:45:19.787607 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:19.787582    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cg5rh" event={"ID":"f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913","Type":"ContainerStarted","Data":"3ad7a923d859781af2fb4f4ce2ff6880c77777ca233666fff566018953ac0855"}
Jul 21 13:45:20.240429 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:20.240402    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5f7fcdffb9-tfpcs"
Jul 21 13:45:22.802560 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:22.802442    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s2n8n" event={"ID":"a700e703-22e3-4a59-b70c-c2a489a2a828","Type":"ContainerStarted","Data":"0c34d29b890d38a383b074cfe4b5b18095599c025abfe2d55619c64869fe2bf1"}
Jul 21 13:45:22.804721 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:22.804693    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lskgb" event={"ID":"9780c38e-e96c-47a2-bd63-283bac4f8f59","Type":"ContainerStarted","Data":"f3750f18b424493f6acefb27a16781f7217e9113c1fd85b38861c64561ea7c30"}
Jul 21 13:45:22.804859 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:22.804744    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lskgb" event={"ID":"9780c38e-e96c-47a2-bd63-283bac4f8f59","Type":"ContainerStarted","Data":"40d568552d6562502a45be33cd71279cdafbabcb8cd8397395eee187adb3bf59"}
Jul 21 13:45:22.822391 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:22.822339    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-s2n8n" podStartSLOduration=17.426425002 podStartE2EDuration="34.822321611s" totalImagesPullingTime="17.395896609s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 13:44:48 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:45:22.821414395 +0000 UTC m=+69.781702954" watchObservedRunningTime="2026-07-21 13:45:22.822321611 +0000 UTC m=+69.782610167"
Jul 21 13:45:22.838020 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:22.837962    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-lskgb" podStartSLOduration=67.397831746 podStartE2EDuration="1m9.837941339s" totalImagesPullingTime="2.440109593s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:44:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:45:22.837263851 +0000 UTC m=+69.797552409" watchObservedRunningTime="2026-07-21 13:45:22.837941339 +0000 UTC m=+69.798229900"
Jul 21 13:45:26.823387 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:26.823343    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8f7c2" event={"ID":"18e5c194-5162-4442-98b5-61bfaf0b8f66","Type":"ContainerStarted","Data":"c53f6dcd202e79e439deb22dd09ffb342619008e55ed92f97eed22717435161f"}
Jul 21 13:45:26.823837 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:26.823446    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-8f7c2"
Jul 21 13:45:26.824731 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:26.824708    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cg5rh" event={"ID":"f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913","Type":"ContainerStarted","Data":"87b988c4709074bbeb985134d40c561f737266333b0ce38b5dfb3d7135b07aa4"}
Jul 21 13:45:26.840652 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:26.840595    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-8f7c2" podStartSLOduration=67.307979532 podStartE2EDuration="1m13.840582062s" totalImagesPullingTime="6.53260253s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:44:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:45:26.839301555 +0000 UTC m=+73.799590112" watchObservedRunningTime="2026-07-21 13:45:26.840582062 +0000 UTC m=+73.800870610"
Jul 21 13:45:33.756431 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:33.756401    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5747f97549-lrm92"
Jul 21 13:45:33.778415 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:33.778356    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-cg5rh" podStartSLOduration=74.208969266 podStartE2EDuration="1m20.778334621s" totalImagesPullingTime="6.569365355s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 13:44:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:45:26.856548293 +0000 UTC m=+73.816836850" watchObservedRunningTime="2026-07-21 13:45:33.778334621 +0000 UTC m=+80.738623181"
Jul 21 13:45:35.252972 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:35.252817    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5f7fcdffb9-tfpcs" podUID="51aea150-df52-44c2-a4f2-1f03c268f070" containerName="registry" containerID="cri-o://3b933687702e412811e102556eb32180b17b6fe3cafd8c13ec2a1ddc4c2bbdad" gracePeriod=30
Jul 21 13:45:35.495400 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:35.495375    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f7fcdffb9-tfpcs"
Jul 21 13:45:35.619635 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:35.619596    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dkzhr\" (UniqueName: \"kubernetes.io/projected/51aea150-df52-44c2-a4f2-1f03c268f070-kube-api-access-dkzhr\") pod \"51aea150-df52-44c2-a4f2-1f03c268f070\" (UID: \"51aea150-df52-44c2-a4f2-1f03c268f070\") "
Jul 21 13:45:35.619635 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:35.619639    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/51aea150-df52-44c2-a4f2-1f03c268f070-image-registry-private-configuration\") pod \"51aea150-df52-44c2-a4f2-1f03c268f070\" (UID: \"51aea150-df52-44c2-a4f2-1f03c268f070\") "
Jul 21 13:45:35.619870 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:35.619660    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/51aea150-df52-44c2-a4f2-1f03c268f070-trusted-ca\") pod \"51aea150-df52-44c2-a4f2-1f03c268f070\" (UID: \"51aea150-df52-44c2-a4f2-1f03c268f070\") "
Jul 21 13:45:35.619870 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:35.619695    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51aea150-df52-44c2-a4f2-1f03c268f070-bound-sa-token\") pod \"51aea150-df52-44c2-a4f2-1f03c268f070\" (UID: \"51aea150-df52-44c2-a4f2-1f03c268f070\") "
Jul 21 13:45:35.619870 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:35.619716    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/51aea150-df52-44c2-a4f2-1f03c268f070-registry-certificates\") pod \"51aea150-df52-44c2-a4f2-1f03c268f070\" (UID: \"51aea150-df52-44c2-a4f2-1f03c268f070\") "
Jul 21 13:45:35.619870 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:35.619736    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/51aea150-df52-44c2-a4f2-1f03c268f070-installation-pull-secrets\") pod \"51aea150-df52-44c2-a4f2-1f03c268f070\" (UID: \"51aea150-df52-44c2-a4f2-1f03c268f070\") "
Jul 21 13:45:35.619870 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:35.619758    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51aea150-df52-44c2-a4f2-1f03c268f070-registry-tls\") pod \"51aea150-df52-44c2-a4f2-1f03c268f070\" (UID: \"51aea150-df52-44c2-a4f2-1f03c268f070\") "
Jul 21 13:45:35.619870 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:35.619807    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/51aea150-df52-44c2-a4f2-1f03c268f070-ca-trust-extracted\") pod \"51aea150-df52-44c2-a4f2-1f03c268f070\" (UID: \"51aea150-df52-44c2-a4f2-1f03c268f070\") "
Jul 21 13:45:35.620280 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:35.620239    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51aea150-df52-44c2-a4f2-1f03c268f070-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "51aea150-df52-44c2-a4f2-1f03c268f070" (UID: "51aea150-df52-44c2-a4f2-1f03c268f070"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:45:35.620346 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:35.620294    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51aea150-df52-44c2-a4f2-1f03c268f070-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "51aea150-df52-44c2-a4f2-1f03c268f070" (UID: "51aea150-df52-44c2-a4f2-1f03c268f070"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:45:35.622550 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:35.622524    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51aea150-df52-44c2-a4f2-1f03c268f070-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "51aea150-df52-44c2-a4f2-1f03c268f070" (UID: "51aea150-df52-44c2-a4f2-1f03c268f070"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:45:35.622652 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:35.622576    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51aea150-df52-44c2-a4f2-1f03c268f070-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "51aea150-df52-44c2-a4f2-1f03c268f070" (UID: "51aea150-df52-44c2-a4f2-1f03c268f070"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 13:45:35.622652 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:35.622613    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51aea150-df52-44c2-a4f2-1f03c268f070-kube-api-access-dkzhr" (OuterVolumeSpecName: "kube-api-access-dkzhr") pod "51aea150-df52-44c2-a4f2-1f03c268f070" (UID: "51aea150-df52-44c2-a4f2-1f03c268f070"). InnerVolumeSpecName "kube-api-access-dkzhr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:45:35.622750 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:35.622646    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51aea150-df52-44c2-a4f2-1f03c268f070-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "51aea150-df52-44c2-a4f2-1f03c268f070" (UID: "51aea150-df52-44c2-a4f2-1f03c268f070"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 13:45:35.622750 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:35.622674    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51aea150-df52-44c2-a4f2-1f03c268f070-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "51aea150-df52-44c2-a4f2-1f03c268f070" (UID: "51aea150-df52-44c2-a4f2-1f03c268f070"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:45:35.627859 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:35.627834    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51aea150-df52-44c2-a4f2-1f03c268f070-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "51aea150-df52-44c2-a4f2-1f03c268f070" (UID: "51aea150-df52-44c2-a4f2-1f03c268f070"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:45:35.721241 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:35.721206    2578 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/51aea150-df52-44c2-a4f2-1f03c268f070-ca-trust-extracted\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 13:45:35.721241 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:35.721236    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dkzhr\" (UniqueName: \"kubernetes.io/projected/51aea150-df52-44c2-a4f2-1f03c268f070-kube-api-access-dkzhr\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 13:45:35.721241 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:35.721247    2578 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/51aea150-df52-44c2-a4f2-1f03c268f070-image-registry-private-configuration\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 13:45:35.721467 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:35.721256    2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/51aea150-df52-44c2-a4f2-1f03c268f070-trusted-ca\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 13:45:35.721467 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:35.721265    2578 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/51aea150-df52-44c2-a4f2-1f03c268f070-bound-sa-token\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 13:45:35.721467 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:35.721273    2578 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/51aea150-df52-44c2-a4f2-1f03c268f070-registry-certificates\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 13:45:35.721467 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:35.721281    2578 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/51aea150-df52-44c2-a4f2-1f03c268f070-installation-pull-secrets\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 13:45:35.721467 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:35.721289    2578 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/51aea150-df52-44c2-a4f2-1f03c268f070-registry-tls\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 13:45:35.853185 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:35.853150    2578 generic.go:356] "Generic (PLEG): container finished" podID="51aea150-df52-44c2-a4f2-1f03c268f070" containerID="3b933687702e412811e102556eb32180b17b6fe3cafd8c13ec2a1ddc4c2bbdad" exitCode=0
Jul 21 13:45:35.853373 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:35.853194    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f7fcdffb9-tfpcs"
Jul 21 13:45:35.853373 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:35.853236    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f7fcdffb9-tfpcs" event={"ID":"51aea150-df52-44c2-a4f2-1f03c268f070","Type":"ContainerDied","Data":"3b933687702e412811e102556eb32180b17b6fe3cafd8c13ec2a1ddc4c2bbdad"}
Jul 21 13:45:35.853373 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:35.853290    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f7fcdffb9-tfpcs" event={"ID":"51aea150-df52-44c2-a4f2-1f03c268f070","Type":"ContainerDied","Data":"31e9baa0ec76285f4d2c6ca136778be60bf53047258bd2d75bb3379eb6f2de94"}
Jul 21 13:45:35.853373 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:35.853318    2578 scope.go:117] "RemoveContainer" containerID="3b933687702e412811e102556eb32180b17b6fe3cafd8c13ec2a1ddc4c2bbdad"
Jul 21 13:45:35.862528 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:35.862508    2578 scope.go:117] "RemoveContainer" containerID="3b933687702e412811e102556eb32180b17b6fe3cafd8c13ec2a1ddc4c2bbdad"
Jul 21 13:45:35.862795 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:45:35.862773    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b933687702e412811e102556eb32180b17b6fe3cafd8c13ec2a1ddc4c2bbdad\": container with ID starting with 3b933687702e412811e102556eb32180b17b6fe3cafd8c13ec2a1ddc4c2bbdad not found: ID does not exist" containerID="3b933687702e412811e102556eb32180b17b6fe3cafd8c13ec2a1ddc4c2bbdad"
Jul 21 13:45:35.862844 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:35.862806    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b933687702e412811e102556eb32180b17b6fe3cafd8c13ec2a1ddc4c2bbdad"} err="failed to get container status \"3b933687702e412811e102556eb32180b17b6fe3cafd8c13ec2a1ddc4c2bbdad\": rpc error: code = NotFound desc = could not find container \"3b933687702e412811e102556eb32180b17b6fe3cafd8c13ec2a1ddc4c2bbdad\": container with ID starting with 3b933687702e412811e102556eb32180b17b6fe3cafd8c13ec2a1ddc4c2bbdad not found: ID does not exist"
Jul 21 13:45:35.877441 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:35.875497    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5f7fcdffb9-tfpcs"]
Jul 21 13:45:35.878859 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:35.878836    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5f7fcdffb9-tfpcs"]
Jul 21 13:45:37.502656 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:37.502624    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51aea150-df52-44c2-a4f2-1f03c268f070" path="/var/lib/kubelet/pods/51aea150-df52-44c2-a4f2-1f03c268f070/volumes"
Jul 21 13:45:38.707724 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:38.707689    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-jhfnl"]
Jul 21 13:45:38.708125 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:38.707919    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51aea150-df52-44c2-a4f2-1f03c268f070" containerName="registry"
Jul 21 13:45:38.708125 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:38.707943    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="51aea150-df52-44c2-a4f2-1f03c268f070" containerName="registry"
Jul 21 13:45:38.708125 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:38.707997    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="51aea150-df52-44c2-a4f2-1f03c268f070" containerName="registry"
Jul 21 13:45:38.712789 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:38.712765    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jhfnl"
Jul 21 13:45:38.715222 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:38.715199    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 21 13:45:38.717003 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:38.716955    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 21 13:45:38.717348 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:38.717327    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-t6whq\""
Jul 21 13:45:38.717348 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:38.717343    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 21 13:45:38.717348 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:38.717354    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 21 13:45:38.717560 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:38.717388    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 21 13:45:38.718241 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:38.718224    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 21 13:45:38.841766 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:38.841727    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9ac7fe8d-1185-41d3-b185-bb7da6062f5a-node-exporter-tls\") pod \"node-exporter-jhfnl\" (UID: \"9ac7fe8d-1185-41d3-b185-bb7da6062f5a\") " pod="openshift-monitoring/node-exporter-jhfnl"
Jul 21 13:45:38.841766 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:38.841770    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9ac7fe8d-1185-41d3-b185-bb7da6062f5a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jhfnl\" (UID: \"9ac7fe8d-1185-41d3-b185-bb7da6062f5a\") " pod="openshift-monitoring/node-exporter-jhfnl"
Jul 21 13:45:38.842052 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:38.841802    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9ac7fe8d-1185-41d3-b185-bb7da6062f5a-root\") pod \"node-exporter-jhfnl\" (UID: \"9ac7fe8d-1185-41d3-b185-bb7da6062f5a\") " pod="openshift-monitoring/node-exporter-jhfnl"
Jul 21 13:45:38.842052 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:38.841825    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9ac7fe8d-1185-41d3-b185-bb7da6062f5a-sys\") pod \"node-exporter-jhfnl\" (UID: \"9ac7fe8d-1185-41d3-b185-bb7da6062f5a\") " pod="openshift-monitoring/node-exporter-jhfnl"
Jul 21 13:45:38.842052 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:38.841854    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9ac7fe8d-1185-41d3-b185-bb7da6062f5a-node-exporter-textfile\") pod \"node-exporter-jhfnl\" (UID: \"9ac7fe8d-1185-41d3-b185-bb7da6062f5a\") " pod="openshift-monitoring/node-exporter-jhfnl"
Jul 21 13:45:38.842052 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:38.841902    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9ac7fe8d-1185-41d3-b185-bb7da6062f5a-node-exporter-accelerators-collector-config\") pod \"node-exporter-jhfnl\" (UID: \"9ac7fe8d-1185-41d3-b185-bb7da6062f5a\") " pod="openshift-monitoring/node-exporter-jhfnl"
Jul 21 13:45:38.842052 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:38.841928    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ac7fe8d-1185-41d3-b185-bb7da6062f5a-metrics-client-ca\") pod \"node-exporter-jhfnl\" (UID: \"9ac7fe8d-1185-41d3-b185-bb7da6062f5a\") " pod="openshift-monitoring/node-exporter-jhfnl"
Jul 21 13:45:38.842052 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:38.841998    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qf966\" (UniqueName: \"kubernetes.io/projected/9ac7fe8d-1185-41d3-b185-bb7da6062f5a-kube-api-access-qf966\") pod \"node-exporter-jhfnl\" (UID: \"9ac7fe8d-1185-41d3-b185-bb7da6062f5a\") " pod="openshift-monitoring/node-exporter-jhfnl"
Jul 21 13:45:38.842052 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:38.842022    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9ac7fe8d-1185-41d3-b185-bb7da6062f5a-node-exporter-wtmp\") pod \"node-exporter-jhfnl\" (UID: \"9ac7fe8d-1185-41d3-b185-bb7da6062f5a\") " pod="openshift-monitoring/node-exporter-jhfnl"
Jul 21 13:45:38.942847 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:38.942810    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9ac7fe8d-1185-41d3-b185-bb7da6062f5a-node-exporter-tls\") pod \"node-exporter-jhfnl\" (UID: \"9ac7fe8d-1185-41d3-b185-bb7da6062f5a\") " pod="openshift-monitoring/node-exporter-jhfnl"
Jul 21 13:45:38.942847 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:38.942847    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9ac7fe8d-1185-41d3-b185-bb7da6062f5a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jhfnl\" (UID: \"9ac7fe8d-1185-41d3-b185-bb7da6062f5a\") " pod="openshift-monitoring/node-exporter-jhfnl"
Jul 21 13:45:38.943121 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:38.942868    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9ac7fe8d-1185-41d3-b185-bb7da6062f5a-root\") pod \"node-exporter-jhfnl\" (UID: \"9ac7fe8d-1185-41d3-b185-bb7da6062f5a\") " pod="openshift-monitoring/node-exporter-jhfnl"
Jul 21 13:45:38.943121 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:38.942900    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9ac7fe8d-1185-41d3-b185-bb7da6062f5a-sys\") pod \"node-exporter-jhfnl\" (UID: \"9ac7fe8d-1185-41d3-b185-bb7da6062f5a\") " pod="openshift-monitoring/node-exporter-jhfnl"
Jul 21 13:45:38.943121 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:38.942942    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9ac7fe8d-1185-41d3-b185-bb7da6062f5a-sys\") pod \"node-exporter-jhfnl\" (UID: \"9ac7fe8d-1185-41d3-b185-bb7da6062f5a\") " pod="openshift-monitoring/node-exporter-jhfnl"
Jul 21 13:45:38.943121 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:38.942975    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9ac7fe8d-1185-41d3-b185-bb7da6062f5a-root\") pod \"node-exporter-jhfnl\" (UID: \"9ac7fe8d-1185-41d3-b185-bb7da6062f5a\") " pod="openshift-monitoring/node-exporter-jhfnl"
Jul 21 13:45:38.943121 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:38.943018    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9ac7fe8d-1185-41d3-b185-bb7da6062f5a-node-exporter-textfile\") pod \"node-exporter-jhfnl\" (UID: \"9ac7fe8d-1185-41d3-b185-bb7da6062f5a\") " pod="openshift-monitoring/node-exporter-jhfnl"
Jul 21 13:45:38.943121 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:38.943043    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9ac7fe8d-1185-41d3-b185-bb7da6062f5a-node-exporter-accelerators-collector-config\") pod \"node-exporter-jhfnl\" (UID: \"9ac7fe8d-1185-41d3-b185-bb7da6062f5a\") " pod="openshift-monitoring/node-exporter-jhfnl"
Jul 21 13:45:38.943121 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:38.943062    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ac7fe8d-1185-41d3-b185-bb7da6062f5a-metrics-client-ca\") pod \"node-exporter-jhfnl\" (UID: \"9ac7fe8d-1185-41d3-b185-bb7da6062f5a\") " pod="openshift-monitoring/node-exporter-jhfnl"
Jul 21 13:45:38.943121 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:38.943112    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qf966\" (UniqueName: \"kubernetes.io/projected/9ac7fe8d-1185-41d3-b185-bb7da6062f5a-kube-api-access-qf966\") pod \"node-exporter-jhfnl\" (UID: \"9ac7fe8d-1185-41d3-b185-bb7da6062f5a\") " pod="openshift-monitoring/node-exporter-jhfnl"
Jul 21 13:45:38.943507 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:38.943131    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9ac7fe8d-1185-41d3-b185-bb7da6062f5a-node-exporter-wtmp\") pod \"node-exporter-jhfnl\" (UID: \"9ac7fe8d-1185-41d3-b185-bb7da6062f5a\") " pod="openshift-monitoring/node-exporter-jhfnl"
Jul 21 13:45:38.943507 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:38.943279    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9ac7fe8d-1185-41d3-b185-bb7da6062f5a-node-exporter-wtmp\") pod \"node-exporter-jhfnl\" (UID: \"9ac7fe8d-1185-41d3-b185-bb7da6062f5a\") " pod="openshift-monitoring/node-exporter-jhfnl"
Jul 21 13:45:38.943507 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:38.943342    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9ac7fe8d-1185-41d3-b185-bb7da6062f5a-node-exporter-textfile\") pod \"node-exporter-jhfnl\" (UID: \"9ac7fe8d-1185-41d3-b185-bb7da6062f5a\") " pod="openshift-monitoring/node-exporter-jhfnl"
Jul 21 13:45:38.943761 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:38.943741    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9ac7fe8d-1185-41d3-b185-bb7da6062f5a-node-exporter-accelerators-collector-config\") pod \"node-exporter-jhfnl\" (UID: \"9ac7fe8d-1185-41d3-b185-bb7da6062f5a\") " pod="openshift-monitoring/node-exporter-jhfnl"
Jul 21 13:45:38.943853 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:38.943771    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ac7fe8d-1185-41d3-b185-bb7da6062f5a-metrics-client-ca\") pod \"node-exporter-jhfnl\" (UID: \"9ac7fe8d-1185-41d3-b185-bb7da6062f5a\") " pod="openshift-monitoring/node-exporter-jhfnl"
Jul 21 13:45:38.945362 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:38.945343    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9ac7fe8d-1185-41d3-b185-bb7da6062f5a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jhfnl\" (UID: \"9ac7fe8d-1185-41d3-b185-bb7da6062f5a\") " pod="openshift-monitoring/node-exporter-jhfnl"
Jul 21 13:45:38.945450 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:38.945367    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9ac7fe8d-1185-41d3-b185-bb7da6062f5a-node-exporter-tls\") pod \"node-exporter-jhfnl\" (UID: \"9ac7fe8d-1185-41d3-b185-bb7da6062f5a\") " pod="openshift-monitoring/node-exporter-jhfnl"
Jul 21 13:45:38.959562 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:38.959501    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qf966\" (UniqueName: \"kubernetes.io/projected/9ac7fe8d-1185-41d3-b185-bb7da6062f5a-kube-api-access-qf966\") pod \"node-exporter-jhfnl\" (UID: \"9ac7fe8d-1185-41d3-b185-bb7da6062f5a\") " pod="openshift-monitoring/node-exporter-jhfnl"
Jul 21 13:45:39.022053 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:39.022021    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jhfnl"
Jul 21 13:45:39.033494 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:45:39.033468    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ac7fe8d_1185_41d3_b185_bb7da6062f5a.slice/crio-32059b7e52bd0f391260fa8e370d9578af1a4c2554dc0efdcb045c3132efd0c4 WatchSource:0}: Error finding container 32059b7e52bd0f391260fa8e370d9578af1a4c2554dc0efdcb045c3132efd0c4: Status 404 returned error can't find the container with id 32059b7e52bd0f391260fa8e370d9578af1a4c2554dc0efdcb045c3132efd0c4
Jul 21 13:45:39.868581 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:39.868549    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jhfnl" event={"ID":"9ac7fe8d-1185-41d3-b185-bb7da6062f5a","Type":"ContainerStarted","Data":"32059b7e52bd0f391260fa8e370d9578af1a4c2554dc0efdcb045c3132efd0c4"}
Jul 21 13:45:40.873774 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:40.873738    2578 generic.go:356] "Generic (PLEG): container finished" podID="9ac7fe8d-1185-41d3-b185-bb7da6062f5a" containerID="9aaf160957492e6361f6374960d5a676ad70ec4f2402e0b5ba17ebea55b4e381" exitCode=0
Jul 21 13:45:40.874188 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:40.873821    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jhfnl" event={"ID":"9ac7fe8d-1185-41d3-b185-bb7da6062f5a","Type":"ContainerDied","Data":"9aaf160957492e6361f6374960d5a676ad70ec4f2402e0b5ba17ebea55b4e381"}
Jul 21 13:45:41.883254 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:41.883214    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jhfnl" event={"ID":"9ac7fe8d-1185-41d3-b185-bb7da6062f5a","Type":"ContainerStarted","Data":"baefffbc004e2882cee6787e24e07c5d61718213edc4ee25b3011ae17c01d8f6"}
Jul 21 13:45:41.883702 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:41.883287    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jhfnl" event={"ID":"9ac7fe8d-1185-41d3-b185-bb7da6062f5a","Type":"ContainerStarted","Data":"eb8e7e3d2970a7bcbfdedd16eadbfd83f0c018763181a9bcbcebc090d65fda35"}
Jul 21 13:45:41.906812 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:41.906759    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-jhfnl" podStartSLOduration=2.525367841 podStartE2EDuration="3.906744724s" totalImagesPullingTime="1.317960951s" totalInitContainerRuntime="63.415932ms" isStatefulPod=true podCreationTimestamp="2026-07-21 13:45:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:45:41.904894816 +0000 UTC m=+88.865183364" watchObservedRunningTime="2026-07-21 13:45:41.906744724 +0000 UTC m=+88.867033280"
Jul 21 13:45:57.831031 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:45:57.831000    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-8f7c2"
Jul 21 13:49:13.364400 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:13.364374    2578 kubelet.go:1648] "Image garbage collection succeeded"
Jul 21 13:49:25.786488 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:25.786452    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d98d8798d-q9bwx"]
Jul 21 13:49:25.789728 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:25.789713    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-q9bwx"
Jul 21 13:49:25.793165 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:25.793137    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 21 13:49:25.793586 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:25.793562    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 21 13:49:25.793712 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:25.793642    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 21 13:49:25.793790 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:25.793777    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 21 13:49:25.794156 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:25.794142    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-hwcdx\""
Jul 21 13:49:25.805270 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:25.805244    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d98d8798d-q9bwx"]
Jul 21 13:49:25.953194 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:25.953155    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/36f99470-322b-4492-91f4-77e433d30910-manifests\") pod \"opendatahub-operator-controller-manager-5d98d8798d-q9bwx\" (UID: \"36f99470-322b-4492-91f4-77e433d30910\") " pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-q9bwx"
Jul 21 13:49:25.953194 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:25.953192    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/36f99470-322b-4492-91f4-77e433d30910-tmp\") pod \"opendatahub-operator-controller-manager-5d98d8798d-q9bwx\" (UID: \"36f99470-322b-4492-91f4-77e433d30910\") " pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-q9bwx"
Jul 21 13:49:25.953424 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:25.953211    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/36f99470-322b-4492-91f4-77e433d30910-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d98d8798d-q9bwx\" (UID: \"36f99470-322b-4492-91f4-77e433d30910\") " pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-q9bwx"
Jul 21 13:49:25.953424 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:25.953290    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m772z\" (UniqueName: \"kubernetes.io/projected/36f99470-322b-4492-91f4-77e433d30910-kube-api-access-m772z\") pod \"opendatahub-operator-controller-manager-5d98d8798d-q9bwx\" (UID: \"36f99470-322b-4492-91f4-77e433d30910\") " pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-q9bwx"
Jul 21 13:49:25.953424 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:25.953318    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/36f99470-322b-4492-91f4-77e433d30910-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d98d8798d-q9bwx\" (UID: \"36f99470-322b-4492-91f4-77e433d30910\") " pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-q9bwx"
Jul 21 13:49:26.054527 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:26.054433    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m772z\" (UniqueName: \"kubernetes.io/projected/36f99470-322b-4492-91f4-77e433d30910-kube-api-access-m772z\") pod \"opendatahub-operator-controller-manager-5d98d8798d-q9bwx\" (UID: \"36f99470-322b-4492-91f4-77e433d30910\") " pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-q9bwx"
Jul 21 13:49:26.054527 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:26.054481    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/36f99470-322b-4492-91f4-77e433d30910-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d98d8798d-q9bwx\" (UID: \"36f99470-322b-4492-91f4-77e433d30910\") " pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-q9bwx"
Jul 21 13:49:26.054711 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:26.054547    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/36f99470-322b-4492-91f4-77e433d30910-manifests\") pod \"opendatahub-operator-controller-manager-5d98d8798d-q9bwx\" (UID: \"36f99470-322b-4492-91f4-77e433d30910\") " pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-q9bwx"
Jul 21 13:49:26.054711 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:26.054580    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/36f99470-322b-4492-91f4-77e433d30910-tmp\") pod \"opendatahub-operator-controller-manager-5d98d8798d-q9bwx\" (UID: \"36f99470-322b-4492-91f4-77e433d30910\") " pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-q9bwx"
Jul 21 13:49:26.054711 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:26.054606    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/36f99470-322b-4492-91f4-77e433d30910-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d98d8798d-q9bwx\" (UID: \"36f99470-322b-4492-91f4-77e433d30910\") " pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-q9bwx"
Jul 21 13:49:26.055051 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:26.055021    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/36f99470-322b-4492-91f4-77e433d30910-manifests\") pod \"opendatahub-operator-controller-manager-5d98d8798d-q9bwx\" (UID: \"36f99470-322b-4492-91f4-77e433d30910\") " pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-q9bwx"
Jul 21 13:49:26.056937 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:26.056914    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/36f99470-322b-4492-91f4-77e433d30910-tmp\") pod \"opendatahub-operator-controller-manager-5d98d8798d-q9bwx\" (UID: \"36f99470-322b-4492-91f4-77e433d30910\") " pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-q9bwx"
Jul 21 13:49:26.057081 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:26.057064    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/36f99470-322b-4492-91f4-77e433d30910-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d98d8798d-q9bwx\" (UID: \"36f99470-322b-4492-91f4-77e433d30910\") " pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-q9bwx"
Jul 21 13:49:26.057128 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:26.057114    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/36f99470-322b-4492-91f4-77e433d30910-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d98d8798d-q9bwx\" (UID: \"36f99470-322b-4492-91f4-77e433d30910\") " pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-q9bwx"
Jul 21 13:49:26.063659 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:26.063636    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m772z\" (UniqueName: \"kubernetes.io/projected/36f99470-322b-4492-91f4-77e433d30910-kube-api-access-m772z\") pod \"opendatahub-operator-controller-manager-5d98d8798d-q9bwx\" (UID: \"36f99470-322b-4492-91f4-77e433d30910\") " pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-q9bwx"
Jul 21 13:49:26.099479 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:26.099449    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-q9bwx"
Jul 21 13:49:26.244176 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:26.244138    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d98d8798d-q9bwx"]
Jul 21 13:49:26.248921 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:49:26.248897    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36f99470_322b_4492_91f4_77e433d30910.slice/crio-8a0b9420a7cbfca9bae7a46343aa1969dfb3ca38ba265b533424dfb2ea94f9f3 WatchSource:0}: Error finding container 8a0b9420a7cbfca9bae7a46343aa1969dfb3ca38ba265b533424dfb2ea94f9f3: Status 404 returned error can't find the container with id 8a0b9420a7cbfca9bae7a46343aa1969dfb3ca38ba265b533424dfb2ea94f9f3
Jul 21 13:49:26.250613 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:26.250597    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 13:49:26.602131 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:26.602096    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-q9bwx" event={"ID":"36f99470-322b-4492-91f4-77e433d30910","Type":"ContainerStarted","Data":"8a0b9420a7cbfca9bae7a46343aa1969dfb3ca38ba265b533424dfb2ea94f9f3"}
Jul 21 13:49:29.614847 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:29.614807    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-q9bwx" event={"ID":"36f99470-322b-4492-91f4-77e433d30910","Type":"ContainerStarted","Data":"72a8d87e8923a0b6009358a1a0143be0f85fcf6ed49124afa52aa4828eb9010d"}
Jul 21 13:49:31.624510 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:31.624478    2578 generic.go:356] "Generic (PLEG): container finished" podID="36f99470-322b-4492-91f4-77e433d30910" containerID="72a8d87e8923a0b6009358a1a0143be0f85fcf6ed49124afa52aa4828eb9010d" exitCode=0
Jul 21 13:49:31.624891 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:31.624557    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-q9bwx" event={"ID":"36f99470-322b-4492-91f4-77e433d30910","Type":"ContainerDied","Data":"72a8d87e8923a0b6009358a1a0143be0f85fcf6ed49124afa52aa4828eb9010d"}
Jul 21 13:49:32.629648 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:32.629607    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-q9bwx" event={"ID":"36f99470-322b-4492-91f4-77e433d30910","Type":"ContainerStarted","Data":"7a2ea355476d68418f9194a6e0e218121bd7b7cdbc3bef07e8846369447a482a"}
Jul 21 13:49:32.630118 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:32.629855    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-q9bwx"
Jul 21 13:49:32.655363 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:32.655315    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-q9bwx" podStartSLOduration=2.526194113 podStartE2EDuration="7.655298751s" totalImagesPullingTime="4.089591275s" totalInitContainerRuntime="1.039513363s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:49:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:49:32.654745427 +0000 UTC m=+319.615033985" watchObservedRunningTime="2026-07-21 13:49:32.655298751 +0000 UTC m=+319.615587308"
Jul 21 13:49:43.635639 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:43.635605    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5d98d8798d-q9bwx"
Jul 21 13:49:46.116845 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:46.116806    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-n2dg9"]
Jul 21 13:49:46.126602 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:46.126577    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-n2dg9"
Jul 21 13:49:46.129712 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:46.129685    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 21 13:49:46.129847 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:46.129781    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-ns8hq\""
Jul 21 13:49:46.145224 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:46.145195    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-n2dg9"]
Jul 21 13:49:46.201131 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:46.201097    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cb3de6f1-5afe-4ecd-afe1-927dd01151bd-cert\") pod \"odh-model-controller-7998c67dc4-n2dg9\" (UID: \"cb3de6f1-5afe-4ecd-afe1-927dd01151bd\") " pod="opendatahub/odh-model-controller-7998c67dc4-n2dg9"
Jul 21 13:49:46.201131 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:46.201129    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4t84\" (UniqueName: \"kubernetes.io/projected/cb3de6f1-5afe-4ecd-afe1-927dd01151bd-kube-api-access-p4t84\") pod \"odh-model-controller-7998c67dc4-n2dg9\" (UID: \"cb3de6f1-5afe-4ecd-afe1-927dd01151bd\") " pod="opendatahub/odh-model-controller-7998c67dc4-n2dg9"
Jul 21 13:49:46.302167 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:46.302131    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cb3de6f1-5afe-4ecd-afe1-927dd01151bd-cert\") pod \"odh-model-controller-7998c67dc4-n2dg9\" (UID: \"cb3de6f1-5afe-4ecd-afe1-927dd01151bd\") " pod="opendatahub/odh-model-controller-7998c67dc4-n2dg9"
Jul 21 13:49:46.302167 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:46.302166    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4t84\" (UniqueName: \"kubernetes.io/projected/cb3de6f1-5afe-4ecd-afe1-927dd01151bd-kube-api-access-p4t84\") pod \"odh-model-controller-7998c67dc4-n2dg9\" (UID: \"cb3de6f1-5afe-4ecd-afe1-927dd01151bd\") " pod="opendatahub/odh-model-controller-7998c67dc4-n2dg9"
Jul 21 13:49:46.302459 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:49:46.302284    2578 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 21 13:49:46.302459 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:49:46.302361    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb3de6f1-5afe-4ecd-afe1-927dd01151bd-cert podName:cb3de6f1-5afe-4ecd-afe1-927dd01151bd nodeName:}" failed. No retries permitted until 2026-07-21 13:49:46.802340306 +0000 UTC m=+333.762628842 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/cb3de6f1-5afe-4ecd-afe1-927dd01151bd-cert") pod "odh-model-controller-7998c67dc4-n2dg9" (UID: "cb3de6f1-5afe-4ecd-afe1-927dd01151bd") : secret "odh-model-controller-webhook-cert" not found
Jul 21 13:49:46.311418 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:46.311385    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4t84\" (UniqueName: \"kubernetes.io/projected/cb3de6f1-5afe-4ecd-afe1-927dd01151bd-kube-api-access-p4t84\") pod \"odh-model-controller-7998c67dc4-n2dg9\" (UID: \"cb3de6f1-5afe-4ecd-afe1-927dd01151bd\") " pod="opendatahub/odh-model-controller-7998c67dc4-n2dg9"
Jul 21 13:49:46.805112 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:46.805077    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cb3de6f1-5afe-4ecd-afe1-927dd01151bd-cert\") pod \"odh-model-controller-7998c67dc4-n2dg9\" (UID: \"cb3de6f1-5afe-4ecd-afe1-927dd01151bd\") " pod="opendatahub/odh-model-controller-7998c67dc4-n2dg9"
Jul 21 13:49:46.807491 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:46.807464    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cb3de6f1-5afe-4ecd-afe1-927dd01151bd-cert\") pod \"odh-model-controller-7998c67dc4-n2dg9\" (UID: \"cb3de6f1-5afe-4ecd-afe1-927dd01151bd\") " pod="opendatahub/odh-model-controller-7998c67dc4-n2dg9"
Jul 21 13:49:46.819096 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:46.819070    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9zp5z"]
Jul 21 13:49:46.822272 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:46.822256    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-9zp5z"
Jul 21 13:49:46.825075 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:46.825057    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 21 13:49:46.834433 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:46.834409    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9zp5z"]
Jul 21 13:49:46.905622 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:46.905582    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60134e12-b9df-4957-a9bd-fdb9224515f1-tls-certs\") pod \"model-serving-api-54d89bd5fc-9zp5z\" (UID: \"60134e12-b9df-4957-a9bd-fdb9224515f1\") " pod="opendatahub/model-serving-api-54d89bd5fc-9zp5z"
Jul 21 13:49:46.905789 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:46.905629    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m52pw\" (UniqueName: \"kubernetes.io/projected/60134e12-b9df-4957-a9bd-fdb9224515f1-kube-api-access-m52pw\") pod \"model-serving-api-54d89bd5fc-9zp5z\" (UID: \"60134e12-b9df-4957-a9bd-fdb9224515f1\") " pod="opendatahub/model-serving-api-54d89bd5fc-9zp5z"
Jul 21 13:49:47.006697 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:47.006663    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60134e12-b9df-4957-a9bd-fdb9224515f1-tls-certs\") pod \"model-serving-api-54d89bd5fc-9zp5z\" (UID: \"60134e12-b9df-4957-a9bd-fdb9224515f1\") " pod="opendatahub/model-serving-api-54d89bd5fc-9zp5z"
Jul 21 13:49:47.006697 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:47.006705    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m52pw\" (UniqueName: \"kubernetes.io/projected/60134e12-b9df-4957-a9bd-fdb9224515f1-kube-api-access-m52pw\") pod \"model-serving-api-54d89bd5fc-9zp5z\" (UID: \"60134e12-b9df-4957-a9bd-fdb9224515f1\") " pod="opendatahub/model-serving-api-54d89bd5fc-9zp5z"
Jul 21 13:49:47.009150 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:47.009129    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60134e12-b9df-4957-a9bd-fdb9224515f1-tls-certs\") pod \"model-serving-api-54d89bd5fc-9zp5z\" (UID: \"60134e12-b9df-4957-a9bd-fdb9224515f1\") " pod="opendatahub/model-serving-api-54d89bd5fc-9zp5z"
Jul 21 13:49:47.017922 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:47.017897    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m52pw\" (UniqueName: \"kubernetes.io/projected/60134e12-b9df-4957-a9bd-fdb9224515f1-kube-api-access-m52pw\") pod \"model-serving-api-54d89bd5fc-9zp5z\" (UID: \"60134e12-b9df-4957-a9bd-fdb9224515f1\") " pod="opendatahub/model-serving-api-54d89bd5fc-9zp5z"
Jul 21 13:49:47.036287 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:47.036257    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-n2dg9"
Jul 21 13:49:47.132241 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:47.132204    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-9zp5z"
Jul 21 13:49:47.162775 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:47.162748    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-n2dg9"]
Jul 21 13:49:47.164938 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:49:47.164908    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb3de6f1_5afe_4ecd_afe1_927dd01151bd.slice/crio-6917d767ea3b226f1815e3c7e539b0aeed253bd4ab293a7e17fd179c32acdf4c WatchSource:0}: Error finding container 6917d767ea3b226f1815e3c7e539b0aeed253bd4ab293a7e17fd179c32acdf4c: Status 404 returned error can't find the container with id 6917d767ea3b226f1815e3c7e539b0aeed253bd4ab293a7e17fd179c32acdf4c
Jul 21 13:49:47.259521 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:47.259487    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9zp5z"]
Jul 21 13:49:47.262085 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:49:47.262057    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60134e12_b9df_4957_a9bd_fdb9224515f1.slice/crio-5fdee6bd797476b4a79584aa1b71d81e2ec0bd2edb31087b3de85133a9583e0f WatchSource:0}: Error finding container 5fdee6bd797476b4a79584aa1b71d81e2ec0bd2edb31087b3de85133a9583e0f: Status 404 returned error can't find the container with id 5fdee6bd797476b4a79584aa1b71d81e2ec0bd2edb31087b3de85133a9583e0f
Jul 21 13:49:47.679528 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:47.679489    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-9zp5z" event={"ID":"60134e12-b9df-4957-a9bd-fdb9224515f1","Type":"ContainerStarted","Data":"5fdee6bd797476b4a79584aa1b71d81e2ec0bd2edb31087b3de85133a9583e0f"}
Jul 21 13:49:47.680545 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:47.680521    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-n2dg9" event={"ID":"cb3de6f1-5afe-4ecd-afe1-927dd01151bd","Type":"ContainerStarted","Data":"6917d767ea3b226f1815e3c7e539b0aeed253bd4ab293a7e17fd179c32acdf4c"}
Jul 21 13:49:50.694197 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:50.694157    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-n2dg9" event={"ID":"cb3de6f1-5afe-4ecd-afe1-927dd01151bd","Type":"ContainerStarted","Data":"2810f828af30b69ffc2afdbb317ad0d4f1ca480c36fafdafecf9adf36c366f39"}
Jul 21 13:49:50.694660 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:50.694255    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-n2dg9"
Jul 21 13:49:50.695512 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:50.695488    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-9zp5z" event={"ID":"60134e12-b9df-4957-a9bd-fdb9224515f1","Type":"ContainerStarted","Data":"81a8bd2453f95591e349c920b9219d99e3357e9b02fdb82b150677dded33b695"}
Jul 21 13:49:50.695626 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:50.695610    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-9zp5z"
Jul 21 13:49:50.711336 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:50.711277    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-n2dg9" podStartSLOduration=1.821833652 podStartE2EDuration="4.711261965s" totalImagesPullingTime="2.889428313s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:49:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:49:50.710481723 +0000 UTC m=+337.670770292" watchObservedRunningTime="2026-07-21 13:49:50.711261965 +0000 UTC m=+337.671550559"
Jul 21 13:49:50.727740 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:50.727689    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-9zp5z" podStartSLOduration=1.9819220579999999 podStartE2EDuration="4.727674668s" totalImagesPullingTime="2.74575261s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:49:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:49:50.726743062 +0000 UTC m=+337.687031619" watchObservedRunningTime="2026-07-21 13:49:50.727674668 +0000 UTC m=+337.687963225"
Jul 21 13:49:55.888460 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:55.888421    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-984j4"]
Jul 21 13:49:55.894869 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:55.894839    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-984j4"
Jul 21 13:49:55.897968 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:55.897948    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 21 13:49:55.898121 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:55.898103    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-j5zh9\""
Jul 21 13:49:55.901824 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:55.901801    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xwfks"]
Jul 21 13:49:55.905107 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:55.905085    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xwfks"
Jul 21 13:49:55.905473 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:55.905447    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-984j4"]
Jul 21 13:49:55.907573 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:55.907551    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-6bx24\""
Jul 21 13:49:55.907573 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:55.907564    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 21 13:49:55.917246 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:55.917222    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xwfks"]
Jul 21 13:49:55.977781 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:55.977743    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0ab4a773-bbc8-480e-82bd-188315ea18da-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xwfks\" (UID: \"0ab4a773-bbc8-480e-82bd-188315ea18da\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xwfks"
Jul 21 13:49:55.978004 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:55.977792    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxqnj\" (UniqueName: \"kubernetes.io/projected/9f04a128-24bd-4084-a222-c6716333f9fd-kube-api-access-dxqnj\") pod \"kserve-controller-manager-d6f4b65b8-984j4\" (UID: \"9f04a128-24bd-4084-a222-c6716333f9fd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-984j4"
Jul 21 13:49:55.978004 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:55.977942    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9f04a128-24bd-4084-a222-c6716333f9fd-cert\") pod \"kserve-controller-manager-d6f4b65b8-984j4\" (UID: \"9f04a128-24bd-4084-a222-c6716333f9fd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-984j4"
Jul 21 13:49:55.978004 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:55.977976    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6ctf\" (UniqueName: \"kubernetes.io/projected/0ab4a773-bbc8-480e-82bd-188315ea18da-kube-api-access-q6ctf\") pod \"llmisvc-controller-manager-777c6bd7d5-xwfks\" (UID: \"0ab4a773-bbc8-480e-82bd-188315ea18da\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xwfks"
Jul 21 13:49:56.078665 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:56.078632    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9f04a128-24bd-4084-a222-c6716333f9fd-cert\") pod \"kserve-controller-manager-d6f4b65b8-984j4\" (UID: \"9f04a128-24bd-4084-a222-c6716333f9fd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-984j4"
Jul 21 13:49:56.078665 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:56.078668    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6ctf\" (UniqueName: \"kubernetes.io/projected/0ab4a773-bbc8-480e-82bd-188315ea18da-kube-api-access-q6ctf\") pod \"llmisvc-controller-manager-777c6bd7d5-xwfks\" (UID: \"0ab4a773-bbc8-480e-82bd-188315ea18da\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xwfks"
Jul 21 13:49:56.078867 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:56.078687    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0ab4a773-bbc8-480e-82bd-188315ea18da-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xwfks\" (UID: \"0ab4a773-bbc8-480e-82bd-188315ea18da\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xwfks"
Jul 21 13:49:56.078867 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:56.078716    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxqnj\" (UniqueName: \"kubernetes.io/projected/9f04a128-24bd-4084-a222-c6716333f9fd-kube-api-access-dxqnj\") pod \"kserve-controller-manager-d6f4b65b8-984j4\" (UID: \"9f04a128-24bd-4084-a222-c6716333f9fd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-984j4"
Jul 21 13:49:56.081111 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:56.081089    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9f04a128-24bd-4084-a222-c6716333f9fd-cert\") pod \"kserve-controller-manager-d6f4b65b8-984j4\" (UID: \"9f04a128-24bd-4084-a222-c6716333f9fd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-984j4"
Jul 21 13:49:56.081271 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:56.081248    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0ab4a773-bbc8-480e-82bd-188315ea18da-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-xwfks\" (UID: \"0ab4a773-bbc8-480e-82bd-188315ea18da\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xwfks"
Jul 21 13:49:56.093793 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:56.093766    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6ctf\" (UniqueName: \"kubernetes.io/projected/0ab4a773-bbc8-480e-82bd-188315ea18da-kube-api-access-q6ctf\") pod \"llmisvc-controller-manager-777c6bd7d5-xwfks\" (UID: \"0ab4a773-bbc8-480e-82bd-188315ea18da\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xwfks"
Jul 21 13:49:56.097521 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:56.097496    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxqnj\" (UniqueName: \"kubernetes.io/projected/9f04a128-24bd-4084-a222-c6716333f9fd-kube-api-access-dxqnj\") pod \"kserve-controller-manager-d6f4b65b8-984j4\" (UID: \"9f04a128-24bd-4084-a222-c6716333f9fd\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-984j4"
Jul 21 13:49:56.205798 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:56.205693    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-984j4"
Jul 21 13:49:56.213990 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:56.213967    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xwfks"
Jul 21 13:49:56.364356 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:56.364318    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-984j4"]
Jul 21 13:49:56.367100 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:49:56.367070    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f04a128_24bd_4084_a222_c6716333f9fd.slice/crio-f9fcd0842dd1f31ebdb569163157c0b3466180c7d2fcdf29f420dd6ffd67d073 WatchSource:0}: Error finding container f9fcd0842dd1f31ebdb569163157c0b3466180c7d2fcdf29f420dd6ffd67d073: Status 404 returned error can't find the container with id f9fcd0842dd1f31ebdb569163157c0b3466180c7d2fcdf29f420dd6ffd67d073
Jul 21 13:49:56.376114 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:56.376088    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-xwfks"]
Jul 21 13:49:56.379052 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:49:56.379028    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0ab4a773_bbc8_480e_82bd_188315ea18da.slice/crio-0804c525fa0fc28d4b101eef8b85673def658de5deae80175a846b9b6826538a WatchSource:0}: Error finding container 0804c525fa0fc28d4b101eef8b85673def658de5deae80175a846b9b6826538a: Status 404 returned error can't find the container with id 0804c525fa0fc28d4b101eef8b85673def658de5deae80175a846b9b6826538a
Jul 21 13:49:56.446135 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:56.446100    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-55c6b9f88d-d87sk"]
Jul 21 13:49:56.450378 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:56.450361    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-55c6b9f88d-d87sk"
Jul 21 13:49:56.453143 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:56.453121    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 21 13:49:56.453443 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:56.453417    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 21 13:49:56.453698 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:56.453680    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 21 13:49:56.454179 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:56.454161    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 21 13:49:56.475084 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:56.475055    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-55c6b9f88d-d87sk"]
Jul 21 13:49:56.481986 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:56.481962    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/07798b24-2c54-4815-bcd4-f8ad1134712c-tmp\") pod \"kube-auth-proxy-55c6b9f88d-d87sk\" (UID: \"07798b24-2c54-4815-bcd4-f8ad1134712c\") " pod="openshift-ingress/kube-auth-proxy-55c6b9f88d-d87sk"
Jul 21 13:49:56.482097 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:56.481994    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07798b24-2c54-4815-bcd4-f8ad1134712c-tls-certs\") pod \"kube-auth-proxy-55c6b9f88d-d87sk\" (UID: \"07798b24-2c54-4815-bcd4-f8ad1134712c\") " pod="openshift-ingress/kube-auth-proxy-55c6b9f88d-d87sk"
Jul 21 13:49:56.482097 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:56.482040    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdrnf\" (UniqueName: \"kubernetes.io/projected/07798b24-2c54-4815-bcd4-f8ad1134712c-kube-api-access-wdrnf\") pod \"kube-auth-proxy-55c6b9f88d-d87sk\" (UID: \"07798b24-2c54-4815-bcd4-f8ad1134712c\") " pod="openshift-ingress/kube-auth-proxy-55c6b9f88d-d87sk"
Jul 21 13:49:56.582698 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:56.582661    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/07798b24-2c54-4815-bcd4-f8ad1134712c-tmp\") pod \"kube-auth-proxy-55c6b9f88d-d87sk\" (UID: \"07798b24-2c54-4815-bcd4-f8ad1134712c\") " pod="openshift-ingress/kube-auth-proxy-55c6b9f88d-d87sk"
Jul 21 13:49:56.582698 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:56.582700    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07798b24-2c54-4815-bcd4-f8ad1134712c-tls-certs\") pod \"kube-auth-proxy-55c6b9f88d-d87sk\" (UID: \"07798b24-2c54-4815-bcd4-f8ad1134712c\") " pod="openshift-ingress/kube-auth-proxy-55c6b9f88d-d87sk"
Jul 21 13:49:56.582935 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:56.582741    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdrnf\" (UniqueName: \"kubernetes.io/projected/07798b24-2c54-4815-bcd4-f8ad1134712c-kube-api-access-wdrnf\") pod \"kube-auth-proxy-55c6b9f88d-d87sk\" (UID: \"07798b24-2c54-4815-bcd4-f8ad1134712c\") " pod="openshift-ingress/kube-auth-proxy-55c6b9f88d-d87sk"
Jul 21 13:49:56.582935 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:49:56.582866    2578 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 21 13:49:56.583002 ip-10-0-132-245 kubenswrapper[2578]: E0721 13:49:56.582966    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07798b24-2c54-4815-bcd4-f8ad1134712c-tls-certs podName:07798b24-2c54-4815-bcd4-f8ad1134712c nodeName:}" failed. No retries permitted until 2026-07-21 13:49:57.082943055 +0000 UTC m=+344.043231612 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/07798b24-2c54-4815-bcd4-f8ad1134712c-tls-certs") pod "kube-auth-proxy-55c6b9f88d-d87sk" (UID: "07798b24-2c54-4815-bcd4-f8ad1134712c") : secret "kube-auth-proxy-tls" not found
Jul 21 13:49:56.584923 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:56.584905    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/07798b24-2c54-4815-bcd4-f8ad1134712c-tmp\") pod \"kube-auth-proxy-55c6b9f88d-d87sk\" (UID: \"07798b24-2c54-4815-bcd4-f8ad1134712c\") " pod="openshift-ingress/kube-auth-proxy-55c6b9f88d-d87sk"
Jul 21 13:49:56.595256 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:56.595231    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdrnf\" (UniqueName: \"kubernetes.io/projected/07798b24-2c54-4815-bcd4-f8ad1134712c-kube-api-access-wdrnf\") pod \"kube-auth-proxy-55c6b9f88d-d87sk\" (UID: \"07798b24-2c54-4815-bcd4-f8ad1134712c\") " pod="openshift-ingress/kube-auth-proxy-55c6b9f88d-d87sk"
Jul 21 13:49:56.715465 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:56.715380    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xwfks" event={"ID":"0ab4a773-bbc8-480e-82bd-188315ea18da","Type":"ContainerStarted","Data":"0804c525fa0fc28d4b101eef8b85673def658de5deae80175a846b9b6826538a"}
Jul 21 13:49:56.716566 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:56.716531    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-984j4" event={"ID":"9f04a128-24bd-4084-a222-c6716333f9fd","Type":"ContainerStarted","Data":"f9fcd0842dd1f31ebdb569163157c0b3466180c7d2fcdf29f420dd6ffd67d073"}
Jul 21 13:49:57.086287 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:57.086245    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07798b24-2c54-4815-bcd4-f8ad1134712c-tls-certs\") pod \"kube-auth-proxy-55c6b9f88d-d87sk\" (UID: \"07798b24-2c54-4815-bcd4-f8ad1134712c\") " pod="openshift-ingress/kube-auth-proxy-55c6b9f88d-d87sk"
Jul 21 13:49:57.088725 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:57.088702    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07798b24-2c54-4815-bcd4-f8ad1134712c-tls-certs\") pod \"kube-auth-proxy-55c6b9f88d-d87sk\" (UID: \"07798b24-2c54-4815-bcd4-f8ad1134712c\") " pod="openshift-ingress/kube-auth-proxy-55c6b9f88d-d87sk"
Jul 21 13:49:57.360564 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:57.360491    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-55c6b9f88d-d87sk"
Jul 21 13:49:57.881768 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:57.881744    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-55c6b9f88d-d87sk"]
Jul 21 13:49:57.884619 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:49:57.884587    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07798b24_2c54_4815_bcd4_f8ad1134712c.slice/crio-5130beaa26cd3f123bc202a6d98a2b37f2fb60a477c38ec96e234cc51644376e WatchSource:0}: Error finding container 5130beaa26cd3f123bc202a6d98a2b37f2fb60a477c38ec96e234cc51644376e: Status 404 returned error can't find the container with id 5130beaa26cd3f123bc202a6d98a2b37f2fb60a477c38ec96e234cc51644376e
Jul 21 13:49:58.466584 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:58.466167    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d86x5"]
Jul 21 13:49:58.469774 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:58.469675    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d86x5"
Jul 21 13:49:58.473641 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:58.473618    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 21 13:49:58.474071 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:58.474052    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 21 13:49:58.474164 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:58.474105    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 21 13:49:58.474446 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:58.474428    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-xv6bs\""
Jul 21 13:49:58.474840 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:58.474805    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 21 13:49:58.495369 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:58.495339    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d86x5"]
Jul 21 13:49:58.497014 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:58.496984    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/289a3b8d-9d4a-462a-ab7b-1b38b94fa868-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-d86x5\" (UID: \"289a3b8d-9d4a-462a-ab7b-1b38b94fa868\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d86x5"
Jul 21 13:49:58.497134 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:58.497031    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/289a3b8d-9d4a-462a-ab7b-1b38b94fa868-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-d86x5\" (UID: \"289a3b8d-9d4a-462a-ab7b-1b38b94fa868\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d86x5"
Jul 21 13:49:58.497134 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:58.497065    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8t4t\" (UniqueName: \"kubernetes.io/projected/289a3b8d-9d4a-462a-ab7b-1b38b94fa868-kube-api-access-p8t4t\") pod \"istiod-openshift-gateway-94bb8fbfd-d86x5\" (UID: \"289a3b8d-9d4a-462a-ab7b-1b38b94fa868\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d86x5"
Jul 21 13:49:58.497134 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:58.497095    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/289a3b8d-9d4a-462a-ab7b-1b38b94fa868-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-d86x5\" (UID: \"289a3b8d-9d4a-462a-ab7b-1b38b94fa868\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d86x5"
Jul 21 13:49:58.497296 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:58.497200    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/289a3b8d-9d4a-462a-ab7b-1b38b94fa868-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-d86x5\" (UID: \"289a3b8d-9d4a-462a-ab7b-1b38b94fa868\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d86x5"
Jul 21 13:49:58.497296 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:58.497244    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/289a3b8d-9d4a-462a-ab7b-1b38b94fa868-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-d86x5\" (UID: \"289a3b8d-9d4a-462a-ab7b-1b38b94fa868\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d86x5"
Jul 21 13:49:58.497296 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:58.497273    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/289a3b8d-9d4a-462a-ab7b-1b38b94fa868-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-d86x5\" (UID: \"289a3b8d-9d4a-462a-ab7b-1b38b94fa868\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d86x5"
Jul 21 13:49:58.599367 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:58.598342    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/289a3b8d-9d4a-462a-ab7b-1b38b94fa868-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-d86x5\" (UID: \"289a3b8d-9d4a-462a-ab7b-1b38b94fa868\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d86x5"
Jul 21 13:49:58.599367 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:58.598393    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/289a3b8d-9d4a-462a-ab7b-1b38b94fa868-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-d86x5\" (UID: \"289a3b8d-9d4a-462a-ab7b-1b38b94fa868\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d86x5"
Jul 21 13:49:58.599367 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:58.598430    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8t4t\" (UniqueName: \"kubernetes.io/projected/289a3b8d-9d4a-462a-ab7b-1b38b94fa868-kube-api-access-p8t4t\") pod \"istiod-openshift-gateway-94bb8fbfd-d86x5\" (UID: \"289a3b8d-9d4a-462a-ab7b-1b38b94fa868\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d86x5"
Jul 21 13:49:58.599367 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:58.598465    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/289a3b8d-9d4a-462a-ab7b-1b38b94fa868-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-d86x5\" (UID: \"289a3b8d-9d4a-462a-ab7b-1b38b94fa868\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d86x5"
Jul 21 13:49:58.599367 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:58.598514    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/289a3b8d-9d4a-462a-ab7b-1b38b94fa868-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-d86x5\" (UID: \"289a3b8d-9d4a-462a-ab7b-1b38b94fa868\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d86x5"
Jul 21 13:49:58.599367 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:58.598578    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/289a3b8d-9d4a-462a-ab7b-1b38b94fa868-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-d86x5\" (UID: \"289a3b8d-9d4a-462a-ab7b-1b38b94fa868\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d86x5"
Jul 21 13:49:58.599367 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:58.598614    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/289a3b8d-9d4a-462a-ab7b-1b38b94fa868-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-d86x5\" (UID: \"289a3b8d-9d4a-462a-ab7b-1b38b94fa868\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d86x5"
Jul 21 13:49:58.599367 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:58.599318    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/289a3b8d-9d4a-462a-ab7b-1b38b94fa868-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-d86x5\" (UID: \"289a3b8d-9d4a-462a-ab7b-1b38b94fa868\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d86x5"
Jul 21 13:49:58.601484 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:58.601423    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/289a3b8d-9d4a-462a-ab7b-1b38b94fa868-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-d86x5\" (UID: \"289a3b8d-9d4a-462a-ab7b-1b38b94fa868\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d86x5"
Jul 21 13:49:58.601814 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:58.601789    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/289a3b8d-9d4a-462a-ab7b-1b38b94fa868-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-d86x5\" (UID: \"289a3b8d-9d4a-462a-ab7b-1b38b94fa868\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d86x5"
Jul 21 13:49:58.602312 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:58.602273    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/289a3b8d-9d4a-462a-ab7b-1b38b94fa868-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-d86x5\" (UID: \"289a3b8d-9d4a-462a-ab7b-1b38b94fa868\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d86x5"
Jul 21 13:49:58.602312 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:58.602293    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/289a3b8d-9d4a-462a-ab7b-1b38b94fa868-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-d86x5\" (UID: \"289a3b8d-9d4a-462a-ab7b-1b38b94fa868\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d86x5"
Jul 21 13:49:58.619109 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:58.619047    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8t4t\" (UniqueName: \"kubernetes.io/projected/289a3b8d-9d4a-462a-ab7b-1b38b94fa868-kube-api-access-p8t4t\") pod \"istiod-openshift-gateway-94bb8fbfd-d86x5\" (UID: \"289a3b8d-9d4a-462a-ab7b-1b38b94fa868\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d86x5"
Jul 21 13:49:58.621533 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:58.621485    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/289a3b8d-9d4a-462a-ab7b-1b38b94fa868-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-d86x5\" (UID: \"289a3b8d-9d4a-462a-ab7b-1b38b94fa868\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d86x5"
Jul 21 13:49:58.732142 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:58.732044    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-55c6b9f88d-d87sk" event={"ID":"07798b24-2c54-4815-bcd4-f8ad1134712c","Type":"ContainerStarted","Data":"5130beaa26cd3f123bc202a6d98a2b37f2fb60a477c38ec96e234cc51644376e"}
Jul 21 13:49:58.783437 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:49:58.783394    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d86x5"
Jul 21 13:50:00.596520 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:50:00.596493    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d86x5"]
Jul 21 13:50:00.609074 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:50:00.609039    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod289a3b8d_9d4a_462a_ab7b_1b38b94fa868.slice/crio-9991156220def3903dbdac2d2a0181ec55d9b125d5cd213e7c7b421b154295a1 WatchSource:0}: Error finding container 9991156220def3903dbdac2d2a0181ec55d9b125d5cd213e7c7b421b154295a1: Status 404 returned error can't find the container with id 9991156220def3903dbdac2d2a0181ec55d9b125d5cd213e7c7b421b154295a1
Jul 21 13:50:00.741925 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:50:00.741824    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xwfks" event={"ID":"0ab4a773-bbc8-480e-82bd-188315ea18da","Type":"ContainerStarted","Data":"fc547e003d37e03e3dab86510b83776fb4352da1b93a279a0dd45368fc66013f"}
Jul 21 13:50:00.742069 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:50:00.741950    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xwfks"
Jul 21 13:50:00.743331 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:50:00.743305    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-984j4" event={"ID":"9f04a128-24bd-4084-a222-c6716333f9fd","Type":"ContainerStarted","Data":"4b5efe0d195b78642dbfe6747cbeb9182a66fcb3c7b9397fddb4987533d8a0f6"}
Jul 21 13:50:00.743438 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:50:00.743350    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-984j4"
Jul 21 13:50:00.744401 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:50:00.744379    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d86x5" event={"ID":"289a3b8d-9d4a-462a-ab7b-1b38b94fa868","Type":"ContainerStarted","Data":"9991156220def3903dbdac2d2a0181ec55d9b125d5cd213e7c7b421b154295a1"}
Jul 21 13:50:00.745802 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:50:00.745781    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-55c6b9f88d-d87sk" event={"ID":"07798b24-2c54-4815-bcd4-f8ad1134712c","Type":"ContainerStarted","Data":"c8f2f4f443245cc2be35d8f35183002468be87a9194d7656278c304ad9efbc8c"}
Jul 21 13:50:00.762028 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:50:00.761981    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xwfks" podStartSLOduration=1.6832850000000001 podStartE2EDuration="5.761962281s" totalImagesPullingTime="4.078677281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:49:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:50:00.7603565 +0000 UTC m=+347.720645058" watchObservedRunningTime="2026-07-21 13:50:00.761962281 +0000 UTC m=+347.722250816"
Jul 21 13:50:00.778354 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:50:00.778299    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-984j4" podStartSLOduration=1.690788296 podStartE2EDuration="5.77828444s" totalImagesPullingTime="4.087496144s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:49:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:50:00.777653313 +0000 UTC m=+347.737941897" watchObservedRunningTime="2026-07-21 13:50:00.77828444 +0000 UTC m=+347.738572996"
Jul 21 13:50:00.796625 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:50:00.796571    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-55c6b9f88d-d87sk" podStartSLOduration=2.220007371 podStartE2EDuration="4.79655688s" totalImagesPullingTime="2.576549509s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:49:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:50:00.795015357 +0000 UTC m=+347.755303913" watchObservedRunningTime="2026-07-21 13:50:00.79655688 +0000 UTC m=+347.756845437"
Jul 21 13:50:01.701572 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:50:01.701530    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-n2dg9"
Jul 21 13:50:01.703731 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:50:01.703707    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-9zp5z"
Jul 21 13:50:02.866026 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:50:02.865986    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 13:50:02.866370 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:50:02.866072    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 13:50:03.762714 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:50:03.762682    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d86x5" event={"ID":"289a3b8d-9d4a-462a-ab7b-1b38b94fa868","Type":"ContainerStarted","Data":"2fa302769194d54ed286d0ac44b952def5de9350a1459a4f39711f299473a8e2"}
Jul 21 13:50:03.762938 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:50:03.762889    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d86x5"
Jul 21 13:50:03.764546 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:50:03.764512    2578 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-d86x5 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 21 13:50:03.764768 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:50:03.764567    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d86x5" podUID="289a3b8d-9d4a-462a-ab7b-1b38b94fa868" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 21 13:50:03.795265 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:50:03.795200    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d86x5" podStartSLOduration=3.54104331 podStartE2EDuration="5.795176563s" totalImagesPullingTime="2.254133253s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:49:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:50:03.793641046 +0000 UTC m=+350.753929604" watchObservedRunningTime="2026-07-21 13:50:03.795176563 +0000 UTC m=+350.755465121"
Jul 21 13:50:04.767709 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:50:04.767685    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-d86x5"
Jul 21 13:50:31.754989 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:50:31.754906    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-xwfks"
Jul 21 13:50:31.757797 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:50:31.757775    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-984j4"
Jul 21 13:51:23.908439 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:51:23.908403    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-wchpf"]
Jul 21 13:51:23.911751 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:51:23.911731    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wchpf"
Jul 21 13:51:23.914497 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:51:23.914475    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 21 13:51:23.915494 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:51:23.915474    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 21 13:51:23.915494 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:51:23.915489    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-b7mgv\""
Jul 21 13:51:23.930186 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:51:23.930160    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-wchpf"]
Jul 21 13:51:23.965244 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:51:23.965217    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5vcr\" (UniqueName: \"kubernetes.io/projected/f07a2504-208f-43ed-9254-fc2a366ca555-kube-api-access-h5vcr\") pod \"authorino-operator-75cc8d5cb7-wchpf\" (UID: \"f07a2504-208f-43ed-9254-fc2a366ca555\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-wchpf"
Jul 21 13:51:24.066199 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:51:24.066166    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5vcr\" (UniqueName: \"kubernetes.io/projected/f07a2504-208f-43ed-9254-fc2a366ca555-kube-api-access-h5vcr\") pod \"authorino-operator-75cc8d5cb7-wchpf\" (UID: \"f07a2504-208f-43ed-9254-fc2a366ca555\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-wchpf"
Jul 21 13:51:24.080502 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:51:24.080470    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5vcr\" (UniqueName: \"kubernetes.io/projected/f07a2504-208f-43ed-9254-fc2a366ca555-kube-api-access-h5vcr\") pod \"authorino-operator-75cc8d5cb7-wchpf\" (UID: \"f07a2504-208f-43ed-9254-fc2a366ca555\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-wchpf"
Jul 21 13:51:24.221208 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:51:24.221109    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wchpf"
Jul 21 13:51:24.367618 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:51:24.367595    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-wchpf"]
Jul 21 13:51:24.370197 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:51:24.370174    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf07a2504_208f_43ed_9254_fc2a366ca555.slice/crio-31a39c0424f14b174ba66e4e5b6c4b9643f93a724d582b00030c64f062bc41b4 WatchSource:0}: Error finding container 31a39c0424f14b174ba66e4e5b6c4b9643f93a724d582b00030c64f062bc41b4: Status 404 returned error can't find the container with id 31a39c0424f14b174ba66e4e5b6c4b9643f93a724d582b00030c64f062bc41b4
Jul 21 13:51:25.075845 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:51:25.075800    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wchpf" event={"ID":"f07a2504-208f-43ed-9254-fc2a366ca555","Type":"ContainerStarted","Data":"31a39c0424f14b174ba66e4e5b6c4b9643f93a724d582b00030c64f062bc41b4"}
Jul 21 13:51:27.087707 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:51:27.087672    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wchpf" event={"ID":"f07a2504-208f-43ed-9254-fc2a366ca555","Type":"ContainerStarted","Data":"66bfcf52a7d268677ab03ccef4d73eefdb7f79a2f38c6b3585c1f6b8eeb5ce36"}
Jul 21 13:51:27.088197 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:51:27.087799    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wchpf"
Jul 21 13:51:27.115645 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:51:27.115297    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wchpf" podStartSLOduration=1.916782462 podStartE2EDuration="4.115277187s" totalImagesPullingTime="2.198494725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:51:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:51:27.113715232 +0000 UTC m=+434.074003788" watchObservedRunningTime="2026-07-21 13:51:27.115277187 +0000 UTC m=+434.075565745"
Jul 21 13:51:38.094040 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:51:38.093965    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wchpf"
Jul 21 13:52:21.690966 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:52:21.690930    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 13:52:21.694128 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:52:21.694111    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-klnkt"
Jul 21 13:52:21.696910 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:52:21.696871    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-sq6kq\""
Jul 21 13:52:21.696910 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:52:21.696900    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 21 13:52:21.704983 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:52:21.704960    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 13:52:21.721238 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:52:21.721211    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 13:52:21.731121 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:52:21.731097    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zg7zz\" (UniqueName: \"kubernetes.io/projected/f622bbe8-bd7a-4d61-8137-c2f66ddcc5b3-kube-api-access-zg7zz\") pod \"limitador-limitador-54d6d988d5-klnkt\" (UID: \"f622bbe8-bd7a-4d61-8137-c2f66ddcc5b3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-klnkt"
Jul 21 13:52:21.731195 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:52:21.731165    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f622bbe8-bd7a-4d61-8137-c2f66ddcc5b3-config-file\") pod \"limitador-limitador-54d6d988d5-klnkt\" (UID: \"f622bbe8-bd7a-4d61-8137-c2f66ddcc5b3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-klnkt"
Jul 21 13:52:21.832068 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:52:21.832030    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f622bbe8-bd7a-4d61-8137-c2f66ddcc5b3-config-file\") pod \"limitador-limitador-54d6d988d5-klnkt\" (UID: \"f622bbe8-bd7a-4d61-8137-c2f66ddcc5b3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-klnkt"
Jul 21 13:52:21.832238 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:52:21.832088    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zg7zz\" (UniqueName: \"kubernetes.io/projected/f622bbe8-bd7a-4d61-8137-c2f66ddcc5b3-kube-api-access-zg7zz\") pod \"limitador-limitador-54d6d988d5-klnkt\" (UID: \"f622bbe8-bd7a-4d61-8137-c2f66ddcc5b3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-klnkt"
Jul 21 13:52:21.832696 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:52:21.832674    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f622bbe8-bd7a-4d61-8137-c2f66ddcc5b3-config-file\") pod \"limitador-limitador-54d6d988d5-klnkt\" (UID: \"f622bbe8-bd7a-4d61-8137-c2f66ddcc5b3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-klnkt"
Jul 21 13:52:21.841149 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:52:21.841124    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zg7zz\" (UniqueName: \"kubernetes.io/projected/f622bbe8-bd7a-4d61-8137-c2f66ddcc5b3-kube-api-access-zg7zz\") pod \"limitador-limitador-54d6d988d5-klnkt\" (UID: \"f622bbe8-bd7a-4d61-8137-c2f66ddcc5b3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-klnkt"
Jul 21 13:52:22.005187 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:52:22.005110    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-klnkt"
Jul 21 13:52:22.144721 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:52:22.144696    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 13:52:22.147317 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:52:22.147290    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf622bbe8_bd7a_4d61_8137_c2f66ddcc5b3.slice/crio-df69cde0e5ad6f61ca8641cc8abbcbd97a1cdc936267b8dcfed3dc0e281ce554 WatchSource:0}: Error finding container df69cde0e5ad6f61ca8641cc8abbcbd97a1cdc936267b8dcfed3dc0e281ce554: Status 404 returned error can't find the container with id df69cde0e5ad6f61ca8641cc8abbcbd97a1cdc936267b8dcfed3dc0e281ce554
Jul 21 13:52:22.312046 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:52:22.311965    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-klnkt" event={"ID":"f622bbe8-bd7a-4d61-8137-c2f66ddcc5b3","Type":"ContainerStarted","Data":"df69cde0e5ad6f61ca8641cc8abbcbd97a1cdc936267b8dcfed3dc0e281ce554"}
Jul 21 13:52:25.326145 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:52:25.326109    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-klnkt" event={"ID":"f622bbe8-bd7a-4d61-8137-c2f66ddcc5b3","Type":"ContainerStarted","Data":"93ff73689f101e5a2cd916dd68a34672c709b6fcbd9e1d44be57557cae110bc5"}
Jul 21 13:52:25.326543 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:52:25.326255    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-klnkt"
Jul 21 13:52:25.349195 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:52:25.349124    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-klnkt" podStartSLOduration=1.684885459 podStartE2EDuration="4.349108234s" totalImagesPullingTime="2.664222775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:52:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:52:25.348986183 +0000 UTC m=+492.309274741" watchObservedRunningTime="2026-07-21 13:52:25.349108234 +0000 UTC m=+492.309396791"
Jul 21 13:52:36.332249 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:52:36.332218    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-klnkt"
Jul 21 13:53:40.970543 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:53:40.970468    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 13:54:03.606101 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:54:03.606064    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 13:54:16.572255 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:54:16.572214    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 13:54:21.176051 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:54:21.176013    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 13:54:32.577786 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:54:32.577750    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 13:54:45.981572 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:54:45.981543    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-c2bzr"]
Jul 21 13:54:45.984650 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:54:45.984628    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2bzr"
Jul 21 13:54:45.988004 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:54:45.987980    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 21 13:54:45.988931 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:54:45.988915    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 21 13:54:45.989043 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:54:45.988930    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 21 13:54:45.989043 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:54:45.988931    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-gr2dk\""
Jul 21 13:54:46.000957 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:54:46.000932    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-c2bzr"]
Jul 21 13:54:46.085468 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:54:46.085431    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c0ea429-ddaf-479e-9122-56272792462c-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c2bzr\" (UID: \"9c0ea429-ddaf-479e-9122-56272792462c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2bzr"
Jul 21 13:54:46.085642 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:54:46.085481    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c0ea429-ddaf-479e-9122-56272792462c-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c2bzr\" (UID: \"9c0ea429-ddaf-479e-9122-56272792462c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2bzr"
Jul 21 13:54:46.085642 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:54:46.085501    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c0ea429-ddaf-479e-9122-56272792462c-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c2bzr\" (UID: \"9c0ea429-ddaf-479e-9122-56272792462c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2bzr"
Jul 21 13:54:46.085642 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:54:46.085564    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c0ea429-ddaf-479e-9122-56272792462c-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c2bzr\" (UID: \"9c0ea429-ddaf-479e-9122-56272792462c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2bzr"
Jul 21 13:54:46.085744 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:54:46.085642    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c0ea429-ddaf-479e-9122-56272792462c-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c2bzr\" (UID: \"9c0ea429-ddaf-479e-9122-56272792462c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2bzr"
Jul 21 13:54:46.085744 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:54:46.085685    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hj55j\" (UniqueName: \"kubernetes.io/projected/9c0ea429-ddaf-479e-9122-56272792462c-kube-api-access-hj55j\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c2bzr\" (UID: \"9c0ea429-ddaf-479e-9122-56272792462c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2bzr"
Jul 21 13:54:46.186437 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:54:46.186401    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c0ea429-ddaf-479e-9122-56272792462c-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c2bzr\" (UID: \"9c0ea429-ddaf-479e-9122-56272792462c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2bzr"
Jul 21 13:54:46.186437 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:54:46.186444    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c0ea429-ddaf-479e-9122-56272792462c-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c2bzr\" (UID: \"9c0ea429-ddaf-479e-9122-56272792462c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2bzr"
Jul 21 13:54:46.186657 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:54:46.186540    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hj55j\" (UniqueName: \"kubernetes.io/projected/9c0ea429-ddaf-479e-9122-56272792462c-kube-api-access-hj55j\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c2bzr\" (UID: \"9c0ea429-ddaf-479e-9122-56272792462c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2bzr"
Jul 21 13:54:46.186657 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:54:46.186575    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c0ea429-ddaf-479e-9122-56272792462c-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c2bzr\" (UID: \"9c0ea429-ddaf-479e-9122-56272792462c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2bzr"
Jul 21 13:54:46.186657 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:54:46.186603    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c0ea429-ddaf-479e-9122-56272792462c-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c2bzr\" (UID: \"9c0ea429-ddaf-479e-9122-56272792462c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2bzr"
Jul 21 13:54:46.186657 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:54:46.186619    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c0ea429-ddaf-479e-9122-56272792462c-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c2bzr\" (UID: \"9c0ea429-ddaf-479e-9122-56272792462c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2bzr"
Jul 21 13:54:46.186862 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:54:46.186795    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c0ea429-ddaf-479e-9122-56272792462c-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c2bzr\" (UID: \"9c0ea429-ddaf-479e-9122-56272792462c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2bzr"
Jul 21 13:54:46.186967 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:54:46.186949    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c0ea429-ddaf-479e-9122-56272792462c-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c2bzr\" (UID: \"9c0ea429-ddaf-479e-9122-56272792462c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2bzr"
Jul 21 13:54:46.187022 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:54:46.187003    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c0ea429-ddaf-479e-9122-56272792462c-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c2bzr\" (UID: \"9c0ea429-ddaf-479e-9122-56272792462c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2bzr"
Jul 21 13:54:46.188713 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:54:46.188688    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c0ea429-ddaf-479e-9122-56272792462c-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c2bzr\" (UID: \"9c0ea429-ddaf-479e-9122-56272792462c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2bzr"
Jul 21 13:54:46.189052 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:54:46.189035    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c0ea429-ddaf-479e-9122-56272792462c-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c2bzr\" (UID: \"9c0ea429-ddaf-479e-9122-56272792462c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2bzr"
Jul 21 13:54:46.195143 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:54:46.195119    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hj55j\" (UniqueName: \"kubernetes.io/projected/9c0ea429-ddaf-479e-9122-56272792462c-kube-api-access-hj55j\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-c2bzr\" (UID: \"9c0ea429-ddaf-479e-9122-56272792462c\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2bzr"
Jul 21 13:54:46.295085 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:54:46.294987    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2bzr"
Jul 21 13:54:46.429291 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:54:46.429268    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-c2bzr"]
Jul 21 13:54:46.431536 ip-10-0-132-245 kubenswrapper[2578]: W0721 13:54:46.431504    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c0ea429_ddaf_479e_9122_56272792462c.slice/crio-5e0902ad9388dfa9ace4216f2496ee7398081954226d093a196a2de3da24f697 WatchSource:0}: Error finding container 5e0902ad9388dfa9ace4216f2496ee7398081954226d093a196a2de3da24f697: Status 404 returned error can't find the container with id 5e0902ad9388dfa9ace4216f2496ee7398081954226d093a196a2de3da24f697
Jul 21 13:54:46.432816 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:54:46.432799    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 13:54:46.884548 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:54:46.884504    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2bzr" event={"ID":"9c0ea429-ddaf-479e-9122-56272792462c","Type":"ContainerStarted","Data":"5e0902ad9388dfa9ace4216f2496ee7398081954226d093a196a2de3da24f697"}
Jul 21 13:54:48.891360 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:54:48.891324    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 13:54:49.898050 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:54:49.898013    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2bzr" event={"ID":"9c0ea429-ddaf-479e-9122-56272792462c","Type":"ContainerStarted","Data":"f7a3402622d728d72e4eee542e4c46fa8c1bd6616431f731c6c77b459519fa1c"}
Jul 21 13:54:49.922117 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:54:49.922067    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2bzr" podStartSLOduration=2.177401798 podStartE2EDuration="4.922050239s" totalImagesPullingTime="2.744648441s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:54:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:54:49.919972168 +0000 UTC m=+636.880260726" watchObservedRunningTime="2026-07-21 13:54:49.922050239 +0000 UTC m=+636.882338795"
Jul 21 13:54:56.295128 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:54:56.295091    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2bzr"
Jul 21 13:54:56.295535 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:54:56.295161    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2bzr"
Jul 21 13:54:56.307374 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:54:56.307344    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2bzr"
Jul 21 13:54:56.938063 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:54:56.938036    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-c2bzr"
Jul 21 13:55:02.181632 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:55:02.181599    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 13:56:24.584633 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:56:24.584541    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 13:56:34.773757 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:56:34.773723    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 13:56:44.973586 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:56:44.973557    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 13:58:28.841956 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:58:28.841920    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 13:58:39.864058 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:58:39.864023    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 13:59:14.663226 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:59:14.663184    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 13:59:25.061178 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:59:25.061139    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 13:59:34.266406 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:59:34.266373    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 13:59:45.374710 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:59:45.374673    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 13:59:57.468887 ip-10-0-132-245 kubenswrapper[2578]: I0721 13:59:57.468850    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:00:00.145397 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:00:00.145352    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744040-7j47p"]
Jul 21 14:00:00.148544 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:00:00.148529    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744040-7j47p"
Jul 21 14:00:00.150998 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:00:00.150979    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5t9ph\""
Jul 21 14:00:00.157686 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:00:00.157668    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744040-7j47p"
Jul 21 14:00:00.166997 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:00:00.166973    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744040-7j47p"]
Jul 21 14:00:00.285090 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:00:00.285065    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744040-7j47p"]
Jul 21 14:00:00.287651 ip-10-0-132-245 kubenswrapper[2578]: W0721 14:00:00.287620    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd0d8136_4bbb_4e33_995a_ba522d813b6b.slice/crio-f008882201b250a745b7eec786c5fc2d0955062ddde180e8c9c8712ca75674bd WatchSource:0}: Error finding container f008882201b250a745b7eec786c5fc2d0955062ddde180e8c9c8712ca75674bd: Status 404 returned error can't find the container with id f008882201b250a745b7eec786c5fc2d0955062ddde180e8c9c8712ca75674bd
Jul 21 14:00:00.289394 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:00:00.289378    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 14:00:01.149720 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:00:01.149688    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744040-7j47p" event={"ID":"fd0d8136-4bbb-4e33-995a-ba522d813b6b","Type":"ContainerStarted","Data":"f008882201b250a745b7eec786c5fc2d0955062ddde180e8c9c8712ca75674bd"}
Jul 21 14:00:05.166698 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:00:05.166659    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744040-7j47p" event={"ID":"fd0d8136-4bbb-4e33-995a-ba522d813b6b","Type":"ContainerStarted","Data":"ebb8657269c693987fc2c8b5fcec6dbc2e11634905dd4975c4e228abfbd5b789"}
Jul 21 14:00:05.182706 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:00:05.182652    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744040-7j47p" podStartSLOduration=0.69349991 podStartE2EDuration="5.182638823s" totalImagesPullingTime="4.489138913s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:00:05.180755736 +0000 UTC m=+952.141044296" watchObservedRunningTime="2026-07-21 14:00:05.182638823 +0000 UTC m=+952.142927358"
Jul 21 14:00:05.749696 ip-10-0-132-245 kubenswrapper[2578]: E0721 14:00:05.749662    2578 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd0d8136_4bbb_4e33_995a_ba522d813b6b.slice/crio-conmon-ebb8657269c693987fc2c8b5fcec6dbc2e11634905dd4975c4e228abfbd5b789.scope\": RecentStats: unable to find data in memory cache]"
Jul 21 14:00:06.172153 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:00:06.172115    2578 generic.go:356] "Generic (PLEG): container finished" podID="fd0d8136-4bbb-4e33-995a-ba522d813b6b" containerID="ebb8657269c693987fc2c8b5fcec6dbc2e11634905dd4975c4e228abfbd5b789" exitCode=0
Jul 21 14:00:06.172508 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:00:06.172174    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744040-7j47p" event={"ID":"fd0d8136-4bbb-4e33-995a-ba522d813b6b","Type":"ContainerDied","Data":"ebb8657269c693987fc2c8b5fcec6dbc2e11634905dd4975c4e228abfbd5b789"}
Jul 21 14:00:06.869783 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:00:06.869741    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:00:07.302350 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:00:07.302329    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744040-7j47p"
Jul 21 14:00:08.186589 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:00:08.186556    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744040-7j47p"
Jul 21 14:00:08.186589 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:00:08.186573    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744040-7j47p" event={"ID":"fd0d8136-4bbb-4e33-995a-ba522d813b6b","Type":"ContainerDied","Data":"f008882201b250a745b7eec786c5fc2d0955062ddde180e8c9c8712ca75674bd"}
Jul 21 14:00:08.186809 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:00:08.186647    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f008882201b250a745b7eec786c5fc2d0955062ddde180e8c9c8712ca75674bd"
Jul 21 14:00:19.559028 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:00:19.558991    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:00:30.068346 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:00:30.068311    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:00:51.567892 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:00:51.567851    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:01:03.064862 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:01:03.064825    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:01:26.466996 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:01:26.466960    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:01:37.869129 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:01:37.869093    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:01:53.381680 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:01:53.381638    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:02:11.977828 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:02:11.977747    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:02:27.967664 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:02:27.967628    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:04:20.778975 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:04:20.778942    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:04:33.570721 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:04:33.570684    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:04:42.967258 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:04:42.967226    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:04:57.874317 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:04:57.874280    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:05:06.969606 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:05:06.969523    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:05:17.962798 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:05:17.962739    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:05:26.820780 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:05:26.820743    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:05:38.692055 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:05:38.692024    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:05:49.258914 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:05:49.258863    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:05:57.791297 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:05:57.791257    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:06:09.080696 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:06:09.080661    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:06:34.787655 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:06:34.787624    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:06:43.467851 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:06:43.467814    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:06:58.157168 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:06:58.157133    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:07:09.769167 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:07:09.769125    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:07:26.970752 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:07:26.970716    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:07:41.661170 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:07:41.661127    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:07:51.276464 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:07:51.276425    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:08:03.764790 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:08:03.764759    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:08:13.865574 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:08:13.865542    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:08:24.267610 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:08:24.267575    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:08:32.270803 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:08:32.270766    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:08:44.172754 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:08:44.172720    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:08:55.776317 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:08:55.776280    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:09:13.870015 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:09:13.869982    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:09:31.575255 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:09:31.575216    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:09:39.681394 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:09:39.681318    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:09:51.178253 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:09:51.178218    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:10:09.799733 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:10:09.799702    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:10:22.082859 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:10:22.082820    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:10:34.466574 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:10:34.466534    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:10:49.125518 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:10:49.125483    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-47dbec60-958d8cd8c-cwgsk"]
Jul 21 14:10:49.125930 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:10:49.125911    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd0d8136-4bbb-4e33-995a-ba522d813b6b" containerName="cleanup"
Jul 21 14:10:49.125930 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:10:49.125927    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd0d8136-4bbb-4e33-995a-ba522d813b6b" containerName="cleanup"
Jul 21 14:10:49.126063 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:10:49.125981    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd0d8136-4bbb-4e33-995a-ba522d813b6b" containerName="cleanup"
Jul 21 14:10:49.129263 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:10:49.129240    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-47dbec60-958d8cd8c-cwgsk"
Jul 21 14:10:49.131901 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:10:49.131854    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 14:10:49.132015 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:10:49.131903    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-nhfmq\""
Jul 21 14:10:49.132947 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:10:49.132930    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 14:10:49.133012 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:10:49.132958    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-47dbec60-serving-cert\""
Jul 21 14:10:49.141863 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:10:49.141840    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-47dbec60-958d8cd8c-cwgsk"]
Jul 21 14:10:49.264468 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:10:49.264436    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b13db41f-1aba-4282-a213-b4f315948ed5-maas-api-tls\") pod \"maas-api-e2e-ait-47dbec60-958d8cd8c-cwgsk\" (UID: \"b13db41f-1aba-4282-a213-b4f315948ed5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-47dbec60-958d8cd8c-cwgsk"
Jul 21 14:10:49.264660 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:10:49.264484    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b13db41f-1aba-4282-a213-b4f315948ed5-openshift-service-ca\") pod \"maas-api-e2e-ait-47dbec60-958d8cd8c-cwgsk\" (UID: \"b13db41f-1aba-4282-a213-b4f315948ed5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-47dbec60-958d8cd8c-cwgsk"
Jul 21 14:10:49.264660 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:10:49.264502    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgq2b\" (UniqueName: \"kubernetes.io/projected/b13db41f-1aba-4282-a213-b4f315948ed5-kube-api-access-rgq2b\") pod \"maas-api-e2e-ait-47dbec60-958d8cd8c-cwgsk\" (UID: \"b13db41f-1aba-4282-a213-b4f315948ed5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-47dbec60-958d8cd8c-cwgsk"
Jul 21 14:10:49.365297 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:10:49.365256    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b13db41f-1aba-4282-a213-b4f315948ed5-maas-api-tls\") pod \"maas-api-e2e-ait-47dbec60-958d8cd8c-cwgsk\" (UID: \"b13db41f-1aba-4282-a213-b4f315948ed5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-47dbec60-958d8cd8c-cwgsk"
Jul 21 14:10:49.365484 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:10:49.365330    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b13db41f-1aba-4282-a213-b4f315948ed5-openshift-service-ca\") pod \"maas-api-e2e-ait-47dbec60-958d8cd8c-cwgsk\" (UID: \"b13db41f-1aba-4282-a213-b4f315948ed5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-47dbec60-958d8cd8c-cwgsk"
Jul 21 14:10:49.365484 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:10:49.365359    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgq2b\" (UniqueName: \"kubernetes.io/projected/b13db41f-1aba-4282-a213-b4f315948ed5-kube-api-access-rgq2b\") pod \"maas-api-e2e-ait-47dbec60-958d8cd8c-cwgsk\" (UID: \"b13db41f-1aba-4282-a213-b4f315948ed5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-47dbec60-958d8cd8c-cwgsk"
Jul 21 14:10:49.366066 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:10:49.366045    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b13db41f-1aba-4282-a213-b4f315948ed5-openshift-service-ca\") pod \"maas-api-e2e-ait-47dbec60-958d8cd8c-cwgsk\" (UID: \"b13db41f-1aba-4282-a213-b4f315948ed5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-47dbec60-958d8cd8c-cwgsk"
Jul 21 14:10:49.367852 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:10:49.367822    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b13db41f-1aba-4282-a213-b4f315948ed5-maas-api-tls\") pod \"maas-api-e2e-ait-47dbec60-958d8cd8c-cwgsk\" (UID: \"b13db41f-1aba-4282-a213-b4f315948ed5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-47dbec60-958d8cd8c-cwgsk"
Jul 21 14:10:49.374512 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:10:49.374490    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgq2b\" (UniqueName: \"kubernetes.io/projected/b13db41f-1aba-4282-a213-b4f315948ed5-kube-api-access-rgq2b\") pod \"maas-api-e2e-ait-47dbec60-958d8cd8c-cwgsk\" (UID: \"b13db41f-1aba-4282-a213-b4f315948ed5\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-47dbec60-958d8cd8c-cwgsk"
Jul 21 14:10:49.439053 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:10:49.438973    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-47dbec60-958d8cd8c-cwgsk"
Jul 21 14:10:49.571618 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:10:49.571591    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-47dbec60-958d8cd8c-cwgsk"]
Jul 21 14:10:49.573126 ip-10-0-132-245 kubenswrapper[2578]: W0721 14:10:49.573104    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb13db41f_1aba_4282_a213_b4f315948ed5.slice/crio-8e09c1a757596bb02f9af8e1bd20802edec5515838c707ad6c403de98d10363a WatchSource:0}: Error finding container 8e09c1a757596bb02f9af8e1bd20802edec5515838c707ad6c403de98d10363a: Status 404 returned error can't find the container with id 8e09c1a757596bb02f9af8e1bd20802edec5515838c707ad6c403de98d10363a
Jul 21 14:10:49.574503 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:10:49.574483    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 14:10:49.753039 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:10:49.752956    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-47dbec60-958d8cd8c-cwgsk" event={"ID":"b13db41f-1aba-4282-a213-b4f315948ed5","Type":"ContainerStarted","Data":"8e09c1a757596bb02f9af8e1bd20802edec5515838c707ad6c403de98d10363a"}
Jul 21 14:10:52.770039 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:10:52.770007    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-47dbec60-958d8cd8c-cwgsk" event={"ID":"b13db41f-1aba-4282-a213-b4f315948ed5","Type":"ContainerStarted","Data":"1fc13fdba36ad73dce2c2b2070e487417e96aefa8e4381fbce7d86d7175025ea"}
Jul 21 14:10:52.770412 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:10:52.770069    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-47dbec60-958d8cd8c-cwgsk"
Jul 21 14:10:52.789294 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:10:52.789248    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-47dbec60-958d8cd8c-cwgsk" podStartSLOduration=1.347762526 podStartE2EDuration="3.789235145s" totalImagesPullingTime="2.441472619s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:10:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:10:52.787605037 +0000 UTC m=+1599.747893591" watchObservedRunningTime="2026-07-21 14:10:52.789235145 +0000 UTC m=+1599.749523702"
Jul 21 14:10:56.366836 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:10:56.366799    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:10:58.779523 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:10:58.779496    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-47dbec60-958d8cd8c-cwgsk"
Jul 21 14:11:14.559087 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:14.559004    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-47dbec60-hjg6q"]
Jul 21 14:11:14.562248 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:14.562231    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-47dbec60-hjg6q"
Jul 21 14:11:14.565114 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:14.565088    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5t9ph\""
Jul 21 14:11:14.570296 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:14.570273    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-47dbec60-hjg6q"]
Jul 21 14:11:14.578138 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:14.578109    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf1144a9-529c-4ecf-8523-d66f9701231d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-47dbec60-hjg6q\" (UID: \"cf1144a9-529c-4ecf-8523-d66f9701231d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-47dbec60-hjg6q"
Jul 21 14:11:14.678703 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:14.678663    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf1144a9-529c-4ecf-8523-d66f9701231d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-47dbec60-hjg6q\" (UID: \"cf1144a9-529c-4ecf-8523-d66f9701231d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-47dbec60-hjg6q"
Jul 21 14:11:14.679306 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:14.679284    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf1144a9-529c-4ecf-8523-d66f9701231d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-47dbec60-hjg6q\" (UID: \"cf1144a9-529c-4ecf-8523-d66f9701231d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-47dbec60-hjg6q"
Jul 21 14:11:14.873139 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:14.873112    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-47dbec60-hjg6q"
Jul 21 14:11:15.003585 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:15.003550    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-47dbec60-hjg6q"]
Jul 21 14:11:15.007021 ip-10-0-132-245 kubenswrapper[2578]: W0721 14:11:15.006993    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf1144a9_529c_4ecf_8523_d66f9701231d.slice/crio-311efe2276f03dcfb0de9b9ac369e69766cca334c6b788117c93a3c6c4d17b32 WatchSource:0}: Error finding container 311efe2276f03dcfb0de9b9ac369e69766cca334c6b788117c93a3c6c4d17b32: Status 404 returned error can't find the container with id 311efe2276f03dcfb0de9b9ac369e69766cca334c6b788117c93a3c6c4d17b32
Jul 21 14:11:15.860786 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:15.860752    2578 generic.go:356] "Generic (PLEG): container finished" podID="cf1144a9-529c-4ecf-8523-d66f9701231d" containerID="d543bf1c07c0c9878d9376f17bbc11bb8220a1251d28fdaee021173b64a3390a" exitCode=0
Jul 21 14:11:15.861139 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:15.860834    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-47dbec60-hjg6q" event={"ID":"cf1144a9-529c-4ecf-8523-d66f9701231d","Type":"ContainerDied","Data":"d543bf1c07c0c9878d9376f17bbc11bb8220a1251d28fdaee021173b64a3390a"}
Jul 21 14:11:15.861139 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:15.860901    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-47dbec60-hjg6q" event={"ID":"cf1144a9-529c-4ecf-8523-d66f9701231d","Type":"ContainerStarted","Data":"311efe2276f03dcfb0de9b9ac369e69766cca334c6b788117c93a3c6c4d17b32"}
Jul 21 14:11:16.992423 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:16.992398    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-47dbec60-hjg6q"
Jul 21 14:11:16.995469 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:16.995449    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf1144a9-529c-4ecf-8523-d66f9701231d-maas-api-service-ca\") pod \"cf1144a9-529c-4ecf-8523-d66f9701231d\" (UID: \"cf1144a9-529c-4ecf-8523-d66f9701231d\") "
Jul 21 14:11:16.995913 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:16.995869    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf1144a9-529c-4ecf-8523-d66f9701231d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cf1144a9-529c-4ecf-8523-d66f9701231d" (UID: "cf1144a9-529c-4ecf-8523-d66f9701231d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:11:17.096069 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:17.096040    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf1144a9-529c-4ecf-8523-d66f9701231d-maas-api-service-ca\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:11:17.870346 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:17.870316    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-47dbec60-hjg6q"
Jul 21 14:11:17.870346 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:17.870339    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-47dbec60-hjg6q" event={"ID":"cf1144a9-529c-4ecf-8523-d66f9701231d","Type":"ContainerDied","Data":"311efe2276f03dcfb0de9b9ac369e69766cca334c6b788117c93a3c6c4d17b32"}
Jul 21 14:11:17.870552 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:17.870387    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="311efe2276f03dcfb0de9b9ac369e69766cca334c6b788117c93a3c6c4d17b32"
Jul 21 14:11:25.158115 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:25.158081    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-47dbec60-958d8cd8c-cwgsk"]
Jul 21 14:11:25.158534 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:25.158302    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-47dbec60-958d8cd8c-cwgsk" podUID="b13db41f-1aba-4282-a213-b4f315948ed5" containerName="maas-api" containerID="cri-o://1fc13fdba36ad73dce2c2b2070e487417e96aefa8e4381fbce7d86d7175025ea" gracePeriod=30
Jul 21 14:11:25.410553 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:25.410495    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-47dbec60-958d8cd8c-cwgsk"
Jul 21 14:11:25.456221 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:25.456192    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b13db41f-1aba-4282-a213-b4f315948ed5-maas-api-tls\") pod \"b13db41f-1aba-4282-a213-b4f315948ed5\" (UID: \"b13db41f-1aba-4282-a213-b4f315948ed5\") "
Jul 21 14:11:25.456395 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:25.456229    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b13db41f-1aba-4282-a213-b4f315948ed5-openshift-service-ca\") pod \"b13db41f-1aba-4282-a213-b4f315948ed5\" (UID: \"b13db41f-1aba-4282-a213-b4f315948ed5\") "
Jul 21 14:11:25.456395 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:25.456283    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rgq2b\" (UniqueName: \"kubernetes.io/projected/b13db41f-1aba-4282-a213-b4f315948ed5-kube-api-access-rgq2b\") pod \"b13db41f-1aba-4282-a213-b4f315948ed5\" (UID: \"b13db41f-1aba-4282-a213-b4f315948ed5\") "
Jul 21 14:11:25.456667 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:25.456641    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b13db41f-1aba-4282-a213-b4f315948ed5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b13db41f-1aba-4282-a213-b4f315948ed5" (UID: "b13db41f-1aba-4282-a213-b4f315948ed5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:11:25.458451 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:25.458425    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b13db41f-1aba-4282-a213-b4f315948ed5-kube-api-access-rgq2b" (OuterVolumeSpecName: "kube-api-access-rgq2b") pod "b13db41f-1aba-4282-a213-b4f315948ed5" (UID: "b13db41f-1aba-4282-a213-b4f315948ed5"). InnerVolumeSpecName "kube-api-access-rgq2b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:11:25.458533 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:25.458435    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b13db41f-1aba-4282-a213-b4f315948ed5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b13db41f-1aba-4282-a213-b4f315948ed5" (UID: "b13db41f-1aba-4282-a213-b4f315948ed5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:11:25.557431 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:25.557391    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b13db41f-1aba-4282-a213-b4f315948ed5-maas-api-tls\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:11:25.557431 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:25.557423    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b13db41f-1aba-4282-a213-b4f315948ed5-openshift-service-ca\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:11:25.557431 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:25.557434    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rgq2b\" (UniqueName: \"kubernetes.io/projected/b13db41f-1aba-4282-a213-b4f315948ed5-kube-api-access-rgq2b\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:11:25.904364 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:25.904332    2578 generic.go:356] "Generic (PLEG): container finished" podID="b13db41f-1aba-4282-a213-b4f315948ed5" containerID="1fc13fdba36ad73dce2c2b2070e487417e96aefa8e4381fbce7d86d7175025ea" exitCode=0
Jul 21 14:11:25.904531 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:25.904388    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-47dbec60-958d8cd8c-cwgsk"
Jul 21 14:11:25.904531 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:25.904421    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-47dbec60-958d8cd8c-cwgsk" event={"ID":"b13db41f-1aba-4282-a213-b4f315948ed5","Type":"ContainerDied","Data":"1fc13fdba36ad73dce2c2b2070e487417e96aefa8e4381fbce7d86d7175025ea"}
Jul 21 14:11:25.904531 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:25.904473    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-47dbec60-958d8cd8c-cwgsk" event={"ID":"b13db41f-1aba-4282-a213-b4f315948ed5","Type":"ContainerDied","Data":"8e09c1a757596bb02f9af8e1bd20802edec5515838c707ad6c403de98d10363a"}
Jul 21 14:11:25.904531 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:25.904495    2578 scope.go:117] "RemoveContainer" containerID="1fc13fdba36ad73dce2c2b2070e487417e96aefa8e4381fbce7d86d7175025ea"
Jul 21 14:11:25.912827 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:25.912804    2578 scope.go:117] "RemoveContainer" containerID="1fc13fdba36ad73dce2c2b2070e487417e96aefa8e4381fbce7d86d7175025ea"
Jul 21 14:11:25.913112 ip-10-0-132-245 kubenswrapper[2578]: E0721 14:11:25.913092    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fc13fdba36ad73dce2c2b2070e487417e96aefa8e4381fbce7d86d7175025ea\": container with ID starting with 1fc13fdba36ad73dce2c2b2070e487417e96aefa8e4381fbce7d86d7175025ea not found: ID does not exist" containerID="1fc13fdba36ad73dce2c2b2070e487417e96aefa8e4381fbce7d86d7175025ea"
Jul 21 14:11:25.913166 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:25.913122    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fc13fdba36ad73dce2c2b2070e487417e96aefa8e4381fbce7d86d7175025ea"} err="failed to get container status \"1fc13fdba36ad73dce2c2b2070e487417e96aefa8e4381fbce7d86d7175025ea\": rpc error: code = NotFound desc = could not find container \"1fc13fdba36ad73dce2c2b2070e487417e96aefa8e4381fbce7d86d7175025ea\": container with ID starting with 1fc13fdba36ad73dce2c2b2070e487417e96aefa8e4381fbce7d86d7175025ea not found: ID does not exist"
Jul 21 14:11:25.920099 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:25.920079    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-47dbec60-958d8cd8c-cwgsk"]
Jul 21 14:11:25.924543 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:25.924509    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-47dbec60-958d8cd8c-cwgsk"]
Jul 21 14:11:27.503505 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:27.503474    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b13db41f-1aba-4282-a213-b4f315948ed5" path="/var/lib/kubelet/pods/b13db41f-1aba-4282-a213-b4f315948ed5/volumes"
Jul 21 14:11:28.073858 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:28.073824    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:11:29.673889 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:29.673844    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-47dbec60-hjg6q"]
Jul 21 14:11:29.676472 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:29.676450    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-47dbec60-hjg6q"]
Jul 21 14:11:31.504601 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:31.504561    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf1144a9-529c-4ecf-8523-d66f9701231d" path="/var/lib/kubelet/pods/cf1144a9-529c-4ecf-8523-d66f9701231d/volumes"
Jul 21 14:11:33.474652 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:33.474620    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-1ebf4d7f-5cb85d585d-fxg94"]
Jul 21 14:11:33.475087 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:33.475071    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf1144a9-529c-4ecf-8523-d66f9701231d" containerName="revoke-keys"
Jul 21 14:11:33.475142 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:33.475091    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf1144a9-529c-4ecf-8523-d66f9701231d" containerName="revoke-keys"
Jul 21 14:11:33.475142 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:33.475110    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b13db41f-1aba-4282-a213-b4f315948ed5" containerName="maas-api"
Jul 21 14:11:33.475142 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:33.475119    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b13db41f-1aba-4282-a213-b4f315948ed5" containerName="maas-api"
Jul 21 14:11:33.475256 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:33.475185    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf1144a9-529c-4ecf-8523-d66f9701231d" containerName="revoke-keys"
Jul 21 14:11:33.475256 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:33.475195    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b13db41f-1aba-4282-a213-b4f315948ed5" containerName="maas-api"
Jul 21 14:11:33.480350 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:33.480331    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1ebf4d7f-5cb85d585d-fxg94"
Jul 21 14:11:33.483504 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:33.483483    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 14:11:33.483504 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:33.483496    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-1ebf4d7f-serving-cert\""
Jul 21 14:11:33.483504 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:33.483500    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 14:11:33.484440 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:33.484425    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-nhfmq\""
Jul 21 14:11:33.492549 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:33.492528    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-1ebf4d7f-5cb85d585d-fxg94"]
Jul 21 14:11:33.532811 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:33.532777    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b77fca8-2794-477a-a8bb-35fb694d6b91-maas-api-tls\") pod \"maas-api-e2e-migrate-1ebf4d7f-5cb85d585d-fxg94\" (UID: \"8b77fca8-2794-477a-a8bb-35fb694d6b91\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1ebf4d7f-5cb85d585d-fxg94"
Jul 21 14:11:33.532811 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:33.532814    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b77fca8-2794-477a-a8bb-35fb694d6b91-openshift-service-ca\") pod \"maas-api-e2e-migrate-1ebf4d7f-5cb85d585d-fxg94\" (UID: \"8b77fca8-2794-477a-a8bb-35fb694d6b91\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1ebf4d7f-5cb85d585d-fxg94"
Jul 21 14:11:33.533051 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:33.532858    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxcmm\" (UniqueName: \"kubernetes.io/projected/8b77fca8-2794-477a-a8bb-35fb694d6b91-kube-api-access-dxcmm\") pod \"maas-api-e2e-migrate-1ebf4d7f-5cb85d585d-fxg94\" (UID: \"8b77fca8-2794-477a-a8bb-35fb694d6b91\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1ebf4d7f-5cb85d585d-fxg94"
Jul 21 14:11:33.634049 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:33.634006    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxcmm\" (UniqueName: \"kubernetes.io/projected/8b77fca8-2794-477a-a8bb-35fb694d6b91-kube-api-access-dxcmm\") pod \"maas-api-e2e-migrate-1ebf4d7f-5cb85d585d-fxg94\" (UID: \"8b77fca8-2794-477a-a8bb-35fb694d6b91\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1ebf4d7f-5cb85d585d-fxg94"
Jul 21 14:11:33.634244 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:33.634156    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b77fca8-2794-477a-a8bb-35fb694d6b91-maas-api-tls\") pod \"maas-api-e2e-migrate-1ebf4d7f-5cb85d585d-fxg94\" (UID: \"8b77fca8-2794-477a-a8bb-35fb694d6b91\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1ebf4d7f-5cb85d585d-fxg94"
Jul 21 14:11:33.634244 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:33.634190    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b77fca8-2794-477a-a8bb-35fb694d6b91-openshift-service-ca\") pod \"maas-api-e2e-migrate-1ebf4d7f-5cb85d585d-fxg94\" (UID: \"8b77fca8-2794-477a-a8bb-35fb694d6b91\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1ebf4d7f-5cb85d585d-fxg94"
Jul 21 14:11:33.634860 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:33.634834    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b77fca8-2794-477a-a8bb-35fb694d6b91-openshift-service-ca\") pod \"maas-api-e2e-migrate-1ebf4d7f-5cb85d585d-fxg94\" (UID: \"8b77fca8-2794-477a-a8bb-35fb694d6b91\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1ebf4d7f-5cb85d585d-fxg94"
Jul 21 14:11:33.636674 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:33.636654    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b77fca8-2794-477a-a8bb-35fb694d6b91-maas-api-tls\") pod \"maas-api-e2e-migrate-1ebf4d7f-5cb85d585d-fxg94\" (UID: \"8b77fca8-2794-477a-a8bb-35fb694d6b91\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1ebf4d7f-5cb85d585d-fxg94"
Jul 21 14:11:33.643663 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:33.643638    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxcmm\" (UniqueName: \"kubernetes.io/projected/8b77fca8-2794-477a-a8bb-35fb694d6b91-kube-api-access-dxcmm\") pod \"maas-api-e2e-migrate-1ebf4d7f-5cb85d585d-fxg94\" (UID: \"8b77fca8-2794-477a-a8bb-35fb694d6b91\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1ebf4d7f-5cb85d585d-fxg94"
Jul 21 14:11:33.790114 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:33.790023    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1ebf4d7f-5cb85d585d-fxg94"
Jul 21 14:11:33.929114 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:33.929084    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-1ebf4d7f-5cb85d585d-fxg94"]
Jul 21 14:11:33.930819 ip-10-0-132-245 kubenswrapper[2578]: W0721 14:11:33.930791    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b77fca8_2794_477a_a8bb_35fb694d6b91.slice/crio-729aac8620916ab31fbab5f64b7f96e522c13724f809ceb22035be2867484940 WatchSource:0}: Error finding container 729aac8620916ab31fbab5f64b7f96e522c13724f809ceb22035be2867484940: Status 404 returned error can't find the container with id 729aac8620916ab31fbab5f64b7f96e522c13724f809ceb22035be2867484940
Jul 21 14:11:33.940688 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:33.940660    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1ebf4d7f-5cb85d585d-fxg94" event={"ID":"8b77fca8-2794-477a-a8bb-35fb694d6b91","Type":"ContainerStarted","Data":"729aac8620916ab31fbab5f64b7f96e522c13724f809ceb22035be2867484940"}
Jul 21 14:11:34.946072 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:34.946040    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1ebf4d7f-5cb85d585d-fxg94" event={"ID":"8b77fca8-2794-477a-a8bb-35fb694d6b91","Type":"ContainerStarted","Data":"64f8705b17926f98a0472fb19c930b9f456a880b37f8b8578620b7cd2c87df48"}
Jul 21 14:11:34.946436 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:34.946158    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1ebf4d7f-5cb85d585d-fxg94"
Jul 21 14:11:34.964471 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:34.964409    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1ebf4d7f-5cb85d585d-fxg94" podStartSLOduration=1.059948082 podStartE2EDuration="1.964392874s" totalImagesPullingTime="904.444792ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:11:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:11:34.963231746 +0000 UTC m=+1641.923520302" watchObservedRunningTime="2026-07-21 14:11:34.964392874 +0000 UTC m=+1641.924681430"
Jul 21 14:11:39.165417 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:39.165379    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:11:40.955463 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:40.955437    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1ebf4d7f-5cb85d585d-fxg94"
Jul 21 14:11:58.439160 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:58.439122    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1ebf4d7f-mmbmb"]
Jul 21 14:11:58.442420 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:58.442403    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1ebf4d7f-mmbmb"
Jul 21 14:11:58.445065 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:58.445047    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5t9ph\""
Jul 21 14:11:58.451266 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:58.451240    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1ebf4d7f-mmbmb"]
Jul 21 14:11:58.554813 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:58.554775    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81e97b80-cc1f-4132-98ec-939000238a67-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1ebf4d7f-mmbmb\" (UID: \"81e97b80-cc1f-4132-98ec-939000238a67\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1ebf4d7f-mmbmb"
Jul 21 14:11:58.656171 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:58.656136    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81e97b80-cc1f-4132-98ec-939000238a67-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1ebf4d7f-mmbmb\" (UID: \"81e97b80-cc1f-4132-98ec-939000238a67\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1ebf4d7f-mmbmb"
Jul 21 14:11:58.656740 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:58.656719    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81e97b80-cc1f-4132-98ec-939000238a67-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-1ebf4d7f-mmbmb\" (UID: \"81e97b80-cc1f-4132-98ec-939000238a67\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1ebf4d7f-mmbmb"
Jul 21 14:11:58.753477 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:58.753399    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1ebf4d7f-mmbmb"
Jul 21 14:11:58.878783 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:58.878753    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1ebf4d7f-mmbmb"]
Jul 21 14:11:58.881378 ip-10-0-132-245 kubenswrapper[2578]: W0721 14:11:58.881345    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81e97b80_cc1f_4132_98ec_939000238a67.slice/crio-7d7ce6571277b87c7730775d21e057f9b458dfdc7c569f308cfcf43092a59d28 WatchSource:0}: Error finding container 7d7ce6571277b87c7730775d21e057f9b458dfdc7c569f308cfcf43092a59d28: Status 404 returned error can't find the container with id 7d7ce6571277b87c7730775d21e057f9b458dfdc7c569f308cfcf43092a59d28
Jul 21 14:11:59.043555 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:11:59.043470    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1ebf4d7f-mmbmb" event={"ID":"81e97b80-cc1f-4132-98ec-939000238a67","Type":"ContainerStarted","Data":"7d7ce6571277b87c7730775d21e057f9b458dfdc7c569f308cfcf43092a59d28"}
Jul 21 14:12:00.049152 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:00.049067    2578 generic.go:356] "Generic (PLEG): container finished" podID="81e97b80-cc1f-4132-98ec-939000238a67" containerID="a145c791a523d53bbd9ad674dec791c03e9a574feaba6079b72a3f3744b908c8" exitCode=0
Jul 21 14:12:00.049495 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:00.049145    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1ebf4d7f-mmbmb" event={"ID":"81e97b80-cc1f-4132-98ec-939000238a67","Type":"ContainerDied","Data":"a145c791a523d53bbd9ad674dec791c03e9a574feaba6079b72a3f3744b908c8"}
Jul 21 14:12:01.180226 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:01.180202    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1ebf4d7f-mmbmb"
Jul 21 14:12:01.277925 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:01.277873    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81e97b80-cc1f-4132-98ec-939000238a67-maas-api-service-ca\") pod \"81e97b80-cc1f-4132-98ec-939000238a67\" (UID: \"81e97b80-cc1f-4132-98ec-939000238a67\") "
Jul 21 14:12:01.278241 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:01.278217    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81e97b80-cc1f-4132-98ec-939000238a67-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "81e97b80-cc1f-4132-98ec-939000238a67" (UID: "81e97b80-cc1f-4132-98ec-939000238a67"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:12:01.379315 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:01.379281    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81e97b80-cc1f-4132-98ec-939000238a67-maas-api-service-ca\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:12:02.059744 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:02.059714    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1ebf4d7f-mmbmb"
Jul 21 14:12:02.059928 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:02.059707    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1ebf4d7f-mmbmb" event={"ID":"81e97b80-cc1f-4132-98ec-939000238a67","Type":"ContainerDied","Data":"7d7ce6571277b87c7730775d21e057f9b458dfdc7c569f308cfcf43092a59d28"}
Jul 21 14:12:02.059928 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:02.059844    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7d7ce6571277b87c7730775d21e057f9b458dfdc7c569f308cfcf43092a59d28"
Jul 21 14:12:09.033722 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:09.033691    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-1ebf4d7f-5cb85d585d-fxg94"]
Jul 21 14:12:09.034312 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:09.033922    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1ebf4d7f-5cb85d585d-fxg94" podUID="8b77fca8-2794-477a-a8bb-35fb694d6b91" containerName="maas-api" containerID="cri-o://64f8705b17926f98a0472fb19c930b9f456a880b37f8b8578620b7cd2c87df48" gracePeriod=30
Jul 21 14:12:09.276770 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:09.276745    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1ebf4d7f-5cb85d585d-fxg94"
Jul 21 14:12:09.342361 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:09.342335    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b77fca8-2794-477a-a8bb-35fb694d6b91-maas-api-tls\") pod \"8b77fca8-2794-477a-a8bb-35fb694d6b91\" (UID: \"8b77fca8-2794-477a-a8bb-35fb694d6b91\") "
Jul 21 14:12:09.342521 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:09.342446    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dxcmm\" (UniqueName: \"kubernetes.io/projected/8b77fca8-2794-477a-a8bb-35fb694d6b91-kube-api-access-dxcmm\") pod \"8b77fca8-2794-477a-a8bb-35fb694d6b91\" (UID: \"8b77fca8-2794-477a-a8bb-35fb694d6b91\") "
Jul 21 14:12:09.342521 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:09.342493    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b77fca8-2794-477a-a8bb-35fb694d6b91-openshift-service-ca\") pod \"8b77fca8-2794-477a-a8bb-35fb694d6b91\" (UID: \"8b77fca8-2794-477a-a8bb-35fb694d6b91\") "
Jul 21 14:12:09.342940 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:09.342908    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b77fca8-2794-477a-a8bb-35fb694d6b91-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8b77fca8-2794-477a-a8bb-35fb694d6b91" (UID: "8b77fca8-2794-477a-a8bb-35fb694d6b91"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:12:09.344483 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:09.344458    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b77fca8-2794-477a-a8bb-35fb694d6b91-kube-api-access-dxcmm" (OuterVolumeSpecName: "kube-api-access-dxcmm") pod "8b77fca8-2794-477a-a8bb-35fb694d6b91" (UID: "8b77fca8-2794-477a-a8bb-35fb694d6b91"). InnerVolumeSpecName "kube-api-access-dxcmm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:12:09.344549 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:09.344517    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b77fca8-2794-477a-a8bb-35fb694d6b91-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8b77fca8-2794-477a-a8bb-35fb694d6b91" (UID: "8b77fca8-2794-477a-a8bb-35fb694d6b91"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:12:09.443439 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:09.443400    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxcmm\" (UniqueName: \"kubernetes.io/projected/8b77fca8-2794-477a-a8bb-35fb694d6b91-kube-api-access-dxcmm\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:12:09.443439 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:09.443434    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b77fca8-2794-477a-a8bb-35fb694d6b91-openshift-service-ca\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:12:09.443439 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:09.443444    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b77fca8-2794-477a-a8bb-35fb694d6b91-maas-api-tls\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:12:10.093974 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:10.093935    2578 generic.go:356] "Generic (PLEG): container finished" podID="8b77fca8-2794-477a-a8bb-35fb694d6b91" containerID="64f8705b17926f98a0472fb19c930b9f456a880b37f8b8578620b7cd2c87df48" exitCode=0
Jul 21 14:12:10.094363 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:10.094001    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1ebf4d7f-5cb85d585d-fxg94"
Jul 21 14:12:10.094363 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:10.094028    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1ebf4d7f-5cb85d585d-fxg94" event={"ID":"8b77fca8-2794-477a-a8bb-35fb694d6b91","Type":"ContainerDied","Data":"64f8705b17926f98a0472fb19c930b9f456a880b37f8b8578620b7cd2c87df48"}
Jul 21 14:12:10.094363 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:10.094107    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-1ebf4d7f-5cb85d585d-fxg94" event={"ID":"8b77fca8-2794-477a-a8bb-35fb694d6b91","Type":"ContainerDied","Data":"729aac8620916ab31fbab5f64b7f96e522c13724f809ceb22035be2867484940"}
Jul 21 14:12:10.094363 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:10.094129    2578 scope.go:117] "RemoveContainer" containerID="64f8705b17926f98a0472fb19c930b9f456a880b37f8b8578620b7cd2c87df48"
Jul 21 14:12:10.102174 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:10.102160    2578 scope.go:117] "RemoveContainer" containerID="64f8705b17926f98a0472fb19c930b9f456a880b37f8b8578620b7cd2c87df48"
Jul 21 14:12:10.102395 ip-10-0-132-245 kubenswrapper[2578]: E0721 14:12:10.102377    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64f8705b17926f98a0472fb19c930b9f456a880b37f8b8578620b7cd2c87df48\": container with ID starting with 64f8705b17926f98a0472fb19c930b9f456a880b37f8b8578620b7cd2c87df48 not found: ID does not exist" containerID="64f8705b17926f98a0472fb19c930b9f456a880b37f8b8578620b7cd2c87df48"
Jul 21 14:12:10.102443 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:10.102402    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64f8705b17926f98a0472fb19c930b9f456a880b37f8b8578620b7cd2c87df48"} err="failed to get container status \"64f8705b17926f98a0472fb19c930b9f456a880b37f8b8578620b7cd2c87df48\": rpc error: code = NotFound desc = could not find container \"64f8705b17926f98a0472fb19c930b9f456a880b37f8b8578620b7cd2c87df48\": container with ID starting with 64f8705b17926f98a0472fb19c930b9f456a880b37f8b8578620b7cd2c87df48 not found: ID does not exist"
Jul 21 14:12:10.110300 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:10.110279    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-1ebf4d7f-5cb85d585d-fxg94"]
Jul 21 14:12:10.114000 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:10.113979    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-1ebf4d7f-5cb85d585d-fxg94"]
Jul 21 14:12:11.503717 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:11.503683    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b77fca8-2794-477a-a8bb-35fb694d6b91" path="/var/lib/kubelet/pods/8b77fca8-2794-477a-a8bb-35fb694d6b91/volumes"
Jul 21 14:12:12.272199 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:12.272163    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:12:13.515030 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:13.514996    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1ebf4d7f-mmbmb"]
Jul 21 14:12:13.520523 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:13.520494    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-1ebf4d7f-mmbmb"]
Jul 21 14:12:15.503254 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:15.503224    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81e97b80-cc1f-4132-98ec-939000238a67" path="/var/lib/kubelet/pods/81e97b80-cc1f-4132-98ec-939000238a67/volumes"
Jul 21 14:12:16.603987 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:16.603952    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-462af97f-77c8c4b98f-4qzs4"]
Jul 21 14:12:16.604451 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:16.604396    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b77fca8-2794-477a-a8bb-35fb694d6b91" containerName="maas-api"
Jul 21 14:12:16.604451 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:16.604414    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b77fca8-2794-477a-a8bb-35fb694d6b91" containerName="maas-api"
Jul 21 14:12:16.604451 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:16.604428    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81e97b80-cc1f-4132-98ec-939000238a67" containerName="revoke-keys"
Jul 21 14:12:16.604451 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:16.604437    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="81e97b80-cc1f-4132-98ec-939000238a67" containerName="revoke-keys"
Jul 21 14:12:16.604669 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:16.604528    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="81e97b80-cc1f-4132-98ec-939000238a67" containerName="revoke-keys"
Jul 21 14:12:16.604669 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:16.604541    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b77fca8-2794-477a-a8bb-35fb694d6b91" containerName="maas-api"
Jul 21 14:12:16.607624 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:16.607600    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-462af97f-77c8c4b98f-4qzs4"
Jul 21 14:12:16.610104 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:16.610080    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-462af97f-serving-cert\""
Jul 21 14:12:16.610404 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:16.610271    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 14:12:16.610404 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:16.610284    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-nhfmq\""
Jul 21 14:12:16.611057 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:16.611042    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 14:12:16.619793 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:16.619765    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-462af97f-77c8c4b98f-4qzs4"]
Jul 21 14:12:16.702610 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:16.702576    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbj8v\" (UniqueName: \"kubernetes.io/projected/105ec901-342c-4026-b36a-9de37c9ffd55-kube-api-access-hbj8v\") pod \"maas-api-e2e-ait-462af97f-77c8c4b98f-4qzs4\" (UID: \"105ec901-342c-4026-b36a-9de37c9ffd55\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-462af97f-77c8c4b98f-4qzs4"
Jul 21 14:12:16.702795 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:16.702619    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/105ec901-342c-4026-b36a-9de37c9ffd55-maas-api-tls\") pod \"maas-api-e2e-ait-462af97f-77c8c4b98f-4qzs4\" (UID: \"105ec901-342c-4026-b36a-9de37c9ffd55\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-462af97f-77c8c4b98f-4qzs4"
Jul 21 14:12:16.702795 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:16.702656    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/105ec901-342c-4026-b36a-9de37c9ffd55-openshift-service-ca\") pod \"maas-api-e2e-ait-462af97f-77c8c4b98f-4qzs4\" (UID: \"105ec901-342c-4026-b36a-9de37c9ffd55\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-462af97f-77c8c4b98f-4qzs4"
Jul 21 14:12:16.803765 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:16.803731    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbj8v\" (UniqueName: \"kubernetes.io/projected/105ec901-342c-4026-b36a-9de37c9ffd55-kube-api-access-hbj8v\") pod \"maas-api-e2e-ait-462af97f-77c8c4b98f-4qzs4\" (UID: \"105ec901-342c-4026-b36a-9de37c9ffd55\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-462af97f-77c8c4b98f-4qzs4"
Jul 21 14:12:16.803765 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:16.803779    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/105ec901-342c-4026-b36a-9de37c9ffd55-maas-api-tls\") pod \"maas-api-e2e-ait-462af97f-77c8c4b98f-4qzs4\" (UID: \"105ec901-342c-4026-b36a-9de37c9ffd55\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-462af97f-77c8c4b98f-4qzs4"
Jul 21 14:12:16.804081 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:16.803817    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/105ec901-342c-4026-b36a-9de37c9ffd55-openshift-service-ca\") pod \"maas-api-e2e-ait-462af97f-77c8c4b98f-4qzs4\" (UID: \"105ec901-342c-4026-b36a-9de37c9ffd55\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-462af97f-77c8c4b98f-4qzs4"
Jul 21 14:12:16.804081 ip-10-0-132-245 kubenswrapper[2578]: E0721 14:12:16.803926    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-462af97f-serving-cert: secret "maas-api-e2e-ait-462af97f-serving-cert" not found
Jul 21 14:12:16.804081 ip-10-0-132-245 kubenswrapper[2578]: E0721 14:12:16.803984    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/105ec901-342c-4026-b36a-9de37c9ffd55-maas-api-tls podName:105ec901-342c-4026-b36a-9de37c9ffd55 nodeName:}" failed. No retries permitted until 2026-07-21 14:12:17.303965776 +0000 UTC m=+1684.264254310 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/105ec901-342c-4026-b36a-9de37c9ffd55-maas-api-tls") pod "maas-api-e2e-ait-462af97f-77c8c4b98f-4qzs4" (UID: "105ec901-342c-4026-b36a-9de37c9ffd55") : secret "maas-api-e2e-ait-462af97f-serving-cert" not found
Jul 21 14:12:16.804472 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:16.804451    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/105ec901-342c-4026-b36a-9de37c9ffd55-openshift-service-ca\") pod \"maas-api-e2e-ait-462af97f-77c8c4b98f-4qzs4\" (UID: \"105ec901-342c-4026-b36a-9de37c9ffd55\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-462af97f-77c8c4b98f-4qzs4"
Jul 21 14:12:16.812442 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:16.812416    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbj8v\" (UniqueName: \"kubernetes.io/projected/105ec901-342c-4026-b36a-9de37c9ffd55-kube-api-access-hbj8v\") pod \"maas-api-e2e-ait-462af97f-77c8c4b98f-4qzs4\" (UID: \"105ec901-342c-4026-b36a-9de37c9ffd55\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-462af97f-77c8c4b98f-4qzs4"
Jul 21 14:12:17.309194 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:17.309159    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/105ec901-342c-4026-b36a-9de37c9ffd55-maas-api-tls\") pod \"maas-api-e2e-ait-462af97f-77c8c4b98f-4qzs4\" (UID: \"105ec901-342c-4026-b36a-9de37c9ffd55\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-462af97f-77c8c4b98f-4qzs4"
Jul 21 14:12:17.311442 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:17.311417    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/105ec901-342c-4026-b36a-9de37c9ffd55-maas-api-tls\") pod \"maas-api-e2e-ait-462af97f-77c8c4b98f-4qzs4\" (UID: \"105ec901-342c-4026-b36a-9de37c9ffd55\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-462af97f-77c8c4b98f-4qzs4"
Jul 21 14:12:17.519974 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:17.519936    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-462af97f-77c8c4b98f-4qzs4"
Jul 21 14:12:17.651225 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:17.651199    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-462af97f-77c8c4b98f-4qzs4"]
Jul 21 14:12:17.653559 ip-10-0-132-245 kubenswrapper[2578]: W0721 14:12:17.653532    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod105ec901_342c_4026_b36a_9de37c9ffd55.slice/crio-c1774e8ecb0b21c4693d41e2a44161fd603c9ef631b31b5f8155ace2fb7166cc WatchSource:0}: Error finding container c1774e8ecb0b21c4693d41e2a44161fd603c9ef631b31b5f8155ace2fb7166cc: Status 404 returned error can't find the container with id c1774e8ecb0b21c4693d41e2a44161fd603c9ef631b31b5f8155ace2fb7166cc
Jul 21 14:12:18.130803 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:18.130767    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-462af97f-77c8c4b98f-4qzs4" event={"ID":"105ec901-342c-4026-b36a-9de37c9ffd55","Type":"ContainerStarted","Data":"fa3da2803b0cabb74984a99da08d0fb3a36700844c038ea855cfcbb252e529a7"}
Jul 21 14:12:18.130982 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:18.130827    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-462af97f-77c8c4b98f-4qzs4" event={"ID":"105ec901-342c-4026-b36a-9de37c9ffd55","Type":"ContainerStarted","Data":"c1774e8ecb0b21c4693d41e2a44161fd603c9ef631b31b5f8155ace2fb7166cc"}
Jul 21 14:12:18.130982 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:18.130856    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-462af97f-77c8c4b98f-4qzs4"
Jul 21 14:12:18.159192 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:18.159129    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-462af97f-77c8c4b98f-4qzs4" podStartSLOduration=1.845005241 podStartE2EDuration="2.159109173s" totalImagesPullingTime="314.103932ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:12:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:12:18.158894109 +0000 UTC m=+1685.119182661" watchObservedRunningTime="2026-07-21 14:12:18.159109173 +0000 UTC m=+1685.119397731"
Jul 21 14:12:23.665467 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:23.665434    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:12:24.141004 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:24.140979    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-462af97f-77c8c4b98f-4qzs4"
Jul 21 14:12:42.388206 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:42.388133    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-462af97f-8wxt5"]
Jul 21 14:12:42.391559 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:42.391544    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-462af97f-8wxt5"
Jul 21 14:12:42.393915 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:42.393891    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5t9ph\""
Jul 21 14:12:42.398461 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:42.398439    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-462af97f-8wxt5"]
Jul 21 14:12:42.522625 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:42.522586    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b56ff8c8-5d29-4305-b26b-c54eb865083a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-462af97f-8wxt5\" (UID: \"b56ff8c8-5d29-4305-b26b-c54eb865083a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-462af97f-8wxt5"
Jul 21 14:12:42.623503 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:42.623473    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b56ff8c8-5d29-4305-b26b-c54eb865083a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-462af97f-8wxt5\" (UID: \"b56ff8c8-5d29-4305-b26b-c54eb865083a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-462af97f-8wxt5"
Jul 21 14:12:42.624109 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:42.624093    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b56ff8c8-5d29-4305-b26b-c54eb865083a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-462af97f-8wxt5\" (UID: \"b56ff8c8-5d29-4305-b26b-c54eb865083a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-462af97f-8wxt5"
Jul 21 14:12:42.702023 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:42.701942    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-462af97f-8wxt5"
Jul 21 14:12:42.829456 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:42.829422    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-462af97f-8wxt5"]
Jul 21 14:12:42.831900 ip-10-0-132-245 kubenswrapper[2578]: W0721 14:12:42.831862    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb56ff8c8_5d29_4305_b26b_c54eb865083a.slice/crio-3d79a95c9d4ae278814c8a93643380e918cb16f06c8925bf24d21ff085be62ae WatchSource:0}: Error finding container 3d79a95c9d4ae278814c8a93643380e918cb16f06c8925bf24d21ff085be62ae: Status 404 returned error can't find the container with id 3d79a95c9d4ae278814c8a93643380e918cb16f06c8925bf24d21ff085be62ae
Jul 21 14:12:43.240564 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:43.240526    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-462af97f-8wxt5" event={"ID":"b56ff8c8-5d29-4305-b26b-c54eb865083a","Type":"ContainerStarted","Data":"f13cb62f553db5f9b41683a26bc35c97460b5e9b1c3bbd3fc82088c6dc242ccd"}
Jul 21 14:12:43.240736 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:43.240617    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-462af97f-8wxt5" event={"ID":"b56ff8c8-5d29-4305-b26b-c54eb865083a","Type":"ContainerStarted","Data":"3d79a95c9d4ae278814c8a93643380e918cb16f06c8925bf24d21ff085be62ae"}
Jul 21 14:12:43.256326 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:43.256277    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-462af97f-8wxt5" podStartSLOduration=1.256261759 podStartE2EDuration="1.256261759s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:12:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:12:43.254617829 +0000 UTC m=+1710.214906388" watchObservedRunningTime="2026-07-21 14:12:43.256261759 +0000 UTC m=+1710.216550315"
Jul 21 14:12:44.246194 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:44.246158    2578 generic.go:356] "Generic (PLEG): container finished" podID="b56ff8c8-5d29-4305-b26b-c54eb865083a" containerID="f13cb62f553db5f9b41683a26bc35c97460b5e9b1c3bbd3fc82088c6dc242ccd" exitCode=0
Jul 21 14:12:44.246564 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:44.246248    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-462af97f-8wxt5" event={"ID":"b56ff8c8-5d29-4305-b26b-c54eb865083a","Type":"ContainerDied","Data":"f13cb62f553db5f9b41683a26bc35c97460b5e9b1c3bbd3fc82088c6dc242ccd"}
Jul 21 14:12:45.379944 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:45.379913    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-462af97f-8wxt5"
Jul 21 14:12:45.448080 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:45.448051    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b56ff8c8-5d29-4305-b26b-c54eb865083a-maas-api-service-ca\") pod \"b56ff8c8-5d29-4305-b26b-c54eb865083a\" (UID: \"b56ff8c8-5d29-4305-b26b-c54eb865083a\") "
Jul 21 14:12:45.448431 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:45.448405    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b56ff8c8-5d29-4305-b26b-c54eb865083a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b56ff8c8-5d29-4305-b26b-c54eb865083a" (UID: "b56ff8c8-5d29-4305-b26b-c54eb865083a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:12:45.549067 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:45.549035    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b56ff8c8-5d29-4305-b26b-c54eb865083a-maas-api-service-ca\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:12:46.255662 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:46.255633    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-462af97f-8wxt5"
Jul 21 14:12:46.255662 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:46.255659    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-462af97f-8wxt5" event={"ID":"b56ff8c8-5d29-4305-b26b-c54eb865083a","Type":"ContainerDied","Data":"3d79a95c9d4ae278814c8a93643380e918cb16f06c8925bf24d21ff085be62ae"}
Jul 21 14:12:46.255964 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:46.255699    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3d79a95c9d4ae278814c8a93643380e918cb16f06c8925bf24d21ff085be62ae"
Jul 21 14:12:52.801388 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:52.801351    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-462af97f-77c8c4b98f-4qzs4"]
Jul 21 14:12:52.802145 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:52.802114    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-462af97f-77c8c4b98f-4qzs4" podUID="105ec901-342c-4026-b36a-9de37c9ffd55" containerName="maas-api" containerID="cri-o://fa3da2803b0cabb74984a99da08d0fb3a36700844c038ea855cfcbb252e529a7" gracePeriod=30
Jul 21 14:12:53.058163 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:53.058103    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-462af97f-77c8c4b98f-4qzs4"
Jul 21 14:12:53.212329 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:53.212295    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/105ec901-342c-4026-b36a-9de37c9ffd55-maas-api-tls\") pod \"105ec901-342c-4026-b36a-9de37c9ffd55\" (UID: \"105ec901-342c-4026-b36a-9de37c9ffd55\") "
Jul 21 14:12:53.212483 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:53.212405    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hbj8v\" (UniqueName: \"kubernetes.io/projected/105ec901-342c-4026-b36a-9de37c9ffd55-kube-api-access-hbj8v\") pod \"105ec901-342c-4026-b36a-9de37c9ffd55\" (UID: \"105ec901-342c-4026-b36a-9de37c9ffd55\") "
Jul 21 14:12:53.212483 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:53.212425    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/105ec901-342c-4026-b36a-9de37c9ffd55-openshift-service-ca\") pod \"105ec901-342c-4026-b36a-9de37c9ffd55\" (UID: \"105ec901-342c-4026-b36a-9de37c9ffd55\") "
Jul 21 14:12:53.212889 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:53.212841    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/105ec901-342c-4026-b36a-9de37c9ffd55-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "105ec901-342c-4026-b36a-9de37c9ffd55" (UID: "105ec901-342c-4026-b36a-9de37c9ffd55"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:12:53.214301 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:53.214272    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/105ec901-342c-4026-b36a-9de37c9ffd55-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "105ec901-342c-4026-b36a-9de37c9ffd55" (UID: "105ec901-342c-4026-b36a-9de37c9ffd55"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:12:53.214394 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:53.214359    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/105ec901-342c-4026-b36a-9de37c9ffd55-kube-api-access-hbj8v" (OuterVolumeSpecName: "kube-api-access-hbj8v") pod "105ec901-342c-4026-b36a-9de37c9ffd55" (UID: "105ec901-342c-4026-b36a-9de37c9ffd55"). InnerVolumeSpecName "kube-api-access-hbj8v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:12:53.294114 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:53.294077    2578 generic.go:356] "Generic (PLEG): container finished" podID="105ec901-342c-4026-b36a-9de37c9ffd55" containerID="fa3da2803b0cabb74984a99da08d0fb3a36700844c038ea855cfcbb252e529a7" exitCode=0
Jul 21 14:12:53.294288 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:53.294123    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-462af97f-77c8c4b98f-4qzs4"
Jul 21 14:12:53.294288 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:53.294164    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-462af97f-77c8c4b98f-4qzs4" event={"ID":"105ec901-342c-4026-b36a-9de37c9ffd55","Type":"ContainerDied","Data":"fa3da2803b0cabb74984a99da08d0fb3a36700844c038ea855cfcbb252e529a7"}
Jul 21 14:12:53.294288 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:53.294211    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-462af97f-77c8c4b98f-4qzs4" event={"ID":"105ec901-342c-4026-b36a-9de37c9ffd55","Type":"ContainerDied","Data":"c1774e8ecb0b21c4693d41e2a44161fd603c9ef631b31b5f8155ace2fb7166cc"}
Jul 21 14:12:53.294288 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:53.294233    2578 scope.go:117] "RemoveContainer" containerID="fa3da2803b0cabb74984a99da08d0fb3a36700844c038ea855cfcbb252e529a7"
Jul 21 14:12:53.305164 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:53.305145    2578 scope.go:117] "RemoveContainer" containerID="fa3da2803b0cabb74984a99da08d0fb3a36700844c038ea855cfcbb252e529a7"
Jul 21 14:12:53.305453 ip-10-0-132-245 kubenswrapper[2578]: E0721 14:12:53.305432    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa3da2803b0cabb74984a99da08d0fb3a36700844c038ea855cfcbb252e529a7\": container with ID starting with fa3da2803b0cabb74984a99da08d0fb3a36700844c038ea855cfcbb252e529a7 not found: ID does not exist" containerID="fa3da2803b0cabb74984a99da08d0fb3a36700844c038ea855cfcbb252e529a7"
Jul 21 14:12:53.305503 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:53.305461    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa3da2803b0cabb74984a99da08d0fb3a36700844c038ea855cfcbb252e529a7"} err="failed to get container status \"fa3da2803b0cabb74984a99da08d0fb3a36700844c038ea855cfcbb252e529a7\": rpc error: code = NotFound desc = could not find container \"fa3da2803b0cabb74984a99da08d0fb3a36700844c038ea855cfcbb252e529a7\": container with ID starting with fa3da2803b0cabb74984a99da08d0fb3a36700844c038ea855cfcbb252e529a7 not found: ID does not exist"
Jul 21 14:12:53.312996 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:53.312941    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/105ec901-342c-4026-b36a-9de37c9ffd55-maas-api-tls\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:12:53.312996 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:53.312963    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hbj8v\" (UniqueName: \"kubernetes.io/projected/105ec901-342c-4026-b36a-9de37c9ffd55-kube-api-access-hbj8v\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:12:53.312996 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:53.312977    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/105ec901-342c-4026-b36a-9de37c9ffd55-openshift-service-ca\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:12:53.316233 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:53.316212    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-462af97f-77c8c4b98f-4qzs4"]
Jul 21 14:12:53.319733 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:53.319709    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-462af97f-77c8c4b98f-4qzs4"]
Jul 21 14:12:53.503077 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:53.503045    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="105ec901-342c-4026-b36a-9de37c9ffd55" path="/var/lib/kubelet/pods/105ec901-342c-4026-b36a-9de37c9ffd55/volumes"
Jul 21 14:12:55.863664 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:55.863627    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:12:57.471175 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:57.471142    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-462af97f-8wxt5"]
Jul 21 14:12:57.473770 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:57.473746    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-462af97f-8wxt5"]
Jul 21 14:12:57.503407 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:12:57.503383    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b56ff8c8-5d29-4305-b26b-c54eb865083a" path="/var/lib/kubelet/pods/b56ff8c8-5d29-4305-b26b-c54eb865083a/volumes"
Jul 21 14:13:02.021133 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:02.021100    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-caef500b-85f9f6765f-d2nl8"]
Jul 21 14:13:02.021552 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:02.021399    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b56ff8c8-5d29-4305-b26b-c54eb865083a" containerName="revoke-keys"
Jul 21 14:13:02.021552 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:02.021410    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b56ff8c8-5d29-4305-b26b-c54eb865083a" containerName="revoke-keys"
Jul 21 14:13:02.021552 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:02.021421    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="105ec901-342c-4026-b36a-9de37c9ffd55" containerName="maas-api"
Jul 21 14:13:02.021552 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:02.021426    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="105ec901-342c-4026-b36a-9de37c9ffd55" containerName="maas-api"
Jul 21 14:13:02.021552 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:02.021479    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="105ec901-342c-4026-b36a-9de37c9ffd55" containerName="maas-api"
Jul 21 14:13:02.021552 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:02.021489    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b56ff8c8-5d29-4305-b26b-c54eb865083a" containerName="revoke-keys"
Jul 21 14:13:02.025412 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:02.025395    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-caef500b-85f9f6765f-d2nl8"
Jul 21 14:13:02.028598 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:02.028565    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-nhfmq\""
Jul 21 14:13:02.028721 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:02.028626    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-caef500b-serving-cert\""
Jul 21 14:13:02.028827 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:02.028808    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 14:13:02.029498 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:02.029481    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 14:13:02.035341 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:02.035312    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-caef500b-85f9f6765f-d2nl8"]
Jul 21 14:13:02.188242 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:02.188210    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jk9m\" (UniqueName: \"kubernetes.io/projected/075af0c0-71e7-406f-a4eb-64d937293bf6-kube-api-access-8jk9m\") pod \"maas-api-e2e-derive-caef500b-85f9f6765f-d2nl8\" (UID: \"075af0c0-71e7-406f-a4eb-64d937293bf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-caef500b-85f9f6765f-d2nl8"
Jul 21 14:13:02.188405 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:02.188269    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/075af0c0-71e7-406f-a4eb-64d937293bf6-openshift-service-ca\") pod \"maas-api-e2e-derive-caef500b-85f9f6765f-d2nl8\" (UID: \"075af0c0-71e7-406f-a4eb-64d937293bf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-caef500b-85f9f6765f-d2nl8"
Jul 21 14:13:02.188405 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:02.188291    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/075af0c0-71e7-406f-a4eb-64d937293bf6-maas-api-tls\") pod \"maas-api-e2e-derive-caef500b-85f9f6765f-d2nl8\" (UID: \"075af0c0-71e7-406f-a4eb-64d937293bf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-caef500b-85f9f6765f-d2nl8"
Jul 21 14:13:02.289712 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:02.289615    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jk9m\" (UniqueName: \"kubernetes.io/projected/075af0c0-71e7-406f-a4eb-64d937293bf6-kube-api-access-8jk9m\") pod \"maas-api-e2e-derive-caef500b-85f9f6765f-d2nl8\" (UID: \"075af0c0-71e7-406f-a4eb-64d937293bf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-caef500b-85f9f6765f-d2nl8"
Jul 21 14:13:02.289712 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:02.289683    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/075af0c0-71e7-406f-a4eb-64d937293bf6-openshift-service-ca\") pod \"maas-api-e2e-derive-caef500b-85f9f6765f-d2nl8\" (UID: \"075af0c0-71e7-406f-a4eb-64d937293bf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-caef500b-85f9f6765f-d2nl8"
Jul 21 14:13:02.289712 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:02.289705    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/075af0c0-71e7-406f-a4eb-64d937293bf6-maas-api-tls\") pod \"maas-api-e2e-derive-caef500b-85f9f6765f-d2nl8\" (UID: \"075af0c0-71e7-406f-a4eb-64d937293bf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-caef500b-85f9f6765f-d2nl8"
Jul 21 14:13:02.290385 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:02.290366    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/075af0c0-71e7-406f-a4eb-64d937293bf6-openshift-service-ca\") pod \"maas-api-e2e-derive-caef500b-85f9f6765f-d2nl8\" (UID: \"075af0c0-71e7-406f-a4eb-64d937293bf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-caef500b-85f9f6765f-d2nl8"
Jul 21 14:13:02.292094 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:02.292079    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/075af0c0-71e7-406f-a4eb-64d937293bf6-maas-api-tls\") pod \"maas-api-e2e-derive-caef500b-85f9f6765f-d2nl8\" (UID: \"075af0c0-71e7-406f-a4eb-64d937293bf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-caef500b-85f9f6765f-d2nl8"
Jul 21 14:13:02.298005 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:02.297982    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jk9m\" (UniqueName: \"kubernetes.io/projected/075af0c0-71e7-406f-a4eb-64d937293bf6-kube-api-access-8jk9m\") pod \"maas-api-e2e-derive-caef500b-85f9f6765f-d2nl8\" (UID: \"075af0c0-71e7-406f-a4eb-64d937293bf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-caef500b-85f9f6765f-d2nl8"
Jul 21 14:13:02.335658 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:02.335616    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-caef500b-85f9f6765f-d2nl8"
Jul 21 14:13:02.476939 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:02.476909    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-caef500b-85f9f6765f-d2nl8"]
Jul 21 14:13:02.480484 ip-10-0-132-245 kubenswrapper[2578]: W0721 14:13:02.480441    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod075af0c0_71e7_406f_a4eb_64d937293bf6.slice/crio-215ba906b03a70349cb8e1170344aa9ac6fdcaa3b91280bbbdba694502c01e36 WatchSource:0}: Error finding container 215ba906b03a70349cb8e1170344aa9ac6fdcaa3b91280bbbdba694502c01e36: Status 404 returned error can't find the container with id 215ba906b03a70349cb8e1170344aa9ac6fdcaa3b91280bbbdba694502c01e36
Jul 21 14:13:03.337551 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:03.337522    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-caef500b-85f9f6765f-d2nl8" event={"ID":"075af0c0-71e7-406f-a4eb-64d937293bf6","Type":"ContainerStarted","Data":"abda055766464165326cfcd7edbdc4e1ec626c5644c90baeeed9a023d0cf1105"}
Jul 21 14:13:03.337948 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:03.337571    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-caef500b-85f9f6765f-d2nl8" event={"ID":"075af0c0-71e7-406f-a4eb-64d937293bf6","Type":"ContainerStarted","Data":"215ba906b03a70349cb8e1170344aa9ac6fdcaa3b91280bbbdba694502c01e36"}
Jul 21 14:13:03.337948 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:03.337601    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-caef500b-85f9f6765f-d2nl8"
Jul 21 14:13:03.358112 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:03.358063    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-caef500b-85f9f6765f-d2nl8" podStartSLOduration=1.94471233 podStartE2EDuration="2.358048286s" totalImagesPullingTime="413.335956ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:13:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:13:03.3561787 +0000 UTC m=+1730.316467257" watchObservedRunningTime="2026-07-21 14:13:03.358048286 +0000 UTC m=+1730.318336843"
Jul 21 14:13:07.773245 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:07.773209    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:13:09.348361 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:09.348334    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-caef500b-85f9f6765f-d2nl8"
Jul 21 14:13:26.674450 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:26.674415    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-caef500b-lj2sb"]
Jul 21 14:13:26.677683 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:26.677667    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-caef500b-lj2sb"
Jul 21 14:13:26.680237 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:26.680218    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5t9ph\""
Jul 21 14:13:26.685920 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:26.685900    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-caef500b-lj2sb"]
Jul 21 14:13:26.779822 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:26.779787    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48f1d116-34fa-4324-81a8-b235660a6d72-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-caef500b-lj2sb\" (UID: \"48f1d116-34fa-4324-81a8-b235660a6d72\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-caef500b-lj2sb"
Jul 21 14:13:26.880864 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:26.880826    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48f1d116-34fa-4324-81a8-b235660a6d72-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-caef500b-lj2sb\" (UID: \"48f1d116-34fa-4324-81a8-b235660a6d72\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-caef500b-lj2sb"
Jul 21 14:13:26.881476 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:26.881454    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48f1d116-34fa-4324-81a8-b235660a6d72-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-caef500b-lj2sb\" (UID: \"48f1d116-34fa-4324-81a8-b235660a6d72\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-caef500b-lj2sb"
Jul 21 14:13:26.988857 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:26.988748    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-caef500b-lj2sb"
Jul 21 14:13:27.118366 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:27.118190    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-caef500b-lj2sb"]
Jul 21 14:13:27.120954 ip-10-0-132-245 kubenswrapper[2578]: W0721 14:13:27.120928    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48f1d116_34fa_4324_81a8_b235660a6d72.slice/crio-e145327857b1150f358f995f597bed4c82509a776c9c768cfe3efa2eb48b73b7 WatchSource:0}: Error finding container e145327857b1150f358f995f597bed4c82509a776c9c768cfe3efa2eb48b73b7: Status 404 returned error can't find the container with id e145327857b1150f358f995f597bed4c82509a776c9c768cfe3efa2eb48b73b7
Jul 21 14:13:27.438737 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:27.438700    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-caef500b-lj2sb" event={"ID":"48f1d116-34fa-4324-81a8-b235660a6d72","Type":"ContainerStarted","Data":"81d9b03c6a52e6d71fca531a03b900b0aec9de02046195da4b5a8fd9bd621508"}
Jul 21 14:13:27.438949 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:27.438758    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-caef500b-lj2sb" event={"ID":"48f1d116-34fa-4324-81a8-b235660a6d72","Type":"ContainerStarted","Data":"e145327857b1150f358f995f597bed4c82509a776c9c768cfe3efa2eb48b73b7"}
Jul 21 14:13:27.455967 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:27.455911    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-caef500b-lj2sb" podStartSLOduration=1.455862316 podStartE2EDuration="1.455862316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:13:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:13:27.455606354 +0000 UTC m=+1754.415894912" watchObservedRunningTime="2026-07-21 14:13:27.455862316 +0000 UTC m=+1754.416150868"
Jul 21 14:13:28.444417 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:28.444385    2578 generic.go:356] "Generic (PLEG): container finished" podID="48f1d116-34fa-4324-81a8-b235660a6d72" containerID="81d9b03c6a52e6d71fca531a03b900b0aec9de02046195da4b5a8fd9bd621508" exitCode=0
Jul 21 14:13:28.444793 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:28.444474    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-caef500b-lj2sb" event={"ID":"48f1d116-34fa-4324-81a8-b235660a6d72","Type":"ContainerDied","Data":"81d9b03c6a52e6d71fca531a03b900b0aec9de02046195da4b5a8fd9bd621508"}
Jul 21 14:13:29.580632 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:29.580609    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-caef500b-lj2sb"
Jul 21 14:13:29.706086 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:29.705860    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48f1d116-34fa-4324-81a8-b235660a6d72-maas-api-service-ca\") pod \"48f1d116-34fa-4324-81a8-b235660a6d72\" (UID: \"48f1d116-34fa-4324-81a8-b235660a6d72\") "
Jul 21 14:13:29.706271 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:29.706249    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48f1d116-34fa-4324-81a8-b235660a6d72-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "48f1d116-34fa-4324-81a8-b235660a6d72" (UID: "48f1d116-34fa-4324-81a8-b235660a6d72"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:13:29.807058 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:29.807028    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48f1d116-34fa-4324-81a8-b235660a6d72-maas-api-service-ca\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:13:30.454504 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:30.454476    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-caef500b-lj2sb"
Jul 21 14:13:30.454678 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:30.454518    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-caef500b-lj2sb" event={"ID":"48f1d116-34fa-4324-81a8-b235660a6d72","Type":"ContainerDied","Data":"e145327857b1150f358f995f597bed4c82509a776c9c768cfe3efa2eb48b73b7"}
Jul 21 14:13:30.454678 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:30.454563    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e145327857b1150f358f995f597bed4c82509a776c9c768cfe3efa2eb48b73b7"
Jul 21 14:13:37.271918 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:37.271839    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-caef500b-85f9f6765f-d2nl8"]
Jul 21 14:13:37.272486 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:37.272099    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-caef500b-85f9f6765f-d2nl8" podUID="075af0c0-71e7-406f-a4eb-64d937293bf6" containerName="maas-api" containerID="cri-o://abda055766464165326cfcd7edbdc4e1ec626c5644c90baeeed9a023d0cf1105" gracePeriod=30
Jul 21 14:13:37.485320 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:37.485291    2578 generic.go:356] "Generic (PLEG): container finished" podID="075af0c0-71e7-406f-a4eb-64d937293bf6" containerID="abda055766464165326cfcd7edbdc4e1ec626c5644c90baeeed9a023d0cf1105" exitCode=0
Jul 21 14:13:37.485486 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:37.485369    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-caef500b-85f9f6765f-d2nl8" event={"ID":"075af0c0-71e7-406f-a4eb-64d937293bf6","Type":"ContainerDied","Data":"abda055766464165326cfcd7edbdc4e1ec626c5644c90baeeed9a023d0cf1105"}
Jul 21 14:13:37.548260 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:37.548234    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-caef500b-85f9f6765f-d2nl8"
Jul 21 14:13:37.672018 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:37.671981    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/075af0c0-71e7-406f-a4eb-64d937293bf6-openshift-service-ca\") pod \"075af0c0-71e7-406f-a4eb-64d937293bf6\" (UID: \"075af0c0-71e7-406f-a4eb-64d937293bf6\") "
Jul 21 14:13:37.672210 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:37.672031    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8jk9m\" (UniqueName: \"kubernetes.io/projected/075af0c0-71e7-406f-a4eb-64d937293bf6-kube-api-access-8jk9m\") pod \"075af0c0-71e7-406f-a4eb-64d937293bf6\" (UID: \"075af0c0-71e7-406f-a4eb-64d937293bf6\") "
Jul 21 14:13:37.672210 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:37.672173    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/075af0c0-71e7-406f-a4eb-64d937293bf6-maas-api-tls\") pod \"075af0c0-71e7-406f-a4eb-64d937293bf6\" (UID: \"075af0c0-71e7-406f-a4eb-64d937293bf6\") "
Jul 21 14:13:37.672348 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:37.672324    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/075af0c0-71e7-406f-a4eb-64d937293bf6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "075af0c0-71e7-406f-a4eb-64d937293bf6" (UID: "075af0c0-71e7-406f-a4eb-64d937293bf6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:13:37.672414 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:37.672399    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/075af0c0-71e7-406f-a4eb-64d937293bf6-openshift-service-ca\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:13:37.674147 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:37.674112    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/075af0c0-71e7-406f-a4eb-64d937293bf6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "075af0c0-71e7-406f-a4eb-64d937293bf6" (UID: "075af0c0-71e7-406f-a4eb-64d937293bf6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:13:37.674254 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:37.674227    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/075af0c0-71e7-406f-a4eb-64d937293bf6-kube-api-access-8jk9m" (OuterVolumeSpecName: "kube-api-access-8jk9m") pod "075af0c0-71e7-406f-a4eb-64d937293bf6" (UID: "075af0c0-71e7-406f-a4eb-64d937293bf6"). InnerVolumeSpecName "kube-api-access-8jk9m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:13:37.773364 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:37.773320    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/075af0c0-71e7-406f-a4eb-64d937293bf6-maas-api-tls\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:13:37.773364 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:37.773355    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8jk9m\" (UniqueName: \"kubernetes.io/projected/075af0c0-71e7-406f-a4eb-64d937293bf6-kube-api-access-8jk9m\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:13:38.490552 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:38.490520    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-caef500b-85f9f6765f-d2nl8"
Jul 21 14:13:38.490552 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:38.490537    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-caef500b-85f9f6765f-d2nl8" event={"ID":"075af0c0-71e7-406f-a4eb-64d937293bf6","Type":"ContainerDied","Data":"215ba906b03a70349cb8e1170344aa9ac6fdcaa3b91280bbbdba694502c01e36"}
Jul 21 14:13:38.491084 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:38.490632    2578 scope.go:117] "RemoveContainer" containerID="abda055766464165326cfcd7edbdc4e1ec626c5644c90baeeed9a023d0cf1105"
Jul 21 14:13:38.513803 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:38.513774    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-caef500b-85f9f6765f-d2nl8"]
Jul 21 14:13:38.515999 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:38.515976    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-caef500b-85f9f6765f-d2nl8"]
Jul 21 14:13:39.502719 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:39.502686    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="075af0c0-71e7-406f-a4eb-64d937293bf6" path="/var/lib/kubelet/pods/075af0c0-71e7-406f-a4eb-64d937293bf6/volumes"
Jul 21 14:13:40.373117 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:40.373086    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:13:41.791337 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:41.791303    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-caef500b-lj2sb"]
Jul 21 14:13:41.794862 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:41.794838    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-caef500b-lj2sb"]
Jul 21 14:13:43.503267 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:13:43.503235    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48f1d116-34fa-4324-81a8-b235660a6d72" path="/var/lib/kubelet/pods/48f1d116-34fa-4324-81a8-b235660a6d72/volumes"
Jul 21 14:15:00.230260 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:00.230223    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744055-lls7w"]
Jul 21 14:15:00.232657 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:00.230507    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="075af0c0-71e7-406f-a4eb-64d937293bf6" containerName="maas-api"
Jul 21 14:15:00.232657 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:00.230518    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="075af0c0-71e7-406f-a4eb-64d937293bf6" containerName="maas-api"
Jul 21 14:15:00.232657 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:00.230536    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48f1d116-34fa-4324-81a8-b235660a6d72" containerName="revoke-keys"
Jul 21 14:15:00.232657 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:00.230542    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="48f1d116-34fa-4324-81a8-b235660a6d72" containerName="revoke-keys"
Jul 21 14:15:00.232657 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:00.230588    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="48f1d116-34fa-4324-81a8-b235660a6d72" containerName="revoke-keys"
Jul 21 14:15:00.232657 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:00.230596    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="075af0c0-71e7-406f-a4eb-64d937293bf6" containerName="maas-api"
Jul 21 14:15:00.233520 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:00.233505    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744055-lls7w"
Jul 21 14:15:00.237841 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:00.237821    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5t9ph\""
Jul 21 14:15:00.242443 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:00.242425    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744055-lls7w"
Jul 21 14:15:00.260010 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:00.259982    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744055-lls7w"]
Jul 21 14:15:00.384074 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:00.384049    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744055-lls7w"]
Jul 21 14:15:00.385798 ip-10-0-132-245 kubenswrapper[2578]: W0721 14:15:00.385768    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61cbdead_b67f_4862_b7bc_d0bd9b05fe90.slice/crio-c5fe04386309eea6b2507ef5f1f7d8d7e55dd1e5df8a7cdd156ff7193338868b WatchSource:0}: Error finding container c5fe04386309eea6b2507ef5f1f7d8d7e55dd1e5df8a7cdd156ff7193338868b: Status 404 returned error can't find the container with id c5fe04386309eea6b2507ef5f1f7d8d7e55dd1e5df8a7cdd156ff7193338868b
Jul 21 14:15:00.843700 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:00.843663    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744055-lls7w" event={"ID":"61cbdead-b67f-4862-b7bc-d0bd9b05fe90","Type":"ContainerStarted","Data":"e5c65b65e22b09b7d7fd30556285247722a33d3b1f36edaef6a6bea7cdfc14f9"}
Jul 21 14:15:00.843872 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:00.843723    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744055-lls7w" event={"ID":"61cbdead-b67f-4862-b7bc-d0bd9b05fe90","Type":"ContainerStarted","Data":"c5fe04386309eea6b2507ef5f1f7d8d7e55dd1e5df8a7cdd156ff7193338868b"}
Jul 21 14:15:01.849493 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:01.849456    2578 generic.go:356] "Generic (PLEG): container finished" podID="61cbdead-b67f-4862-b7bc-d0bd9b05fe90" containerID="e5c65b65e22b09b7d7fd30556285247722a33d3b1f36edaef6a6bea7cdfc14f9" exitCode=0
Jul 21 14:15:01.849889 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:01.849537    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744055-lls7w" event={"ID":"61cbdead-b67f-4862-b7bc-d0bd9b05fe90","Type":"ContainerDied","Data":"e5c65b65e22b09b7d7fd30556285247722a33d3b1f36edaef6a6bea7cdfc14f9"}
Jul 21 14:15:02.983105 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:02.983084    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744055-lls7w"
Jul 21 14:15:03.859389 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:03.859361    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744055-lls7w"
Jul 21 14:15:03.859549 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:03.859392    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744055-lls7w" event={"ID":"61cbdead-b67f-4862-b7bc-d0bd9b05fe90","Type":"ContainerDied","Data":"c5fe04386309eea6b2507ef5f1f7d8d7e55dd1e5df8a7cdd156ff7193338868b"}
Jul 21 14:15:03.859549 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:03.859438    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c5fe04386309eea6b2507ef5f1f7d8d7e55dd1e5df8a7cdd156ff7193338868b"
Jul 21 14:15:10.878949 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:10.878916    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:15:24.286389 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:24.286351    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:15:27.445938 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:27.445908    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5b75b902-777b56ff69-f7psr"]
Jul 21 14:15:27.446290 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:27.446206    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61cbdead-b67f-4862-b7bc-d0bd9b05fe90" containerName="cleanup"
Jul 21 14:15:27.446290 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:27.446216    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="61cbdead-b67f-4862-b7bc-d0bd9b05fe90" containerName="cleanup"
Jul 21 14:15:27.446290 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:27.446271    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="61cbdead-b67f-4862-b7bc-d0bd9b05fe90" containerName="cleanup"
Jul 21 14:15:27.449229 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:27.449209    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5b75b902-777b56ff69-f7psr"
Jul 21 14:15:27.454008 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:27.453984    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 14:15:27.454915 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:27.454872    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 14:15:27.455029 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:27.454896    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-5b75b902-serving-cert\""
Jul 21 14:15:27.455029 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:27.454945    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-nhfmq\""
Jul 21 14:15:27.463200 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:27.463155    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5b75b902-777b56ff69-f7psr"]
Jul 21 14:15:27.551999 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:27.551967    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/905843d7-7a9f-4ea1-9688-2e3ab8c25ece-maas-api-tls\") pod \"maas-api-e2e-mt-5b75b902-777b56ff69-f7psr\" (UID: \"905843d7-7a9f-4ea1-9688-2e3ab8c25ece\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5b75b902-777b56ff69-f7psr"
Jul 21 14:15:27.552178 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:27.552100    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92dtg\" (UniqueName: \"kubernetes.io/projected/905843d7-7a9f-4ea1-9688-2e3ab8c25ece-kube-api-access-92dtg\") pod \"maas-api-e2e-mt-5b75b902-777b56ff69-f7psr\" (UID: \"905843d7-7a9f-4ea1-9688-2e3ab8c25ece\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5b75b902-777b56ff69-f7psr"
Jul 21 14:15:27.552178 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:27.552156    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/905843d7-7a9f-4ea1-9688-2e3ab8c25ece-openshift-service-ca\") pod \"maas-api-e2e-mt-5b75b902-777b56ff69-f7psr\" (UID: \"905843d7-7a9f-4ea1-9688-2e3ab8c25ece\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5b75b902-777b56ff69-f7psr"
Jul 21 14:15:27.653559 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:27.653519    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92dtg\" (UniqueName: \"kubernetes.io/projected/905843d7-7a9f-4ea1-9688-2e3ab8c25ece-kube-api-access-92dtg\") pod \"maas-api-e2e-mt-5b75b902-777b56ff69-f7psr\" (UID: \"905843d7-7a9f-4ea1-9688-2e3ab8c25ece\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5b75b902-777b56ff69-f7psr"
Jul 21 14:15:27.653559 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:27.653561    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/905843d7-7a9f-4ea1-9688-2e3ab8c25ece-openshift-service-ca\") pod \"maas-api-e2e-mt-5b75b902-777b56ff69-f7psr\" (UID: \"905843d7-7a9f-4ea1-9688-2e3ab8c25ece\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5b75b902-777b56ff69-f7psr"
Jul 21 14:15:27.653771 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:27.653674    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/905843d7-7a9f-4ea1-9688-2e3ab8c25ece-maas-api-tls\") pod \"maas-api-e2e-mt-5b75b902-777b56ff69-f7psr\" (UID: \"905843d7-7a9f-4ea1-9688-2e3ab8c25ece\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5b75b902-777b56ff69-f7psr"
Jul 21 14:15:27.653824 ip-10-0-132-245 kubenswrapper[2578]: E0721 14:15:27.653804    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-5b75b902-serving-cert: secret "maas-api-e2e-mt-5b75b902-serving-cert" not found
Jul 21 14:15:27.653923 ip-10-0-132-245 kubenswrapper[2578]: E0721 14:15:27.653911    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/905843d7-7a9f-4ea1-9688-2e3ab8c25ece-maas-api-tls podName:905843d7-7a9f-4ea1-9688-2e3ab8c25ece nodeName:}" failed. No retries permitted until 2026-07-21 14:15:28.153886286 +0000 UTC m=+1875.114174838 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/905843d7-7a9f-4ea1-9688-2e3ab8c25ece-maas-api-tls") pod "maas-api-e2e-mt-5b75b902-777b56ff69-f7psr" (UID: "905843d7-7a9f-4ea1-9688-2e3ab8c25ece") : secret "maas-api-e2e-mt-5b75b902-serving-cert" not found
Jul 21 14:15:27.654214 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:27.654197    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/905843d7-7a9f-4ea1-9688-2e3ab8c25ece-openshift-service-ca\") pod \"maas-api-e2e-mt-5b75b902-777b56ff69-f7psr\" (UID: \"905843d7-7a9f-4ea1-9688-2e3ab8c25ece\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5b75b902-777b56ff69-f7psr"
Jul 21 14:15:27.665478 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:27.665452    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92dtg\" (UniqueName: \"kubernetes.io/projected/905843d7-7a9f-4ea1-9688-2e3ab8c25ece-kube-api-access-92dtg\") pod \"maas-api-e2e-mt-5b75b902-777b56ff69-f7psr\" (UID: \"905843d7-7a9f-4ea1-9688-2e3ab8c25ece\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5b75b902-777b56ff69-f7psr"
Jul 21 14:15:28.158558 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:28.158521    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/905843d7-7a9f-4ea1-9688-2e3ab8c25ece-maas-api-tls\") pod \"maas-api-e2e-mt-5b75b902-777b56ff69-f7psr\" (UID: \"905843d7-7a9f-4ea1-9688-2e3ab8c25ece\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5b75b902-777b56ff69-f7psr"
Jul 21 14:15:28.160978 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:28.160949    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/905843d7-7a9f-4ea1-9688-2e3ab8c25ece-maas-api-tls\") pod \"maas-api-e2e-mt-5b75b902-777b56ff69-f7psr\" (UID: \"905843d7-7a9f-4ea1-9688-2e3ab8c25ece\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5b75b902-777b56ff69-f7psr"
Jul 21 14:15:28.359600 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:28.359555    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5b75b902-777b56ff69-f7psr"
Jul 21 14:15:28.494391 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:28.494352    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5b75b902-777b56ff69-f7psr"]
Jul 21 14:15:28.496521 ip-10-0-132-245 kubenswrapper[2578]: W0721 14:15:28.496479    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod905843d7_7a9f_4ea1_9688_2e3ab8c25ece.slice/crio-a6f628fadf1bdc25ac27839893ffb8bd2547b360cc31cdf433201a3cd4f686dd WatchSource:0}: Error finding container a6f628fadf1bdc25ac27839893ffb8bd2547b360cc31cdf433201a3cd4f686dd: Status 404 returned error can't find the container with id a6f628fadf1bdc25ac27839893ffb8bd2547b360cc31cdf433201a3cd4f686dd
Jul 21 14:15:28.968929 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:28.968903    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5b75b902-777b56ff69-f7psr" event={"ID":"905843d7-7a9f-4ea1-9688-2e3ab8c25ece","Type":"ContainerStarted","Data":"a6f628fadf1bdc25ac27839893ffb8bd2547b360cc31cdf433201a3cd4f686dd"}
Jul 21 14:15:29.974766 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:29.974731    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5b75b902-777b56ff69-f7psr" event={"ID":"905843d7-7a9f-4ea1-9688-2e3ab8c25ece","Type":"ContainerStarted","Data":"f058c2db058765b1c8d272bee2e64330e7e17414917925c9128c4892f6aed8f2"}
Jul 21 14:15:29.975183 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:29.974807    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5b75b902-777b56ff69-f7psr"
Jul 21 14:15:29.995387 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:29.995338    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5b75b902-777b56ff69-f7psr" podStartSLOduration=2.567548656 podStartE2EDuration="2.995321979s" totalImagesPullingTime="427.773323ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:15:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:15:29.993470241 +0000 UTC m=+1876.953758798" watchObservedRunningTime="2026-07-21 14:15:29.995321979 +0000 UTC m=+1876.955610535"
Jul 21 14:15:35.985085 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:35.985010    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5b75b902-777b56ff69-f7psr"
Jul 21 14:15:39.737789 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:39.737747    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5b75b902-bkmr5"]
Jul 21 14:15:39.741293 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:39.741267    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5b75b902-bkmr5"
Jul 21 14:15:39.744067 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:39.744049    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5t9ph\""
Jul 21 14:15:39.756420 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:39.756382    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5b75b902-bkmr5"]
Jul 21 14:15:39.858013 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:39.857965    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/420bca2b-bc62-4320-a586-a0abbd3226b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-5b75b902-bkmr5\" (UID: \"420bca2b-bc62-4320-a586-a0abbd3226b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5b75b902-bkmr5"
Jul 21 14:15:39.958366 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:39.958324    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/420bca2b-bc62-4320-a586-a0abbd3226b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-5b75b902-bkmr5\" (UID: \"420bca2b-bc62-4320-a586-a0abbd3226b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5b75b902-bkmr5"
Jul 21 14:15:39.958993 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:39.958970    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/420bca2b-bc62-4320-a586-a0abbd3226b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-5b75b902-bkmr5\" (UID: \"420bca2b-bc62-4320-a586-a0abbd3226b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5b75b902-bkmr5"
Jul 21 14:15:40.050533 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:40.050440    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5b75b902-bkmr5"
Jul 21 14:15:40.083604 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:40.083567    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:15:40.182518 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:40.182485    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5b75b902-bkmr5"]
Jul 21 14:15:40.185854 ip-10-0-132-245 kubenswrapper[2578]: W0721 14:15:40.185828    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod420bca2b_bc62_4320_a586_a0abbd3226b3.slice/crio-a2fb7a9bc499c7b7e56f6e5f411c4a89d4ea5bf6214dfbeb30a5af49fc0d537a WatchSource:0}: Error finding container a2fb7a9bc499c7b7e56f6e5f411c4a89d4ea5bf6214dfbeb30a5af49fc0d537a: Status 404 returned error can't find the container with id a2fb7a9bc499c7b7e56f6e5f411c4a89d4ea5bf6214dfbeb30a5af49fc0d537a
Jul 21 14:15:41.021602 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:41.021565    2578 generic.go:356] "Generic (PLEG): container finished" podID="420bca2b-bc62-4320-a586-a0abbd3226b3" containerID="4bab398e5ee4189525b150d247d90c77e966e83774955b13ba4ae6f6d3476226" exitCode=0
Jul 21 14:15:41.021998 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:41.021608    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5b75b902-bkmr5" event={"ID":"420bca2b-bc62-4320-a586-a0abbd3226b3","Type":"ContainerDied","Data":"4bab398e5ee4189525b150d247d90c77e966e83774955b13ba4ae6f6d3476226"}
Jul 21 14:15:41.021998 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:41.021656    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5b75b902-bkmr5" event={"ID":"420bca2b-bc62-4320-a586-a0abbd3226b3","Type":"ContainerStarted","Data":"a2fb7a9bc499c7b7e56f6e5f411c4a89d4ea5bf6214dfbeb30a5af49fc0d537a"}
Jul 21 14:15:42.158538 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:42.158513    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5b75b902-bkmr5"
Jul 21 14:15:42.279138 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:42.279036    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/420bca2b-bc62-4320-a586-a0abbd3226b3-maas-api-service-ca\") pod \"420bca2b-bc62-4320-a586-a0abbd3226b3\" (UID: \"420bca2b-bc62-4320-a586-a0abbd3226b3\") "
Jul 21 14:15:42.279391 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:42.279367    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/420bca2b-bc62-4320-a586-a0abbd3226b3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "420bca2b-bc62-4320-a586-a0abbd3226b3" (UID: "420bca2b-bc62-4320-a586-a0abbd3226b3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:15:42.379746 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:42.379710    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/420bca2b-bc62-4320-a586-a0abbd3226b3-maas-api-service-ca\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:15:43.030995 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:43.030955    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5b75b902-bkmr5" event={"ID":"420bca2b-bc62-4320-a586-a0abbd3226b3","Type":"ContainerDied","Data":"a2fb7a9bc499c7b7e56f6e5f411c4a89d4ea5bf6214dfbeb30a5af49fc0d537a"}
Jul 21 14:15:43.031167 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:43.031002    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5b75b902-bkmr5"
Jul 21 14:15:43.031167 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:43.031006    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a2fb7a9bc499c7b7e56f6e5f411c4a89d4ea5bf6214dfbeb30a5af49fc0d537a"
Jul 21 14:15:50.246925 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:50.246869    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5b75b902-777b56ff69-f7psr"]
Jul 21 14:15:50.247342 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:50.247092    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5b75b902-777b56ff69-f7psr" podUID="905843d7-7a9f-4ea1-9688-2e3ab8c25ece" containerName="maas-api" containerID="cri-o://f058c2db058765b1c8d272bee2e64330e7e17414917925c9128c4892f6aed8f2" gracePeriod=30
Jul 21 14:15:50.500212 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:50.500159    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5b75b902-777b56ff69-f7psr"
Jul 21 14:15:50.549501 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:50.549472    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/905843d7-7a9f-4ea1-9688-2e3ab8c25ece-openshift-service-ca\") pod \"905843d7-7a9f-4ea1-9688-2e3ab8c25ece\" (UID: \"905843d7-7a9f-4ea1-9688-2e3ab8c25ece\") "
Jul 21 14:15:50.549501 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:50.549507    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-92dtg\" (UniqueName: \"kubernetes.io/projected/905843d7-7a9f-4ea1-9688-2e3ab8c25ece-kube-api-access-92dtg\") pod \"905843d7-7a9f-4ea1-9688-2e3ab8c25ece\" (UID: \"905843d7-7a9f-4ea1-9688-2e3ab8c25ece\") "
Jul 21 14:15:50.549692 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:50.549549    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/905843d7-7a9f-4ea1-9688-2e3ab8c25ece-maas-api-tls\") pod \"905843d7-7a9f-4ea1-9688-2e3ab8c25ece\" (UID: \"905843d7-7a9f-4ea1-9688-2e3ab8c25ece\") "
Jul 21 14:15:50.549922 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:50.549865    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/905843d7-7a9f-4ea1-9688-2e3ab8c25ece-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "905843d7-7a9f-4ea1-9688-2e3ab8c25ece" (UID: "905843d7-7a9f-4ea1-9688-2e3ab8c25ece"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:15:50.551637 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:50.551614    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/905843d7-7a9f-4ea1-9688-2e3ab8c25ece-kube-api-access-92dtg" (OuterVolumeSpecName: "kube-api-access-92dtg") pod "905843d7-7a9f-4ea1-9688-2e3ab8c25ece" (UID: "905843d7-7a9f-4ea1-9688-2e3ab8c25ece"). InnerVolumeSpecName "kube-api-access-92dtg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:15:50.551748 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:50.551643    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/905843d7-7a9f-4ea1-9688-2e3ab8c25ece-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "905843d7-7a9f-4ea1-9688-2e3ab8c25ece" (UID: "905843d7-7a9f-4ea1-9688-2e3ab8c25ece"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:15:50.650820 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:50.650788    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/905843d7-7a9f-4ea1-9688-2e3ab8c25ece-openshift-service-ca\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:15:50.650820 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:50.650815    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92dtg\" (UniqueName: \"kubernetes.io/projected/905843d7-7a9f-4ea1-9688-2e3ab8c25ece-kube-api-access-92dtg\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:15:50.650820 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:50.650825    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/905843d7-7a9f-4ea1-9688-2e3ab8c25ece-maas-api-tls\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:15:51.067726 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:51.067686    2578 generic.go:356] "Generic (PLEG): container finished" podID="905843d7-7a9f-4ea1-9688-2e3ab8c25ece" containerID="f058c2db058765b1c8d272bee2e64330e7e17414917925c9128c4892f6aed8f2" exitCode=0
Jul 21 14:15:51.067919 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:51.067737    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5b75b902-777b56ff69-f7psr"
Jul 21 14:15:51.067919 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:51.067756    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5b75b902-777b56ff69-f7psr" event={"ID":"905843d7-7a9f-4ea1-9688-2e3ab8c25ece","Type":"ContainerDied","Data":"f058c2db058765b1c8d272bee2e64330e7e17414917925c9128c4892f6aed8f2"}
Jul 21 14:15:51.067919 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:51.067808    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5b75b902-777b56ff69-f7psr" event={"ID":"905843d7-7a9f-4ea1-9688-2e3ab8c25ece","Type":"ContainerDied","Data":"a6f628fadf1bdc25ac27839893ffb8bd2547b360cc31cdf433201a3cd4f686dd"}
Jul 21 14:15:51.067919 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:51.067838    2578 scope.go:117] "RemoveContainer" containerID="f058c2db058765b1c8d272bee2e64330e7e17414917925c9128c4892f6aed8f2"
Jul 21 14:15:51.076404 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:51.076385    2578 scope.go:117] "RemoveContainer" containerID="f058c2db058765b1c8d272bee2e64330e7e17414917925c9128c4892f6aed8f2"
Jul 21 14:15:51.076691 ip-10-0-132-245 kubenswrapper[2578]: E0721 14:15:51.076672    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f058c2db058765b1c8d272bee2e64330e7e17414917925c9128c4892f6aed8f2\": container with ID starting with f058c2db058765b1c8d272bee2e64330e7e17414917925c9128c4892f6aed8f2 not found: ID does not exist" containerID="f058c2db058765b1c8d272bee2e64330e7e17414917925c9128c4892f6aed8f2"
Jul 21 14:15:51.076743 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:51.076700    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f058c2db058765b1c8d272bee2e64330e7e17414917925c9128c4892f6aed8f2"} err="failed to get container status \"f058c2db058765b1c8d272bee2e64330e7e17414917925c9128c4892f6aed8f2\": rpc error: code = NotFound desc = could not find container \"f058c2db058765b1c8d272bee2e64330e7e17414917925c9128c4892f6aed8f2\": container with ID starting with f058c2db058765b1c8d272bee2e64330e7e17414917925c9128c4892f6aed8f2 not found: ID does not exist"
Jul 21 14:15:51.095648 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:51.095615    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5b75b902-777b56ff69-f7psr"]
Jul 21 14:15:51.098072 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:51.098048    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5b75b902-777b56ff69-f7psr"]
Jul 21 14:15:51.503384 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:51.503345    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="905843d7-7a9f-4ea1-9688-2e3ab8c25ece" path="/var/lib/kubelet/pods/905843d7-7a9f-4ea1-9688-2e3ab8c25ece/volumes"
Jul 21 14:15:52.667321 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:52.667284    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:15:54.813290 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:54.813256    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5b75b902-bkmr5"]
Jul 21 14:15:54.815510 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:54.815484    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5b75b902-bkmr5"]
Jul 21 14:15:55.503028 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:15:55.502995    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="420bca2b-bc62-4320-a586-a0abbd3226b3" path="/var/lib/kubelet/pods/420bca2b-bc62-4320-a586-a0abbd3226b3/volumes"
Jul 21 14:16:25.787916 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:25.787861    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9b3e9882-57d7d5485c-z7pwr"]
Jul 21 14:16:25.788399 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:25.788313    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="905843d7-7a9f-4ea1-9688-2e3ab8c25ece" containerName="maas-api"
Jul 21 14:16:25.788399 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:25.788331    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="905843d7-7a9f-4ea1-9688-2e3ab8c25ece" containerName="maas-api"
Jul 21 14:16:25.788399 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:25.788357    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="420bca2b-bc62-4320-a586-a0abbd3226b3" containerName="revoke-keys"
Jul 21 14:16:25.788399 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:25.788366    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="420bca2b-bc62-4320-a586-a0abbd3226b3" containerName="revoke-keys"
Jul 21 14:16:25.788616 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:25.788444    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="420bca2b-bc62-4320-a586-a0abbd3226b3" containerName="revoke-keys"
Jul 21 14:16:25.788616 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:25.788459    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="905843d7-7a9f-4ea1-9688-2e3ab8c25ece" containerName="maas-api"
Jul 21 14:16:25.791330 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:25.791311    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b3e9882-57d7d5485c-z7pwr"
Jul 21 14:16:25.793993 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:25.793970    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-9b3e9882-serving-cert\""
Jul 21 14:16:25.794103 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:25.794010    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 14:16:25.794930 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:25.794912    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-nhfmq\""
Jul 21 14:16:25.795104 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:25.795084    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 14:16:25.801064 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:25.801022    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9b3e9882-57d7d5485c-z7pwr"]
Jul 21 14:16:25.831774 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:25.831748    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b65d4244-ebf8-4f39-9808-89edbeb09108-openshift-service-ca\") pod \"maas-api-e2e-mt-9b3e9882-57d7d5485c-z7pwr\" (UID: \"b65d4244-ebf8-4f39-9808-89edbeb09108\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b3e9882-57d7d5485c-z7pwr"
Jul 21 14:16:25.831955 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:25.831797    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxc6x\" (UniqueName: \"kubernetes.io/projected/b65d4244-ebf8-4f39-9808-89edbeb09108-kube-api-access-pxc6x\") pod \"maas-api-e2e-mt-9b3e9882-57d7d5485c-z7pwr\" (UID: \"b65d4244-ebf8-4f39-9808-89edbeb09108\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b3e9882-57d7d5485c-z7pwr"
Jul 21 14:16:25.831955 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:25.831833    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b65d4244-ebf8-4f39-9808-89edbeb09108-maas-api-tls\") pod \"maas-api-e2e-mt-9b3e9882-57d7d5485c-z7pwr\" (UID: \"b65d4244-ebf8-4f39-9808-89edbeb09108\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b3e9882-57d7d5485c-z7pwr"
Jul 21 14:16:25.932469 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:25.932431    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b65d4244-ebf8-4f39-9808-89edbeb09108-maas-api-tls\") pod \"maas-api-e2e-mt-9b3e9882-57d7d5485c-z7pwr\" (UID: \"b65d4244-ebf8-4f39-9808-89edbeb09108\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b3e9882-57d7d5485c-z7pwr"
Jul 21 14:16:25.932641 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:25.932481    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b65d4244-ebf8-4f39-9808-89edbeb09108-openshift-service-ca\") pod \"maas-api-e2e-mt-9b3e9882-57d7d5485c-z7pwr\" (UID: \"b65d4244-ebf8-4f39-9808-89edbeb09108\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b3e9882-57d7d5485c-z7pwr"
Jul 21 14:16:25.932641 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:25.932532    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxc6x\" (UniqueName: \"kubernetes.io/projected/b65d4244-ebf8-4f39-9808-89edbeb09108-kube-api-access-pxc6x\") pod \"maas-api-e2e-mt-9b3e9882-57d7d5485c-z7pwr\" (UID: \"b65d4244-ebf8-4f39-9808-89edbeb09108\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b3e9882-57d7d5485c-z7pwr"
Jul 21 14:16:25.932641 ip-10-0-132-245 kubenswrapper[2578]: E0721 14:16:25.932604    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-9b3e9882-serving-cert: secret "maas-api-e2e-mt-9b3e9882-serving-cert" not found
Jul 21 14:16:25.932747 ip-10-0-132-245 kubenswrapper[2578]: E0721 14:16:25.932696    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b65d4244-ebf8-4f39-9808-89edbeb09108-maas-api-tls podName:b65d4244-ebf8-4f39-9808-89edbeb09108 nodeName:}" failed. No retries permitted until 2026-07-21 14:16:26.432678502 +0000 UTC m=+1933.392967037 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b65d4244-ebf8-4f39-9808-89edbeb09108-maas-api-tls") pod "maas-api-e2e-mt-9b3e9882-57d7d5485c-z7pwr" (UID: "b65d4244-ebf8-4f39-9808-89edbeb09108") : secret "maas-api-e2e-mt-9b3e9882-serving-cert" not found
Jul 21 14:16:25.933196 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:25.933177    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b65d4244-ebf8-4f39-9808-89edbeb09108-openshift-service-ca\") pod \"maas-api-e2e-mt-9b3e9882-57d7d5485c-z7pwr\" (UID: \"b65d4244-ebf8-4f39-9808-89edbeb09108\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b3e9882-57d7d5485c-z7pwr"
Jul 21 14:16:25.941818 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:25.941794    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxc6x\" (UniqueName: \"kubernetes.io/projected/b65d4244-ebf8-4f39-9808-89edbeb09108-kube-api-access-pxc6x\") pod \"maas-api-e2e-mt-9b3e9882-57d7d5485c-z7pwr\" (UID: \"b65d4244-ebf8-4f39-9808-89edbeb09108\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b3e9882-57d7d5485c-z7pwr"
Jul 21 14:16:26.435284 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:26.435250    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b65d4244-ebf8-4f39-9808-89edbeb09108-maas-api-tls\") pod \"maas-api-e2e-mt-9b3e9882-57d7d5485c-z7pwr\" (UID: \"b65d4244-ebf8-4f39-9808-89edbeb09108\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b3e9882-57d7d5485c-z7pwr"
Jul 21 14:16:26.437701 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:26.437667    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b65d4244-ebf8-4f39-9808-89edbeb09108-maas-api-tls\") pod \"maas-api-e2e-mt-9b3e9882-57d7d5485c-z7pwr\" (UID: \"b65d4244-ebf8-4f39-9808-89edbeb09108\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b3e9882-57d7d5485c-z7pwr"
Jul 21 14:16:26.702234 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:26.702149    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b3e9882-57d7d5485c-z7pwr"
Jul 21 14:16:26.833546 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:26.833517    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9b3e9882-57d7d5485c-z7pwr"]
Jul 21 14:16:26.836144 ip-10-0-132-245 kubenswrapper[2578]: W0721 14:16:26.836106    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb65d4244_ebf8_4f39_9808_89edbeb09108.slice/crio-a195b141417eb14a72345364ae080e81790473d34f8f2b8a1cd1924747db2d2f WatchSource:0}: Error finding container a195b141417eb14a72345364ae080e81790473d34f8f2b8a1cd1924747db2d2f: Status 404 returned error can't find the container with id a195b141417eb14a72345364ae080e81790473d34f8f2b8a1cd1924747db2d2f
Jul 21 14:16:26.837454 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:26.837435    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 14:16:27.231476 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:27.231449    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b3e9882-57d7d5485c-z7pwr" event={"ID":"b65d4244-ebf8-4f39-9808-89edbeb09108","Type":"ContainerStarted","Data":"a195b141417eb14a72345364ae080e81790473d34f8f2b8a1cd1924747db2d2f"}
Jul 21 14:16:28.236809 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:28.236772    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b3e9882-57d7d5485c-z7pwr" event={"ID":"b65d4244-ebf8-4f39-9808-89edbeb09108","Type":"ContainerStarted","Data":"b66c6f2c21a46519d3aeaf8be5d68ef91f90b0fb2aa6e90f323ff0a368a4c6cf"}
Jul 21 14:16:28.237339 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:28.236904    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b3e9882-57d7d5485c-z7pwr"
Jul 21 14:16:28.257715 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:28.257667    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b3e9882-57d7d5485c-z7pwr" podStartSLOduration=2.877446167 podStartE2EDuration="3.257649177s" totalImagesPullingTime="380.20301ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:16:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:16:28.255433237 +0000 UTC m=+1935.215721794" watchObservedRunningTime="2026-07-21 14:16:28.257649177 +0000 UTC m=+1935.217937737"
Jul 21 14:16:30.571378 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:30.571337    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:16:34.246776 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:34.246745    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b3e9882-57d7d5485c-z7pwr"
Jul 21 14:16:37.467130 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:37.467096    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-9b3e9882/e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6"]
Jul 21 14:16:37.470510 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:37.470489    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-9b3e9882/e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6"
Jul 21 14:16:37.474149 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:37.474118    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-9b3e9882\"/\"kube-root-ca.crt\""
Jul 21 14:16:37.474413 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:37.474393    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-9b3e9882\"/\"e2e-lifecycle-model-9b3e9882-kserve-self-signed-certs\""
Jul 21 14:16:37.474646 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:37.474629    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-9b3e9882\"/\"default-dockercfg-n5ttw\""
Jul 21 14:16:37.474916 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:37.474899    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-9b3e9882\"/\"openshift-service-ca.crt\""
Jul 21 14:16:37.481829 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:37.481799    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-9b3e9882/e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6"]
Jul 21 14:16:37.527434 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:37.527398    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9598e003-9e30-4da8-b722-ea71cc09dfb2-dshm\") pod \"e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6\" (UID: \"9598e003-9e30-4da8-b722-ea71cc09dfb2\") " pod="ai-tenant-e2e-mt-9b3e9882/e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6"
Jul 21 14:16:37.527629 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:37.527460    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9598e003-9e30-4da8-b722-ea71cc09dfb2-model-cache\") pod \"e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6\" (UID: \"9598e003-9e30-4da8-b722-ea71cc09dfb2\") " pod="ai-tenant-e2e-mt-9b3e9882/e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6"
Jul 21 14:16:37.527629 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:37.527483    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9598e003-9e30-4da8-b722-ea71cc09dfb2-tls-certs\") pod \"e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6\" (UID: \"9598e003-9e30-4da8-b722-ea71cc09dfb2\") " pod="ai-tenant-e2e-mt-9b3e9882/e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6"
Jul 21 14:16:37.528097 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:37.527708    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9598e003-9e30-4da8-b722-ea71cc09dfb2-home\") pod \"e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6\" (UID: \"9598e003-9e30-4da8-b722-ea71cc09dfb2\") " pod="ai-tenant-e2e-mt-9b3e9882/e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6"
Jul 21 14:16:37.528097 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:37.527760    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9598e003-9e30-4da8-b722-ea71cc09dfb2-tmp-dir\") pod \"e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6\" (UID: \"9598e003-9e30-4da8-b722-ea71cc09dfb2\") " pod="ai-tenant-e2e-mt-9b3e9882/e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6"
Jul 21 14:16:37.528097 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:37.527848    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5scpm\" (UniqueName: \"kubernetes.io/projected/9598e003-9e30-4da8-b722-ea71cc09dfb2-kube-api-access-5scpm\") pod \"e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6\" (UID: \"9598e003-9e30-4da8-b722-ea71cc09dfb2\") " pod="ai-tenant-e2e-mt-9b3e9882/e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6"
Jul 21 14:16:37.628532 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:37.628495    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9598e003-9e30-4da8-b722-ea71cc09dfb2-model-cache\") pod \"e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6\" (UID: \"9598e003-9e30-4da8-b722-ea71cc09dfb2\") " pod="ai-tenant-e2e-mt-9b3e9882/e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6"
Jul 21 14:16:37.628532 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:37.628533    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9598e003-9e30-4da8-b722-ea71cc09dfb2-tls-certs\") pod \"e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6\" (UID: \"9598e003-9e30-4da8-b722-ea71cc09dfb2\") " pod="ai-tenant-e2e-mt-9b3e9882/e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6"
Jul 21 14:16:37.628759 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:37.628558    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9598e003-9e30-4da8-b722-ea71cc09dfb2-home\") pod \"e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6\" (UID: \"9598e003-9e30-4da8-b722-ea71cc09dfb2\") " pod="ai-tenant-e2e-mt-9b3e9882/e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6"
Jul 21 14:16:37.628759 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:37.628581    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9598e003-9e30-4da8-b722-ea71cc09dfb2-tmp-dir\") pod \"e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6\" (UID: \"9598e003-9e30-4da8-b722-ea71cc09dfb2\") " pod="ai-tenant-e2e-mt-9b3e9882/e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6"
Jul 21 14:16:37.628759 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:37.628609    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5scpm\" (UniqueName: \"kubernetes.io/projected/9598e003-9e30-4da8-b722-ea71cc09dfb2-kube-api-access-5scpm\") pod \"e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6\" (UID: \"9598e003-9e30-4da8-b722-ea71cc09dfb2\") " pod="ai-tenant-e2e-mt-9b3e9882/e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6"
Jul 21 14:16:37.628759 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:37.628645    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9598e003-9e30-4da8-b722-ea71cc09dfb2-dshm\") pod \"e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6\" (UID: \"9598e003-9e30-4da8-b722-ea71cc09dfb2\") " pod="ai-tenant-e2e-mt-9b3e9882/e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6"
Jul 21 14:16:37.629054 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:37.629026    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9598e003-9e30-4da8-b722-ea71cc09dfb2-tmp-dir\") pod \"e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6\" (UID: \"9598e003-9e30-4da8-b722-ea71cc09dfb2\") " pod="ai-tenant-e2e-mt-9b3e9882/e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6"
Jul 21 14:16:37.629124 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:37.629088    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9598e003-9e30-4da8-b722-ea71cc09dfb2-home\") pod \"e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6\" (UID: \"9598e003-9e30-4da8-b722-ea71cc09dfb2\") " pod="ai-tenant-e2e-mt-9b3e9882/e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6"
Jul 21 14:16:37.629124 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:37.629104    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9598e003-9e30-4da8-b722-ea71cc09dfb2-model-cache\") pod \"e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6\" (UID: \"9598e003-9e30-4da8-b722-ea71cc09dfb2\") " pod="ai-tenant-e2e-mt-9b3e9882/e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6"
Jul 21 14:16:37.630950 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:37.630933    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9598e003-9e30-4da8-b722-ea71cc09dfb2-dshm\") pod \"e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6\" (UID: \"9598e003-9e30-4da8-b722-ea71cc09dfb2\") " pod="ai-tenant-e2e-mt-9b3e9882/e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6"
Jul 21 14:16:37.631371 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:37.631349    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9598e003-9e30-4da8-b722-ea71cc09dfb2-tls-certs\") pod \"e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6\" (UID: \"9598e003-9e30-4da8-b722-ea71cc09dfb2\") " pod="ai-tenant-e2e-mt-9b3e9882/e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6"
Jul 21 14:16:37.641151 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:37.641132    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5scpm\" (UniqueName: \"kubernetes.io/projected/9598e003-9e30-4da8-b722-ea71cc09dfb2-kube-api-access-5scpm\") pod \"e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6\" (UID: \"9598e003-9e30-4da8-b722-ea71cc09dfb2\") " pod="ai-tenant-e2e-mt-9b3e9882/e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6"
Jul 21 14:16:37.781097 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:37.781005    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-9b3e9882/e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6"
Jul 21 14:16:37.913869 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:37.913842    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-9b3e9882/e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6"]
Jul 21 14:16:37.916094 ip-10-0-132-245 kubenswrapper[2578]: W0721 14:16:37.916058    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9598e003_9e30_4da8_b722_ea71cc09dfb2.slice/crio-1697dc66e22ece67afb9eae99cebc12dfd12671f3ba449cf82a43bf7e45097c0 WatchSource:0}: Error finding container 1697dc66e22ece67afb9eae99cebc12dfd12671f3ba449cf82a43bf7e45097c0: Status 404 returned error can't find the container with id 1697dc66e22ece67afb9eae99cebc12dfd12671f3ba449cf82a43bf7e45097c0
Jul 21 14:16:38.284167 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:38.284133    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-9b3e9882/e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6" event={"ID":"9598e003-9e30-4da8-b722-ea71cc09dfb2","Type":"ContainerStarted","Data":"dd9f24e29fc9aa9c84de0e5deba940d0475f6fcddb7c5e6d2b561294bc8214f7"}
Jul 21 14:16:38.284370 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:38.284176    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-9b3e9882/e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6" event={"ID":"9598e003-9e30-4da8-b722-ea71cc09dfb2","Type":"ContainerStarted","Data":"1697dc66e22ece67afb9eae99cebc12dfd12671f3ba449cf82a43bf7e45097c0"}
Jul 21 14:16:38.303742 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:38.303690    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-9b3e9882/e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6" podStartSLOduration=1.3036759039999999 podStartE2EDuration="1.303675904s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:16:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:16:38.301515191 +0000 UTC m=+1945.261803773" watchObservedRunningTime="2026-07-21 14:16:38.303675904 +0000 UTC m=+1945.263964511"
Jul 21 14:16:43.071926 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:43.071888    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:16:47.781159 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:47.781118    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-9b3e9882/e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6"
Jul 21 14:16:47.781647 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:47.781418    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-9b3e9882/e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6"
Jul 21 14:16:47.793809 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:47.793782    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-9b3e9882/e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6"
Jul 21 14:16:48.338389 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:48.338363    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-9b3e9882/e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6"
Jul 21 14:16:56.569180 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:16:56.569146    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:17:03.646783 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:03.646751    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9b3e9882-qzdmn"]
Jul 21 14:17:03.650110 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:03.650093    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9b3e9882-qzdmn"
Jul 21 14:17:03.653651 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:03.653630    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5t9ph\""
Jul 21 14:17:03.660412 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:03.660387    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9b3e9882-qzdmn"]
Jul 21 14:17:03.768316 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:03.768280    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6776fd61-75c0-4cfb-adc9-acee4e5ce5bc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-9b3e9882-qzdmn\" (UID: \"6776fd61-75c0-4cfb-adc9-acee4e5ce5bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9b3e9882-qzdmn"
Jul 21 14:17:03.868716 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:03.868686    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6776fd61-75c0-4cfb-adc9-acee4e5ce5bc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-9b3e9882-qzdmn\" (UID: \"6776fd61-75c0-4cfb-adc9-acee4e5ce5bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9b3e9882-qzdmn"
Jul 21 14:17:03.869539 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:03.869511    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6776fd61-75c0-4cfb-adc9-acee4e5ce5bc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-9b3e9882-qzdmn\" (UID: \"6776fd61-75c0-4cfb-adc9-acee4e5ce5bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9b3e9882-qzdmn"
Jul 21 14:17:03.960624 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:03.960529    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9b3e9882-qzdmn"
Jul 21 14:17:04.094761 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:04.094077    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9b3e9882-qzdmn"]
Jul 21 14:17:04.401644 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:04.401607    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9b3e9882-qzdmn" event={"ID":"6776fd61-75c0-4cfb-adc9-acee4e5ce5bc","Type":"ContainerStarted","Data":"7b6a52c264b13f591a7a82db030cdb9245a06fdc0263e647e19b12c4ddaa0408"}
Jul 21 14:17:04.401821 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:04.401662    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9b3e9882-qzdmn" event={"ID":"6776fd61-75c0-4cfb-adc9-acee4e5ce5bc","Type":"ContainerStarted","Data":"74666cb52965cf7fa69f04541db5bbe695b4bd06530710031295cafe6e6535c5"}
Jul 21 14:17:04.420972 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:04.420921    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9b3e9882-qzdmn" podStartSLOduration=1.420905062 podStartE2EDuration="1.420905062s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:17:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:17:04.419227691 +0000 UTC m=+1971.379516249" watchObservedRunningTime="2026-07-21 14:17:04.420905062 +0000 UTC m=+1971.381193612"
Jul 21 14:17:05.407668 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:05.407593    2578 generic.go:356] "Generic (PLEG): container finished" podID="6776fd61-75c0-4cfb-adc9-acee4e5ce5bc" containerID="7b6a52c264b13f591a7a82db030cdb9245a06fdc0263e647e19b12c4ddaa0408" exitCode=0
Jul 21 14:17:05.407668 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:05.407657    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9b3e9882-qzdmn" event={"ID":"6776fd61-75c0-4cfb-adc9-acee4e5ce5bc","Type":"ContainerDied","Data":"7b6a52c264b13f591a7a82db030cdb9245a06fdc0263e647e19b12c4ddaa0408"}
Jul 21 14:17:06.540308 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:06.540286    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9b3e9882-qzdmn"
Jul 21 14:17:06.689690 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:06.689609    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6776fd61-75c0-4cfb-adc9-acee4e5ce5bc-maas-api-service-ca\") pod \"6776fd61-75c0-4cfb-adc9-acee4e5ce5bc\" (UID: \"6776fd61-75c0-4cfb-adc9-acee4e5ce5bc\") "
Jul 21 14:17:06.689970 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:06.689945    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6776fd61-75c0-4cfb-adc9-acee4e5ce5bc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6776fd61-75c0-4cfb-adc9-acee4e5ce5bc" (UID: "6776fd61-75c0-4cfb-adc9-acee4e5ce5bc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:17:06.790327 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:06.790289    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6776fd61-75c0-4cfb-adc9-acee4e5ce5bc-maas-api-service-ca\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:17:07.417514 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:07.417486    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9b3e9882-qzdmn"
Jul 21 14:17:07.417514 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:07.417512    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9b3e9882-qzdmn" event={"ID":"6776fd61-75c0-4cfb-adc9-acee4e5ce5bc","Type":"ContainerDied","Data":"74666cb52965cf7fa69f04541db5bbe695b4bd06530710031295cafe6e6535c5"}
Jul 21 14:17:07.417780 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:07.417553    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="74666cb52965cf7fa69f04541db5bbe695b4bd06530710031295cafe6e6535c5"
Jul 21 14:17:16.769274 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:16.769237    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:17:19.088558 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:19.088519    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9b3e9882-57d7d5485c-z7pwr"]
Jul 21 14:17:19.089050 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:19.088779    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b3e9882-57d7d5485c-z7pwr" podUID="b65d4244-ebf8-4f39-9808-89edbeb09108" containerName="maas-api" containerID="cri-o://b66c6f2c21a46519d3aeaf8be5d68ef91f90b0fb2aa6e90f323ff0a368a4c6cf" gracePeriod=30
Jul 21 14:17:19.242919 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:19.242834    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b3e9882-57d7d5485c-z7pwr" podUID="b65d4244-ebf8-4f39-9808-89edbeb09108" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.36:8443/health\": dial tcp 10.133.0.36:8443: connect: connection refused"
Jul 21 14:17:19.336574 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:19.336540    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b3e9882-57d7d5485c-z7pwr"
Jul 21 14:17:19.394871 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:19.394785    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pxc6x\" (UniqueName: \"kubernetes.io/projected/b65d4244-ebf8-4f39-9808-89edbeb09108-kube-api-access-pxc6x\") pod \"b65d4244-ebf8-4f39-9808-89edbeb09108\" (UID: \"b65d4244-ebf8-4f39-9808-89edbeb09108\") "
Jul 21 14:17:19.394871 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:19.394837    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b65d4244-ebf8-4f39-9808-89edbeb09108-openshift-service-ca\") pod \"b65d4244-ebf8-4f39-9808-89edbeb09108\" (UID: \"b65d4244-ebf8-4f39-9808-89edbeb09108\") "
Jul 21 14:17:19.394871 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:19.394856    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b65d4244-ebf8-4f39-9808-89edbeb09108-maas-api-tls\") pod \"b65d4244-ebf8-4f39-9808-89edbeb09108\" (UID: \"b65d4244-ebf8-4f39-9808-89edbeb09108\") "
Jul 21 14:17:19.395274 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:19.395249    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b65d4244-ebf8-4f39-9808-89edbeb09108-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b65d4244-ebf8-4f39-9808-89edbeb09108" (UID: "b65d4244-ebf8-4f39-9808-89edbeb09108"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:17:19.397008 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:19.396984    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b65d4244-ebf8-4f39-9808-89edbeb09108-kube-api-access-pxc6x" (OuterVolumeSpecName: "kube-api-access-pxc6x") pod "b65d4244-ebf8-4f39-9808-89edbeb09108" (UID: "b65d4244-ebf8-4f39-9808-89edbeb09108"). InnerVolumeSpecName "kube-api-access-pxc6x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:17:19.397169 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:19.397148    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b65d4244-ebf8-4f39-9808-89edbeb09108-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b65d4244-ebf8-4f39-9808-89edbeb09108" (UID: "b65d4244-ebf8-4f39-9808-89edbeb09108"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:17:19.471175 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:19.471143    2578 generic.go:356] "Generic (PLEG): container finished" podID="b65d4244-ebf8-4f39-9808-89edbeb09108" containerID="b66c6f2c21a46519d3aeaf8be5d68ef91f90b0fb2aa6e90f323ff0a368a4c6cf" exitCode=0
Jul 21 14:17:19.471326 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:19.471192    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b3e9882-57d7d5485c-z7pwr"
Jul 21 14:17:19.471326 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:19.471226    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b3e9882-57d7d5485c-z7pwr" event={"ID":"b65d4244-ebf8-4f39-9808-89edbeb09108","Type":"ContainerDied","Data":"b66c6f2c21a46519d3aeaf8be5d68ef91f90b0fb2aa6e90f323ff0a368a4c6cf"}
Jul 21 14:17:19.471326 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:19.471282    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-9b3e9882-57d7d5485c-z7pwr" event={"ID":"b65d4244-ebf8-4f39-9808-89edbeb09108","Type":"ContainerDied","Data":"a195b141417eb14a72345364ae080e81790473d34f8f2b8a1cd1924747db2d2f"}
Jul 21 14:17:19.471326 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:19.471310    2578 scope.go:117] "RemoveContainer" containerID="b66c6f2c21a46519d3aeaf8be5d68ef91f90b0fb2aa6e90f323ff0a368a4c6cf"
Jul 21 14:17:19.479525 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:19.479503    2578 scope.go:117] "RemoveContainer" containerID="b66c6f2c21a46519d3aeaf8be5d68ef91f90b0fb2aa6e90f323ff0a368a4c6cf"
Jul 21 14:17:19.479833 ip-10-0-132-245 kubenswrapper[2578]: E0721 14:17:19.479812    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b66c6f2c21a46519d3aeaf8be5d68ef91f90b0fb2aa6e90f323ff0a368a4c6cf\": container with ID starting with b66c6f2c21a46519d3aeaf8be5d68ef91f90b0fb2aa6e90f323ff0a368a4c6cf not found: ID does not exist" containerID="b66c6f2c21a46519d3aeaf8be5d68ef91f90b0fb2aa6e90f323ff0a368a4c6cf"
Jul 21 14:17:19.479942 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:19.479841    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b66c6f2c21a46519d3aeaf8be5d68ef91f90b0fb2aa6e90f323ff0a368a4c6cf"} err="failed to get container status \"b66c6f2c21a46519d3aeaf8be5d68ef91f90b0fb2aa6e90f323ff0a368a4c6cf\": rpc error: code = NotFound desc = could not find container \"b66c6f2c21a46519d3aeaf8be5d68ef91f90b0fb2aa6e90f323ff0a368a4c6cf\": container with ID starting with b66c6f2c21a46519d3aeaf8be5d68ef91f90b0fb2aa6e90f323ff0a368a4c6cf not found: ID does not exist"
Jul 21 14:17:19.494424 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:19.494397    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9b3e9882-57d7d5485c-z7pwr"]
Jul 21 14:17:19.496174 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:19.496155    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxc6x\" (UniqueName: \"kubernetes.io/projected/b65d4244-ebf8-4f39-9808-89edbeb09108-kube-api-access-pxc6x\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:17:19.496240 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:19.496179    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b65d4244-ebf8-4f39-9808-89edbeb09108-openshift-service-ca\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:17:19.496240 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:19.496190    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b65d4244-ebf8-4f39-9808-89edbeb09108-maas-api-tls\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:17:19.503451 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:19.503430    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-9b3e9882-57d7d5485c-z7pwr"]
Jul 21 14:17:21.504098 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:21.504065    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b65d4244-ebf8-4f39-9808-89edbeb09108" path="/var/lib/kubelet/pods/b65d4244-ebf8-4f39-9808-89edbeb09108/volumes"
Jul 21 14:17:23.759726 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:23.759688    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9b3e9882-qzdmn"]
Jul 21 14:17:23.762196 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:23.762169    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9b3e9882-qzdmn"]
Jul 21 14:17:25.502661 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:25.502622    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6776fd61-75c0-4cfb-adc9-acee4e5ce5bc" path="/var/lib/kubelet/pods/6776fd61-75c0-4cfb-adc9-acee4e5ce5bc/volumes"
Jul 21 14:17:27.975762 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:27.975725    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:17:30.969645 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:30.969609    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-9b3e9882/e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6"]
Jul 21 14:17:45.989533 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:45.989485    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-9b3e9882/e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6" podUID="9598e003-9e30-4da8-b722-ea71cc09dfb2" containerName="main" containerID="cri-o://dd9f24e29fc9aa9c84de0e5deba940d0475f6fcddb7c5e6d2b561294bc8214f7" gracePeriod=45
Jul 21 14:17:46.237848 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:46.237826    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-9b3e9882/e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6"
Jul 21 14:17:46.320059 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:46.319973    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9598e003-9e30-4da8-b722-ea71cc09dfb2-tmp-dir\") pod \"9598e003-9e30-4da8-b722-ea71cc09dfb2\" (UID: \"9598e003-9e30-4da8-b722-ea71cc09dfb2\") "
Jul 21 14:17:46.320059 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:46.320006    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9598e003-9e30-4da8-b722-ea71cc09dfb2-tls-certs\") pod \"9598e003-9e30-4da8-b722-ea71cc09dfb2\" (UID: \"9598e003-9e30-4da8-b722-ea71cc09dfb2\") "
Jul 21 14:17:46.320059 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:46.320030    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9598e003-9e30-4da8-b722-ea71cc09dfb2-dshm\") pod \"9598e003-9e30-4da8-b722-ea71cc09dfb2\" (UID: \"9598e003-9e30-4da8-b722-ea71cc09dfb2\") "
Jul 21 14:17:46.320342 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:46.320074    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5scpm\" (UniqueName: \"kubernetes.io/projected/9598e003-9e30-4da8-b722-ea71cc09dfb2-kube-api-access-5scpm\") pod \"9598e003-9e30-4da8-b722-ea71cc09dfb2\" (UID: \"9598e003-9e30-4da8-b722-ea71cc09dfb2\") "
Jul 21 14:17:46.320342 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:46.320169    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9598e003-9e30-4da8-b722-ea71cc09dfb2-model-cache\") pod \"9598e003-9e30-4da8-b722-ea71cc09dfb2\" (UID: \"9598e003-9e30-4da8-b722-ea71cc09dfb2\") "
Jul 21 14:17:46.320342 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:46.320258    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9598e003-9e30-4da8-b722-ea71cc09dfb2-home\") pod \"9598e003-9e30-4da8-b722-ea71cc09dfb2\" (UID: \"9598e003-9e30-4da8-b722-ea71cc09dfb2\") "
Jul 21 14:17:46.320342 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:46.320332    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9598e003-9e30-4da8-b722-ea71cc09dfb2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9598e003-9e30-4da8-b722-ea71cc09dfb2" (UID: "9598e003-9e30-4da8-b722-ea71cc09dfb2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 14:17:46.320529 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:46.320418    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9598e003-9e30-4da8-b722-ea71cc09dfb2-model-cache" (OuterVolumeSpecName: "model-cache") pod "9598e003-9e30-4da8-b722-ea71cc09dfb2" (UID: "9598e003-9e30-4da8-b722-ea71cc09dfb2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 14:17:46.320529 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:46.320505    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9598e003-9e30-4da8-b722-ea71cc09dfb2-home" (OuterVolumeSpecName: "home") pod "9598e003-9e30-4da8-b722-ea71cc09dfb2" (UID: "9598e003-9e30-4da8-b722-ea71cc09dfb2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 14:17:46.320618 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:46.320557    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9598e003-9e30-4da8-b722-ea71cc09dfb2-tmp-dir\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:17:46.320618 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:46.320579    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9598e003-9e30-4da8-b722-ea71cc09dfb2-model-cache\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:17:46.320618 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:46.320590    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9598e003-9e30-4da8-b722-ea71cc09dfb2-home\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:17:46.322219 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:46.322200    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9598e003-9e30-4da8-b722-ea71cc09dfb2-kube-api-access-5scpm" (OuterVolumeSpecName: "kube-api-access-5scpm") pod "9598e003-9e30-4da8-b722-ea71cc09dfb2" (UID: "9598e003-9e30-4da8-b722-ea71cc09dfb2"). InnerVolumeSpecName "kube-api-access-5scpm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:17:46.322580 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:46.322558    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9598e003-9e30-4da8-b722-ea71cc09dfb2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9598e003-9e30-4da8-b722-ea71cc09dfb2" (UID: "9598e003-9e30-4da8-b722-ea71cc09dfb2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:17:46.322580 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:46.322562    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9598e003-9e30-4da8-b722-ea71cc09dfb2-dshm" (OuterVolumeSpecName: "dshm") pod "9598e003-9e30-4da8-b722-ea71cc09dfb2" (UID: "9598e003-9e30-4da8-b722-ea71cc09dfb2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 14:17:46.421498 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:46.421455    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9598e003-9e30-4da8-b722-ea71cc09dfb2-tls-certs\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:17:46.421498 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:46.421491    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9598e003-9e30-4da8-b722-ea71cc09dfb2-dshm\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:17:46.421498 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:46.421500    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5scpm\" (UniqueName: \"kubernetes.io/projected/9598e003-9e30-4da8-b722-ea71cc09dfb2-kube-api-access-5scpm\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:17:46.590566 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:46.590538    2578 generic.go:356] "Generic (PLEG): container finished" podID="9598e003-9e30-4da8-b722-ea71cc09dfb2" containerID="dd9f24e29fc9aa9c84de0e5deba940d0475f6fcddb7c5e6d2b561294bc8214f7" exitCode=0
Jul 21 14:17:46.590731 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:46.590578    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-9b3e9882/e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6"
Jul 21 14:17:46.590731 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:46.590611    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-9b3e9882/e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6" event={"ID":"9598e003-9e30-4da8-b722-ea71cc09dfb2","Type":"ContainerDied","Data":"dd9f24e29fc9aa9c84de0e5deba940d0475f6fcddb7c5e6d2b561294bc8214f7"}
Jul 21 14:17:46.590731 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:46.590642    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-9b3e9882/e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6" event={"ID":"9598e003-9e30-4da8-b722-ea71cc09dfb2","Type":"ContainerDied","Data":"1697dc66e22ece67afb9eae99cebc12dfd12671f3ba449cf82a43bf7e45097c0"}
Jul 21 14:17:46.590731 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:46.590657    2578 scope.go:117] "RemoveContainer" containerID="dd9f24e29fc9aa9c84de0e5deba940d0475f6fcddb7c5e6d2b561294bc8214f7"
Jul 21 14:17:46.602172 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:46.602121    2578 scope.go:117] "RemoveContainer" containerID="dd9f24e29fc9aa9c84de0e5deba940d0475f6fcddb7c5e6d2b561294bc8214f7"
Jul 21 14:17:46.602625 ip-10-0-132-245 kubenswrapper[2578]: E0721 14:17:46.602602    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd9f24e29fc9aa9c84de0e5deba940d0475f6fcddb7c5e6d2b561294bc8214f7\": container with ID starting with dd9f24e29fc9aa9c84de0e5deba940d0475f6fcddb7c5e6d2b561294bc8214f7 not found: ID does not exist" containerID="dd9f24e29fc9aa9c84de0e5deba940d0475f6fcddb7c5e6d2b561294bc8214f7"
Jul 21 14:17:46.602728 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:46.602633    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd9f24e29fc9aa9c84de0e5deba940d0475f6fcddb7c5e6d2b561294bc8214f7"} err="failed to get container status \"dd9f24e29fc9aa9c84de0e5deba940d0475f6fcddb7c5e6d2b561294bc8214f7\": rpc error: code = NotFound desc = could not find container \"dd9f24e29fc9aa9c84de0e5deba940d0475f6fcddb7c5e6d2b561294bc8214f7\": container with ID starting with dd9f24e29fc9aa9c84de0e5deba940d0475f6fcddb7c5e6d2b561294bc8214f7 not found: ID does not exist"
Jul 21 14:17:46.612234 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:46.612210    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-9b3e9882/e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6"]
Jul 21 14:17:46.615945 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:46.615924    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-9b3e9882/e2e-lifecycle-model-9b3e9882-kserve-7757f5ddb7-mqbx6"]
Jul 21 14:17:47.503891 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:17:47.503835    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9598e003-9e30-4da8-b722-ea71cc09dfb2" path="/var/lib/kubelet/pods/9598e003-9e30-4da8-b722-ea71cc09dfb2/volumes"
Jul 21 14:18:13.581429 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:18:13.581395    2578 scope.go:117] "RemoveContainer" containerID="d543bf1c07c0c9878d9376f17bbc11bb8220a1251d28fdaee021173b64a3390a"
Jul 21 14:18:13.589378 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:18:13.589361    2578 scope.go:117] "RemoveContainer" containerID="a145c791a523d53bbd9ad674dec791c03e9a574feaba6079b72a3f3744b908c8"
Jul 21 14:18:25.473400 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:18:25.473359    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:18:38.869001 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:18:38.868917    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:19:13.617450 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:13.617420    2578 scope.go:117] "RemoveContainer" containerID="f13cb62f553db5f9b41683a26bc35c97460b5e9b1c3bbd3fc82088c6dc242ccd"
Jul 21 14:19:22.785434 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:22.785395    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ce81f3-7c9fcb8f87-q9bpx"]
Jul 21 14:19:22.785797 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:22.785717    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b65d4244-ebf8-4f39-9808-89edbeb09108" containerName="maas-api"
Jul 21 14:19:22.785797 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:22.785730    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b65d4244-ebf8-4f39-9808-89edbeb09108" containerName="maas-api"
Jul 21 14:19:22.785797 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:22.785752    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6776fd61-75c0-4cfb-adc9-acee4e5ce5bc" containerName="revoke-keys"
Jul 21 14:19:22.785797 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:22.785758    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6776fd61-75c0-4cfb-adc9-acee4e5ce5bc" containerName="revoke-keys"
Jul 21 14:19:22.785797 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:22.785764    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9598e003-9e30-4da8-b722-ea71cc09dfb2" containerName="main"
Jul 21 14:19:22.785797 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:22.785770    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9598e003-9e30-4da8-b722-ea71cc09dfb2" containerName="main"
Jul 21 14:19:22.786004 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:22.785817    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9598e003-9e30-4da8-b722-ea71cc09dfb2" containerName="main"
Jul 21 14:19:22.786004 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:22.785828    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b65d4244-ebf8-4f39-9808-89edbeb09108" containerName="maas-api"
Jul 21 14:19:22.786004 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:22.785835    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6776fd61-75c0-4cfb-adc9-acee4e5ce5bc" containerName="revoke-keys"
Jul 21 14:19:22.788953 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:22.788934    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce81f3-7c9fcb8f87-q9bpx"
Jul 21 14:19:22.791572 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:22.791550    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-ce81f3-serving-cert\""
Jul 21 14:19:22.791699 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:22.791647    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 14:19:22.791985 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:22.791966    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 14:19:22.792682 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:22.792651    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-nhfmq\""
Jul 21 14:19:22.801393 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:22.801371    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ce81f3-7c9fcb8f87-q9bpx"]
Jul 21 14:19:22.898279 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:22.898243    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d42ce96-1ba2-48e7-84d5-80de72e13096-maas-api-tls\") pod \"maas-api-e2e-inf-a-ce81f3-7c9fcb8f87-q9bpx\" (UID: \"4d42ce96-1ba2-48e7-84d5-80de72e13096\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce81f3-7c9fcb8f87-q9bpx"
Jul 21 14:19:22.898458 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:22.898286    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d42ce96-1ba2-48e7-84d5-80de72e13096-openshift-service-ca\") pod \"maas-api-e2e-inf-a-ce81f3-7c9fcb8f87-q9bpx\" (UID: \"4d42ce96-1ba2-48e7-84d5-80de72e13096\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce81f3-7c9fcb8f87-q9bpx"
Jul 21 14:19:22.898458 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:22.898419    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mlqs\" (UniqueName: \"kubernetes.io/projected/4d42ce96-1ba2-48e7-84d5-80de72e13096-kube-api-access-8mlqs\") pod \"maas-api-e2e-inf-a-ce81f3-7c9fcb8f87-q9bpx\" (UID: \"4d42ce96-1ba2-48e7-84d5-80de72e13096\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce81f3-7c9fcb8f87-q9bpx"
Jul 21 14:19:22.998825 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:22.998787    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mlqs\" (UniqueName: \"kubernetes.io/projected/4d42ce96-1ba2-48e7-84d5-80de72e13096-kube-api-access-8mlqs\") pod \"maas-api-e2e-inf-a-ce81f3-7c9fcb8f87-q9bpx\" (UID: \"4d42ce96-1ba2-48e7-84d5-80de72e13096\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce81f3-7c9fcb8f87-q9bpx"
Jul 21 14:19:22.999049 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:22.998847    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d42ce96-1ba2-48e7-84d5-80de72e13096-maas-api-tls\") pod \"maas-api-e2e-inf-a-ce81f3-7c9fcb8f87-q9bpx\" (UID: \"4d42ce96-1ba2-48e7-84d5-80de72e13096\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce81f3-7c9fcb8f87-q9bpx"
Jul 21 14:19:22.999049 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:22.998888    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d42ce96-1ba2-48e7-84d5-80de72e13096-openshift-service-ca\") pod \"maas-api-e2e-inf-a-ce81f3-7c9fcb8f87-q9bpx\" (UID: \"4d42ce96-1ba2-48e7-84d5-80de72e13096\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce81f3-7c9fcb8f87-q9bpx"
Jul 21 14:19:22.999049 ip-10-0-132-245 kubenswrapper[2578]: E0721 14:19:22.998970    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-ce81f3-serving-cert: secret "maas-api-e2e-inf-a-ce81f3-serving-cert" not found
Jul 21 14:19:22.999049 ip-10-0-132-245 kubenswrapper[2578]: E0721 14:19:22.999050    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d42ce96-1ba2-48e7-84d5-80de72e13096-maas-api-tls podName:4d42ce96-1ba2-48e7-84d5-80de72e13096 nodeName:}" failed. No retries permitted until 2026-07-21 14:19:23.499033423 +0000 UTC m=+2110.459321958 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4d42ce96-1ba2-48e7-84d5-80de72e13096-maas-api-tls") pod "maas-api-e2e-inf-a-ce81f3-7c9fcb8f87-q9bpx" (UID: "4d42ce96-1ba2-48e7-84d5-80de72e13096") : secret "maas-api-e2e-inf-a-ce81f3-serving-cert" not found
Jul 21 14:19:22.999461 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:22.999440    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d42ce96-1ba2-48e7-84d5-80de72e13096-openshift-service-ca\") pod \"maas-api-e2e-inf-a-ce81f3-7c9fcb8f87-q9bpx\" (UID: \"4d42ce96-1ba2-48e7-84d5-80de72e13096\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce81f3-7c9fcb8f87-q9bpx"
Jul 21 14:19:23.009395 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:23.009358    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mlqs\" (UniqueName: \"kubernetes.io/projected/4d42ce96-1ba2-48e7-84d5-80de72e13096-kube-api-access-8mlqs\") pod \"maas-api-e2e-inf-a-ce81f3-7c9fcb8f87-q9bpx\" (UID: \"4d42ce96-1ba2-48e7-84d5-80de72e13096\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce81f3-7c9fcb8f87-q9bpx"
Jul 21 14:19:23.502686 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:23.502653    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d42ce96-1ba2-48e7-84d5-80de72e13096-maas-api-tls\") pod \"maas-api-e2e-inf-a-ce81f3-7c9fcb8f87-q9bpx\" (UID: \"4d42ce96-1ba2-48e7-84d5-80de72e13096\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce81f3-7c9fcb8f87-q9bpx"
Jul 21 14:19:23.505260 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:23.505233    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d42ce96-1ba2-48e7-84d5-80de72e13096-maas-api-tls\") pod \"maas-api-e2e-inf-a-ce81f3-7c9fcb8f87-q9bpx\" (UID: \"4d42ce96-1ba2-48e7-84d5-80de72e13096\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce81f3-7c9fcb8f87-q9bpx"
Jul 21 14:19:23.699308 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:23.699268    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce81f3-7c9fcb8f87-q9bpx"
Jul 21 14:19:23.833068 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:23.833043    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ce81f3-7c9fcb8f87-q9bpx"]
Jul 21 14:19:23.835066 ip-10-0-132-245 kubenswrapper[2578]: W0721 14:19:23.835040    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d42ce96_1ba2_48e7_84d5_80de72e13096.slice/crio-ad31be03269b90a65fb235ebc1bf40b9e55b9e2a70d2b0f252919d86669543ab WatchSource:0}: Error finding container ad31be03269b90a65fb235ebc1bf40b9e55b9e2a70d2b0f252919d86669543ab: Status 404 returned error can't find the container with id ad31be03269b90a65fb235ebc1bf40b9e55b9e2a70d2b0f252919d86669543ab
Jul 21 14:19:24.016763 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:24.016725    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce81f3-7c9fcb8f87-q9bpx" event={"ID":"4d42ce96-1ba2-48e7-84d5-80de72e13096","Type":"ContainerStarted","Data":"ad31be03269b90a65fb235ebc1bf40b9e55b9e2a70d2b0f252919d86669543ab"}
Jul 21 14:19:25.022354 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:25.022317    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce81f3-7c9fcb8f87-q9bpx" event={"ID":"4d42ce96-1ba2-48e7-84d5-80de72e13096","Type":"ContainerStarted","Data":"08195dc5105e4668604d93c315361cdcde976ecd1fd38b19f7e047de3a18a588"}
Jul 21 14:19:25.022709 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:25.022384    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce81f3-7c9fcb8f87-q9bpx"
Jul 21 14:19:25.040923 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:25.040852    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce81f3-7c9fcb8f87-q9bpx" podStartSLOduration=2.664296173 podStartE2EDuration="3.040837594s" totalImagesPullingTime="376.541421ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:19:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:19:25.039932145 +0000 UTC m=+2112.000220700" watchObservedRunningTime="2026-07-21 14:19:25.040837594 +0000 UTC m=+2112.001126150"
Jul 21 14:19:28.075552 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:28.075516    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:19:30.823679 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:30.823645    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-faeda6-594d7f6c5-n44k8"]
Jul 21 14:19:30.827139 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:30.827106    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faeda6-594d7f6c5-n44k8"
Jul 21 14:19:30.830588 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:30.830567    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-faeda6-serving-cert\""
Jul 21 14:19:30.839540 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:30.839514    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-faeda6-594d7f6c5-n44k8"]
Jul 21 14:19:30.863045 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:30.863017    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9-maas-api-tls\") pod \"maas-api-e2e-inf-b-faeda6-594d7f6c5-n44k8\" (UID: \"81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faeda6-594d7f6c5-n44k8"
Jul 21 14:19:30.863200 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:30.863053    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbg8j\" (UniqueName: \"kubernetes.io/projected/81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9-kube-api-access-mbg8j\") pod \"maas-api-e2e-inf-b-faeda6-594d7f6c5-n44k8\" (UID: \"81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faeda6-594d7f6c5-n44k8"
Jul 21 14:19:30.863200 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:30.863160    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9-openshift-service-ca\") pod \"maas-api-e2e-inf-b-faeda6-594d7f6c5-n44k8\" (UID: \"81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faeda6-594d7f6c5-n44k8"
Jul 21 14:19:30.963627 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:30.963591    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9-openshift-service-ca\") pod \"maas-api-e2e-inf-b-faeda6-594d7f6c5-n44k8\" (UID: \"81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faeda6-594d7f6c5-n44k8"
Jul 21 14:19:30.963826 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:30.963657    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9-maas-api-tls\") pod \"maas-api-e2e-inf-b-faeda6-594d7f6c5-n44k8\" (UID: \"81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faeda6-594d7f6c5-n44k8"
Jul 21 14:19:30.963826 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:30.963684    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbg8j\" (UniqueName: \"kubernetes.io/projected/81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9-kube-api-access-mbg8j\") pod \"maas-api-e2e-inf-b-faeda6-594d7f6c5-n44k8\" (UID: \"81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faeda6-594d7f6c5-n44k8"
Jul 21 14:19:30.963957 ip-10-0-132-245 kubenswrapper[2578]: E0721 14:19:30.963869    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-faeda6-serving-cert: secret "maas-api-e2e-inf-b-faeda6-serving-cert" not found
Jul 21 14:19:30.964030 ip-10-0-132-245 kubenswrapper[2578]: E0721 14:19:30.964017    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9-maas-api-tls podName:81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9 nodeName:}" failed. No retries permitted until 2026-07-21 14:19:31.463984296 +0000 UTC m=+2118.424272846 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9-maas-api-tls") pod "maas-api-e2e-inf-b-faeda6-594d7f6c5-n44k8" (UID: "81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9") : secret "maas-api-e2e-inf-b-faeda6-serving-cert" not found
Jul 21 14:19:30.964397 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:30.964377    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9-openshift-service-ca\") pod \"maas-api-e2e-inf-b-faeda6-594d7f6c5-n44k8\" (UID: \"81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faeda6-594d7f6c5-n44k8"
Jul 21 14:19:30.976479 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:30.976455    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbg8j\" (UniqueName: \"kubernetes.io/projected/81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9-kube-api-access-mbg8j\") pod \"maas-api-e2e-inf-b-faeda6-594d7f6c5-n44k8\" (UID: \"81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faeda6-594d7f6c5-n44k8"
Jul 21 14:19:31.032174 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:31.032143    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce81f3-7c9fcb8f87-q9bpx"
Jul 21 14:19:31.468254 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:31.468216    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9-maas-api-tls\") pod \"maas-api-e2e-inf-b-faeda6-594d7f6c5-n44k8\" (UID: \"81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faeda6-594d7f6c5-n44k8"
Jul 21 14:19:31.470781 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:31.470757    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9-maas-api-tls\") pod \"maas-api-e2e-inf-b-faeda6-594d7f6c5-n44k8\" (UID: \"81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faeda6-594d7f6c5-n44k8"
Jul 21 14:19:31.737976 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:31.737858    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faeda6-594d7f6c5-n44k8"
Jul 21 14:19:31.873692 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:31.873667    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-faeda6-594d7f6c5-n44k8"]
Jul 21 14:19:31.876220 ip-10-0-132-245 kubenswrapper[2578]: W0721 14:19:31.876190    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81b1a0bf_676b_4b9b_aa0e_d7fe3d2643e9.slice/crio-149148f72b6f590ff6a16019db9e630954693a0fca44909f9ff86cbb69b20432 WatchSource:0}: Error finding container 149148f72b6f590ff6a16019db9e630954693a0fca44909f9ff86cbb69b20432: Status 404 returned error can't find the container with id 149148f72b6f590ff6a16019db9e630954693a0fca44909f9ff86cbb69b20432
Jul 21 14:19:32.050995 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:32.050906    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faeda6-594d7f6c5-n44k8" event={"ID":"81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9","Type":"ContainerStarted","Data":"149148f72b6f590ff6a16019db9e630954693a0fca44909f9ff86cbb69b20432"}
Jul 21 14:19:33.056197 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:33.056162    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faeda6-594d7f6c5-n44k8" event={"ID":"81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9","Type":"ContainerStarted","Data":"0bfe3c25ecb2cee17347fdd543b06b7eefd220b4207b5e9e1143f4437acd2ea8"}
Jul 21 14:19:33.056680 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:33.056293    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faeda6-594d7f6c5-n44k8"
Jul 21 14:19:33.075018 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:33.074968    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faeda6-594d7f6c5-n44k8" podStartSLOduration=2.716903739 podStartE2EDuration="3.074954942s" totalImagesPullingTime="358.051203ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:19:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:19:33.074065307 +0000 UTC m=+2120.034353876" watchObservedRunningTime="2026-07-21 14:19:33.074954942 +0000 UTC m=+2120.035243498"
Jul 21 14:19:39.066135 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:39.066107    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faeda6-594d7f6c5-n44k8"
Jul 21 14:19:44.564615 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:44.564570    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:19:51.467478 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:51.467438    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-ce81f3/test-model-ce81f3-kserve-774f99b857-wcpkg"]
Jul 21 14:19:51.471354 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:51.471317    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-ce81f3/test-model-ce81f3-kserve-774f99b857-wcpkg"
Jul 21 14:19:51.475894 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:51.475861    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-ce81f3\"/\"default-dockercfg-shxvb\""
Jul 21 14:19:51.476062 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:51.475862    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-ce81f3\"/\"test-model-ce81f3-kserve-self-signed-certs\""
Jul 21 14:19:51.476062 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:51.475863    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-ce81f3\"/\"openshift-service-ca.crt\""
Jul 21 14:19:51.476062 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:51.476049    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-ce81f3\"/\"kube-root-ca.crt\""
Jul 21 14:19:51.482443 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:51.482419    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-ce81f3/test-model-ce81f3-kserve-774f99b857-wcpkg"]
Jul 21 14:19:51.545148 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:51.545116    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03328917-9996-4a60-ae00-04ac5cee33f6-home\") pod \"test-model-ce81f3-kserve-774f99b857-wcpkg\" (UID: \"03328917-9996-4a60-ae00-04ac5cee33f6\") " pod="ai-tenant-e2e-inf-a-ce81f3/test-model-ce81f3-kserve-774f99b857-wcpkg"
Jul 21 14:19:51.545148 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:51.545149    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03328917-9996-4a60-ae00-04ac5cee33f6-model-cache\") pod \"test-model-ce81f3-kserve-774f99b857-wcpkg\" (UID: \"03328917-9996-4a60-ae00-04ac5cee33f6\") " pod="ai-tenant-e2e-inf-a-ce81f3/test-model-ce81f3-kserve-774f99b857-wcpkg"
Jul 21 14:19:51.545348 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:51.545181    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03328917-9996-4a60-ae00-04ac5cee33f6-tls-certs\") pod \"test-model-ce81f3-kserve-774f99b857-wcpkg\" (UID: \"03328917-9996-4a60-ae00-04ac5cee33f6\") " pod="ai-tenant-e2e-inf-a-ce81f3/test-model-ce81f3-kserve-774f99b857-wcpkg"
Jul 21 14:19:51.545348 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:51.545208    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxzr2\" (UniqueName: \"kubernetes.io/projected/03328917-9996-4a60-ae00-04ac5cee33f6-kube-api-access-jxzr2\") pod \"test-model-ce81f3-kserve-774f99b857-wcpkg\" (UID: \"03328917-9996-4a60-ae00-04ac5cee33f6\") " pod="ai-tenant-e2e-inf-a-ce81f3/test-model-ce81f3-kserve-774f99b857-wcpkg"
Jul 21 14:19:51.545348 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:51.545249    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03328917-9996-4a60-ae00-04ac5cee33f6-dshm\") pod \"test-model-ce81f3-kserve-774f99b857-wcpkg\" (UID: \"03328917-9996-4a60-ae00-04ac5cee33f6\") " pod="ai-tenant-e2e-inf-a-ce81f3/test-model-ce81f3-kserve-774f99b857-wcpkg"
Jul 21 14:19:51.545348 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:51.545266    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03328917-9996-4a60-ae00-04ac5cee33f6-tmp-dir\") pod \"test-model-ce81f3-kserve-774f99b857-wcpkg\" (UID: \"03328917-9996-4a60-ae00-04ac5cee33f6\") " pod="ai-tenant-e2e-inf-a-ce81f3/test-model-ce81f3-kserve-774f99b857-wcpkg"
Jul 21 14:19:51.646293 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:51.646261    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03328917-9996-4a60-ae00-04ac5cee33f6-home\") pod \"test-model-ce81f3-kserve-774f99b857-wcpkg\" (UID: \"03328917-9996-4a60-ae00-04ac5cee33f6\") " pod="ai-tenant-e2e-inf-a-ce81f3/test-model-ce81f3-kserve-774f99b857-wcpkg"
Jul 21 14:19:51.646293 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:51.646297    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03328917-9996-4a60-ae00-04ac5cee33f6-model-cache\") pod \"test-model-ce81f3-kserve-774f99b857-wcpkg\" (UID: \"03328917-9996-4a60-ae00-04ac5cee33f6\") " pod="ai-tenant-e2e-inf-a-ce81f3/test-model-ce81f3-kserve-774f99b857-wcpkg"
Jul 21 14:19:51.646545 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:51.646340    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03328917-9996-4a60-ae00-04ac5cee33f6-tls-certs\") pod \"test-model-ce81f3-kserve-774f99b857-wcpkg\" (UID: \"03328917-9996-4a60-ae00-04ac5cee33f6\") " pod="ai-tenant-e2e-inf-a-ce81f3/test-model-ce81f3-kserve-774f99b857-wcpkg"
Jul 21 14:19:51.646545 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:51.646364    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxzr2\" (UniqueName: \"kubernetes.io/projected/03328917-9996-4a60-ae00-04ac5cee33f6-kube-api-access-jxzr2\") pod \"test-model-ce81f3-kserve-774f99b857-wcpkg\" (UID: \"03328917-9996-4a60-ae00-04ac5cee33f6\") " pod="ai-tenant-e2e-inf-a-ce81f3/test-model-ce81f3-kserve-774f99b857-wcpkg"
Jul 21 14:19:51.646545 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:51.646413    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03328917-9996-4a60-ae00-04ac5cee33f6-dshm\") pod \"test-model-ce81f3-kserve-774f99b857-wcpkg\" (UID: \"03328917-9996-4a60-ae00-04ac5cee33f6\") " pod="ai-tenant-e2e-inf-a-ce81f3/test-model-ce81f3-kserve-774f99b857-wcpkg"
Jul 21 14:19:51.646545 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:51.646434    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03328917-9996-4a60-ae00-04ac5cee33f6-tmp-dir\") pod \"test-model-ce81f3-kserve-774f99b857-wcpkg\" (UID: \"03328917-9996-4a60-ae00-04ac5cee33f6\") " pod="ai-tenant-e2e-inf-a-ce81f3/test-model-ce81f3-kserve-774f99b857-wcpkg"
Jul 21 14:19:51.646754 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:51.646702    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03328917-9996-4a60-ae00-04ac5cee33f6-home\") pod \"test-model-ce81f3-kserve-774f99b857-wcpkg\" (UID: \"03328917-9996-4a60-ae00-04ac5cee33f6\") " pod="ai-tenant-e2e-inf-a-ce81f3/test-model-ce81f3-kserve-774f99b857-wcpkg"
Jul 21 14:19:51.646793 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:51.646760    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03328917-9996-4a60-ae00-04ac5cee33f6-model-cache\") pod \"test-model-ce81f3-kserve-774f99b857-wcpkg\" (UID: \"03328917-9996-4a60-ae00-04ac5cee33f6\") " pod="ai-tenant-e2e-inf-a-ce81f3/test-model-ce81f3-kserve-774f99b857-wcpkg"
Jul 21 14:19:51.646895 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:51.646854    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03328917-9996-4a60-ae00-04ac5cee33f6-tmp-dir\") pod \"test-model-ce81f3-kserve-774f99b857-wcpkg\" (UID: \"03328917-9996-4a60-ae00-04ac5cee33f6\") " pod="ai-tenant-e2e-inf-a-ce81f3/test-model-ce81f3-kserve-774f99b857-wcpkg"
Jul 21 14:19:51.648723 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:51.648703    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03328917-9996-4a60-ae00-04ac5cee33f6-dshm\") pod \"test-model-ce81f3-kserve-774f99b857-wcpkg\" (UID: \"03328917-9996-4a60-ae00-04ac5cee33f6\") " pod="ai-tenant-e2e-inf-a-ce81f3/test-model-ce81f3-kserve-774f99b857-wcpkg"
Jul 21 14:19:51.649032 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:51.649012    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03328917-9996-4a60-ae00-04ac5cee33f6-tls-certs\") pod \"test-model-ce81f3-kserve-774f99b857-wcpkg\" (UID: \"03328917-9996-4a60-ae00-04ac5cee33f6\") " pod="ai-tenant-e2e-inf-a-ce81f3/test-model-ce81f3-kserve-774f99b857-wcpkg"
Jul 21 14:19:51.656984 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:51.656961    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxzr2\" (UniqueName: \"kubernetes.io/projected/03328917-9996-4a60-ae00-04ac5cee33f6-kube-api-access-jxzr2\") pod \"test-model-ce81f3-kserve-774f99b857-wcpkg\" (UID: \"03328917-9996-4a60-ae00-04ac5cee33f6\") " pod="ai-tenant-e2e-inf-a-ce81f3/test-model-ce81f3-kserve-774f99b857-wcpkg"
Jul 21 14:19:51.782300 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:51.782203    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-ce81f3/test-model-ce81f3-kserve-774f99b857-wcpkg"
Jul 21 14:19:51.912132 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:51.911974    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-ce81f3/test-model-ce81f3-kserve-774f99b857-wcpkg"]
Jul 21 14:19:51.914694 ip-10-0-132-245 kubenswrapper[2578]: W0721 14:19:51.914668    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod03328917_9996_4a60_ae00_04ac5cee33f6.slice/crio-106f490dc5561ae428ebc3ab1340912e3522ffb871dff8d8e0d994c5ee689f8b WatchSource:0}: Error finding container 106f490dc5561ae428ebc3ab1340912e3522ffb871dff8d8e0d994c5ee689f8b: Status 404 returned error can't find the container with id 106f490dc5561ae428ebc3ab1340912e3522ffb871dff8d8e0d994c5ee689f8b
Jul 21 14:19:52.142647 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:52.142606    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-ce81f3/test-model-ce81f3-kserve-774f99b857-wcpkg" event={"ID":"03328917-9996-4a60-ae00-04ac5cee33f6","Type":"ContainerStarted","Data":"faee9205f16e6d0379e96a9007ecfd39dd3c7f43abc3fbc70b36e989b3734387"}
Jul 21 14:19:52.142647 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:52.142653    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-ce81f3/test-model-ce81f3-kserve-774f99b857-wcpkg" event={"ID":"03328917-9996-4a60-ae00-04ac5cee33f6","Type":"ContainerStarted","Data":"106f490dc5561ae428ebc3ab1340912e3522ffb871dff8d8e0d994c5ee689f8b"}
Jul 21 14:19:52.167379 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:52.167328    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-ce81f3/test-model-ce81f3-kserve-774f99b857-wcpkg" podStartSLOduration=1.167312675 podStartE2EDuration="1.167312675s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:19:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:19:52.165641957 +0000 UTC m=+2139.125930516" watchObservedRunningTime="2026-07-21 14:19:52.167312675 +0000 UTC m=+2139.127601232"
Jul 21 14:19:53.371319 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:19:53.371284    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:20:01.782459 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:20:01.782418    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-ce81f3/test-model-ce81f3-kserve-774f99b857-wcpkg"
Jul 21 14:20:01.782459 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:20:01.782467    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-ce81f3/test-model-ce81f3-kserve-774f99b857-wcpkg"
Jul 21 14:20:01.794939 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:20:01.794918    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-ce81f3/test-model-ce81f3-kserve-774f99b857-wcpkg"
Jul 21 14:20:02.195922 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:20:02.195871    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-ce81f3/test-model-ce81f3-kserve-774f99b857-wcpkg"
Jul 21 14:20:08.166475 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:20:08.166385    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:20:13.634369 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:20:13.634339    2578 scope.go:117] "RemoveContainer" containerID="81d9b03c6a52e6d71fca531a03b900b0aec9de02046195da4b5a8fd9bd621508"
Jul 21 14:20:19.362525 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:20:19.362488    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:20:31.570936 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:20:31.570898    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:20:54.684764 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:20:54.684730    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-ce81f3/test-model-ce81f3-kserve-774f99b857-wcpkg"]
Jul 21 14:20:55.785213 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:20:55.785180    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ce81f3-ms5sd"]
Jul 21 14:20:55.788329 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:20:55.788312    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ce81f3-ms5sd"
Jul 21 14:20:55.792034 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:20:55.792015    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5t9ph\""
Jul 21 14:20:55.811508 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:20:55.811482    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ce81f3-ms5sd"]
Jul 21 14:20:55.892277 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:20:55.892234    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64126af8-66c9-48d1-83dd-0f3bff4b07c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-ce81f3-ms5sd\" (UID: \"64126af8-66c9-48d1-83dd-0f3bff4b07c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ce81f3-ms5sd"
Jul 21 14:20:55.993707 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:20:55.993671    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64126af8-66c9-48d1-83dd-0f3bff4b07c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-ce81f3-ms5sd\" (UID: \"64126af8-66c9-48d1-83dd-0f3bff4b07c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ce81f3-ms5sd"
Jul 21 14:20:55.994327 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:20:55.994306    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64126af8-66c9-48d1-83dd-0f3bff4b07c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-ce81f3-ms5sd\" (UID: \"64126af8-66c9-48d1-83dd-0f3bff4b07c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ce81f3-ms5sd"
Jul 21 14:20:56.098104 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:20:56.098076    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ce81f3-ms5sd"
Jul 21 14:20:56.227054 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:20:56.227032    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ce81f3-ms5sd"]
Jul 21 14:20:56.229732 ip-10-0-132-245 kubenswrapper[2578]: W0721 14:20:56.229691    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64126af8_66c9_48d1_83dd_0f3bff4b07c3.slice/crio-f121e38ab27f27f219a4e5d48a35fb6e8519f9c11ad5fbea9b9870c0782f0c08 WatchSource:0}: Error finding container f121e38ab27f27f219a4e5d48a35fb6e8519f9c11ad5fbea9b9870c0782f0c08: Status 404 returned error can't find the container with id f121e38ab27f27f219a4e5d48a35fb6e8519f9c11ad5fbea9b9870c0782f0c08
Jul 21 14:20:56.419586 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:20:56.419553    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ce81f3-ms5sd" event={"ID":"64126af8-66c9-48d1-83dd-0f3bff4b07c3","Type":"ContainerStarted","Data":"f121e38ab27f27f219a4e5d48a35fb6e8519f9c11ad5fbea9b9870c0782f0c08"}
Jul 21 14:20:57.425486 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:20:57.425448    2578 generic.go:356] "Generic (PLEG): container finished" podID="64126af8-66c9-48d1-83dd-0f3bff4b07c3" containerID="ed5abb27daf8e76300adb82796a7db35206f5e8594f4cc44c49927b60b936d74" exitCode=0
Jul 21 14:20:57.425870 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:20:57.425516    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ce81f3-ms5sd" event={"ID":"64126af8-66c9-48d1-83dd-0f3bff4b07c3","Type":"ContainerDied","Data":"ed5abb27daf8e76300adb82796a7db35206f5e8594f4cc44c49927b60b936d74"}
Jul 21 14:20:58.565335 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:20:58.565305    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ce81f3-ms5sd"
Jul 21 14:20:58.717311 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:20:58.717213    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64126af8-66c9-48d1-83dd-0f3bff4b07c3-maas-api-service-ca\") pod \"64126af8-66c9-48d1-83dd-0f3bff4b07c3\" (UID: \"64126af8-66c9-48d1-83dd-0f3bff4b07c3\") "
Jul 21 14:20:58.717598 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:20:58.717575    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64126af8-66c9-48d1-83dd-0f3bff4b07c3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "64126af8-66c9-48d1-83dd-0f3bff4b07c3" (UID: "64126af8-66c9-48d1-83dd-0f3bff4b07c3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:20:58.818484 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:20:58.818457    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64126af8-66c9-48d1-83dd-0f3bff4b07c3-maas-api-service-ca\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:20:59.365173 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:20:59.365138    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:20:59.434785 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:20:59.434753    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ce81f3-ms5sd"
Jul 21 14:20:59.434972 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:20:59.434794    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ce81f3-ms5sd" event={"ID":"64126af8-66c9-48d1-83dd-0f3bff4b07c3","Type":"ContainerDied","Data":"f121e38ab27f27f219a4e5d48a35fb6e8519f9c11ad5fbea9b9870c0782f0c08"}
Jul 21 14:20:59.434972 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:20:59.434835    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f121e38ab27f27f219a4e5d48a35fb6e8519f9c11ad5fbea9b9870c0782f0c08"
Jul 21 14:21:06.227696 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:06.227659    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ce81f3-7c9fcb8f87-q9bpx"]
Jul 21 14:21:06.228120 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:06.227941    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce81f3-7c9fcb8f87-q9bpx" podUID="4d42ce96-1ba2-48e7-84d5-80de72e13096" containerName="maas-api" containerID="cri-o://08195dc5105e4668604d93c315361cdcde976ecd1fd38b19f7e047de3a18a588" gracePeriod=30
Jul 21 14:21:06.679200 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:06.679176    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce81f3-7c9fcb8f87-q9bpx"
Jul 21 14:21:06.681174 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:06.681155    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d42ce96-1ba2-48e7-84d5-80de72e13096-openshift-service-ca\") pod \"4d42ce96-1ba2-48e7-84d5-80de72e13096\" (UID: \"4d42ce96-1ba2-48e7-84d5-80de72e13096\") "
Jul 21 14:21:06.681223 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:06.681195    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d42ce96-1ba2-48e7-84d5-80de72e13096-maas-api-tls\") pod \"4d42ce96-1ba2-48e7-84d5-80de72e13096\" (UID: \"4d42ce96-1ba2-48e7-84d5-80de72e13096\") "
Jul 21 14:21:06.681223 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:06.681216    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mlqs\" (UniqueName: \"kubernetes.io/projected/4d42ce96-1ba2-48e7-84d5-80de72e13096-kube-api-access-8mlqs\") pod \"4d42ce96-1ba2-48e7-84d5-80de72e13096\" (UID: \"4d42ce96-1ba2-48e7-84d5-80de72e13096\") "
Jul 21 14:21:06.681491 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:06.681466    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d42ce96-1ba2-48e7-84d5-80de72e13096-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4d42ce96-1ba2-48e7-84d5-80de72e13096" (UID: "4d42ce96-1ba2-48e7-84d5-80de72e13096"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:21:06.683158 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:06.683130    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d42ce96-1ba2-48e7-84d5-80de72e13096-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4d42ce96-1ba2-48e7-84d5-80de72e13096" (UID: "4d42ce96-1ba2-48e7-84d5-80de72e13096"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:21:06.683249 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:06.683208    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d42ce96-1ba2-48e7-84d5-80de72e13096-kube-api-access-8mlqs" (OuterVolumeSpecName: "kube-api-access-8mlqs") pod "4d42ce96-1ba2-48e7-84d5-80de72e13096" (UID: "4d42ce96-1ba2-48e7-84d5-80de72e13096"). InnerVolumeSpecName "kube-api-access-8mlqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:21:06.782069 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:06.781975    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4d42ce96-1ba2-48e7-84d5-80de72e13096-openshift-service-ca\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:21:06.782069 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:06.782026    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4d42ce96-1ba2-48e7-84d5-80de72e13096-maas-api-tls\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:21:06.782069 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:06.782036    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mlqs\" (UniqueName: \"kubernetes.io/projected/4d42ce96-1ba2-48e7-84d5-80de72e13096-kube-api-access-8mlqs\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:21:07.470799 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:07.470765    2578 generic.go:356] "Generic (PLEG): container finished" podID="4d42ce96-1ba2-48e7-84d5-80de72e13096" containerID="08195dc5105e4668604d93c315361cdcde976ecd1fd38b19f7e047de3a18a588" exitCode=0
Jul 21 14:21:07.471202 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:07.470807    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce81f3-7c9fcb8f87-q9bpx"
Jul 21 14:21:07.471202 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:07.470852    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce81f3-7c9fcb8f87-q9bpx" event={"ID":"4d42ce96-1ba2-48e7-84d5-80de72e13096","Type":"ContainerDied","Data":"08195dc5105e4668604d93c315361cdcde976ecd1fd38b19f7e047de3a18a588"}
Jul 21 14:21:07.471202 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:07.470915    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-ce81f3-7c9fcb8f87-q9bpx" event={"ID":"4d42ce96-1ba2-48e7-84d5-80de72e13096","Type":"ContainerDied","Data":"ad31be03269b90a65fb235ebc1bf40b9e55b9e2a70d2b0f252919d86669543ab"}
Jul 21 14:21:07.471202 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:07.470944    2578 scope.go:117] "RemoveContainer" containerID="08195dc5105e4668604d93c315361cdcde976ecd1fd38b19f7e047de3a18a588"
Jul 21 14:21:07.479958 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:07.479943    2578 scope.go:117] "RemoveContainer" containerID="08195dc5105e4668604d93c315361cdcde976ecd1fd38b19f7e047de3a18a588"
Jul 21 14:21:07.480218 ip-10-0-132-245 kubenswrapper[2578]: E0721 14:21:07.480197    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08195dc5105e4668604d93c315361cdcde976ecd1fd38b19f7e047de3a18a588\": container with ID starting with 08195dc5105e4668604d93c315361cdcde976ecd1fd38b19f7e047de3a18a588 not found: ID does not exist" containerID="08195dc5105e4668604d93c315361cdcde976ecd1fd38b19f7e047de3a18a588"
Jul 21 14:21:07.480264 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:07.480229    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08195dc5105e4668604d93c315361cdcde976ecd1fd38b19f7e047de3a18a588"} err="failed to get container status \"08195dc5105e4668604d93c315361cdcde976ecd1fd38b19f7e047de3a18a588\": rpc error: code = NotFound desc = could not find container \"08195dc5105e4668604d93c315361cdcde976ecd1fd38b19f7e047de3a18a588\": container with ID starting with 08195dc5105e4668604d93c315361cdcde976ecd1fd38b19f7e047de3a18a588 not found: ID does not exist"
Jul 21 14:21:07.493321 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:07.493295    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ce81f3-7c9fcb8f87-q9bpx"]
Jul 21 14:21:07.503213 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:07.503186    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-ce81f3-7c9fcb8f87-q9bpx"]
Jul 21 14:21:09.502674 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:09.502638    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d42ce96-1ba2-48e7-84d5-80de72e13096" path="/var/lib/kubelet/pods/4d42ce96-1ba2-48e7-84d5-80de72e13096/volumes"
Jul 21 14:21:09.704304 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:09.704248    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-ce81f3/test-model-ce81f3-kserve-774f99b857-wcpkg" podUID="03328917-9996-4a60-ae00-04ac5cee33f6" containerName="main" containerID="cri-o://faee9205f16e6d0379e96a9007ecfd39dd3c7f43abc3fbc70b36e989b3734387" gracePeriod=45
Jul 21 14:21:09.962765 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:09.962742    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-ce81f3/test-model-ce81f3-kserve-774f99b857-wcpkg"
Jul 21 14:21:10.009189 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:10.009155    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03328917-9996-4a60-ae00-04ac5cee33f6-tls-certs\") pod \"03328917-9996-4a60-ae00-04ac5cee33f6\" (UID: \"03328917-9996-4a60-ae00-04ac5cee33f6\") "
Jul 21 14:21:10.009352 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:10.009198    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03328917-9996-4a60-ae00-04ac5cee33f6-dshm\") pod \"03328917-9996-4a60-ae00-04ac5cee33f6\" (UID: \"03328917-9996-4a60-ae00-04ac5cee33f6\") "
Jul 21 14:21:10.009352 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:10.009232    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03328917-9996-4a60-ae00-04ac5cee33f6-model-cache\") pod \"03328917-9996-4a60-ae00-04ac5cee33f6\" (UID: \"03328917-9996-4a60-ae00-04ac5cee33f6\") "
Jul 21 14:21:10.009352 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:10.009283    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03328917-9996-4a60-ae00-04ac5cee33f6-tmp-dir\") pod \"03328917-9996-4a60-ae00-04ac5cee33f6\" (UID: \"03328917-9996-4a60-ae00-04ac5cee33f6\") "
Jul 21 14:21:10.009352 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:10.009320    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jxzr2\" (UniqueName: \"kubernetes.io/projected/03328917-9996-4a60-ae00-04ac5cee33f6-kube-api-access-jxzr2\") pod \"03328917-9996-4a60-ae00-04ac5cee33f6\" (UID: \"03328917-9996-4a60-ae00-04ac5cee33f6\") "
Jul 21 14:21:10.009352 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:10.009348    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03328917-9996-4a60-ae00-04ac5cee33f6-home\") pod \"03328917-9996-4a60-ae00-04ac5cee33f6\" (UID: \"03328917-9996-4a60-ae00-04ac5cee33f6\") "
Jul 21 14:21:10.009600 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:10.009565    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03328917-9996-4a60-ae00-04ac5cee33f6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "03328917-9996-4a60-ae00-04ac5cee33f6" (UID: "03328917-9996-4a60-ae00-04ac5cee33f6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 14:21:10.009798 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:10.009767    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03328917-9996-4a60-ae00-04ac5cee33f6-home" (OuterVolumeSpecName: "home") pod "03328917-9996-4a60-ae00-04ac5cee33f6" (UID: "03328917-9996-4a60-ae00-04ac5cee33f6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 14:21:10.009905 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:10.009833    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03328917-9996-4a60-ae00-04ac5cee33f6-model-cache" (OuterVolumeSpecName: "model-cache") pod "03328917-9996-4a60-ae00-04ac5cee33f6" (UID: "03328917-9996-4a60-ae00-04ac5cee33f6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 14:21:10.011482 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:10.011453    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03328917-9996-4a60-ae00-04ac5cee33f6-kube-api-access-jxzr2" (OuterVolumeSpecName: "kube-api-access-jxzr2") pod "03328917-9996-4a60-ae00-04ac5cee33f6" (UID: "03328917-9996-4a60-ae00-04ac5cee33f6"). InnerVolumeSpecName "kube-api-access-jxzr2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:21:10.011482 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:10.011454    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03328917-9996-4a60-ae00-04ac5cee33f6-dshm" (OuterVolumeSpecName: "dshm") pod "03328917-9996-4a60-ae00-04ac5cee33f6" (UID: "03328917-9996-4a60-ae00-04ac5cee33f6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 14:21:10.011613 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:10.011538    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03328917-9996-4a60-ae00-04ac5cee33f6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "03328917-9996-4a60-ae00-04ac5cee33f6" (UID: "03328917-9996-4a60-ae00-04ac5cee33f6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:21:10.110171 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:10.110133    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03328917-9996-4a60-ae00-04ac5cee33f6-model-cache\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:21:10.110171 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:10.110163    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03328917-9996-4a60-ae00-04ac5cee33f6-tmp-dir\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:21:10.110171 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:10.110173    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jxzr2\" (UniqueName: \"kubernetes.io/projected/03328917-9996-4a60-ae00-04ac5cee33f6-kube-api-access-jxzr2\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:21:10.110396 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:10.110183    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03328917-9996-4a60-ae00-04ac5cee33f6-home\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:21:10.110396 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:10.110192    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03328917-9996-4a60-ae00-04ac5cee33f6-tls-certs\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:21:10.110396 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:10.110201    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03328917-9996-4a60-ae00-04ac5cee33f6-dshm\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:21:10.485417 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:10.485323    2578 generic.go:356] "Generic (PLEG): container finished" podID="03328917-9996-4a60-ae00-04ac5cee33f6" containerID="faee9205f16e6d0379e96a9007ecfd39dd3c7f43abc3fbc70b36e989b3734387" exitCode=0
Jul 21 14:21:10.485417 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:10.485377    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-ce81f3/test-model-ce81f3-kserve-774f99b857-wcpkg"
Jul 21 14:21:10.485417 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:10.485390    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-ce81f3/test-model-ce81f3-kserve-774f99b857-wcpkg" event={"ID":"03328917-9996-4a60-ae00-04ac5cee33f6","Type":"ContainerDied","Data":"faee9205f16e6d0379e96a9007ecfd39dd3c7f43abc3fbc70b36e989b3734387"}
Jul 21 14:21:10.485687 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:10.485423    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-ce81f3/test-model-ce81f3-kserve-774f99b857-wcpkg" event={"ID":"03328917-9996-4a60-ae00-04ac5cee33f6","Type":"ContainerDied","Data":"106f490dc5561ae428ebc3ab1340912e3522ffb871dff8d8e0d994c5ee689f8b"}
Jul 21 14:21:10.485687 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:10.485439    2578 scope.go:117] "RemoveContainer" containerID="faee9205f16e6d0379e96a9007ecfd39dd3c7f43abc3fbc70b36e989b3734387"
Jul 21 14:21:10.493953 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:10.493933    2578 scope.go:117] "RemoveContainer" containerID="faee9205f16e6d0379e96a9007ecfd39dd3c7f43abc3fbc70b36e989b3734387"
Jul 21 14:21:10.494212 ip-10-0-132-245 kubenswrapper[2578]: E0721 14:21:10.494192    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"faee9205f16e6d0379e96a9007ecfd39dd3c7f43abc3fbc70b36e989b3734387\": container with ID starting with faee9205f16e6d0379e96a9007ecfd39dd3c7f43abc3fbc70b36e989b3734387 not found: ID does not exist" containerID="faee9205f16e6d0379e96a9007ecfd39dd3c7f43abc3fbc70b36e989b3734387"
Jul 21 14:21:10.494276 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:10.494225    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"faee9205f16e6d0379e96a9007ecfd39dd3c7f43abc3fbc70b36e989b3734387"} err="failed to get container status \"faee9205f16e6d0379e96a9007ecfd39dd3c7f43abc3fbc70b36e989b3734387\": rpc error: code = NotFound desc = could not find container \"faee9205f16e6d0379e96a9007ecfd39dd3c7f43abc3fbc70b36e989b3734387\": container with ID starting with faee9205f16e6d0379e96a9007ecfd39dd3c7f43abc3fbc70b36e989b3734387 not found: ID does not exist"
Jul 21 14:21:10.505421 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:10.505394    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-ce81f3/test-model-ce81f3-kserve-774f99b857-wcpkg"]
Jul 21 14:21:10.509559 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:10.509534    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-ce81f3/test-model-ce81f3-kserve-774f99b857-wcpkg"]
Jul 21 14:21:10.875659 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:10.875616    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ce81f3-ms5sd"]
Jul 21 14:21:10.880439 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:10.880413    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-ce81f3-ms5sd"]
Jul 21 14:21:11.502797 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:11.502762    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03328917-9996-4a60-ae00-04ac5cee33f6" path="/var/lib/kubelet/pods/03328917-9996-4a60-ae00-04ac5cee33f6/volumes"
Jul 21 14:21:11.503208 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:11.503188    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64126af8-66c9-48d1-83dd-0f3bff4b07c3" path="/var/lib/kubelet/pods/64126af8-66c9-48d1-83dd-0f3bff4b07c3/volumes"
Jul 21 14:21:15.468407 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:15.468370    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:21:18.424699 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:18.424663    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-faeda6-qd7h8"]
Jul 21 14:21:18.425103 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:18.424997    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64126af8-66c9-48d1-83dd-0f3bff4b07c3" containerName="revoke-keys"
Jul 21 14:21:18.425103 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:18.425008    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="64126af8-66c9-48d1-83dd-0f3bff4b07c3" containerName="revoke-keys"
Jul 21 14:21:18.425103 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:18.425023    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03328917-9996-4a60-ae00-04ac5cee33f6" containerName="main"
Jul 21 14:21:18.425103 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:18.425028    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="03328917-9996-4a60-ae00-04ac5cee33f6" containerName="main"
Jul 21 14:21:18.425103 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:18.425039    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d42ce96-1ba2-48e7-84d5-80de72e13096" containerName="maas-api"
Jul 21 14:21:18.425103 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:18.425045    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d42ce96-1ba2-48e7-84d5-80de72e13096" containerName="maas-api"
Jul 21 14:21:18.425103 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:18.425101    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d42ce96-1ba2-48e7-84d5-80de72e13096" containerName="maas-api"
Jul 21 14:21:18.425309 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:18.425107    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="64126af8-66c9-48d1-83dd-0f3bff4b07c3" containerName="revoke-keys"
Jul 21 14:21:18.425309 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:18.425114    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="03328917-9996-4a60-ae00-04ac5cee33f6" containerName="main"
Jul 21 14:21:18.427998 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:18.427975    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-faeda6-qd7h8"
Jul 21 14:21:18.430748 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:18.430725    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5t9ph\""
Jul 21 14:21:18.436387 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:18.436353    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-faeda6-qd7h8"]
Jul 21 14:21:18.484591 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:18.484552    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae97e0ed-60fa-4f55-94ec-82261df4c2e4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-faeda6-qd7h8\" (UID: \"ae97e0ed-60fa-4f55-94ec-82261df4c2e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-faeda6-qd7h8"
Jul 21 14:21:18.585237 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:18.585195    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae97e0ed-60fa-4f55-94ec-82261df4c2e4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-faeda6-qd7h8\" (UID: \"ae97e0ed-60fa-4f55-94ec-82261df4c2e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-faeda6-qd7h8"
Jul 21 14:21:18.585780 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:18.585762    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae97e0ed-60fa-4f55-94ec-82261df4c2e4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-faeda6-qd7h8\" (UID: \"ae97e0ed-60fa-4f55-94ec-82261df4c2e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-faeda6-qd7h8"
Jul 21 14:21:18.739402 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:18.739300    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-faeda6-qd7h8"
Jul 21 14:21:18.885352 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:18.885320    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-faeda6-qd7h8"]
Jul 21 14:21:18.888305 ip-10-0-132-245 kubenswrapper[2578]: W0721 14:21:18.888280    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae97e0ed_60fa_4f55_94ec_82261df4c2e4.slice/crio-e4b7a0d4b42e8f9853a98555472649a94a463d41f8c8b2776e45fee1a0b3b244 WatchSource:0}: Error finding container e4b7a0d4b42e8f9853a98555472649a94a463d41f8c8b2776e45fee1a0b3b244: Status 404 returned error can't find the container with id e4b7a0d4b42e8f9853a98555472649a94a463d41f8c8b2776e45fee1a0b3b244
Jul 21 14:21:19.524284 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:19.524202    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-faeda6-qd7h8" event={"ID":"ae97e0ed-60fa-4f55-94ec-82261df4c2e4","Type":"ContainerStarted","Data":"dcef9db40e2d9a16bca24fff6efe13c75fb5e25d414b69adc54ae9d582850f68"}
Jul 21 14:21:19.524284 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:19.524257    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-faeda6-qd7h8" event={"ID":"ae97e0ed-60fa-4f55-94ec-82261df4c2e4","Type":"ContainerStarted","Data":"e4b7a0d4b42e8f9853a98555472649a94a463d41f8c8b2776e45fee1a0b3b244"}
Jul 21 14:21:19.541306 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:19.541252    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-faeda6-qd7h8" podStartSLOduration=1.541233182 podStartE2EDuration="1.541233182s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:21:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:21:19.540807161 +0000 UTC m=+2226.501095714" watchObservedRunningTime="2026-07-21 14:21:19.541233182 +0000 UTC m=+2226.501521739"
Jul 21 14:21:20.529762 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:20.529724    2578 generic.go:356] "Generic (PLEG): container finished" podID="ae97e0ed-60fa-4f55-94ec-82261df4c2e4" containerID="dcef9db40e2d9a16bca24fff6efe13c75fb5e25d414b69adc54ae9d582850f68" exitCode=0
Jul 21 14:21:20.530161 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:20.529802    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-faeda6-qd7h8" event={"ID":"ae97e0ed-60fa-4f55-94ec-82261df4c2e4","Type":"ContainerDied","Data":"dcef9db40e2d9a16bca24fff6efe13c75fb5e25d414b69adc54ae9d582850f68"}
Jul 21 14:21:21.663992 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:21.663966    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-faeda6-qd7h8"
Jul 21 14:21:21.709572 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:21.709536    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae97e0ed-60fa-4f55-94ec-82261df4c2e4-maas-api-service-ca\") pod \"ae97e0ed-60fa-4f55-94ec-82261df4c2e4\" (UID: \"ae97e0ed-60fa-4f55-94ec-82261df4c2e4\") "
Jul 21 14:21:21.709897 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:21.709860    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae97e0ed-60fa-4f55-94ec-82261df4c2e4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ae97e0ed-60fa-4f55-94ec-82261df4c2e4" (UID: "ae97e0ed-60fa-4f55-94ec-82261df4c2e4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:21:21.810851 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:21.810767    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae97e0ed-60fa-4f55-94ec-82261df4c2e4-maas-api-service-ca\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:21:22.540134 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:22.540102    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-faeda6-qd7h8" event={"ID":"ae97e0ed-60fa-4f55-94ec-82261df4c2e4","Type":"ContainerDied","Data":"e4b7a0d4b42e8f9853a98555472649a94a463d41f8c8b2776e45fee1a0b3b244"}
Jul 21 14:21:22.540134 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:22.540118    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-faeda6-qd7h8"
Jul 21 14:21:22.540338 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:22.540148    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e4b7a0d4b42e8f9853a98555472649a94a463d41f8c8b2776e45fee1a0b3b244"
Jul 21 14:21:28.887065 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:28.887033    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-faeda6-594d7f6c5-n44k8"]
Jul 21 14:21:28.887557 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:28.887270    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faeda6-594d7f6c5-n44k8" podUID="81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9" containerName="maas-api" containerID="cri-o://0bfe3c25ecb2cee17347fdd543b06b7eefd220b4207b5e9e1143f4437acd2ea8" gracePeriod=30
Jul 21 14:21:29.131049 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:29.131027    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faeda6-594d7f6c5-n44k8"
Jul 21 14:21:29.172553 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:29.172482    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbg8j\" (UniqueName: \"kubernetes.io/projected/81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9-kube-api-access-mbg8j\") pod \"81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9\" (UID: \"81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9\") "
Jul 21 14:21:29.172725 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:29.172582    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9-openshift-service-ca\") pod \"81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9\" (UID: \"81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9\") "
Jul 21 14:21:29.172725 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:29.172625    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9-maas-api-tls\") pod \"81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9\" (UID: \"81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9\") "
Jul 21 14:21:29.172942 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:29.172917    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9" (UID: "81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:21:29.174538 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:29.174515    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9" (UID: "81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:21:29.174628 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:29.174584    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9-kube-api-access-mbg8j" (OuterVolumeSpecName: "kube-api-access-mbg8j") pod "81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9" (UID: "81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9"). InnerVolumeSpecName "kube-api-access-mbg8j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:21:29.273739 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:29.273708    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9-openshift-service-ca\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:21:29.273739 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:29.273735    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9-maas-api-tls\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:21:29.273739 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:29.273747    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbg8j\" (UniqueName: \"kubernetes.io/projected/81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9-kube-api-access-mbg8j\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:21:29.574994 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:29.574963    2578 generic.go:356] "Generic (PLEG): container finished" podID="81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9" containerID="0bfe3c25ecb2cee17347fdd543b06b7eefd220b4207b5e9e1143f4437acd2ea8" exitCode=0
Jul 21 14:21:29.575191 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:29.575003    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faeda6-594d7f6c5-n44k8"
Jul 21 14:21:29.575191 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:29.575051    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faeda6-594d7f6c5-n44k8" event={"ID":"81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9","Type":"ContainerDied","Data":"0bfe3c25ecb2cee17347fdd543b06b7eefd220b4207b5e9e1143f4437acd2ea8"}
Jul 21 14:21:29.575191 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:29.575107    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faeda6-594d7f6c5-n44k8" event={"ID":"81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9","Type":"ContainerDied","Data":"149148f72b6f590ff6a16019db9e630954693a0fca44909f9ff86cbb69b20432"}
Jul 21 14:21:29.575191 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:29.575137    2578 scope.go:117] "RemoveContainer" containerID="0bfe3c25ecb2cee17347fdd543b06b7eefd220b4207b5e9e1143f4437acd2ea8"
Jul 21 14:21:29.583250 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:29.583234    2578 scope.go:117] "RemoveContainer" containerID="0bfe3c25ecb2cee17347fdd543b06b7eefd220b4207b5e9e1143f4437acd2ea8"
Jul 21 14:21:29.583499 ip-10-0-132-245 kubenswrapper[2578]: E0721 14:21:29.583481    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bfe3c25ecb2cee17347fdd543b06b7eefd220b4207b5e9e1143f4437acd2ea8\": container with ID starting with 0bfe3c25ecb2cee17347fdd543b06b7eefd220b4207b5e9e1143f4437acd2ea8 not found: ID does not exist" containerID="0bfe3c25ecb2cee17347fdd543b06b7eefd220b4207b5e9e1143f4437acd2ea8"
Jul 21 14:21:29.583544 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:29.583511    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bfe3c25ecb2cee17347fdd543b06b7eefd220b4207b5e9e1143f4437acd2ea8"} err="failed to get container status \"0bfe3c25ecb2cee17347fdd543b06b7eefd220b4207b5e9e1143f4437acd2ea8\": rpc error: code = NotFound desc = could not find container \"0bfe3c25ecb2cee17347fdd543b06b7eefd220b4207b5e9e1143f4437acd2ea8\": container with ID starting with 0bfe3c25ecb2cee17347fdd543b06b7eefd220b4207b5e9e1143f4437acd2ea8 not found: ID does not exist"
Jul 21 14:21:29.591353 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:29.591324    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-faeda6-594d7f6c5-n44k8"]
Jul 21 14:21:29.593377 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:29.593353    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-faeda6-594d7f6c5-n44k8"]
Jul 21 14:21:31.504037 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:31.504004    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9" path="/var/lib/kubelet/pods/81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9/volumes"
Jul 21 14:21:32.062263 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:32.062202    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-faeda6-594d7f6c5-n44k8" podUID="81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.40:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Jul 21 14:21:32.164081 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:32.164047    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:21:33.507624 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:33.507596    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-faeda6-qd7h8"]
Jul 21 14:21:33.507624 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:33.507632    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-faeda6-qd7h8"]
Jul 21 14:21:35.502814 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:35.502740    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae97e0ed-60fa-4f55-94ec-82261df4c2e4" path="/var/lib/kubelet/pods/ae97e0ed-60fa-4f55-94ec-82261df4c2e4/volumes"
Jul 21 14:21:44.730396 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:44.730360    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-44f385-848bc79b86-2b2xg"]
Jul 21 14:21:44.730767 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:44.730672    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae97e0ed-60fa-4f55-94ec-82261df4c2e4" containerName="revoke-keys"
Jul 21 14:21:44.730767 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:44.730684    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae97e0ed-60fa-4f55-94ec-82261df4c2e4" containerName="revoke-keys"
Jul 21 14:21:44.730767 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:44.730701    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9" containerName="maas-api"
Jul 21 14:21:44.730767 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:44.730709    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9" containerName="maas-api"
Jul 21 14:21:44.730767 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:44.730755    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae97e0ed-60fa-4f55-94ec-82261df4c2e4" containerName="revoke-keys"
Jul 21 14:21:44.730767 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:44.730764    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="81b1a0bf-676b-4b9b-aa0e-d7fe3d2643e9" containerName="maas-api"
Jul 21 14:21:44.735042 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:44.735025    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-44f385-848bc79b86-2b2xg"
Jul 21 14:21:44.737658 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:44.737633    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 14:21:44.737809 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:44.737638    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-nhfmq\""
Jul 21 14:21:44.738524 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:44.738506    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 14:21:44.738632 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:44.738524    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-44f385-serving-cert\""
Jul 21 14:21:44.744316 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:44.744293    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-44f385-848bc79b86-2b2xg"]
Jul 21 14:21:44.812016 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:44.811983    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c03193e-d75b-41a4-8481-774b5abcdd42-maas-api-tls\") pod \"maas-api-e2e-api-a-44f385-848bc79b86-2b2xg\" (UID: \"2c03193e-d75b-41a4-8481-774b5abcdd42\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-44f385-848bc79b86-2b2xg"
Jul 21 14:21:44.812184 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:44.812028    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c03193e-d75b-41a4-8481-774b5abcdd42-openshift-service-ca\") pod \"maas-api-e2e-api-a-44f385-848bc79b86-2b2xg\" (UID: \"2c03193e-d75b-41a4-8481-774b5abcdd42\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-44f385-848bc79b86-2b2xg"
Jul 21 14:21:44.812184 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:44.812090    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsgr4\" (UniqueName: \"kubernetes.io/projected/2c03193e-d75b-41a4-8481-774b5abcdd42-kube-api-access-bsgr4\") pod \"maas-api-e2e-api-a-44f385-848bc79b86-2b2xg\" (UID: \"2c03193e-d75b-41a4-8481-774b5abcdd42\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-44f385-848bc79b86-2b2xg"
Jul 21 14:21:44.912961 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:44.912921    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c03193e-d75b-41a4-8481-774b5abcdd42-openshift-service-ca\") pod \"maas-api-e2e-api-a-44f385-848bc79b86-2b2xg\" (UID: \"2c03193e-d75b-41a4-8481-774b5abcdd42\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-44f385-848bc79b86-2b2xg"
Jul 21 14:21:44.913110 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:44.912984    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsgr4\" (UniqueName: \"kubernetes.io/projected/2c03193e-d75b-41a4-8481-774b5abcdd42-kube-api-access-bsgr4\") pod \"maas-api-e2e-api-a-44f385-848bc79b86-2b2xg\" (UID: \"2c03193e-d75b-41a4-8481-774b5abcdd42\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-44f385-848bc79b86-2b2xg"
Jul 21 14:21:44.913110 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:44.913027    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c03193e-d75b-41a4-8481-774b5abcdd42-maas-api-tls\") pod \"maas-api-e2e-api-a-44f385-848bc79b86-2b2xg\" (UID: \"2c03193e-d75b-41a4-8481-774b5abcdd42\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-44f385-848bc79b86-2b2xg"
Jul 21 14:21:44.913615 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:44.913596    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c03193e-d75b-41a4-8481-774b5abcdd42-openshift-service-ca\") pod \"maas-api-e2e-api-a-44f385-848bc79b86-2b2xg\" (UID: \"2c03193e-d75b-41a4-8481-774b5abcdd42\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-44f385-848bc79b86-2b2xg"
Jul 21 14:21:44.915482 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:44.915461    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c03193e-d75b-41a4-8481-774b5abcdd42-maas-api-tls\") pod \"maas-api-e2e-api-a-44f385-848bc79b86-2b2xg\" (UID: \"2c03193e-d75b-41a4-8481-774b5abcdd42\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-44f385-848bc79b86-2b2xg"
Jul 21 14:21:44.931933 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:44.931903    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsgr4\" (UniqueName: \"kubernetes.io/projected/2c03193e-d75b-41a4-8481-774b5abcdd42-kube-api-access-bsgr4\") pod \"maas-api-e2e-api-a-44f385-848bc79b86-2b2xg\" (UID: \"2c03193e-d75b-41a4-8481-774b5abcdd42\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-44f385-848bc79b86-2b2xg"
Jul 21 14:21:45.046160 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:45.046061    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-44f385-848bc79b86-2b2xg"
Jul 21 14:21:45.182301 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:45.182274    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-44f385-848bc79b86-2b2xg"]
Jul 21 14:21:45.184488 ip-10-0-132-245 kubenswrapper[2578]: W0721 14:21:45.184463    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c03193e_d75b_41a4_8481_774b5abcdd42.slice/crio-0dbc5a695b03784ec45af34b65efebb41d9e56e3574a85f39dc58e0f073e1fd2 WatchSource:0}: Error finding container 0dbc5a695b03784ec45af34b65efebb41d9e56e3574a85f39dc58e0f073e1fd2: Status 404 returned error can't find the container with id 0dbc5a695b03784ec45af34b65efebb41d9e56e3574a85f39dc58e0f073e1fd2
Jul 21 14:21:45.185945 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:45.185924    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 14:21:45.649214 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:45.649184    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-44f385-848bc79b86-2b2xg" event={"ID":"2c03193e-d75b-41a4-8481-774b5abcdd42","Type":"ContainerStarted","Data":"0dbc5a695b03784ec45af34b65efebb41d9e56e3574a85f39dc58e0f073e1fd2"}
Jul 21 14:21:46.654742 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:46.654709    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-44f385-848bc79b86-2b2xg" event={"ID":"2c03193e-d75b-41a4-8481-774b5abcdd42","Type":"ContainerStarted","Data":"2dddd29b7a7a9e2967e57b32b12b32d2da9f4e82c270b49aab170f1f251ffc40"}
Jul 21 14:21:46.655160 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:46.654783    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-44f385-848bc79b86-2b2xg"
Jul 21 14:21:46.674685 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:46.674605    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-44f385-848bc79b86-2b2xg" podStartSLOduration=1.5329315449999998 podStartE2EDuration="2.674581448s" totalImagesPullingTime="1.141649903s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:21:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:21:46.674304323 +0000 UTC m=+2253.634592881" watchObservedRunningTime="2026-07-21 14:21:46.674581448 +0000 UTC m=+2253.634870005"
Jul 21 14:21:50.371894 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:50.371846    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:21:52.665191 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:52.665158    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-44f385-848bc79b86-2b2xg"
Jul 21 14:21:52.811544 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:52.811505    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ce2e87-5b75b565d9-2sdzc"]
Jul 21 14:21:52.814955 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:52.814930    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ce2e87-5b75b565d9-2sdzc"
Jul 21 14:21:52.823271 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:52.823249    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-ce2e87-serving-cert\""
Jul 21 14:21:52.840215 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:52.840188    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ce2e87-5b75b565d9-2sdzc"]
Jul 21 14:21:52.981967 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:52.981867    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8pwn\" (UniqueName: \"kubernetes.io/projected/209a4247-2cb7-47ac-9f1c-ccb30048cb2e-kube-api-access-q8pwn\") pod \"maas-api-e2e-api-b-ce2e87-5b75b565d9-2sdzc\" (UID: \"209a4247-2cb7-47ac-9f1c-ccb30048cb2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ce2e87-5b75b565d9-2sdzc"
Jul 21 14:21:52.981967 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:52.981942    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/209a4247-2cb7-47ac-9f1c-ccb30048cb2e-maas-api-tls\") pod \"maas-api-e2e-api-b-ce2e87-5b75b565d9-2sdzc\" (UID: \"209a4247-2cb7-47ac-9f1c-ccb30048cb2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ce2e87-5b75b565d9-2sdzc"
Jul 21 14:21:52.982141 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:52.981970    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/209a4247-2cb7-47ac-9f1c-ccb30048cb2e-openshift-service-ca\") pod \"maas-api-e2e-api-b-ce2e87-5b75b565d9-2sdzc\" (UID: \"209a4247-2cb7-47ac-9f1c-ccb30048cb2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ce2e87-5b75b565d9-2sdzc"
Jul 21 14:21:53.083105 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:53.083073    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/209a4247-2cb7-47ac-9f1c-ccb30048cb2e-openshift-service-ca\") pod \"maas-api-e2e-api-b-ce2e87-5b75b565d9-2sdzc\" (UID: \"209a4247-2cb7-47ac-9f1c-ccb30048cb2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ce2e87-5b75b565d9-2sdzc"
Jul 21 14:21:53.083247 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:53.083156    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8pwn\" (UniqueName: \"kubernetes.io/projected/209a4247-2cb7-47ac-9f1c-ccb30048cb2e-kube-api-access-q8pwn\") pod \"maas-api-e2e-api-b-ce2e87-5b75b565d9-2sdzc\" (UID: \"209a4247-2cb7-47ac-9f1c-ccb30048cb2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ce2e87-5b75b565d9-2sdzc"
Jul 21 14:21:53.083247 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:53.083189    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/209a4247-2cb7-47ac-9f1c-ccb30048cb2e-maas-api-tls\") pod \"maas-api-e2e-api-b-ce2e87-5b75b565d9-2sdzc\" (UID: \"209a4247-2cb7-47ac-9f1c-ccb30048cb2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ce2e87-5b75b565d9-2sdzc"
Jul 21 14:21:53.083729 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:53.083701    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/209a4247-2cb7-47ac-9f1c-ccb30048cb2e-openshift-service-ca\") pod \"maas-api-e2e-api-b-ce2e87-5b75b565d9-2sdzc\" (UID: \"209a4247-2cb7-47ac-9f1c-ccb30048cb2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ce2e87-5b75b565d9-2sdzc"
Jul 21 14:21:53.085515 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:53.085497    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/209a4247-2cb7-47ac-9f1c-ccb30048cb2e-maas-api-tls\") pod \"maas-api-e2e-api-b-ce2e87-5b75b565d9-2sdzc\" (UID: \"209a4247-2cb7-47ac-9f1c-ccb30048cb2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ce2e87-5b75b565d9-2sdzc"
Jul 21 14:21:53.091037 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:53.091015    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8pwn\" (UniqueName: \"kubernetes.io/projected/209a4247-2cb7-47ac-9f1c-ccb30048cb2e-kube-api-access-q8pwn\") pod \"maas-api-e2e-api-b-ce2e87-5b75b565d9-2sdzc\" (UID: \"209a4247-2cb7-47ac-9f1c-ccb30048cb2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ce2e87-5b75b565d9-2sdzc"
Jul 21 14:21:53.126898 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:53.126852    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ce2e87-5b75b565d9-2sdzc"
Jul 21 14:21:53.265752 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:53.265728    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ce2e87-5b75b565d9-2sdzc"]
Jul 21 14:21:53.267450 ip-10-0-132-245 kubenswrapper[2578]: W0721 14:21:53.267424    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod209a4247_2cb7_47ac_9f1c_ccb30048cb2e.slice/crio-a34460c58298f263f1db80394683ef6303e62eab6bdde89e03d244c7cdd36fa7 WatchSource:0}: Error finding container a34460c58298f263f1db80394683ef6303e62eab6bdde89e03d244c7cdd36fa7: Status 404 returned error can't find the container with id a34460c58298f263f1db80394683ef6303e62eab6bdde89e03d244c7cdd36fa7
Jul 21 14:21:53.686035 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:53.685999    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ce2e87-5b75b565d9-2sdzc" event={"ID":"209a4247-2cb7-47ac-9f1c-ccb30048cb2e","Type":"ContainerStarted","Data":"a34460c58298f263f1db80394683ef6303e62eab6bdde89e03d244c7cdd36fa7"}
Jul 21 14:21:54.691851 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:54.691807    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ce2e87-5b75b565d9-2sdzc" event={"ID":"209a4247-2cb7-47ac-9f1c-ccb30048cb2e","Type":"ContainerStarted","Data":"553c017239f6f49f166075fe264cf95b917202352c7215efcbafa539783950aa"}
Jul 21 14:21:54.692273 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:54.691935    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ce2e87-5b75b565d9-2sdzc"
Jul 21 14:21:54.709849 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:21:54.709799    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ce2e87-5b75b565d9-2sdzc" podStartSLOduration=2.200985253 podStartE2EDuration="2.709783353s" totalImagesPullingTime="508.7981ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:21:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:21:54.708313537 +0000 UTC m=+2261.668602106" watchObservedRunningTime="2026-07-21 14:21:54.709783353 +0000 UTC m=+2261.670071910"
Jul 21 14:22:00.702163 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:00.702135    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ce2e87-5b75b565d9-2sdzc"
Jul 21 14:22:05.178296 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:05.178258    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-44f385-9vsm7"]
Jul 21 14:22:05.181637 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:05.181615    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-44f385-9vsm7"
Jul 21 14:22:05.182014 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:05.181984    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/22e2f6ef-b7fd-40f1-a069-5f5ce17c8cb9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-44f385-9vsm7\" (UID: \"22e2f6ef-b7fd-40f1-a069-5f5ce17c8cb9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-44f385-9vsm7"
Jul 21 14:22:05.184182 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:05.184163    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5t9ph\""
Jul 21 14:22:05.189355 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:05.189333    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-44f385-9vsm7"]
Jul 21 14:22:05.282485 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:05.282449    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/22e2f6ef-b7fd-40f1-a069-5f5ce17c8cb9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-44f385-9vsm7\" (UID: \"22e2f6ef-b7fd-40f1-a069-5f5ce17c8cb9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-44f385-9vsm7"
Jul 21 14:22:05.283081 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:05.283061    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/22e2f6ef-b7fd-40f1-a069-5f5ce17c8cb9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-44f385-9vsm7\" (UID: \"22e2f6ef-b7fd-40f1-a069-5f5ce17c8cb9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-44f385-9vsm7"
Jul 21 14:22:05.492825 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:05.492737    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-44f385-9vsm7"
Jul 21 14:22:05.627811 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:05.627742    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-44f385-9vsm7"]
Jul 21 14:22:05.630526 ip-10-0-132-245 kubenswrapper[2578]: W0721 14:22:05.630503    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22e2f6ef_b7fd_40f1_a069_5f5ce17c8cb9.slice/crio-661130faede864e82b262da5f81efd6366e0782222a5a524f3784227064ab016 WatchSource:0}: Error finding container 661130faede864e82b262da5f81efd6366e0782222a5a524f3784227064ab016: Status 404 returned error can't find the container with id 661130faede864e82b262da5f81efd6366e0782222a5a524f3784227064ab016
Jul 21 14:22:05.741788 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:05.741753    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-44f385-9vsm7" event={"ID":"22e2f6ef-b7fd-40f1-a069-5f5ce17c8cb9","Type":"ContainerStarted","Data":"661130faede864e82b262da5f81efd6366e0782222a5a524f3784227064ab016"}
Jul 21 14:22:06.747553 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:06.747521    2578 generic.go:356] "Generic (PLEG): container finished" podID="22e2f6ef-b7fd-40f1-a069-5f5ce17c8cb9" containerID="f62023ada8b72c479a4b74f7c7c9d04a81379975eae331ff8e14df6b4371fb88" exitCode=0
Jul 21 14:22:06.747984 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:06.747593    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-44f385-9vsm7" event={"ID":"22e2f6ef-b7fd-40f1-a069-5f5ce17c8cb9","Type":"ContainerDied","Data":"f62023ada8b72c479a4b74f7c7c9d04a81379975eae331ff8e14df6b4371fb88"}
Jul 21 14:22:07.890833 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:07.890802    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-44f385-9vsm7"
Jul 21 14:22:07.901375 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:07.901349    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/22e2f6ef-b7fd-40f1-a069-5f5ce17c8cb9-maas-api-service-ca\") pod \"22e2f6ef-b7fd-40f1-a069-5f5ce17c8cb9\" (UID: \"22e2f6ef-b7fd-40f1-a069-5f5ce17c8cb9\") "
Jul 21 14:22:07.901677 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:07.901652    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22e2f6ef-b7fd-40f1-a069-5f5ce17c8cb9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "22e2f6ef-b7fd-40f1-a069-5f5ce17c8cb9" (UID: "22e2f6ef-b7fd-40f1-a069-5f5ce17c8cb9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:22:08.001910 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:08.001863    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/22e2f6ef-b7fd-40f1-a069-5f5ce17c8cb9-maas-api-service-ca\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:22:08.757091 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:08.757049    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-44f385-9vsm7"
Jul 21 14:22:08.757091 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:08.757084    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-44f385-9vsm7" event={"ID":"22e2f6ef-b7fd-40f1-a069-5f5ce17c8cb9","Type":"ContainerDied","Data":"661130faede864e82b262da5f81efd6366e0782222a5a524f3784227064ab016"}
Jul 21 14:22:08.757353 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:08.757129    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="661130faede864e82b262da5f81efd6366e0782222a5a524f3784227064ab016"
Jul 21 14:22:09.578423 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:09.578385    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:22:13.667746 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:13.667715    2578 scope.go:117] "RemoveContainer" containerID="4bab398e5ee4189525b150d247d90c77e966e83774955b13ba4ae6f6d3476226"
Jul 21 14:22:20.529019 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:20.528979    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-44f385-848bc79b86-2b2xg"]
Jul 21 14:22:20.529487 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:20.529258    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-44f385-848bc79b86-2b2xg" podUID="2c03193e-d75b-41a4-8481-774b5abcdd42" containerName="maas-api" containerID="cri-o://2dddd29b7a7a9e2967e57b32b12b32d2da9f4e82c270b49aab170f1f251ffc40" gracePeriod=30
Jul 21 14:22:20.779211 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:20.779153    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-44f385-848bc79b86-2b2xg"
Jul 21 14:22:20.791935 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:20.791906    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bsgr4\" (UniqueName: \"kubernetes.io/projected/2c03193e-d75b-41a4-8481-774b5abcdd42-kube-api-access-bsgr4\") pod \"2c03193e-d75b-41a4-8481-774b5abcdd42\" (UID: \"2c03193e-d75b-41a4-8481-774b5abcdd42\") "
Jul 21 14:22:20.792100 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:20.791943    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c03193e-d75b-41a4-8481-774b5abcdd42-openshift-service-ca\") pod \"2c03193e-d75b-41a4-8481-774b5abcdd42\" (UID: \"2c03193e-d75b-41a4-8481-774b5abcdd42\") "
Jul 21 14:22:20.792100 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:20.791965    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c03193e-d75b-41a4-8481-774b5abcdd42-maas-api-tls\") pod \"2c03193e-d75b-41a4-8481-774b5abcdd42\" (UID: \"2c03193e-d75b-41a4-8481-774b5abcdd42\") "
Jul 21 14:22:20.792329 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:20.792305    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c03193e-d75b-41a4-8481-774b5abcdd42-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2c03193e-d75b-41a4-8481-774b5abcdd42" (UID: "2c03193e-d75b-41a4-8481-774b5abcdd42"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:22:20.794104 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:20.794082    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c03193e-d75b-41a4-8481-774b5abcdd42-kube-api-access-bsgr4" (OuterVolumeSpecName: "kube-api-access-bsgr4") pod "2c03193e-d75b-41a4-8481-774b5abcdd42" (UID: "2c03193e-d75b-41a4-8481-774b5abcdd42"). InnerVolumeSpecName "kube-api-access-bsgr4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:22:20.794104 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:20.794098    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c03193e-d75b-41a4-8481-774b5abcdd42-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2c03193e-d75b-41a4-8481-774b5abcdd42" (UID: "2c03193e-d75b-41a4-8481-774b5abcdd42"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:22:20.812194 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:20.812066    2578 generic.go:356] "Generic (PLEG): container finished" podID="2c03193e-d75b-41a4-8481-774b5abcdd42" containerID="2dddd29b7a7a9e2967e57b32b12b32d2da9f4e82c270b49aab170f1f251ffc40" exitCode=0
Jul 21 14:22:20.812194 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:20.812110    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-44f385-848bc79b86-2b2xg"
Jul 21 14:22:20.812194 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:20.812154    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-44f385-848bc79b86-2b2xg" event={"ID":"2c03193e-d75b-41a4-8481-774b5abcdd42","Type":"ContainerDied","Data":"2dddd29b7a7a9e2967e57b32b12b32d2da9f4e82c270b49aab170f1f251ffc40"}
Jul 21 14:22:20.812466 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:20.812213    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-44f385-848bc79b86-2b2xg" event={"ID":"2c03193e-d75b-41a4-8481-774b5abcdd42","Type":"ContainerDied","Data":"0dbc5a695b03784ec45af34b65efebb41d9e56e3574a85f39dc58e0f073e1fd2"}
Jul 21 14:22:20.812466 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:20.812235    2578 scope.go:117] "RemoveContainer" containerID="2dddd29b7a7a9e2967e57b32b12b32d2da9f4e82c270b49aab170f1f251ffc40"
Jul 21 14:22:20.821903 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:20.821869    2578 scope.go:117] "RemoveContainer" containerID="2dddd29b7a7a9e2967e57b32b12b32d2da9f4e82c270b49aab170f1f251ffc40"
Jul 21 14:22:20.822200 ip-10-0-132-245 kubenswrapper[2578]: E0721 14:22:20.822174    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2dddd29b7a7a9e2967e57b32b12b32d2da9f4e82c270b49aab170f1f251ffc40\": container with ID starting with 2dddd29b7a7a9e2967e57b32b12b32d2da9f4e82c270b49aab170f1f251ffc40 not found: ID does not exist" containerID="2dddd29b7a7a9e2967e57b32b12b32d2da9f4e82c270b49aab170f1f251ffc40"
Jul 21 14:22:20.822244 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:20.822201    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2dddd29b7a7a9e2967e57b32b12b32d2da9f4e82c270b49aab170f1f251ffc40"} err="failed to get container status \"2dddd29b7a7a9e2967e57b32b12b32d2da9f4e82c270b49aab170f1f251ffc40\": rpc error: code = NotFound desc = could not find container \"2dddd29b7a7a9e2967e57b32b12b32d2da9f4e82c270b49aab170f1f251ffc40\": container with ID starting with 2dddd29b7a7a9e2967e57b32b12b32d2da9f4e82c270b49aab170f1f251ffc40 not found: ID does not exist"
Jul 21 14:22:20.832498 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:20.832472    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-44f385-848bc79b86-2b2xg"]
Jul 21 14:22:20.836788 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:20.836757    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-44f385-848bc79b86-2b2xg"]
Jul 21 14:22:20.892446 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:20.892415    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c03193e-d75b-41a4-8481-774b5abcdd42-maas-api-tls\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:22:20.892446 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:20.892445    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bsgr4\" (UniqueName: \"kubernetes.io/projected/2c03193e-d75b-41a4-8481-774b5abcdd42-kube-api-access-bsgr4\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:22:20.892629 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:20.892458    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c03193e-d75b-41a4-8481-774b5abcdd42-openshift-service-ca\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:22:21.503807 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:21.503772    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c03193e-d75b-41a4-8481-774b5abcdd42" path="/var/lib/kubelet/pods/2c03193e-d75b-41a4-8481-774b5abcdd42/volumes"
Jul 21 14:22:23.262778 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:23.262738    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:22:25.264597 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:25.264556    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-44f385-9vsm7"]
Jul 21 14:22:25.273002 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:25.272969    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-44f385-9vsm7"]
Jul 21 14:22:25.504124 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:25.504087    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22e2f6ef-b7fd-40f1-a069-5f5ce17c8cb9" path="/var/lib/kubelet/pods/22e2f6ef-b7fd-40f1-a069-5f5ce17c8cb9/volumes"
Jul 21 14:22:32.421536 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:32.421499    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ce2e87-h7jdr"]
Jul 21 14:22:32.422046 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:32.421841    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22e2f6ef-b7fd-40f1-a069-5f5ce17c8cb9" containerName="revoke-keys"
Jul 21 14:22:32.422046 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:32.421854    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="22e2f6ef-b7fd-40f1-a069-5f5ce17c8cb9" containerName="revoke-keys"
Jul 21 14:22:32.422046 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:32.421863    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c03193e-d75b-41a4-8481-774b5abcdd42" containerName="maas-api"
Jul 21 14:22:32.422046 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:32.421869    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c03193e-d75b-41a4-8481-774b5abcdd42" containerName="maas-api"
Jul 21 14:22:32.422046 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:32.421940    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="22e2f6ef-b7fd-40f1-a069-5f5ce17c8cb9" containerName="revoke-keys"
Jul 21 14:22:32.422046 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:32.421952    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c03193e-d75b-41a4-8481-774b5abcdd42" containerName="maas-api"
Jul 21 14:22:32.426046 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:32.426029    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ce2e87-h7jdr"
Jul 21 14:22:32.429383 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:32.429360    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5t9ph\""
Jul 21 14:22:32.446814 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:32.446786    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ce2e87-h7jdr"]
Jul 21 14:22:32.485525 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:32.485489    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4fd51a2-8b78-4f1a-b462-89cce7b3ae94-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-ce2e87-h7jdr\" (UID: \"f4fd51a2-8b78-4f1a-b462-89cce7b3ae94\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ce2e87-h7jdr"
Jul 21 14:22:32.586848 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:32.586804    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4fd51a2-8b78-4f1a-b462-89cce7b3ae94-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-ce2e87-h7jdr\" (UID: \"f4fd51a2-8b78-4f1a-b462-89cce7b3ae94\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ce2e87-h7jdr"
Jul 21 14:22:32.587494 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:32.587474    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4fd51a2-8b78-4f1a-b462-89cce7b3ae94-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-ce2e87-h7jdr\" (UID: \"f4fd51a2-8b78-4f1a-b462-89cce7b3ae94\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ce2e87-h7jdr"
Jul 21 14:22:32.735845 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:32.735766    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ce2e87-h7jdr"
Jul 21 14:22:32.869252 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:32.869226    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ce2e87-h7jdr"]
Jul 21 14:22:32.871621 ip-10-0-132-245 kubenswrapper[2578]: W0721 14:22:32.871598    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4fd51a2_8b78_4f1a_b462_89cce7b3ae94.slice/crio-99ed7553f1c50aed6c1f5e69f4f7c2d1a4128d9f06211509dcd85427c9f360ea WatchSource:0}: Error finding container 99ed7553f1c50aed6c1f5e69f4f7c2d1a4128d9f06211509dcd85427c9f360ea: Status 404 returned error can't find the container with id 99ed7553f1c50aed6c1f5e69f4f7c2d1a4128d9f06211509dcd85427c9f360ea
Jul 21 14:22:33.872076 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:33.871974    2578 generic.go:356] "Generic (PLEG): container finished" podID="f4fd51a2-8b78-4f1a-b462-89cce7b3ae94" containerID="da041ab947b0031a5b18035a63ba4772d19b6d376789bc63df092d6ea399e81e" exitCode=0
Jul 21 14:22:33.872076 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:33.872018    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ce2e87-h7jdr" event={"ID":"f4fd51a2-8b78-4f1a-b462-89cce7b3ae94","Type":"ContainerDied","Data":"da041ab947b0031a5b18035a63ba4772d19b6d376789bc63df092d6ea399e81e"}
Jul 21 14:22:33.872076 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:33.872068    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ce2e87-h7jdr" event={"ID":"f4fd51a2-8b78-4f1a-b462-89cce7b3ae94","Type":"ContainerStarted","Data":"99ed7553f1c50aed6c1f5e69f4f7c2d1a4128d9f06211509dcd85427c9f360ea"}
Jul 21 14:22:35.001690 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:35.001664    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ce2e87-h7jdr"
Jul 21 14:22:35.106508 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:35.106466    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4fd51a2-8b78-4f1a-b462-89cce7b3ae94-maas-api-service-ca\") pod \"f4fd51a2-8b78-4f1a-b462-89cce7b3ae94\" (UID: \"f4fd51a2-8b78-4f1a-b462-89cce7b3ae94\") "
Jul 21 14:22:35.106832 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:35.106809    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4fd51a2-8b78-4f1a-b462-89cce7b3ae94-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f4fd51a2-8b78-4f1a-b462-89cce7b3ae94" (UID: "f4fd51a2-8b78-4f1a-b462-89cce7b3ae94"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:22:35.207467 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:35.207381    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4fd51a2-8b78-4f1a-b462-89cce7b3ae94-maas-api-service-ca\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:22:35.880888 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:35.880858    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ce2e87-h7jdr"
Jul 21 14:22:35.881039 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:35.880870    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ce2e87-h7jdr" event={"ID":"f4fd51a2-8b78-4f1a-b462-89cce7b3ae94","Type":"ContainerDied","Data":"99ed7553f1c50aed6c1f5e69f4f7c2d1a4128d9f06211509dcd85427c9f360ea"}
Jul 21 14:22:35.881039 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:35.880959    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="99ed7553f1c50aed6c1f5e69f4f7c2d1a4128d9f06211509dcd85427c9f360ea"
Jul 21 14:22:47.850412 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:47.850379    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ce2e87-5b75b565d9-2sdzc"]
Jul 21 14:22:47.850981 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:47.850609    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ce2e87-5b75b565d9-2sdzc" podUID="209a4247-2cb7-47ac-9f1c-ccb30048cb2e" containerName="maas-api" containerID="cri-o://553c017239f6f49f166075fe264cf95b917202352c7215efcbafa539783950aa" gracePeriod=30
Jul 21 14:22:48.095824 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:48.095801    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ce2e87-5b75b565d9-2sdzc"
Jul 21 14:22:48.209625 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:48.209542    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/209a4247-2cb7-47ac-9f1c-ccb30048cb2e-openshift-service-ca\") pod \"209a4247-2cb7-47ac-9f1c-ccb30048cb2e\" (UID: \"209a4247-2cb7-47ac-9f1c-ccb30048cb2e\") "
Jul 21 14:22:48.209625 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:48.209595    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8pwn\" (UniqueName: \"kubernetes.io/projected/209a4247-2cb7-47ac-9f1c-ccb30048cb2e-kube-api-access-q8pwn\") pod \"209a4247-2cb7-47ac-9f1c-ccb30048cb2e\" (UID: \"209a4247-2cb7-47ac-9f1c-ccb30048cb2e\") "
Jul 21 14:22:48.209889 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:48.209635    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/209a4247-2cb7-47ac-9f1c-ccb30048cb2e-maas-api-tls\") pod \"209a4247-2cb7-47ac-9f1c-ccb30048cb2e\" (UID: \"209a4247-2cb7-47ac-9f1c-ccb30048cb2e\") "
Jul 21 14:22:48.209959 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:48.209935    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/209a4247-2cb7-47ac-9f1c-ccb30048cb2e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "209a4247-2cb7-47ac-9f1c-ccb30048cb2e" (UID: "209a4247-2cb7-47ac-9f1c-ccb30048cb2e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:22:48.211783 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:48.211757    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/209a4247-2cb7-47ac-9f1c-ccb30048cb2e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "209a4247-2cb7-47ac-9f1c-ccb30048cb2e" (UID: "209a4247-2cb7-47ac-9f1c-ccb30048cb2e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:22:48.211917 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:48.211787    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/209a4247-2cb7-47ac-9f1c-ccb30048cb2e-kube-api-access-q8pwn" (OuterVolumeSpecName: "kube-api-access-q8pwn") pod "209a4247-2cb7-47ac-9f1c-ccb30048cb2e" (UID: "209a4247-2cb7-47ac-9f1c-ccb30048cb2e"). InnerVolumeSpecName "kube-api-access-q8pwn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:22:48.310409 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:48.310366    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8pwn\" (UniqueName: \"kubernetes.io/projected/209a4247-2cb7-47ac-9f1c-ccb30048cb2e-kube-api-access-q8pwn\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:22:48.310409 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:48.310409    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/209a4247-2cb7-47ac-9f1c-ccb30048cb2e-maas-api-tls\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:22:48.310608 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:48.310424    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/209a4247-2cb7-47ac-9f1c-ccb30048cb2e-openshift-service-ca\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:22:48.941345 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:48.941311    2578 generic.go:356] "Generic (PLEG): container finished" podID="209a4247-2cb7-47ac-9f1c-ccb30048cb2e" containerID="553c017239f6f49f166075fe264cf95b917202352c7215efcbafa539783950aa" exitCode=0
Jul 21 14:22:48.941753 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:48.941366    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ce2e87-5b75b565d9-2sdzc"
Jul 21 14:22:48.941753 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:48.941406    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ce2e87-5b75b565d9-2sdzc" event={"ID":"209a4247-2cb7-47ac-9f1c-ccb30048cb2e","Type":"ContainerDied","Data":"553c017239f6f49f166075fe264cf95b917202352c7215efcbafa539783950aa"}
Jul 21 14:22:48.941753 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:48.941482    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ce2e87-5b75b565d9-2sdzc" event={"ID":"209a4247-2cb7-47ac-9f1c-ccb30048cb2e","Type":"ContainerDied","Data":"a34460c58298f263f1db80394683ef6303e62eab6bdde89e03d244c7cdd36fa7"}
Jul 21 14:22:48.941753 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:48.941505    2578 scope.go:117] "RemoveContainer" containerID="553c017239f6f49f166075fe264cf95b917202352c7215efcbafa539783950aa"
Jul 21 14:22:48.950556 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:48.950538    2578 scope.go:117] "RemoveContainer" containerID="553c017239f6f49f166075fe264cf95b917202352c7215efcbafa539783950aa"
Jul 21 14:22:48.950824 ip-10-0-132-245 kubenswrapper[2578]: E0721 14:22:48.950806    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"553c017239f6f49f166075fe264cf95b917202352c7215efcbafa539783950aa\": container with ID starting with 553c017239f6f49f166075fe264cf95b917202352c7215efcbafa539783950aa not found: ID does not exist" containerID="553c017239f6f49f166075fe264cf95b917202352c7215efcbafa539783950aa"
Jul 21 14:22:48.950899 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:48.950834    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"553c017239f6f49f166075fe264cf95b917202352c7215efcbafa539783950aa"} err="failed to get container status \"553c017239f6f49f166075fe264cf95b917202352c7215efcbafa539783950aa\": rpc error: code = NotFound desc = could not find container \"553c017239f6f49f166075fe264cf95b917202352c7215efcbafa539783950aa\": container with ID starting with 553c017239f6f49f166075fe264cf95b917202352c7215efcbafa539783950aa not found: ID does not exist"
Jul 21 14:22:48.963302 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:48.963273    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ce2e87-5b75b565d9-2sdzc"]
Jul 21 14:22:48.969976 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:48.969956    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ce2e87-5b75b565d9-2sdzc"]
Jul 21 14:22:49.503431 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:49.503396    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="209a4247-2cb7-47ac-9f1c-ccb30048cb2e" path="/var/lib/kubelet/pods/209a4247-2cb7-47ac-9f1c-ccb30048cb2e/volumes"
Jul 21 14:22:52.462666 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:52.462631    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ce2e87-h7jdr"]
Jul 21 14:22:52.465433 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:52.465405    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-ce2e87-h7jdr"]
Jul 21 14:22:53.504524 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:53.504486    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4fd51a2-8b78-4f1a-b462-89cce7b3ae94" path="/var/lib/kubelet/pods/f4fd51a2-8b78-4f1a-b462-89cce7b3ae94/volumes"
Jul 21 14:22:53.670745 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:22:53.670712    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:23:03.231049 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:03.231019    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-67be07-65949f64b8-r2j75"]
Jul 21 14:23:03.231429 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:03.231317    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="209a4247-2cb7-47ac-9f1c-ccb30048cb2e" containerName="maas-api"
Jul 21 14:23:03.231429 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:03.231328    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="209a4247-2cb7-47ac-9f1c-ccb30048cb2e" containerName="maas-api"
Jul 21 14:23:03.231429 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:03.231340    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4fd51a2-8b78-4f1a-b462-89cce7b3ae94" containerName="revoke-keys"
Jul 21 14:23:03.231429 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:03.231346    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4fd51a2-8b78-4f1a-b462-89cce7b3ae94" containerName="revoke-keys"
Jul 21 14:23:03.231429 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:03.231394    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="209a4247-2cb7-47ac-9f1c-ccb30048cb2e" containerName="maas-api"
Jul 21 14:23:03.231429 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:03.231401    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4fd51a2-8b78-4f1a-b462-89cce7b3ae94" containerName="revoke-keys"
Jul 21 14:23:03.235555 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:03.235535    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67be07-65949f64b8-r2j75"
Jul 21 14:23:03.238278 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:03.238254    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 14:23:03.238399 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:03.238309    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-nhfmq\""
Jul 21 14:23:03.239230 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:03.239209    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 14:23:03.239610 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:03.239594    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-67be07-serving-cert\""
Jul 21 14:23:03.249309 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:03.249283    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-67be07-65949f64b8-r2j75"]
Jul 21 14:23:03.328650 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:03.328606    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2513f979-f9ab-4d92-b912-9ab5eeb6dcc9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-67be07-65949f64b8-r2j75\" (UID: \"2513f979-f9ab-4d92-b912-9ab5eeb6dcc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67be07-65949f64b8-r2j75"
Jul 21 14:23:03.328837 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:03.328663    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2513f979-f9ab-4d92-b912-9ab5eeb6dcc9-maas-api-tls\") pod \"maas-api-e2e-shared-a-67be07-65949f64b8-r2j75\" (UID: \"2513f979-f9ab-4d92-b912-9ab5eeb6dcc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67be07-65949f64b8-r2j75"
Jul 21 14:23:03.328837 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:03.328693    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrtqw\" (UniqueName: \"kubernetes.io/projected/2513f979-f9ab-4d92-b912-9ab5eeb6dcc9-kube-api-access-mrtqw\") pod \"maas-api-e2e-shared-a-67be07-65949f64b8-r2j75\" (UID: \"2513f979-f9ab-4d92-b912-9ab5eeb6dcc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67be07-65949f64b8-r2j75"
Jul 21 14:23:03.430159 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:03.430121    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2513f979-f9ab-4d92-b912-9ab5eeb6dcc9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-67be07-65949f64b8-r2j75\" (UID: \"2513f979-f9ab-4d92-b912-9ab5eeb6dcc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67be07-65949f64b8-r2j75"
Jul 21 14:23:03.430159 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:03.430167    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2513f979-f9ab-4d92-b912-9ab5eeb6dcc9-maas-api-tls\") pod \"maas-api-e2e-shared-a-67be07-65949f64b8-r2j75\" (UID: \"2513f979-f9ab-4d92-b912-9ab5eeb6dcc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67be07-65949f64b8-r2j75"
Jul 21 14:23:03.430395 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:03.430184    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrtqw\" (UniqueName: \"kubernetes.io/projected/2513f979-f9ab-4d92-b912-9ab5eeb6dcc9-kube-api-access-mrtqw\") pod \"maas-api-e2e-shared-a-67be07-65949f64b8-r2j75\" (UID: \"2513f979-f9ab-4d92-b912-9ab5eeb6dcc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67be07-65949f64b8-r2j75"
Jul 21 14:23:03.430395 ip-10-0-132-245 kubenswrapper[2578]: E0721 14:23:03.430301    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-67be07-serving-cert: secret "maas-api-e2e-shared-a-67be07-serving-cert" not found
Jul 21 14:23:03.430395 ip-10-0-132-245 kubenswrapper[2578]: E0721 14:23:03.430384    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2513f979-f9ab-4d92-b912-9ab5eeb6dcc9-maas-api-tls podName:2513f979-f9ab-4d92-b912-9ab5eeb6dcc9 nodeName:}" failed. No retries permitted until 2026-07-21 14:23:03.930363783 +0000 UTC m=+2330.890652325 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2513f979-f9ab-4d92-b912-9ab5eeb6dcc9-maas-api-tls") pod "maas-api-e2e-shared-a-67be07-65949f64b8-r2j75" (UID: "2513f979-f9ab-4d92-b912-9ab5eeb6dcc9") : secret "maas-api-e2e-shared-a-67be07-serving-cert" not found
Jul 21 14:23:03.430768 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:03.430748    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2513f979-f9ab-4d92-b912-9ab5eeb6dcc9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-67be07-65949f64b8-r2j75\" (UID: \"2513f979-f9ab-4d92-b912-9ab5eeb6dcc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67be07-65949f64b8-r2j75"
Jul 21 14:23:03.441392 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:03.441359    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrtqw\" (UniqueName: \"kubernetes.io/projected/2513f979-f9ab-4d92-b912-9ab5eeb6dcc9-kube-api-access-mrtqw\") pod \"maas-api-e2e-shared-a-67be07-65949f64b8-r2j75\" (UID: \"2513f979-f9ab-4d92-b912-9ab5eeb6dcc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67be07-65949f64b8-r2j75"
Jul 21 14:23:03.935905 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:03.935856    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2513f979-f9ab-4d92-b912-9ab5eeb6dcc9-maas-api-tls\") pod \"maas-api-e2e-shared-a-67be07-65949f64b8-r2j75\" (UID: \"2513f979-f9ab-4d92-b912-9ab5eeb6dcc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67be07-65949f64b8-r2j75"
Jul 21 14:23:03.938254 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:03.938230    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2513f979-f9ab-4d92-b912-9ab5eeb6dcc9-maas-api-tls\") pod \"maas-api-e2e-shared-a-67be07-65949f64b8-r2j75\" (UID: \"2513f979-f9ab-4d92-b912-9ab5eeb6dcc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67be07-65949f64b8-r2j75"
Jul 21 14:23:04.145736 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:04.145705    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67be07-65949f64b8-r2j75"
Jul 21 14:23:04.277912 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:04.277867    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-67be07-65949f64b8-r2j75"]
Jul 21 14:23:04.279768 ip-10-0-132-245 kubenswrapper[2578]: W0721 14:23:04.279742    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2513f979_f9ab_4d92_b912_9ab5eeb6dcc9.slice/crio-ac5c30965fefdd14fcffeaf05e3a625a42780b8b7a4682cf3c13d1cf8b4df60f WatchSource:0}: Error finding container ac5c30965fefdd14fcffeaf05e3a625a42780b8b7a4682cf3c13d1cf8b4df60f: Status 404 returned error can't find the container with id ac5c30965fefdd14fcffeaf05e3a625a42780b8b7a4682cf3c13d1cf8b4df60f
Jul 21 14:23:05.014360 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:05.014326    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67be07-65949f64b8-r2j75" event={"ID":"2513f979-f9ab-4d92-b912-9ab5eeb6dcc9","Type":"ContainerStarted","Data":"ac5c30965fefdd14fcffeaf05e3a625a42780b8b7a4682cf3c13d1cf8b4df60f"}
Jul 21 14:23:06.020025 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:06.019995    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67be07-65949f64b8-r2j75" event={"ID":"2513f979-f9ab-4d92-b912-9ab5eeb6dcc9","Type":"ContainerStarted","Data":"ae43c95a42df414211fe5d10c0d7df19a9712330518fd60c720b30931eeeae76"}
Jul 21 14:23:06.020401 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:06.020067    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67be07-65949f64b8-r2j75"
Jul 21 14:23:06.038431 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:06.038387    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67be07-65949f64b8-r2j75" podStartSLOduration=2.147675256 podStartE2EDuration="3.038371993s" totalImagesPullingTime="890.696737ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:23:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:23:06.037374128 +0000 UTC m=+2332.997662684" watchObservedRunningTime="2026-07-21 14:23:06.038371993 +0000 UTC m=+2332.998660550"
Jul 21 14:23:11.171561 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:11.171516    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:23:11.331098 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:11.331070    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-52967c-99dd4dc86-d6nhj"]
Jul 21 14:23:11.334701 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:11.334673    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52967c-99dd4dc86-d6nhj"
Jul 21 14:23:11.337188 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:11.337161    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-52967c-serving-cert\""
Jul 21 14:23:11.346758 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:11.346733    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-52967c-99dd4dc86-d6nhj"]
Jul 21 14:23:11.498378 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:11.498278    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-466lc\" (UniqueName: \"kubernetes.io/projected/f1f9ab77-cf72-4335-81ed-816a28c7e971-kube-api-access-466lc\") pod \"maas-api-e2e-shared-b-52967c-99dd4dc86-d6nhj\" (UID: \"f1f9ab77-cf72-4335-81ed-816a28c7e971\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52967c-99dd4dc86-d6nhj"
Jul 21 14:23:11.498378 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:11.498337    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1f9ab77-cf72-4335-81ed-816a28c7e971-openshift-service-ca\") pod \"maas-api-e2e-shared-b-52967c-99dd4dc86-d6nhj\" (UID: \"f1f9ab77-cf72-4335-81ed-816a28c7e971\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52967c-99dd4dc86-d6nhj"
Jul 21 14:23:11.498606 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:11.498391    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1f9ab77-cf72-4335-81ed-816a28c7e971-maas-api-tls\") pod \"maas-api-e2e-shared-b-52967c-99dd4dc86-d6nhj\" (UID: \"f1f9ab77-cf72-4335-81ed-816a28c7e971\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52967c-99dd4dc86-d6nhj"
Jul 21 14:23:11.599579 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:11.599530    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1f9ab77-cf72-4335-81ed-816a28c7e971-maas-api-tls\") pod \"maas-api-e2e-shared-b-52967c-99dd4dc86-d6nhj\" (UID: \"f1f9ab77-cf72-4335-81ed-816a28c7e971\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52967c-99dd4dc86-d6nhj"
Jul 21 14:23:11.599788 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:11.599595    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-466lc\" (UniqueName: \"kubernetes.io/projected/f1f9ab77-cf72-4335-81ed-816a28c7e971-kube-api-access-466lc\") pod \"maas-api-e2e-shared-b-52967c-99dd4dc86-d6nhj\" (UID: \"f1f9ab77-cf72-4335-81ed-816a28c7e971\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52967c-99dd4dc86-d6nhj"
Jul 21 14:23:11.599788 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:11.599640    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1f9ab77-cf72-4335-81ed-816a28c7e971-openshift-service-ca\") pod \"maas-api-e2e-shared-b-52967c-99dd4dc86-d6nhj\" (UID: \"f1f9ab77-cf72-4335-81ed-816a28c7e971\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52967c-99dd4dc86-d6nhj"
Jul 21 14:23:11.600323 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:11.600303    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1f9ab77-cf72-4335-81ed-816a28c7e971-openshift-service-ca\") pod \"maas-api-e2e-shared-b-52967c-99dd4dc86-d6nhj\" (UID: \"f1f9ab77-cf72-4335-81ed-816a28c7e971\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52967c-99dd4dc86-d6nhj"
Jul 21 14:23:11.602024 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:11.602006    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1f9ab77-cf72-4335-81ed-816a28c7e971-maas-api-tls\") pod \"maas-api-e2e-shared-b-52967c-99dd4dc86-d6nhj\" (UID: \"f1f9ab77-cf72-4335-81ed-816a28c7e971\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52967c-99dd4dc86-d6nhj"
Jul 21 14:23:11.609533 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:11.609509    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-466lc\" (UniqueName: \"kubernetes.io/projected/f1f9ab77-cf72-4335-81ed-816a28c7e971-kube-api-access-466lc\") pod \"maas-api-e2e-shared-b-52967c-99dd4dc86-d6nhj\" (UID: \"f1f9ab77-cf72-4335-81ed-816a28c7e971\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52967c-99dd4dc86-d6nhj"
Jul 21 14:23:11.648172 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:11.648133    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52967c-99dd4dc86-d6nhj"
Jul 21 14:23:11.817018 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:11.816977    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-52967c-99dd4dc86-d6nhj"]
Jul 21 14:23:11.819430 ip-10-0-132-245 kubenswrapper[2578]: W0721 14:23:11.819403    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1f9ab77_cf72_4335_81ed_816a28c7e971.slice/crio-8fc02901126e6b74b5ab163339015ecbe11ce51d20915a87fbdfafbd56be4f26 WatchSource:0}: Error finding container 8fc02901126e6b74b5ab163339015ecbe11ce51d20915a87fbdfafbd56be4f26: Status 404 returned error can't find the container with id 8fc02901126e6b74b5ab163339015ecbe11ce51d20915a87fbdfafbd56be4f26
Jul 21 14:23:12.030711 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:12.030632    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67be07-65949f64b8-r2j75"
Jul 21 14:23:12.049571 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:12.049523    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52967c-99dd4dc86-d6nhj" event={"ID":"f1f9ab77-cf72-4335-81ed-816a28c7e971","Type":"ContainerStarted","Data":"8fc02901126e6b74b5ab163339015ecbe11ce51d20915a87fbdfafbd56be4f26"}
Jul 21 14:23:13.055370 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:13.055336    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52967c-99dd4dc86-d6nhj" event={"ID":"f1f9ab77-cf72-4335-81ed-816a28c7e971","Type":"ContainerStarted","Data":"8dc8db2018cd18a990823a8de154183353460e6e0e7bda09d0eed6676ec3a54a"}
Jul 21 14:23:13.055791 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:13.055457    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52967c-99dd4dc86-d6nhj"
Jul 21 14:23:13.075025 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:13.074978    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52967c-99dd4dc86-d6nhj" podStartSLOduration=1.408698327 podStartE2EDuration="2.07496273s" totalImagesPullingTime="666.264403ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:23:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:23:13.074093639 +0000 UTC m=+2340.034382197" watchObservedRunningTime="2026-07-21 14:23:13.07496273 +0000 UTC m=+2340.035251290"
Jul 21 14:23:13.689603 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:13.689572    2578 scope.go:117] "RemoveContainer" containerID="7b6a52c264b13f591a7a82db030cdb9245a06fdc0263e647e19b12c4ddaa0408"
Jul 21 14:23:19.065123 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:19.065094    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52967c-99dd4dc86-d6nhj"
Jul 21 14:23:26.569974 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:26.569941    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:23:32.386354 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:32.386321    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-67be07/auth-test-model-67be07-kserve-7cb77b4d99-97hdt"]
Jul 21 14:23:32.389732 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:32.389712    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-67be07/auth-test-model-67be07-kserve-7cb77b4d99-97hdt"
Jul 21 14:23:32.392521 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:32.392498    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-67be07\"/\"openshift-service-ca.crt\""
Jul 21 14:23:32.392725 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:32.392707    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-67be07\"/\"auth-test-model-67be07-kserve-self-signed-certs\""
Jul 21 14:23:32.393733 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:32.393715    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-67be07\"/\"default-dockercfg-s45j2\""
Jul 21 14:23:32.393994 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:32.393977    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-67be07\"/\"kube-root-ca.crt\""
Jul 21 14:23:32.400857 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:32.400837    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-67be07/auth-test-model-67be07-kserve-7cb77b4d99-97hdt"]
Jul 21 14:23:32.475408 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:32.475375    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b59337d6-9e7a-46ec-8a2a-de299945583e-home\") pod \"auth-test-model-67be07-kserve-7cb77b4d99-97hdt\" (UID: \"b59337d6-9e7a-46ec-8a2a-de299945583e\") " pod="ai-tenant-e2e-shared-a-67be07/auth-test-model-67be07-kserve-7cb77b4d99-97hdt"
Jul 21 14:23:32.475602 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:32.475426    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b59337d6-9e7a-46ec-8a2a-de299945583e-tmp-dir\") pod \"auth-test-model-67be07-kserve-7cb77b4d99-97hdt\" (UID: \"b59337d6-9e7a-46ec-8a2a-de299945583e\") " pod="ai-tenant-e2e-shared-a-67be07/auth-test-model-67be07-kserve-7cb77b4d99-97hdt"
Jul 21 14:23:32.475602 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:32.475449    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nr8xl\" (UniqueName: \"kubernetes.io/projected/b59337d6-9e7a-46ec-8a2a-de299945583e-kube-api-access-nr8xl\") pod \"auth-test-model-67be07-kserve-7cb77b4d99-97hdt\" (UID: \"b59337d6-9e7a-46ec-8a2a-de299945583e\") " pod="ai-tenant-e2e-shared-a-67be07/auth-test-model-67be07-kserve-7cb77b4d99-97hdt"
Jul 21 14:23:32.475602 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:32.475523    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b59337d6-9e7a-46ec-8a2a-de299945583e-dshm\") pod \"auth-test-model-67be07-kserve-7cb77b4d99-97hdt\" (UID: \"b59337d6-9e7a-46ec-8a2a-de299945583e\") " pod="ai-tenant-e2e-shared-a-67be07/auth-test-model-67be07-kserve-7cb77b4d99-97hdt"
Jul 21 14:23:32.475602 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:32.475559    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b59337d6-9e7a-46ec-8a2a-de299945583e-model-cache\") pod \"auth-test-model-67be07-kserve-7cb77b4d99-97hdt\" (UID: \"b59337d6-9e7a-46ec-8a2a-de299945583e\") " pod="ai-tenant-e2e-shared-a-67be07/auth-test-model-67be07-kserve-7cb77b4d99-97hdt"
Jul 21 14:23:32.475602 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:32.475582    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b59337d6-9e7a-46ec-8a2a-de299945583e-tls-certs\") pod \"auth-test-model-67be07-kserve-7cb77b4d99-97hdt\" (UID: \"b59337d6-9e7a-46ec-8a2a-de299945583e\") " pod="ai-tenant-e2e-shared-a-67be07/auth-test-model-67be07-kserve-7cb77b4d99-97hdt"
Jul 21 14:23:32.577043 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:32.577005    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b59337d6-9e7a-46ec-8a2a-de299945583e-home\") pod \"auth-test-model-67be07-kserve-7cb77b4d99-97hdt\" (UID: \"b59337d6-9e7a-46ec-8a2a-de299945583e\") " pod="ai-tenant-e2e-shared-a-67be07/auth-test-model-67be07-kserve-7cb77b4d99-97hdt"
Jul 21 14:23:32.577229 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:32.577069    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b59337d6-9e7a-46ec-8a2a-de299945583e-tmp-dir\") pod \"auth-test-model-67be07-kserve-7cb77b4d99-97hdt\" (UID: \"b59337d6-9e7a-46ec-8a2a-de299945583e\") " pod="ai-tenant-e2e-shared-a-67be07/auth-test-model-67be07-kserve-7cb77b4d99-97hdt"
Jul 21 14:23:32.577229 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:32.577092    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nr8xl\" (UniqueName: \"kubernetes.io/projected/b59337d6-9e7a-46ec-8a2a-de299945583e-kube-api-access-nr8xl\") pod \"auth-test-model-67be07-kserve-7cb77b4d99-97hdt\" (UID: \"b59337d6-9e7a-46ec-8a2a-de299945583e\") " pod="ai-tenant-e2e-shared-a-67be07/auth-test-model-67be07-kserve-7cb77b4d99-97hdt"
Jul 21 14:23:32.577229 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:32.577114    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b59337d6-9e7a-46ec-8a2a-de299945583e-dshm\") pod \"auth-test-model-67be07-kserve-7cb77b4d99-97hdt\" (UID: \"b59337d6-9e7a-46ec-8a2a-de299945583e\") " pod="ai-tenant-e2e-shared-a-67be07/auth-test-model-67be07-kserve-7cb77b4d99-97hdt"
Jul 21 14:23:32.577229 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:32.577143    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b59337d6-9e7a-46ec-8a2a-de299945583e-model-cache\") pod \"auth-test-model-67be07-kserve-7cb77b4d99-97hdt\" (UID: \"b59337d6-9e7a-46ec-8a2a-de299945583e\") " pod="ai-tenant-e2e-shared-a-67be07/auth-test-model-67be07-kserve-7cb77b4d99-97hdt"
Jul 21 14:23:32.577229 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:32.577161    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b59337d6-9e7a-46ec-8a2a-de299945583e-tls-certs\") pod \"auth-test-model-67be07-kserve-7cb77b4d99-97hdt\" (UID: \"b59337d6-9e7a-46ec-8a2a-de299945583e\") " pod="ai-tenant-e2e-shared-a-67be07/auth-test-model-67be07-kserve-7cb77b4d99-97hdt"
Jul 21 14:23:32.577555 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:32.577468    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b59337d6-9e7a-46ec-8a2a-de299945583e-home\") pod \"auth-test-model-67be07-kserve-7cb77b4d99-97hdt\" (UID: \"b59337d6-9e7a-46ec-8a2a-de299945583e\") " pod="ai-tenant-e2e-shared-a-67be07/auth-test-model-67be07-kserve-7cb77b4d99-97hdt"
Jul 21 14:23:32.577555 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:32.577514    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b59337d6-9e7a-46ec-8a2a-de299945583e-tmp-dir\") pod \"auth-test-model-67be07-kserve-7cb77b4d99-97hdt\" (UID: \"b59337d6-9e7a-46ec-8a2a-de299945583e\") " pod="ai-tenant-e2e-shared-a-67be07/auth-test-model-67be07-kserve-7cb77b4d99-97hdt"
Jul 21 14:23:32.577641 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:32.577603    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b59337d6-9e7a-46ec-8a2a-de299945583e-model-cache\") pod \"auth-test-model-67be07-kserve-7cb77b4d99-97hdt\" (UID: \"b59337d6-9e7a-46ec-8a2a-de299945583e\") " pod="ai-tenant-e2e-shared-a-67be07/auth-test-model-67be07-kserve-7cb77b4d99-97hdt"
Jul 21 14:23:32.580134 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:32.580105    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b59337d6-9e7a-46ec-8a2a-de299945583e-dshm\") pod \"auth-test-model-67be07-kserve-7cb77b4d99-97hdt\" (UID: \"b59337d6-9e7a-46ec-8a2a-de299945583e\") " pod="ai-tenant-e2e-shared-a-67be07/auth-test-model-67be07-kserve-7cb77b4d99-97hdt"
Jul 21 14:23:32.580345 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:32.580328    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b59337d6-9e7a-46ec-8a2a-de299945583e-tls-certs\") pod \"auth-test-model-67be07-kserve-7cb77b4d99-97hdt\" (UID: \"b59337d6-9e7a-46ec-8a2a-de299945583e\") " pod="ai-tenant-e2e-shared-a-67be07/auth-test-model-67be07-kserve-7cb77b4d99-97hdt"
Jul 21 14:23:32.586605 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:32.586584    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nr8xl\" (UniqueName: \"kubernetes.io/projected/b59337d6-9e7a-46ec-8a2a-de299945583e-kube-api-access-nr8xl\") pod \"auth-test-model-67be07-kserve-7cb77b4d99-97hdt\" (UID: \"b59337d6-9e7a-46ec-8a2a-de299945583e\") " pod="ai-tenant-e2e-shared-a-67be07/auth-test-model-67be07-kserve-7cb77b4d99-97hdt"
Jul 21 14:23:32.700028 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:32.699940    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-67be07/auth-test-model-67be07-kserve-7cb77b4d99-97hdt"
Jul 21 14:23:32.838648 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:32.838601    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-67be07/auth-test-model-67be07-kserve-7cb77b4d99-97hdt"]
Jul 21 14:23:32.841380 ip-10-0-132-245 kubenswrapper[2578]: W0721 14:23:32.841341    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb59337d6_9e7a_46ec_8a2a_de299945583e.slice/crio-37eeda8af6607908f330c07883442cc36c83c30aed979c42dac5309cadbbcda2 WatchSource:0}: Error finding container 37eeda8af6607908f330c07883442cc36c83c30aed979c42dac5309cadbbcda2: Status 404 returned error can't find the container with id 37eeda8af6607908f330c07883442cc36c83c30aed979c42dac5309cadbbcda2
Jul 21 14:23:33.146472 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:33.146439    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-67be07/auth-test-model-67be07-kserve-7cb77b4d99-97hdt" event={"ID":"b59337d6-9e7a-46ec-8a2a-de299945583e","Type":"ContainerStarted","Data":"b21fe85e1d41e920e38aa56b2fe935ac58c22a3d044f288722f8e2b141d3787f"}
Jul 21 14:23:33.146666 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:33.146482    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-67be07/auth-test-model-67be07-kserve-7cb77b4d99-97hdt" event={"ID":"b59337d6-9e7a-46ec-8a2a-de299945583e","Type":"ContainerStarted","Data":"37eeda8af6607908f330c07883442cc36c83c30aed979c42dac5309cadbbcda2"}
Jul 21 14:23:33.168715 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:33.168657    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-67be07/auth-test-model-67be07-kserve-7cb77b4d99-97hdt" podStartSLOduration=1.168637834 podStartE2EDuration="1.168637834s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:23:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:23:33.167370024 +0000 UTC m=+2360.127658583" watchObservedRunningTime="2026-07-21 14:23:33.168637834 +0000 UTC m=+2360.128926392"
Jul 21 14:23:36.765820 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:36.765780    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:23:42.701034 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:42.700995    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-67be07/auth-test-model-67be07-kserve-7cb77b4d99-97hdt"
Jul 21 14:23:42.701034 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:42.701042    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-67be07/auth-test-model-67be07-kserve-7cb77b4d99-97hdt"
Jul 21 14:23:42.713552 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:42.713522    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-67be07/auth-test-model-67be07-kserve-7cb77b4d99-97hdt"
Jul 21 14:23:43.200921 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:23:43.200873    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-67be07/auth-test-model-67be07-kserve-7cb77b4d99-97hdt"
Jul 21 14:24:07.561696 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:24:07.561662    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:24:23.275473 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:24:23.275435    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:24:40.367926 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:24:40.367823    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:24:58.783264 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:24:58.783231    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:25:15.861757 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:25:15.861722    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:25:34.964156 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:25:34.964122    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:25:55.071891 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:25:55.071829    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:26:10.489640 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:26:10.489558    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-67be07/auth-test-model-67be07-kserve-7cb77b4d99-97hdt"]
Jul 21 14:26:14.083944 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:26:14.083908    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:26:25.509609 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:26:25.509554    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-67be07/auth-test-model-67be07-kserve-7cb77b4d99-97hdt" podUID="b59337d6-9e7a-46ec-8a2a-de299945583e" containerName="main" containerID="cri-o://b21fe85e1d41e920e38aa56b2fe935ac58c22a3d044f288722f8e2b141d3787f" gracePeriod=45
Jul 21 14:26:25.767208 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:26:25.767149    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-67be07/auth-test-model-67be07-kserve-7cb77b4d99-97hdt"
Jul 21 14:26:25.829653 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:26:25.829624    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b59337d6-9e7a-46ec-8a2a-de299945583e-model-cache\") pod \"b59337d6-9e7a-46ec-8a2a-de299945583e\" (UID: \"b59337d6-9e7a-46ec-8a2a-de299945583e\") "
Jul 21 14:26:25.829841 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:26:25.829670    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b59337d6-9e7a-46ec-8a2a-de299945583e-tmp-dir\") pod \"b59337d6-9e7a-46ec-8a2a-de299945583e\" (UID: \"b59337d6-9e7a-46ec-8a2a-de299945583e\") "
Jul 21 14:26:25.829841 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:26:25.829686    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b59337d6-9e7a-46ec-8a2a-de299945583e-dshm\") pod \"b59337d6-9e7a-46ec-8a2a-de299945583e\" (UID: \"b59337d6-9e7a-46ec-8a2a-de299945583e\") "
Jul 21 14:26:25.829841 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:26:25.829708    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b59337d6-9e7a-46ec-8a2a-de299945583e-home\") pod \"b59337d6-9e7a-46ec-8a2a-de299945583e\" (UID: \"b59337d6-9e7a-46ec-8a2a-de299945583e\") "
Jul 21 14:26:25.829841 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:26:25.829735    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nr8xl\" (UniqueName: \"kubernetes.io/projected/b59337d6-9e7a-46ec-8a2a-de299945583e-kube-api-access-nr8xl\") pod \"b59337d6-9e7a-46ec-8a2a-de299945583e\" (UID: \"b59337d6-9e7a-46ec-8a2a-de299945583e\") "
Jul 21 14:26:25.829841 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:26:25.829762    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b59337d6-9e7a-46ec-8a2a-de299945583e-tls-certs\") pod \"b59337d6-9e7a-46ec-8a2a-de299945583e\" (UID: \"b59337d6-9e7a-46ec-8a2a-de299945583e\") "
Jul 21 14:26:25.830138 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:26:25.829958    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b59337d6-9e7a-46ec-8a2a-de299945583e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b59337d6-9e7a-46ec-8a2a-de299945583e" (UID: "b59337d6-9e7a-46ec-8a2a-de299945583e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 14:26:25.830138 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:26:25.829966    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b59337d6-9e7a-46ec-8a2a-de299945583e-model-cache" (OuterVolumeSpecName: "model-cache") pod "b59337d6-9e7a-46ec-8a2a-de299945583e" (UID: "b59337d6-9e7a-46ec-8a2a-de299945583e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 14:26:25.830138 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:26:25.830004    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b59337d6-9e7a-46ec-8a2a-de299945583e-home" (OuterVolumeSpecName: "home") pod "b59337d6-9e7a-46ec-8a2a-de299945583e" (UID: "b59337d6-9e7a-46ec-8a2a-de299945583e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 14:26:25.831925 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:26:25.831858    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b59337d6-9e7a-46ec-8a2a-de299945583e-dshm" (OuterVolumeSpecName: "dshm") pod "b59337d6-9e7a-46ec-8a2a-de299945583e" (UID: "b59337d6-9e7a-46ec-8a2a-de299945583e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 14:26:25.832039 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:26:25.831973    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b59337d6-9e7a-46ec-8a2a-de299945583e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b59337d6-9e7a-46ec-8a2a-de299945583e" (UID: "b59337d6-9e7a-46ec-8a2a-de299945583e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:26:25.832039 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:26:25.831998    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b59337d6-9e7a-46ec-8a2a-de299945583e-kube-api-access-nr8xl" (OuterVolumeSpecName: "kube-api-access-nr8xl") pod "b59337d6-9e7a-46ec-8a2a-de299945583e" (UID: "b59337d6-9e7a-46ec-8a2a-de299945583e"). InnerVolumeSpecName "kube-api-access-nr8xl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:26:25.911448 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:26:25.911414    2578 generic.go:356] "Generic (PLEG): container finished" podID="b59337d6-9e7a-46ec-8a2a-de299945583e" containerID="b21fe85e1d41e920e38aa56b2fe935ac58c22a3d044f288722f8e2b141d3787f" exitCode=0
Jul 21 14:26:25.911636 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:26:25.911454    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-67be07/auth-test-model-67be07-kserve-7cb77b4d99-97hdt"
Jul 21 14:26:25.911636 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:26:25.911503    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-67be07/auth-test-model-67be07-kserve-7cb77b4d99-97hdt" event={"ID":"b59337d6-9e7a-46ec-8a2a-de299945583e","Type":"ContainerDied","Data":"b21fe85e1d41e920e38aa56b2fe935ac58c22a3d044f288722f8e2b141d3787f"}
Jul 21 14:26:25.911636 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:26:25.911549    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-67be07/auth-test-model-67be07-kserve-7cb77b4d99-97hdt" event={"ID":"b59337d6-9e7a-46ec-8a2a-de299945583e","Type":"ContainerDied","Data":"37eeda8af6607908f330c07883442cc36c83c30aed979c42dac5309cadbbcda2"}
Jul 21 14:26:25.911636 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:26:25.911572    2578 scope.go:117] "RemoveContainer" containerID="b21fe85e1d41e920e38aa56b2fe935ac58c22a3d044f288722f8e2b141d3787f"
Jul 21 14:26:25.920648 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:26:25.920632    2578 scope.go:117] "RemoveContainer" containerID="b21fe85e1d41e920e38aa56b2fe935ac58c22a3d044f288722f8e2b141d3787f"
Jul 21 14:26:25.920921 ip-10-0-132-245 kubenswrapper[2578]: E0721 14:26:25.920902    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b21fe85e1d41e920e38aa56b2fe935ac58c22a3d044f288722f8e2b141d3787f\": container with ID starting with b21fe85e1d41e920e38aa56b2fe935ac58c22a3d044f288722f8e2b141d3787f not found: ID does not exist" containerID="b21fe85e1d41e920e38aa56b2fe935ac58c22a3d044f288722f8e2b141d3787f"
Jul 21 14:26:25.920976 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:26:25.920929    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b21fe85e1d41e920e38aa56b2fe935ac58c22a3d044f288722f8e2b141d3787f"} err="failed to get container status \"b21fe85e1d41e920e38aa56b2fe935ac58c22a3d044f288722f8e2b141d3787f\": rpc error: code = NotFound desc = could not find container \"b21fe85e1d41e920e38aa56b2fe935ac58c22a3d044f288722f8e2b141d3787f\": container with ID starting with b21fe85e1d41e920e38aa56b2fe935ac58c22a3d044f288722f8e2b141d3787f not found: ID does not exist"
Jul 21 14:26:25.931083 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:26:25.931054    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b59337d6-9e7a-46ec-8a2a-de299945583e-tls-certs\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:26:25.931083 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:26:25.931077    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b59337d6-9e7a-46ec-8a2a-de299945583e-model-cache\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:26:25.931292 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:26:25.931090    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b59337d6-9e7a-46ec-8a2a-de299945583e-tmp-dir\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:26:25.931292 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:26:25.931102    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b59337d6-9e7a-46ec-8a2a-de299945583e-dshm\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:26:25.931292 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:26:25.931112    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b59337d6-9e7a-46ec-8a2a-de299945583e-home\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:26:25.931292 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:26:25.931125    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nr8xl\" (UniqueName: \"kubernetes.io/projected/b59337d6-9e7a-46ec-8a2a-de299945583e-kube-api-access-nr8xl\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:26:25.932756 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:26:25.932735    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-67be07/auth-test-model-67be07-kserve-7cb77b4d99-97hdt"]
Jul 21 14:26:25.937030 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:26:25.937012    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-67be07/auth-test-model-67be07-kserve-7cb77b4d99-97hdt"]
Jul 21 14:26:27.504786 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:26:27.504752    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b59337d6-9e7a-46ec-8a2a-de299945583e" path="/var/lib/kubelet/pods/b59337d6-9e7a-46ec-8a2a-de299945583e/volumes"
Jul 21 14:26:27.873909 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:26:27.873858    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:26:50.472386 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:26:50.472353    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:27:11.679817 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:27:11.679784    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:27:13.736992 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:27:13.736948    2578 scope.go:117] "RemoveContainer" containerID="ed5abb27daf8e76300adb82796a7db35206f5e8594f4cc44c49927b60b936d74"
Jul 21 14:27:30.077023 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:27:30.076986    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:27:49.569793 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:27:49.569754    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:27:53.075454 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:27:53.075422    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-67be07/rate-test-model-67be07-kserve-6f7f95d455-sxbvz"]
Jul 21 14:27:53.075822 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:27:53.075752    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b59337d6-9e7a-46ec-8a2a-de299945583e" containerName="main"
Jul 21 14:27:53.075822 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:27:53.075763    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b59337d6-9e7a-46ec-8a2a-de299945583e" containerName="main"
Jul 21 14:27:53.075918 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:27:53.075827    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b59337d6-9e7a-46ec-8a2a-de299945583e" containerName="main"
Jul 21 14:27:53.078771 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:27:53.078745    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-67be07/rate-test-model-67be07-kserve-6f7f95d455-sxbvz"
Jul 21 14:27:53.082800 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:27:53.082775    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-67be07\"/\"default-dockercfg-s45j2\""
Jul 21 14:27:53.083817 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:27:53.083796    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-67be07\"/\"kube-root-ca.crt\""
Jul 21 14:27:53.083954 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:27:53.083799    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-67be07\"/\"openshift-service-ca.crt\""
Jul 21 14:27:53.083954 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:27:53.083798    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-67be07\"/\"rate-test-model-67be07-kserve-self-signed-certs\""
Jul 21 14:27:53.091005 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:27:53.090984    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-67be07/rate-test-model-67be07-kserve-6f7f95d455-sxbvz"]
Jul 21 14:27:53.168535 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:27:53.168503    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb-model-cache\") pod \"rate-test-model-67be07-kserve-6f7f95d455-sxbvz\" (UID: \"e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb\") " pod="ai-tenant-e2e-shared-a-67be07/rate-test-model-67be07-kserve-6f7f95d455-sxbvz"
Jul 21 14:27:53.168535 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:27:53.168539    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb-home\") pod \"rate-test-model-67be07-kserve-6f7f95d455-sxbvz\" (UID: \"e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb\") " pod="ai-tenant-e2e-shared-a-67be07/rate-test-model-67be07-kserve-6f7f95d455-sxbvz"
Jul 21 14:27:53.168770 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:27:53.168557    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb-tmp-dir\") pod \"rate-test-model-67be07-kserve-6f7f95d455-sxbvz\" (UID: \"e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb\") " pod="ai-tenant-e2e-shared-a-67be07/rate-test-model-67be07-kserve-6f7f95d455-sxbvz"
Jul 21 14:27:53.168770 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:27:53.168672    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb-dshm\") pod \"rate-test-model-67be07-kserve-6f7f95d455-sxbvz\" (UID: \"e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb\") " pod="ai-tenant-e2e-shared-a-67be07/rate-test-model-67be07-kserve-6f7f95d455-sxbvz"
Jul 21 14:27:53.168770 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:27:53.168701    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb-tls-certs\") pod \"rate-test-model-67be07-kserve-6f7f95d455-sxbvz\" (UID: \"e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb\") " pod="ai-tenant-e2e-shared-a-67be07/rate-test-model-67be07-kserve-6f7f95d455-sxbvz"
Jul 21 14:27:53.168770 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:27:53.168733    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sq2r\" (UniqueName: \"kubernetes.io/projected/e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb-kube-api-access-9sq2r\") pod \"rate-test-model-67be07-kserve-6f7f95d455-sxbvz\" (UID: \"e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb\") " pod="ai-tenant-e2e-shared-a-67be07/rate-test-model-67be07-kserve-6f7f95d455-sxbvz"
Jul 21 14:27:53.269476 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:27:53.269439    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb-model-cache\") pod \"rate-test-model-67be07-kserve-6f7f95d455-sxbvz\" (UID: \"e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb\") " pod="ai-tenant-e2e-shared-a-67be07/rate-test-model-67be07-kserve-6f7f95d455-sxbvz"
Jul 21 14:27:53.269476 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:27:53.269477    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb-home\") pod \"rate-test-model-67be07-kserve-6f7f95d455-sxbvz\" (UID: \"e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb\") " pod="ai-tenant-e2e-shared-a-67be07/rate-test-model-67be07-kserve-6f7f95d455-sxbvz"
Jul 21 14:27:53.269690 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:27:53.269497    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb-tmp-dir\") pod \"rate-test-model-67be07-kserve-6f7f95d455-sxbvz\" (UID: \"e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb\") " pod="ai-tenant-e2e-shared-a-67be07/rate-test-model-67be07-kserve-6f7f95d455-sxbvz"
Jul 21 14:27:53.269690 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:27:53.269549    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb-dshm\") pod \"rate-test-model-67be07-kserve-6f7f95d455-sxbvz\" (UID: \"e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb\") " pod="ai-tenant-e2e-shared-a-67be07/rate-test-model-67be07-kserve-6f7f95d455-sxbvz"
Jul 21 14:27:53.269690 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:27:53.269564    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb-tls-certs\") pod \"rate-test-model-67be07-kserve-6f7f95d455-sxbvz\" (UID: \"e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb\") " pod="ai-tenant-e2e-shared-a-67be07/rate-test-model-67be07-kserve-6f7f95d455-sxbvz"
Jul 21 14:27:53.269690 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:27:53.269591    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sq2r\" (UniqueName: \"kubernetes.io/projected/e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb-kube-api-access-9sq2r\") pod \"rate-test-model-67be07-kserve-6f7f95d455-sxbvz\" (UID: \"e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb\") " pod="ai-tenant-e2e-shared-a-67be07/rate-test-model-67be07-kserve-6f7f95d455-sxbvz"
Jul 21 14:27:53.269946 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:27:53.269921    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb-home\") pod \"rate-test-model-67be07-kserve-6f7f95d455-sxbvz\" (UID: \"e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb\") " pod="ai-tenant-e2e-shared-a-67be07/rate-test-model-67be07-kserve-6f7f95d455-sxbvz"
Jul 21 14:27:53.269997 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:27:53.269951    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb-model-cache\") pod \"rate-test-model-67be07-kserve-6f7f95d455-sxbvz\" (UID: \"e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb\") " pod="ai-tenant-e2e-shared-a-67be07/rate-test-model-67be07-kserve-6f7f95d455-sxbvz"
Jul 21 14:27:53.270036 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:27:53.270014    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb-tmp-dir\") pod \"rate-test-model-67be07-kserve-6f7f95d455-sxbvz\" (UID: \"e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb\") " pod="ai-tenant-e2e-shared-a-67be07/rate-test-model-67be07-kserve-6f7f95d455-sxbvz"
Jul 21 14:27:53.271846 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:27:53.271816    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb-dshm\") pod \"rate-test-model-67be07-kserve-6f7f95d455-sxbvz\" (UID: \"e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb\") " pod="ai-tenant-e2e-shared-a-67be07/rate-test-model-67be07-kserve-6f7f95d455-sxbvz"
Jul 21 14:27:53.272121 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:27:53.272102    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb-tls-certs\") pod \"rate-test-model-67be07-kserve-6f7f95d455-sxbvz\" (UID: \"e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb\") " pod="ai-tenant-e2e-shared-a-67be07/rate-test-model-67be07-kserve-6f7f95d455-sxbvz"
Jul 21 14:27:53.278675 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:27:53.278654    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sq2r\" (UniqueName: \"kubernetes.io/projected/e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb-kube-api-access-9sq2r\") pod \"rate-test-model-67be07-kserve-6f7f95d455-sxbvz\" (UID: \"e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb\") " pod="ai-tenant-e2e-shared-a-67be07/rate-test-model-67be07-kserve-6f7f95d455-sxbvz"
Jul 21 14:27:53.387941 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:27:53.387842    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-67be07/rate-test-model-67be07-kserve-6f7f95d455-sxbvz"
Jul 21 14:27:53.520895 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:27:53.520831    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-67be07/rate-test-model-67be07-kserve-6f7f95d455-sxbvz"]
Jul 21 14:27:53.523620 ip-10-0-132-245 kubenswrapper[2578]: W0721 14:27:53.523593    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode95e1f3f_bb21_41f7_a35b_7ecdd42a4eeb.slice/crio-390be041f16159f6e9636e2df85469947a69cc092870895c536cd3a9b1827b35 WatchSource:0}: Error finding container 390be041f16159f6e9636e2df85469947a69cc092870895c536cd3a9b1827b35: Status 404 returned error can't find the container with id 390be041f16159f6e9636e2df85469947a69cc092870895c536cd3a9b1827b35
Jul 21 14:27:54.306547 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:27:54.306512    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-67be07/rate-test-model-67be07-kserve-6f7f95d455-sxbvz" event={"ID":"e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb","Type":"ContainerStarted","Data":"d153553e81d7575797419619452e6141ba3fa5b11db1dc44f3f461959c6df0b6"}
Jul 21 14:27:54.306547 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:27:54.306552    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-67be07/rate-test-model-67be07-kserve-6f7f95d455-sxbvz" event={"ID":"e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb","Type":"ContainerStarted","Data":"390be041f16159f6e9636e2df85469947a69cc092870895c536cd3a9b1827b35"}
Jul 21 14:27:54.324525 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:27:54.324477    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-67be07/rate-test-model-67be07-kserve-6f7f95d455-sxbvz" podStartSLOduration=1.324463326 podStartE2EDuration="1.324463326s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:27:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:27:54.323412096 +0000 UTC m=+2621.283700683" watchObservedRunningTime="2026-07-21 14:27:54.324463326 +0000 UTC m=+2621.284751882"
Jul 21 14:27:56.780951 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:27:56.780911    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:28:03.388538 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:28:03.388497    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-67be07/rate-test-model-67be07-kserve-6f7f95d455-sxbvz"
Jul 21 14:28:03.389063 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:28:03.388790    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-67be07/rate-test-model-67be07-kserve-6f7f95d455-sxbvz"
Jul 21 14:28:03.401045 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:28:03.401021    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-67be07/rate-test-model-67be07-kserve-6f7f95d455-sxbvz"
Jul 21 14:28:04.360658 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:28:04.360628    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-67be07/rate-test-model-67be07-kserve-6f7f95d455-sxbvz"
Jul 21 14:28:13.757915 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:28:13.757861    2578 scope.go:117] "RemoveContainer" containerID="f62023ada8b72c479a4b74f7c7c9d04a81379975eae331ff8e14df6b4371fb88"
Jul 21 14:28:13.767906 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:28:13.767826    2578 scope.go:117] "RemoveContainer" containerID="dcef9db40e2d9a16bca24fff6efe13c75fb5e25d414b69adc54ae9d582850f68"
Jul 21 14:28:36.470581 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:28:36.470544    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:29:07.766836 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:07.766759    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:29:13.788228 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:13.788195    2578 scope.go:117] "RemoveContainer" containerID="da041ab947b0031a5b18035a63ba4772d19b6d376789bc63df092d6ea399e81e"
Jul 21 14:29:23.966310 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:23.966270    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:29:32.458519 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:32.458486    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-67be07/rate-test-model-67be07-kserve-6f7f95d455-sxbvz"]
Jul 21 14:29:45.168867 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:45.168833    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:29:47.477690 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:47.477649    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-67be07/rate-test-model-67be07-kserve-6f7f95d455-sxbvz" podUID="e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb" containerName="main" containerID="cri-o://d153553e81d7575797419619452e6141ba3fa5b11db1dc44f3f461959c6df0b6" gracePeriod=45
Jul 21 14:29:47.723085 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:47.723063    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-67be07/rate-test-model-67be07-kserve-6f7f95d455-sxbvz"
Jul 21 14:29:47.783387 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:47.783292    2578 generic.go:356] "Generic (PLEG): container finished" podID="e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb" containerID="d153553e81d7575797419619452e6141ba3fa5b11db1dc44f3f461959c6df0b6" exitCode=0
Jul 21 14:29:47.783387 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:47.783344    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-67be07/rate-test-model-67be07-kserve-6f7f95d455-sxbvz"
Jul 21 14:29:47.783387 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:47.783351    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-67be07/rate-test-model-67be07-kserve-6f7f95d455-sxbvz" event={"ID":"e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb","Type":"ContainerDied","Data":"d153553e81d7575797419619452e6141ba3fa5b11db1dc44f3f461959c6df0b6"}
Jul 21 14:29:47.783647 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:47.783415    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-67be07/rate-test-model-67be07-kserve-6f7f95d455-sxbvz" event={"ID":"e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb","Type":"ContainerDied","Data":"390be041f16159f6e9636e2df85469947a69cc092870895c536cd3a9b1827b35"}
Jul 21 14:29:47.783647 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:47.783439    2578 scope.go:117] "RemoveContainer" containerID="d153553e81d7575797419619452e6141ba3fa5b11db1dc44f3f461959c6df0b6"
Jul 21 14:29:47.791479 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:47.791461    2578 scope.go:117] "RemoveContainer" containerID="d153553e81d7575797419619452e6141ba3fa5b11db1dc44f3f461959c6df0b6"
Jul 21 14:29:47.791762 ip-10-0-132-245 kubenswrapper[2578]: E0721 14:29:47.791743    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d153553e81d7575797419619452e6141ba3fa5b11db1dc44f3f461959c6df0b6\": container with ID starting with d153553e81d7575797419619452e6141ba3fa5b11db1dc44f3f461959c6df0b6 not found: ID does not exist" containerID="d153553e81d7575797419619452e6141ba3fa5b11db1dc44f3f461959c6df0b6"
Jul 21 14:29:47.791814 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:47.791772    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d153553e81d7575797419619452e6141ba3fa5b11db1dc44f3f461959c6df0b6"} err="failed to get container status \"d153553e81d7575797419619452e6141ba3fa5b11db1dc44f3f461959c6df0b6\": rpc error: code = NotFound desc = could not find container \"d153553e81d7575797419619452e6141ba3fa5b11db1dc44f3f461959c6df0b6\": container with ID starting with d153553e81d7575797419619452e6141ba3fa5b11db1dc44f3f461959c6df0b6 not found: ID does not exist"
Jul 21 14:29:47.853207 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:47.853173    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb-model-cache\") pod \"e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb\" (UID: \"e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb\") "
Jul 21 14:29:47.853400 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:47.853262    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb-tmp-dir\") pod \"e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb\" (UID: \"e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb\") "
Jul 21 14:29:47.853400 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:47.853286    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb-dshm\") pod \"e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb\" (UID: \"e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb\") "
Jul 21 14:29:47.853400 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:47.853337    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb-tls-certs\") pod \"e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb\" (UID: \"e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb\") "
Jul 21 14:29:47.853400 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:47.853369    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb-home\") pod \"e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb\" (UID: \"e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb\") "
Jul 21 14:29:47.853400 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:47.853392    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9sq2r\" (UniqueName: \"kubernetes.io/projected/e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb-kube-api-access-9sq2r\") pod \"e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb\" (UID: \"e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb\") "
Jul 21 14:29:47.853654 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:47.853456    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb-model-cache" (OuterVolumeSpecName: "model-cache") pod "e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb" (UID: "e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 14:29:47.853654 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:47.853547    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb" (UID: "e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 14:29:47.853654 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:47.853613    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb-home" (OuterVolumeSpecName: "home") pod "e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb" (UID: "e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 14:29:47.853654 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:47.853630    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb-model-cache\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:29:47.853654 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:47.853646    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb-tmp-dir\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:29:47.855437 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:47.855408    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb" (UID: "e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:29:47.855541 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:47.855443    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb-dshm" (OuterVolumeSpecName: "dshm") pod "e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb" (UID: "e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 14:29:47.855541 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:47.855525    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb-kube-api-access-9sq2r" (OuterVolumeSpecName: "kube-api-access-9sq2r") pod "e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb" (UID: "e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb"). InnerVolumeSpecName "kube-api-access-9sq2r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:29:47.955043 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:47.954999    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb-dshm\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:29:47.955043 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:47.955034    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb-tls-certs\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:29:47.955043 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:47.955049    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb-home\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:29:47.955282 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:47.955062    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9sq2r\" (UniqueName: \"kubernetes.io/projected/e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb-kube-api-access-9sq2r\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:29:48.106296 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:48.106264    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-67be07/rate-test-model-67be07-kserve-6f7f95d455-sxbvz"]
Jul 21 14:29:48.110054 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:48.110033    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-67be07/rate-test-model-67be07-kserve-6f7f95d455-sxbvz"]
Jul 21 14:29:49.503336 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:49.503300    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb" path="/var/lib/kubelet/pods/e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb/volumes"
Jul 21 14:29:52.641652 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:52.641617    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-67be07-7mcp4"]
Jul 21 14:29:52.642019 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:52.641937    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb" containerName="main"
Jul 21 14:29:52.642019 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:52.641949    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb" containerName="main"
Jul 21 14:29:52.642019 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:52.642011    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e95e1f3f-bb21-41f7-a35b-7ecdd42a4eeb" containerName="main"
Jul 21 14:29:52.644849 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:52.644830    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-67be07-7mcp4"
Jul 21 14:29:52.647297 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:52.647277    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5t9ph\""
Jul 21 14:29:52.652124 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:52.652103    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-67be07-7mcp4"]
Jul 21 14:29:52.793990 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:52.793951    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c97f73f-19e1-44b7-ac96-d8bf3bb5b812-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-67be07-7mcp4\" (UID: \"1c97f73f-19e1-44b7-ac96-d8bf3bb5b812\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-67be07-7mcp4"
Jul 21 14:29:52.894855 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:52.894769    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c97f73f-19e1-44b7-ac96-d8bf3bb5b812-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-67be07-7mcp4\" (UID: \"1c97f73f-19e1-44b7-ac96-d8bf3bb5b812\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-67be07-7mcp4"
Jul 21 14:29:52.895409 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:52.895390    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c97f73f-19e1-44b7-ac96-d8bf3bb5b812-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-67be07-7mcp4\" (UID: \"1c97f73f-19e1-44b7-ac96-d8bf3bb5b812\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-67be07-7mcp4"
Jul 21 14:29:52.955519 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:52.955489    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-67be07-7mcp4"
Jul 21 14:29:53.086608 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:53.086574    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-67be07-7mcp4"]
Jul 21 14:29:53.089358 ip-10-0-132-245 kubenswrapper[2578]: W0721 14:29:53.089333    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c97f73f_19e1_44b7_ac96_d8bf3bb5b812.slice/crio-e548e1a2488d35fe9ecc3fe1335778d21f4d05a5f2101a6f43b9b0984c6ed413 WatchSource:0}: Error finding container e548e1a2488d35fe9ecc3fe1335778d21f4d05a5f2101a6f43b9b0984c6ed413: Status 404 returned error can't find the container with id e548e1a2488d35fe9ecc3fe1335778d21f4d05a5f2101a6f43b9b0984c6ed413
Jul 21 14:29:53.810770 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:53.810742    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-67be07-7mcp4" event={"ID":"1c97f73f-19e1-44b7-ac96-d8bf3bb5b812","Type":"ContainerStarted","Data":"e445165205e3af174aa6797b2bb910b2598440bfeb68b4cd535f77d248588c07"}
Jul 21 14:29:53.811106 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:53.810790    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-67be07-7mcp4" event={"ID":"1c97f73f-19e1-44b7-ac96-d8bf3bb5b812","Type":"ContainerStarted","Data":"e548e1a2488d35fe9ecc3fe1335778d21f4d05a5f2101a6f43b9b0984c6ed413"}
Jul 21 14:29:53.826282 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:53.826237    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-67be07-7mcp4" podStartSLOduration=1.8262240429999999 podStartE2EDuration="1.826224043s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:29:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:29:53.824816942 +0000 UTC m=+2740.785105510" watchObservedRunningTime="2026-07-21 14:29:53.826224043 +0000 UTC m=+2740.786512600"
Jul 21 14:29:54.816414 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:54.816380    2578 generic.go:356] "Generic (PLEG): container finished" podID="1c97f73f-19e1-44b7-ac96-d8bf3bb5b812" containerID="e445165205e3af174aa6797b2bb910b2598440bfeb68b4cd535f77d248588c07" exitCode=0
Jul 21 14:29:54.816810 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:54.816452    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-67be07-7mcp4" event={"ID":"1c97f73f-19e1-44b7-ac96-d8bf3bb5b812","Type":"ContainerDied","Data":"e445165205e3af174aa6797b2bb910b2598440bfeb68b4cd535f77d248588c07"}
Jul 21 14:29:55.949299 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:55.949273    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-67be07-7mcp4"
Jul 21 14:29:56.123409 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:56.123376    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c97f73f-19e1-44b7-ac96-d8bf3bb5b812-maas-api-service-ca\") pod \"1c97f73f-19e1-44b7-ac96-d8bf3bb5b812\" (UID: \"1c97f73f-19e1-44b7-ac96-d8bf3bb5b812\") "
Jul 21 14:29:56.123773 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:56.123747    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c97f73f-19e1-44b7-ac96-d8bf3bb5b812-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1c97f73f-19e1-44b7-ac96-d8bf3bb5b812" (UID: "1c97f73f-19e1-44b7-ac96-d8bf3bb5b812"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:29:56.224776 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:56.224740    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c97f73f-19e1-44b7-ac96-d8bf3bb5b812-maas-api-service-ca\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:29:56.826106 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:56.826076    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-67be07-7mcp4"
Jul 21 14:29:56.826106 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:56.826085    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-67be07-7mcp4" event={"ID":"1c97f73f-19e1-44b7-ac96-d8bf3bb5b812","Type":"ContainerDied","Data":"e548e1a2488d35fe9ecc3fe1335778d21f4d05a5f2101a6f43b9b0984c6ed413"}
Jul 21 14:29:56.826314 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:29:56.826131    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e548e1a2488d35fe9ecc3fe1335778d21f4d05a5f2101a6f43b9b0984c6ed413"
Jul 21 14:30:00.174789 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:00.174740    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744070-lffm7"]
Jul 21 14:30:00.175194 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:00.175085    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c97f73f-19e1-44b7-ac96-d8bf3bb5b812" containerName="revoke-keys"
Jul 21 14:30:00.175194 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:00.175097    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c97f73f-19e1-44b7-ac96-d8bf3bb5b812" containerName="revoke-keys"
Jul 21 14:30:00.175194 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:00.175173    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c97f73f-19e1-44b7-ac96-d8bf3bb5b812" containerName="revoke-keys"
Jul 21 14:30:00.178105 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:00.178091    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744070-lffm7"
Jul 21 14:30:00.182250 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:00.182229    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5t9ph\""
Jul 21 14:30:00.185123 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:00.185103    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-52967c-29744070-5vwsm"]
Jul 21 14:30:00.187582 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:00.187565    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744070-lffm7"
Jul 21 14:30:00.188063 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:00.188047    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-52967c-29744070-5vwsm"
Jul 21 14:30:00.192251 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:00.192233    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-67be07-29744070-b27l2"]
Jul 21 14:30:00.196051 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:00.196033    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-67be07-29744070-b27l2"
Jul 21 14:30:00.198179 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:00.198157    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744070-lffm7"]
Jul 21 14:30:00.202950 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:00.202933    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-52967c-29744070-5vwsm"
Jul 21 14:30:00.205341 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:00.205316    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-52967c-29744070-5vwsm"]
Jul 21 14:30:00.212555 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:00.212529    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-67be07-29744070-b27l2"]
Jul 21 14:30:00.240677 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:00.240652    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-67be07-29744070-b27l2"
Jul 21 14:30:00.358058 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:00.357856    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744070-lffm7"]
Jul 21 14:30:00.360706 ip-10-0-132-245 kubenswrapper[2578]: W0721 14:30:00.360676    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b0a8575_c8e6_467c_8764_beebea4427b4.slice/crio-a320174b193cefc985b35e554cbff196a140c7c3fdfef6bb1c6c79c9e8a9cb1c WatchSource:0}: Error finding container a320174b193cefc985b35e554cbff196a140c7c3fdfef6bb1c6c79c9e8a9cb1c: Status 404 returned error can't find the container with id a320174b193cefc985b35e554cbff196a140c7c3fdfef6bb1c6c79c9e8a9cb1c
Jul 21 14:30:00.385677 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:00.385634    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-52967c-29744070-5vwsm"]
Jul 21 14:30:00.387786 ip-10-0-132-245 kubenswrapper[2578]: W0721 14:30:00.387760    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2bb2843_7373_451a_939c_01de0a0f7d79.slice/crio-40742b8a69181c44d364e05917bdb216a889995f9e563a7bf09888a983f806a4 WatchSource:0}: Error finding container 40742b8a69181c44d364e05917bdb216a889995f9e563a7bf09888a983f806a4: Status 404 returned error can't find the container with id 40742b8a69181c44d364e05917bdb216a889995f9e563a7bf09888a983f806a4
Jul 21 14:30:00.412486 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:00.412465    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-67be07-29744070-b27l2"]
Jul 21 14:30:00.420901 ip-10-0-132-245 kubenswrapper[2578]: W0721 14:30:00.420857    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa2b56b6_4527_46e7_8b84_83d599b989e8.slice/crio-94fb76a6815ef4591a9dc23735d9d6fc8c2ab5e191175746e002b16edc92162b WatchSource:0}: Error finding container 94fb76a6815ef4591a9dc23735d9d6fc8c2ab5e191175746e002b16edc92162b: Status 404 returned error can't find the container with id 94fb76a6815ef4591a9dc23735d9d6fc8c2ab5e191175746e002b16edc92162b
Jul 21 14:30:00.844522 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:00.844481    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-67be07-29744070-b27l2" event={"ID":"fa2b56b6-4527-46e7-8b84-83d599b989e8","Type":"ContainerStarted","Data":"958ab0fa06dd16e59b0e517423b1004d4365e4f6b8d65bb7b438fbc94222dae6"}
Jul 21 14:30:00.844740 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:00.844537    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-67be07-29744070-b27l2" event={"ID":"fa2b56b6-4527-46e7-8b84-83d599b989e8","Type":"ContainerStarted","Data":"94fb76a6815ef4591a9dc23735d9d6fc8c2ab5e191175746e002b16edc92162b"}
Jul 21 14:30:00.845962 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:00.845939    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744070-lffm7" event={"ID":"9b0a8575-c8e6-467c-8764-beebea4427b4","Type":"ContainerStarted","Data":"13f20fb5eb2e3f866bf14fbe4f55ab137b0563c256bc78fa4afb346b53cd535d"}
Jul 21 14:30:00.846100 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:00.845981    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744070-lffm7" event={"ID":"9b0a8575-c8e6-467c-8764-beebea4427b4","Type":"ContainerStarted","Data":"a320174b193cefc985b35e554cbff196a140c7c3fdfef6bb1c6c79c9e8a9cb1c"}
Jul 21 14:30:00.847433 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:00.847405    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-52967c-29744070-5vwsm" event={"ID":"a2bb2843-7373-451a-939c-01de0a0f7d79","Type":"ContainerStarted","Data":"d7436bf464faa778e4219d354ff416dd9724dfa018b1abc8dbfecdcbb139c39e"}
Jul 21 14:30:00.847547 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:00.847449    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-52967c-29744070-5vwsm" event={"ID":"a2bb2843-7373-451a-939c-01de0a0f7d79","Type":"ContainerStarted","Data":"40742b8a69181c44d364e05917bdb216a889995f9e563a7bf09888a983f806a4"}
Jul 21 14:30:00.862317 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:00.862276    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-67be07-29744070-b27l2" podStartSLOduration=0.862260088 podStartE2EDuration="862.260088ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:30:00.861451608 +0000 UTC m=+2747.821740167" watchObservedRunningTime="2026-07-21 14:30:00.862260088 +0000 UTC m=+2747.822548649"
Jul 21 14:30:00.880317 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:00.880269    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744070-lffm7" podStartSLOduration=0.880256285 podStartE2EDuration="880.256285ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:30:00.878353478 +0000 UTC m=+2747.838642036" watchObservedRunningTime="2026-07-21 14:30:00.880256285 +0000 UTC m=+2747.840544841"
Jul 21 14:30:00.901193 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:00.901148    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-52967c-29744070-5vwsm" podStartSLOduration=0.901133426 podStartE2EDuration="901.133426ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:30:00.89891475 +0000 UTC m=+2747.859203303" watchObservedRunningTime="2026-07-21 14:30:00.901133426 +0000 UTC m=+2747.861421982"
Jul 21 14:30:01.858416 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:01.858378    2578 generic.go:356] "Generic (PLEG): container finished" podID="fa2b56b6-4527-46e7-8b84-83d599b989e8" containerID="958ab0fa06dd16e59b0e517423b1004d4365e4f6b8d65bb7b438fbc94222dae6" exitCode=0
Jul 21 14:30:01.858867 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:01.858430    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-67be07-29744070-b27l2" event={"ID":"fa2b56b6-4527-46e7-8b84-83d599b989e8","Type":"ContainerDied","Data":"958ab0fa06dd16e59b0e517423b1004d4365e4f6b8d65bb7b438fbc94222dae6"}
Jul 21 14:30:01.859975 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:01.859953    2578 generic.go:356] "Generic (PLEG): container finished" podID="9b0a8575-c8e6-467c-8764-beebea4427b4" containerID="13f20fb5eb2e3f866bf14fbe4f55ab137b0563c256bc78fa4afb346b53cd535d" exitCode=0
Jul 21 14:30:01.860073 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:01.859980    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744070-lffm7" event={"ID":"9b0a8575-c8e6-467c-8764-beebea4427b4","Type":"ContainerDied","Data":"13f20fb5eb2e3f866bf14fbe4f55ab137b0563c256bc78fa4afb346b53cd535d"}
Jul 21 14:30:01.861310 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:01.861288    2578 generic.go:356] "Generic (PLEG): container finished" podID="a2bb2843-7373-451a-939c-01de0a0f7d79" containerID="d7436bf464faa778e4219d354ff416dd9724dfa018b1abc8dbfecdcbb139c39e" exitCode=0
Jul 21 14:30:01.861401 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:01.861352    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-52967c-29744070-5vwsm" event={"ID":"a2bb2843-7373-451a-939c-01de0a0f7d79","Type":"ContainerDied","Data":"d7436bf464faa778e4219d354ff416dd9724dfa018b1abc8dbfecdcbb139c39e"}
Jul 21 14:30:03.001676 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:03.001648    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-67be07-29744070-b27l2"
Jul 21 14:30:03.031822 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:03.031796    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744070-lffm7"
Jul 21 14:30:03.034765 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:03.034744    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-52967c-29744070-5vwsm"
Jul 21 14:30:03.147843 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:03.147770    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-67be07-65949f64b8-r2j75"]
Jul 21 14:30:03.148010 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:03.147987    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67be07-65949f64b8-r2j75" podUID="2513f979-f9ab-4d92-b912-9ab5eeb6dcc9" containerName="maas-api" containerID="cri-o://ae43c95a42df414211fe5d10c0d7df19a9712330518fd60c720b30931eeeae76" gracePeriod=30
Jul 21 14:30:03.185055 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:03.185027    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-67be07-29744070-b27l2"]
Jul 21 14:30:03.197257 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:03.197229    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-67be07-29744070-b27l2"]
Jul 21 14:30:03.388471 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:03.388444    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67be07-65949f64b8-r2j75"
Jul 21 14:30:03.489459 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:03.489365    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2513f979-f9ab-4d92-b912-9ab5eeb6dcc9-openshift-service-ca\") pod \"2513f979-f9ab-4d92-b912-9ab5eeb6dcc9\" (UID: \"2513f979-f9ab-4d92-b912-9ab5eeb6dcc9\") "
Jul 21 14:30:03.489459 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:03.489420    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mrtqw\" (UniqueName: \"kubernetes.io/projected/2513f979-f9ab-4d92-b912-9ab5eeb6dcc9-kube-api-access-mrtqw\") pod \"2513f979-f9ab-4d92-b912-9ab5eeb6dcc9\" (UID: \"2513f979-f9ab-4d92-b912-9ab5eeb6dcc9\") "
Jul 21 14:30:03.489682 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:03.489464    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2513f979-f9ab-4d92-b912-9ab5eeb6dcc9-maas-api-tls\") pod \"2513f979-f9ab-4d92-b912-9ab5eeb6dcc9\" (UID: \"2513f979-f9ab-4d92-b912-9ab5eeb6dcc9\") "
Jul 21 14:30:03.489720 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:03.489702    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2513f979-f9ab-4d92-b912-9ab5eeb6dcc9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2513f979-f9ab-4d92-b912-9ab5eeb6dcc9" (UID: "2513f979-f9ab-4d92-b912-9ab5eeb6dcc9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:30:03.491560 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:03.491533    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2513f979-f9ab-4d92-b912-9ab5eeb6dcc9-kube-api-access-mrtqw" (OuterVolumeSpecName: "kube-api-access-mrtqw") pod "2513f979-f9ab-4d92-b912-9ab5eeb6dcc9" (UID: "2513f979-f9ab-4d92-b912-9ab5eeb6dcc9"). InnerVolumeSpecName "kube-api-access-mrtqw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:30:03.491560 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:03.491541    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2513f979-f9ab-4d92-b912-9ab5eeb6dcc9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2513f979-f9ab-4d92-b912-9ab5eeb6dcc9" (UID: "2513f979-f9ab-4d92-b912-9ab5eeb6dcc9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:30:03.504077 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:03.504047    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa2b56b6-4527-46e7-8b84-83d599b989e8" path="/var/lib/kubelet/pods/fa2b56b6-4527-46e7-8b84-83d599b989e8/volumes"
Jul 21 14:30:03.589998 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:03.589960    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2513f979-f9ab-4d92-b912-9ab5eeb6dcc9-openshift-service-ca\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:30:03.589998 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:03.589991    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrtqw\" (UniqueName: \"kubernetes.io/projected/2513f979-f9ab-4d92-b912-9ab5eeb6dcc9-kube-api-access-mrtqw\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:30:03.589998 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:03.590003    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2513f979-f9ab-4d92-b912-9ab5eeb6dcc9-maas-api-tls\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:30:03.872061 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:03.872036    2578 scope.go:117] "RemoveContainer" containerID="958ab0fa06dd16e59b0e517423b1004d4365e4f6b8d65bb7b438fbc94222dae6"
Jul 21 14:30:03.872259 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:03.872035    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-67be07-29744070-b27l2"
Jul 21 14:30:03.873653 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:03.873493    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744070-lffm7"
Jul 21 14:30:03.873653 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:03.873513    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744070-lffm7" event={"ID":"9b0a8575-c8e6-467c-8764-beebea4427b4","Type":"ContainerDied","Data":"a320174b193cefc985b35e554cbff196a140c7c3fdfef6bb1c6c79c9e8a9cb1c"}
Jul 21 14:30:03.873653 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:03.873566    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a320174b193cefc985b35e554cbff196a140c7c3fdfef6bb1c6c79c9e8a9cb1c"
Jul 21 14:30:03.875132 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:03.875106    2578 generic.go:356] "Generic (PLEG): container finished" podID="2513f979-f9ab-4d92-b912-9ab5eeb6dcc9" containerID="ae43c95a42df414211fe5d10c0d7df19a9712330518fd60c720b30931eeeae76" exitCode=0
Jul 21 14:30:03.875248 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:03.875151    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67be07-65949f64b8-r2j75"
Jul 21 14:30:03.875248 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:03.875148    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67be07-65949f64b8-r2j75" event={"ID":"2513f979-f9ab-4d92-b912-9ab5eeb6dcc9","Type":"ContainerDied","Data":"ae43c95a42df414211fe5d10c0d7df19a9712330518fd60c720b30931eeeae76"}
Jul 21 14:30:03.875380 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:03.875275    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-67be07-65949f64b8-r2j75" event={"ID":"2513f979-f9ab-4d92-b912-9ab5eeb6dcc9","Type":"ContainerDied","Data":"ac5c30965fefdd14fcffeaf05e3a625a42780b8b7a4682cf3c13d1cf8b4df60f"}
Jul 21 14:30:03.876701 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:03.876676    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-52967c-29744070-5vwsm" event={"ID":"a2bb2843-7373-451a-939c-01de0a0f7d79","Type":"ContainerDied","Data":"40742b8a69181c44d364e05917bdb216a889995f9e563a7bf09888a983f806a4"}
Jul 21 14:30:03.876701 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:03.876692    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-52967c-29744070-5vwsm"
Jul 21 14:30:03.876894 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:03.876715    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="40742b8a69181c44d364e05917bdb216a889995f9e563a7bf09888a983f806a4"
Jul 21 14:30:03.882431 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:03.882403    2578 scope.go:117] "RemoveContainer" containerID="ae43c95a42df414211fe5d10c0d7df19a9712330518fd60c720b30931eeeae76"
Jul 21 14:30:03.889816 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:03.889793    2578 scope.go:117] "RemoveContainer" containerID="ae43c95a42df414211fe5d10c0d7df19a9712330518fd60c720b30931eeeae76"
Jul 21 14:30:03.890150 ip-10-0-132-245 kubenswrapper[2578]: E0721 14:30:03.890128    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae43c95a42df414211fe5d10c0d7df19a9712330518fd60c720b30931eeeae76\": container with ID starting with ae43c95a42df414211fe5d10c0d7df19a9712330518fd60c720b30931eeeae76 not found: ID does not exist" containerID="ae43c95a42df414211fe5d10c0d7df19a9712330518fd60c720b30931eeeae76"
Jul 21 14:30:03.890219 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:03.890157    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae43c95a42df414211fe5d10c0d7df19a9712330518fd60c720b30931eeeae76"} err="failed to get container status \"ae43c95a42df414211fe5d10c0d7df19a9712330518fd60c720b30931eeeae76\": rpc error: code = NotFound desc = could not find container \"ae43c95a42df414211fe5d10c0d7df19a9712330518fd60c720b30931eeeae76\": container with ID starting with ae43c95a42df414211fe5d10c0d7df19a9712330518fd60c720b30931eeeae76 not found: ID does not exist"
Jul 21 14:30:03.897100 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:03.897078    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-67be07-65949f64b8-r2j75"]
Jul 21 14:30:03.901140 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:03.901118    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-67be07-65949f64b8-r2j75"]
Jul 21 14:30:05.503638 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:05.503607    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2513f979-f9ab-4d92-b912-9ab5eeb6dcc9" path="/var/lib/kubelet/pods/2513f979-f9ab-4d92-b912-9ab5eeb6dcc9/volumes"
Jul 21 14:30:06.462516 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:06.462477    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:30:07.745867 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:07.745833    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-67be07-7mcp4"]
Jul 21 14:30:07.748634 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:07.748608    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-67be07-7mcp4"]
Jul 21 14:30:09.502913 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:09.502857    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c97f73f-19e1-44b7-ac96-d8bf3bb5b812" path="/var/lib/kubelet/pods/1c97f73f-19e1-44b7-ac96-d8bf3bb5b812/volumes"
Jul 21 14:30:15.592241 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:15.592203    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-52967c-rwszk"]
Jul 21 14:30:15.592705 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:15.592528    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2bb2843-7373-451a-939c-01de0a0f7d79" containerName="cleanup"
Jul 21 14:30:15.592705 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:15.592540    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2bb2843-7373-451a-939c-01de0a0f7d79" containerName="cleanup"
Jul 21 14:30:15.592705 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:15.592553    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa2b56b6-4527-46e7-8b84-83d599b989e8" containerName="cleanup"
Jul 21 14:30:15.592705 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:15.592558    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa2b56b6-4527-46e7-8b84-83d599b989e8" containerName="cleanup"
Jul 21 14:30:15.592705 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:15.592566    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b0a8575-c8e6-467c-8764-beebea4427b4" containerName="cleanup"
Jul 21 14:30:15.592705 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:15.592572    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b0a8575-c8e6-467c-8764-beebea4427b4" containerName="cleanup"
Jul 21 14:30:15.592705 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:15.592586    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2513f979-f9ab-4d92-b912-9ab5eeb6dcc9" containerName="maas-api"
Jul 21 14:30:15.592705 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:15.592602    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2513f979-f9ab-4d92-b912-9ab5eeb6dcc9" containerName="maas-api"
Jul 21 14:30:15.592705 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:15.592648    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa2b56b6-4527-46e7-8b84-83d599b989e8" containerName="cleanup"
Jul 21 14:30:15.592705 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:15.592656    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2513f979-f9ab-4d92-b912-9ab5eeb6dcc9" containerName="maas-api"
Jul 21 14:30:15.592705 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:15.592663    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2bb2843-7373-451a-939c-01de0a0f7d79" containerName="cleanup"
Jul 21 14:30:15.592705 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:15.592671    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b0a8575-c8e6-467c-8764-beebea4427b4" containerName="cleanup"
Jul 21 14:30:15.596797 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:15.596779    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-52967c-rwszk"
Jul 21 14:30:15.600282 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:15.600260    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5t9ph\""
Jul 21 14:30:15.609082 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:15.609059    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-52967c-rwszk"]
Jul 21 14:30:15.697119 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:15.697084    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b52f081-7be8-4363-a37e-dd14abbbf95e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-52967c-rwszk\" (UID: \"5b52f081-7be8-4363-a37e-dd14abbbf95e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-52967c-rwszk"
Jul 21 14:30:15.798308 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:15.798256    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b52f081-7be8-4363-a37e-dd14abbbf95e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-52967c-rwszk\" (UID: \"5b52f081-7be8-4363-a37e-dd14abbbf95e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-52967c-rwszk"
Jul 21 14:30:15.798923 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:15.798902    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b52f081-7be8-4363-a37e-dd14abbbf95e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-52967c-rwszk\" (UID: \"5b52f081-7be8-4363-a37e-dd14abbbf95e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-52967c-rwszk"
Jul 21 14:30:15.906634 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:15.906548    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-52967c-rwszk"
Jul 21 14:30:16.043740 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:16.043718    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-52967c-rwszk"]
Jul 21 14:30:16.046059 ip-10-0-132-245 kubenswrapper[2578]: W0721 14:30:16.046022    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b52f081_7be8_4363_a37e_dd14abbbf95e.slice/crio-ef3efdff3ce28dcf23e645b6c26696ff5cf042844c63dfbc7575f1075adfa1f1 WatchSource:0}: Error finding container ef3efdff3ce28dcf23e645b6c26696ff5cf042844c63dfbc7575f1075adfa1f1: Status 404 returned error can't find the container with id ef3efdff3ce28dcf23e645b6c26696ff5cf042844c63dfbc7575f1075adfa1f1
Jul 21 14:30:16.943698 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:16.943613    2578 generic.go:356] "Generic (PLEG): container finished" podID="5b52f081-7be8-4363-a37e-dd14abbbf95e" containerID="b30fafb211a1f9292499060fc8b78512fcb4e0b504b96364117616f93725b1dd" exitCode=0
Jul 21 14:30:16.944112 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:16.943696    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-52967c-rwszk" event={"ID":"5b52f081-7be8-4363-a37e-dd14abbbf95e","Type":"ContainerDied","Data":"b30fafb211a1f9292499060fc8b78512fcb4e0b504b96364117616f93725b1dd"}
Jul 21 14:30:16.944112 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:16.943745    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-52967c-rwszk" event={"ID":"5b52f081-7be8-4363-a37e-dd14abbbf95e","Type":"ContainerStarted","Data":"ef3efdff3ce28dcf23e645b6c26696ff5cf042844c63dfbc7575f1075adfa1f1"}
Jul 21 14:30:18.079710 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:18.079685    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-52967c-rwszk"
Jul 21 14:30:18.119010 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:18.118980    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b52f081-7be8-4363-a37e-dd14abbbf95e-maas-api-service-ca\") pod \"5b52f081-7be8-4363-a37e-dd14abbbf95e\" (UID: \"5b52f081-7be8-4363-a37e-dd14abbbf95e\") "
Jul 21 14:30:18.119353 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:18.119329    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b52f081-7be8-4363-a37e-dd14abbbf95e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5b52f081-7be8-4363-a37e-dd14abbbf95e" (UID: "5b52f081-7be8-4363-a37e-dd14abbbf95e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:30:18.220471 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:18.220378    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b52f081-7be8-4363-a37e-dd14abbbf95e-maas-api-service-ca\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:30:18.953433 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:18.953392    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-52967c-rwszk" event={"ID":"5b52f081-7be8-4363-a37e-dd14abbbf95e","Type":"ContainerDied","Data":"ef3efdff3ce28dcf23e645b6c26696ff5cf042844c63dfbc7575f1075adfa1f1"}
Jul 21 14:30:18.953614 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:18.953448    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ef3efdff3ce28dcf23e645b6c26696ff5cf042844c63dfbc7575f1075adfa1f1"
Jul 21 14:30:18.953614 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:18.953410    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-52967c-rwszk"
Jul 21 14:30:26.100267 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:26.100227    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-52967c-99dd4dc86-d6nhj"]
Jul 21 14:30:26.100694 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:26.100454    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52967c-99dd4dc86-d6nhj" podUID="f1f9ab77-cf72-4335-81ed-816a28c7e971" containerName="maas-api" containerID="cri-o://8dc8db2018cd18a990823a8de154183353460e6e0e7bda09d0eed6676ec3a54a" gracePeriod=30
Jul 21 14:30:26.131286 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:26.131253    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-52967c-29744070-5vwsm"]
Jul 21 14:30:26.139183 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:26.139151    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-52967c-29744070-5vwsm"]
Jul 21 14:30:26.349039 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:26.349009    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52967c-99dd4dc86-d6nhj"
Jul 21 14:30:26.494936 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:26.494833    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-466lc\" (UniqueName: \"kubernetes.io/projected/f1f9ab77-cf72-4335-81ed-816a28c7e971-kube-api-access-466lc\") pod \"f1f9ab77-cf72-4335-81ed-816a28c7e971\" (UID: \"f1f9ab77-cf72-4335-81ed-816a28c7e971\") "
Jul 21 14:30:26.494936 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:26.494922    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1f9ab77-cf72-4335-81ed-816a28c7e971-maas-api-tls\") pod \"f1f9ab77-cf72-4335-81ed-816a28c7e971\" (UID: \"f1f9ab77-cf72-4335-81ed-816a28c7e971\") "
Jul 21 14:30:26.495124 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:26.494956    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1f9ab77-cf72-4335-81ed-816a28c7e971-openshift-service-ca\") pod \"f1f9ab77-cf72-4335-81ed-816a28c7e971\" (UID: \"f1f9ab77-cf72-4335-81ed-816a28c7e971\") "
Jul 21 14:30:26.495340 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:26.495304    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1f9ab77-cf72-4335-81ed-816a28c7e971-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f1f9ab77-cf72-4335-81ed-816a28c7e971" (UID: "f1f9ab77-cf72-4335-81ed-816a28c7e971"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:30:26.496971 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:26.496947    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1f9ab77-cf72-4335-81ed-816a28c7e971-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f1f9ab77-cf72-4335-81ed-816a28c7e971" (UID: "f1f9ab77-cf72-4335-81ed-816a28c7e971"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:30:26.497096 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:26.497007    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1f9ab77-cf72-4335-81ed-816a28c7e971-kube-api-access-466lc" (OuterVolumeSpecName: "kube-api-access-466lc") pod "f1f9ab77-cf72-4335-81ed-816a28c7e971" (UID: "f1f9ab77-cf72-4335-81ed-816a28c7e971"). InnerVolumeSpecName "kube-api-access-466lc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:30:26.596026 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:26.595982    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-466lc\" (UniqueName: \"kubernetes.io/projected/f1f9ab77-cf72-4335-81ed-816a28c7e971-kube-api-access-466lc\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:30:26.596026 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:26.596021    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f1f9ab77-cf72-4335-81ed-816a28c7e971-maas-api-tls\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:30:26.596026 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:26.596034    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1f9ab77-cf72-4335-81ed-816a28c7e971-openshift-service-ca\") on node \"ip-10-0-132-245.ec2.internal\" DevicePath \"\""
Jul 21 14:30:26.990632 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:26.990598    2578 generic.go:356] "Generic (PLEG): container finished" podID="f1f9ab77-cf72-4335-81ed-816a28c7e971" containerID="8dc8db2018cd18a990823a8de154183353460e6e0e7bda09d0eed6676ec3a54a" exitCode=0
Jul 21 14:30:26.990845 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:26.990645    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52967c-99dd4dc86-d6nhj"
Jul 21 14:30:26.990845 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:26.990684    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52967c-99dd4dc86-d6nhj" event={"ID":"f1f9ab77-cf72-4335-81ed-816a28c7e971","Type":"ContainerDied","Data":"8dc8db2018cd18a990823a8de154183353460e6e0e7bda09d0eed6676ec3a54a"}
Jul 21 14:30:26.990845 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:26.990742    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52967c-99dd4dc86-d6nhj" event={"ID":"f1f9ab77-cf72-4335-81ed-816a28c7e971","Type":"ContainerDied","Data":"8fc02901126e6b74b5ab163339015ecbe11ce51d20915a87fbdfafbd56be4f26"}
Jul 21 14:30:26.990845 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:26.990770    2578 scope.go:117] "RemoveContainer" containerID="8dc8db2018cd18a990823a8de154183353460e6e0e7bda09d0eed6676ec3a54a"
Jul 21 14:30:26.999731 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:26.999712    2578 scope.go:117] "RemoveContainer" containerID="8dc8db2018cd18a990823a8de154183353460e6e0e7bda09d0eed6676ec3a54a"
Jul 21 14:30:27.000038 ip-10-0-132-245 kubenswrapper[2578]: E0721 14:30:27.000018    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8dc8db2018cd18a990823a8de154183353460e6e0e7bda09d0eed6676ec3a54a\": container with ID starting with 8dc8db2018cd18a990823a8de154183353460e6e0e7bda09d0eed6676ec3a54a not found: ID does not exist" containerID="8dc8db2018cd18a990823a8de154183353460e6e0e7bda09d0eed6676ec3a54a"
Jul 21 14:30:27.000106 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:27.000048    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dc8db2018cd18a990823a8de154183353460e6e0e7bda09d0eed6676ec3a54a"} err="failed to get container status \"8dc8db2018cd18a990823a8de154183353460e6e0e7bda09d0eed6676ec3a54a\": rpc error: code = NotFound desc = could not find container \"8dc8db2018cd18a990823a8de154183353460e6e0e7bda09d0eed6676ec3a54a\": container with ID starting with 8dc8db2018cd18a990823a8de154183353460e6e0e7bda09d0eed6676ec3a54a not found: ID does not exist"
Jul 21 14:30:27.019144 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:27.019119    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-52967c-99dd4dc86-d6nhj"]
Jul 21 14:30:27.039978 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:27.039951    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-52967c-99dd4dc86-d6nhj"]
Jul 21 14:30:27.502924 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:27.502892    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2bb2843-7373-451a-939c-01de0a0f7d79" path="/var/lib/kubelet/pods/a2bb2843-7373-451a-939c-01de0a0f7d79/volumes"
Jul 21 14:30:27.503299 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:27.503239    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1f9ab77-cf72-4335-81ed-816a28c7e971" path="/var/lib/kubelet/pods/f1f9ab77-cf72-4335-81ed-816a28c7e971/volumes"
Jul 21 14:30:29.170973 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:29.170940    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-klnkt"]
Jul 21 14:30:30.670126 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:30.670086    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-52967c-rwszk"]
Jul 21 14:30:30.675083 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:30.675051    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-52967c-rwszk"]
Jul 21 14:30:31.503164 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:31.503131    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b52f081-7be8-4363-a37e-dd14abbbf95e" path="/var/lib/kubelet/pods/5b52f081-7be8-4363-a37e-dd14abbbf95e/volumes"
Jul 21 14:30:43.807743 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:43.807711    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-984j4_9f04a128-24bd-4084-a222-c6716333f9fd/manager/0.log"
Jul 21 14:30:43.919929 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:43.919889    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-xwfks_0ab4a773-bbc8-480e-82bd-188315ea18da/manager/0.log"
Jul 21 14:30:44.162039 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:44.162000    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-9zp5z_60134e12-b9df-4957-a9bd-fdb9224515f1/server/0.log"
Jul 21 14:30:44.274493 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:44.274469    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-n2dg9_cb3de6f1-5afe-4ecd-afe1-927dd01151bd/manager/0.log"
Jul 21 14:30:44.632985 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:44.632958    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d98d8798d-q9bwx_36f99470-322b-4492-91f4-77e433d30910/copy-manifests/0.log"
Jul 21 14:30:44.639074 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:44.639048    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d98d8798d-q9bwx_36f99470-322b-4492-91f4-77e433d30910/manager/0.log"
Jul 21 14:30:45.110376 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:45.110351    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29744040-7j47p_fd0d8136-4bbb-4e33-995a-ba522d813b6b/cleanup/0.log"
Jul 21 14:30:45.217318 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:45.217279    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29744055-lls7w_61cbdead-b67f-4862-b7bc-d0bd9b05fe90/cleanup/0.log"
Jul 21 14:30:45.323505 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:45.323476    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29744070-lffm7_9b0a8575-c8e6-467c-8764-beebea4427b4/cleanup/0.log"
Jul 21 14:30:46.831588 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:46.831560    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-wchpf_f07a2504-208f-43ed-9254-fc2a366ca555/manager/0.log"
Jul 21 14:30:47.409381 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:47.409357    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-klnkt_f622bbe8-bd7a-4d61-8137-c2f66ddcc5b3/limitador/0.log"
Jul 21 14:30:47.976728 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:47.976696    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-d86x5_289a3b8d-9d4a-462a-ab7b-1b38b94fa868/discovery/0.log"
Jul 21 14:30:48.201933 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:48.201905    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-55c6b9f88d-d87sk_07798b24-2c54-4815-bcd4-f8ad1134712c/kube-auth-proxy/0.log"
Jul 21 14:30:49.473771 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:49.473744    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-c2bzr_9c0ea429-ddaf-479e-9122-56272792462c/main/0.log"
Jul 21 14:30:59.066846 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:59.066811    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-cg5rh_f7e1e1d1-eab3-43fb-ad47-cb6b5fd5b913/global-pull-secret-syncer/0.log"
Jul 21 14:30:59.294282 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:59.294256    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-mnstv_b63df369-72fd-4d11-805c-5204b768db57/konnectivity-agent/0.log"
Jul 21 14:30:59.373543 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:30:59.373516    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-245.ec2.internal_de55c8b128a47fa32f279f57a92fe691/haproxy/0.log"
Jul 21 14:31:05.496230 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:05.496203    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-wchpf_f07a2504-208f-43ed-9254-fc2a366ca555/manager/0.log"
Jul 21 14:31:05.868069 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:05.868041    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-klnkt_f622bbe8-bd7a-4d61-8137-c2f66ddcc5b3/limitador/0.log"
Jul 21 14:31:08.228556 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:08.228524    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jhfnl_9ac7fe8d-1185-41d3-b185-bb7da6062f5a/node-exporter/0.log"
Jul 21 14:31:08.250125 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:08.250096    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jhfnl_9ac7fe8d-1185-41d3-b185-bb7da6062f5a/kube-rbac-proxy/0.log"
Jul 21 14:31:08.282613 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:08.282587    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jhfnl_9ac7fe8d-1185-41d3-b185-bb7da6062f5a/init-textfile/0.log"
Jul 21 14:31:10.010564 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:10.010529    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fv4cl/perf-node-gather-daemonset-kdp4z"]
Jul 21 14:31:10.011029 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:10.010813    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1f9ab77-cf72-4335-81ed-816a28c7e971" containerName="maas-api"
Jul 21 14:31:10.011029 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:10.010823    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1f9ab77-cf72-4335-81ed-816a28c7e971" containerName="maas-api"
Jul 21 14:31:10.011029 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:10.010834    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b52f081-7be8-4363-a37e-dd14abbbf95e" containerName="revoke-keys"
Jul 21 14:31:10.011029 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:10.010839    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b52f081-7be8-4363-a37e-dd14abbbf95e" containerName="revoke-keys"
Jul 21 14:31:10.011029 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:10.010912    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b52f081-7be8-4363-a37e-dd14abbbf95e" containerName="revoke-keys"
Jul 21 14:31:10.011029 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:10.010920    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1f9ab77-cf72-4335-81ed-816a28c7e971" containerName="maas-api"
Jul 21 14:31:10.013971 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:10.013949    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-kdp4z"
Jul 21 14:31:10.016766 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:10.016745    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-fv4cl\"/\"default-dockercfg-zm8bc\""
Jul 21 14:31:10.016766 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:10.016758    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fv4cl\"/\"openshift-service-ca.crt\""
Jul 21 14:31:10.017796 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:10.017781    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fv4cl\"/\"kube-root-ca.crt\""
Jul 21 14:31:10.023200 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:10.023180    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fv4cl/perf-node-gather-daemonset-kdp4z"]
Jul 21 14:31:10.049284 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:10.049256    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a13ae4c9-77cc-41c8-b922-ee0c3369194d-lib-modules\") pod \"perf-node-gather-daemonset-kdp4z\" (UID: \"a13ae4c9-77cc-41c8-b922-ee0c3369194d\") " pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-kdp4z"
Jul 21 14:31:10.049434 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:10.049290    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a13ae4c9-77cc-41c8-b922-ee0c3369194d-proc\") pod \"perf-node-gather-daemonset-kdp4z\" (UID: \"a13ae4c9-77cc-41c8-b922-ee0c3369194d\") " pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-kdp4z"
Jul 21 14:31:10.049434 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:10.049321    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a13ae4c9-77cc-41c8-b922-ee0c3369194d-sys\") pod \"perf-node-gather-daemonset-kdp4z\" (UID: \"a13ae4c9-77cc-41c8-b922-ee0c3369194d\") " pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-kdp4z"
Jul 21 14:31:10.049434 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:10.049403    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a13ae4c9-77cc-41c8-b922-ee0c3369194d-podres\") pod \"perf-node-gather-daemonset-kdp4z\" (UID: \"a13ae4c9-77cc-41c8-b922-ee0c3369194d\") " pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-kdp4z"
Jul 21 14:31:10.049601 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:10.049470    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmlvm\" (UniqueName: \"kubernetes.io/projected/a13ae4c9-77cc-41c8-b922-ee0c3369194d-kube-api-access-nmlvm\") pod \"perf-node-gather-daemonset-kdp4z\" (UID: \"a13ae4c9-77cc-41c8-b922-ee0c3369194d\") " pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-kdp4z"
Jul 21 14:31:10.150330 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:10.150288    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a13ae4c9-77cc-41c8-b922-ee0c3369194d-podres\") pod \"perf-node-gather-daemonset-kdp4z\" (UID: \"a13ae4c9-77cc-41c8-b922-ee0c3369194d\") " pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-kdp4z"
Jul 21 14:31:10.150506 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:10.150351    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmlvm\" (UniqueName: \"kubernetes.io/projected/a13ae4c9-77cc-41c8-b922-ee0c3369194d-kube-api-access-nmlvm\") pod \"perf-node-gather-daemonset-kdp4z\" (UID: \"a13ae4c9-77cc-41c8-b922-ee0c3369194d\") " pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-kdp4z"
Jul 21 14:31:10.150506 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:10.150391    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a13ae4c9-77cc-41c8-b922-ee0c3369194d-lib-modules\") pod \"perf-node-gather-daemonset-kdp4z\" (UID: \"a13ae4c9-77cc-41c8-b922-ee0c3369194d\") " pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-kdp4z"
Jul 21 14:31:10.150506 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:10.150409    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a13ae4c9-77cc-41c8-b922-ee0c3369194d-proc\") pod \"perf-node-gather-daemonset-kdp4z\" (UID: \"a13ae4c9-77cc-41c8-b922-ee0c3369194d\") " pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-kdp4z"
Jul 21 14:31:10.150506 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:10.150435    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a13ae4c9-77cc-41c8-b922-ee0c3369194d-sys\") pod \"perf-node-gather-daemonset-kdp4z\" (UID: \"a13ae4c9-77cc-41c8-b922-ee0c3369194d\") " pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-kdp4z"
Jul 21 14:31:10.150506 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:10.150472    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a13ae4c9-77cc-41c8-b922-ee0c3369194d-podres\") pod \"perf-node-gather-daemonset-kdp4z\" (UID: \"a13ae4c9-77cc-41c8-b922-ee0c3369194d\") " pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-kdp4z"
Jul 21 14:31:10.150677 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:10.150514    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a13ae4c9-77cc-41c8-b922-ee0c3369194d-sys\") pod \"perf-node-gather-daemonset-kdp4z\" (UID: \"a13ae4c9-77cc-41c8-b922-ee0c3369194d\") " pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-kdp4z"
Jul 21 14:31:10.150677 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:10.150520    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a13ae4c9-77cc-41c8-b922-ee0c3369194d-proc\") pod \"perf-node-gather-daemonset-kdp4z\" (UID: \"a13ae4c9-77cc-41c8-b922-ee0c3369194d\") " pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-kdp4z"
Jul 21 14:31:10.150677 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:10.150552    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a13ae4c9-77cc-41c8-b922-ee0c3369194d-lib-modules\") pod \"perf-node-gather-daemonset-kdp4z\" (UID: \"a13ae4c9-77cc-41c8-b922-ee0c3369194d\") " pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-kdp4z"
Jul 21 14:31:10.159707 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:10.159679    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmlvm\" (UniqueName: \"kubernetes.io/projected/a13ae4c9-77cc-41c8-b922-ee0c3369194d-kube-api-access-nmlvm\") pod \"perf-node-gather-daemonset-kdp4z\" (UID: \"a13ae4c9-77cc-41c8-b922-ee0c3369194d\") " pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-kdp4z"
Jul 21 14:31:10.323971 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:10.323860    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-kdp4z"
Jul 21 14:31:10.454235 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:10.454207    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fv4cl/perf-node-gather-daemonset-kdp4z"]
Jul 21 14:31:10.456291 ip-10-0-132-245 kubenswrapper[2578]: W0721 14:31:10.456256    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda13ae4c9_77cc_41c8_b922_ee0c3369194d.slice/crio-1c4ce0d4342203a389149027f46dd1625af60ed03bec743681cb79aa2b8d1c54 WatchSource:0}: Error finding container 1c4ce0d4342203a389149027f46dd1625af60ed03bec743681cb79aa2b8d1c54: Status 404 returned error can't find the container with id 1c4ce0d4342203a389149027f46dd1625af60ed03bec743681cb79aa2b8d1c54
Jul 21 14:31:11.174944 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:11.174910    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-kdp4z" event={"ID":"a13ae4c9-77cc-41c8-b922-ee0c3369194d","Type":"ContainerStarted","Data":"60dd43642865b0c4bcd03e5e904b321a7cc595d66391f84c5df7c0bd85764ba3"}
Jul 21 14:31:11.175376 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:11.174960    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-kdp4z" event={"ID":"a13ae4c9-77cc-41c8-b922-ee0c3369194d","Type":"ContainerStarted","Data":"1c4ce0d4342203a389149027f46dd1625af60ed03bec743681cb79aa2b8d1c54"}
Jul 21 14:31:11.175376 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:11.175041    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-kdp4z"
Jul 21 14:31:11.192325 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:11.192276    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-kdp4z" podStartSLOduration=2.192262164 podStartE2EDuration="2.192262164s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 14:31:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:31:11.191245458 +0000 UTC m=+2818.151534015" watchObservedRunningTime="2026-07-21 14:31:11.192262164 +0000 UTC m=+2818.152550720"
Jul 21 14:31:13.471557 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:13.471527    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vvrf8_79957af5-da1d-4787-9965-1b536cb01bca/dns/0.log"
Jul 21 14:31:13.494621 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:13.494593    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vvrf8_79957af5-da1d-4787-9965-1b536cb01bca/kube-rbac-proxy/0.log"
Jul 21 14:31:13.618088 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:13.618057    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hprzk_86868b9f-3bee-44f9-b662-ed05ab2b323a/dns-node-resolver/0.log"
Jul 21 14:31:14.220647 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:14.220616    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5747f97549-lrm92_7b6a97cd-7c40-40f9-8280-18fdd3e5f38c/registry/0.log"
Jul 21 14:31:14.271894 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:14.271856    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-swmgt_30032785-7cc9-421c-b65a-674a329a954f/node-ca/0.log"
Jul 21 14:31:15.430749 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:15.430714    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-d86x5_289a3b8d-9d4a-462a-ab7b-1b38b94fa868/discovery/0.log"
Jul 21 14:31:15.484319 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:15.484291    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-55c6b9f88d-d87sk_07798b24-2c54-4815-bcd4-f8ad1134712c/kube-auth-proxy/0.log"
Jul 21 14:31:16.409426 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:16.409400    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gxpzz_6dfb88f6-dd03-45f2-96cf-3adc22acbd81/serve-healthcheck-canary/0.log"
Jul 21 14:31:17.051448 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:17.051420    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s2n8n_a700e703-22e3-4a59-b70c-c2a489a2a828/kube-rbac-proxy/0.log"
Jul 21 14:31:17.082566 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:17.082535    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s2n8n_a700e703-22e3-4a59-b70c-c2a489a2a828/exporter/0.log"
Jul 21 14:31:17.120229 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:17.120197    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s2n8n_a700e703-22e3-4a59-b70c-c2a489a2a828/extractor/0.log"
Jul 21 14:31:17.189033 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:17.189010    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-fv4cl/perf-node-gather-daemonset-kdp4z"
Jul 21 14:31:19.773027 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:19.772948    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-984j4_9f04a128-24bd-4084-a222-c6716333f9fd/manager/0.log"
Jul 21 14:31:19.958501 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:19.958470    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-xwfks_0ab4a773-bbc8-480e-82bd-188315ea18da/manager/0.log"
Jul 21 14:31:20.093161 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:20.093132    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-9zp5z_60134e12-b9df-4957-a9bd-fdb9224515f1/server/0.log"
Jul 21 14:31:20.171548 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:20.171516    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-n2dg9_cb3de6f1-5afe-4ecd-afe1-927dd01151bd/manager/0.log"
Jul 21 14:31:20.389313 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:20.389224    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d98d8798d-q9bwx_36f99470-322b-4492-91f4-77e433d30910/manager/0.log"
Jul 21 14:31:20.419471 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:20.419449    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d98d8798d-q9bwx_36f99470-322b-4492-91f4-77e433d30910/copy-manifests/0.log"
Jul 21 14:31:21.794045 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:21.794012    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-d86x5_289a3b8d-9d4a-462a-ab7b-1b38b94fa868/discovery/0.log"
Jul 21 14:31:21.820488 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:21.820454    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-55c6b9f88d-d87sk_07798b24-2c54-4815-bcd4-f8ad1134712c/kube-auth-proxy/0.log"
Jul 21 14:31:23.303379 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:23.303346    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-d86x5_289a3b8d-9d4a-462a-ab7b-1b38b94fa868/discovery/0.log"
Jul 21 14:31:23.345324 ip-10-0-132-245 kubenswrapper[2578]: I0721 14:31:23.345290    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-55c6b9f88d-d87sk_07798b24-2c54-4815-bcd4-f8ad1134712c/kube-auth-proxy/0.log"