Jul 21 14:32:21.723734 ip-10-0-130-83 systemd[1]: Starting Kubernetes Kubelet...
Jul 21 14:32:22.149017 ip-10-0-130-83 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 21 14:32:22.149017 ip-10-0-130-83 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 21 14:32:22.149017 ip-10-0-130-83 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 21 14:32:22.149017 ip-10-0-130-83 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 21 14:32:22.149017 ip-10-0-130-83 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 21 14:32:22.149017 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.144403    2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 21 14:32:22.279658 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.153279    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 14:32:22.279658 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167082    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 14:32:22.279658 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167086    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 14:32:22.279658 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167090    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 14:32:22.279658 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167093    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 14:32:22.279658 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167097    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 14:32:22.279658 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167102    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 14:32:22.279658 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167107    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 14:32:22.279658 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167111    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 14:32:22.279658 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167114    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 14:32:22.279658 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167117    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 14:32:22.279658 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167120    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 14:32:22.279658 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167123    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 14:32:22.279658 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167125    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 14:32:22.279658 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167128    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 14:32:22.279658 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167131    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 14:32:22.279658 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167135    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 14:32:22.279658 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167138    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 14:32:22.279658 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167141    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 14:32:22.236147 ip-10-0-130-83 systemd[1]: Started Kubernetes Kubelet.
Jul 21 14:32:22.280816 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167144    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 14:32:22.280816 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167148    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 14:32:22.280816 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167151    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 14:32:22.280816 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167153    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 14:32:22.280816 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167156    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 14:32:22.280816 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167159    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 14:32:22.280816 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167161    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 14:32:22.280816 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167164    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 14:32:22.280816 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167166    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 14:32:22.280816 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167169    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 14:32:22.280816 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167172    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 14:32:22.280816 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167175    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 14:32:22.280816 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167179    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 14:32:22.280816 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167181    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 14:32:22.280816 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167184    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 14:32:22.280816 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167187    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 14:32:22.280816 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167190    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 14:32:22.280816 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167192    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 14:32:22.280816 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167195    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 14:32:22.280816 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167198    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 14:32:22.281973 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167201    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 14:32:22.281973 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167203    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 14:32:22.281973 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167205    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 14:32:22.281973 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167208    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 14:32:22.281973 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167211    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 14:32:22.281973 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167214    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 14:32:22.281973 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167216    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 14:32:22.281973 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167219    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 14:32:22.281973 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167221    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 14:32:22.281973 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167223    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 14:32:22.281973 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167226    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 14:32:22.281973 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167230    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 14:32:22.281973 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167232    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 14:32:22.281973 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167235    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 14:32:22.281973 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167238    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 14:32:22.281973 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167240    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 14:32:22.281973 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167243    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 14:32:22.281973 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167245    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 14:32:22.281973 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167249    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 14:32:22.283026 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167252    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 14:32:22.283026 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167255    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 14:32:22.283026 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167258    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 14:32:22.283026 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167260    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 14:32:22.283026 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167263    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 14:32:22.283026 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167265    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 14:32:22.283026 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167268    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 14:32:22.283026 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167271    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 14:32:22.283026 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167273    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 14:32:22.283026 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167276    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 14:32:22.283026 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167279    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 14:32:22.283026 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167281    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 14:32:22.283026 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167283    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 14:32:22.283026 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167286    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 14:32:22.283026 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167288    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 14:32:22.283026 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167291    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 14:32:22.283026 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167293    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 14:32:22.283026 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167297    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 14:32:22.283026 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167300    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 14:32:22.283026 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167303    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 14:32:22.285257 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167305    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 14:32:22.285257 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167308    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 14:32:22.285257 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167310    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 14:32:22.285257 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167313    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 14:32:22.285257 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167316    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 14:32:22.285257 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167319    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 14:32:22.285257 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167322    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 14:32:22.285257 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167324    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 14:32:22.285257 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167327    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 14:32:22.285257 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167330    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 14:32:22.285257 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167332    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 14:32:22.285257 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167335    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 14:32:22.285257 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167337    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 14:32:22.285257 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167339    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 14:32:22.285257 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167342    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 14:32:22.285257 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167344    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 14:32:22.285257 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167347    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 14:32:22.285257 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167350    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 14:32:22.285257 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167354    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 14:32:22.286071 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167356    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 14:32:22.286071 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.167359    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 14:32:22.286071 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169106    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 14:32:22.286071 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169114    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 14:32:22.286071 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169119    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 14:32:22.286071 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169122    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 14:32:22.286071 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169126    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 14:32:22.286071 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169129    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 14:32:22.286071 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169131    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 14:32:22.286071 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169134    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 14:32:22.286071 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169137    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 14:32:22.286071 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169139    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 14:32:22.286071 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169142    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 14:32:22.286071 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169144    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 14:32:22.286071 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169148    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 14:32:22.286071 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169152    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 14:32:22.286071 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169155    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 14:32:22.286071 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169158    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 14:32:22.286071 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169161    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 14:32:22.286863 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169164    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 14:32:22.286863 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169167    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 14:32:22.286863 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169169    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 14:32:22.286863 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169172    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 14:32:22.286863 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169174    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 14:32:22.286863 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169177    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 14:32:22.286863 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169180    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 14:32:22.286863 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169182    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 14:32:22.286863 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169185    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 14:32:22.286863 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169187    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 14:32:22.286863 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169190    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 14:32:22.286863 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169192    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 14:32:22.286863 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169195    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 14:32:22.286863 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169198    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 14:32:22.286863 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169200    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 14:32:22.286863 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169203    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 14:32:22.286863 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169206    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 14:32:22.286863 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169209    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 14:32:22.286863 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169212    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 14:32:22.286863 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169214    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 14:32:22.287730 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169217    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 14:32:22.287730 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169219    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 14:32:22.287730 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169222    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 14:32:22.287730 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169224    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 14:32:22.287730 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169226    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 14:32:22.287730 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169229    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 14:32:22.287730 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169231    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 14:32:22.287730 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169234    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 14:32:22.287730 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169237    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 14:32:22.287730 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169239    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 14:32:22.287730 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169244    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 14:32:22.287730 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169247    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 14:32:22.287730 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169249    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 14:32:22.287730 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169253    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 14:32:22.287730 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169257    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 14:32:22.287730 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169260    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 14:32:22.287730 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169262    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 14:32:22.287730 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169265    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 14:32:22.287730 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169269    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 14:32:22.288339 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169271    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 14:32:22.288339 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169274    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 14:32:22.288339 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169277    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 14:32:22.288339 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169280    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 14:32:22.288339 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169282    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 14:32:22.288339 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169285    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 14:32:22.288339 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169287    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 14:32:22.288339 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169290    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 14:32:22.288339 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169292    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 14:32:22.288339 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169295    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 14:32:22.288339 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169297    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 14:32:22.288339 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169301    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 14:32:22.288339 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169303    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 14:32:22.288339 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169306    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 14:32:22.288339 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169308    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 14:32:22.288339 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169311    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 14:32:22.288339 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169313    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 14:32:22.288339 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169316    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 14:32:22.288339 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169318    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 14:32:22.288947 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169322    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 14:32:22.288947 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169325    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 14:32:22.288947 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169327    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 14:32:22.288947 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169330    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 14:32:22.288947 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169333    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 14:32:22.288947 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169336    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 14:32:22.288947 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169338    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 14:32:22.288947 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169341    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 14:32:22.288947 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169344    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 14:32:22.288947 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169346    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 14:32:22.288947 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169351    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 14:32:22.288947 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169353    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 14:32:22.288947 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169356    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 14:32:22.288947 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169358    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 14:32:22.288947 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169361    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 14:32:22.288947 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169363    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 14:32:22.288947 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169366    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 14:32:22.288947 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169369    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 14:32:22.288947 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169371    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 14:32:22.288947 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169374    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 14:32:22.289629 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169376    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 14:32:22.289629 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169379    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 14:32:22.289629 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169381    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 14:32:22.289629 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169384    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 14:32:22.289629 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169778    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 14:32:22.289629 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169783    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 14:32:22.289629 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169786    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 14:32:22.289629 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169789    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 14:32:22.289629 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169791    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 14:32:22.289629 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169794    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 14:32:22.289629 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169797    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 14:32:22.289629 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169799    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 14:32:22.289629 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169802    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 14:32:22.289629 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169805    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 14:32:22.289629 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169807    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 14:32:22.289629 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169810    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 14:32:22.289629 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169813    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 14:32:22.289629 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169816    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 14:32:22.289629 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169818    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 14:32:22.289629 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169820    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 14:32:22.290395 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169823    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 14:32:22.290395 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169826    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 14:32:22.290395 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169830    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 14:32:22.290395 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169835    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 14:32:22.290395 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169838    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 14:32:22.290395 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169841    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 14:32:22.290395 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169844    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 14:32:22.290395 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169847    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 14:32:22.290395 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169850    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 14:32:22.290395 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169852    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 14:32:22.290395 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169855    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 14:32:22.290395 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169858    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 14:32:22.290395 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169861    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 14:32:22.290395 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169863    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 14:32:22.290395 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169866    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 14:32:22.290395 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169868    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 14:32:22.290395 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169871    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 14:32:22.290395 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169873    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 14:32:22.290395 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169876    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 14:32:22.291280 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169878    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 14:32:22.291280 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169881    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 14:32:22.291280 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169883    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 14:32:22.291280 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169888    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 14:32:22.291280 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169891    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 14:32:22.291280 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169893    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 14:32:22.291280 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169896    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 14:32:22.291280 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169898    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 14:32:22.291280 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169901    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 14:32:22.291280 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169903    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 14:32:22.291280 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169905    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 14:32:22.291280 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169908    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 14:32:22.291280 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169910    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 14:32:22.291280 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169913    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 14:32:22.291280 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169916    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 14:32:22.291280 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169920    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 14:32:22.291280 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169923    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 14:32:22.291280 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169925    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 14:32:22.291280 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169928    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 14:32:22.292319 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169930    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 14:32:22.292319 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169933    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 14:32:22.292319 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169935    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 14:32:22.292319 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169938    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 14:32:22.292319 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169940    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 14:32:22.292319 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169943    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 14:32:22.292319 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169945    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 14:32:22.292319 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169948    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 14:32:22.292319 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169950    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 14:32:22.292319 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169953    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 14:32:22.292319 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169955    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 14:32:22.292319 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169958    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 14:32:22.292319 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169960    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 14:32:22.292319 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169963    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 14:32:22.292319 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169965    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 14:32:22.292319 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169967    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 14:32:22.292319 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169972    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 14:32:22.292319 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169976    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 14:32:22.292319 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169979    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 14:32:22.292319 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169982    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 14:32:22.293322 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169984    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 14:32:22.293322 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169987    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 14:32:22.293322 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169990    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 14:32:22.293322 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169992    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 14:32:22.293322 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169996    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 14:32:22.293322 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.169999    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 14:32:22.293322 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170001    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 14:32:22.293322 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170003    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 14:32:22.293322 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170008    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 14:32:22.293322 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170011    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 14:32:22.293322 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170013    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 14:32:22.293322 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170016    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 14:32:22.293322 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170020    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 14:32:22.293322 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170023    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 14:32:22.293322 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170025    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 14:32:22.293322 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170028    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 14:32:22.293322 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170031    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 14:32:22.293322 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170033    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 14:32:22.293322 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170036    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 14:32:22.293322 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170038    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 14:32:22.294121 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170041    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 14:32:22.294121 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170043    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 14:32:22.294121 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170046    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 14:32:22.294121 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170048    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 14:32:22.294121 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170051    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 14:32:22.294121 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170117    2579 flags.go:64] FLAG: --address="0.0.0.0"
Jul 21 14:32:22.294121 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170124    2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 21 14:32:22.294121 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170130    2579 flags.go:64] FLAG: --anonymous-auth="true"
Jul 21 14:32:22.294121 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170136    2579 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 21 14:32:22.294121 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170140    2579 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 21 14:32:22.294121 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170143    2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 21 14:32:22.294121 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170147    2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 21 14:32:22.294121 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170151    2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 21 14:32:22.294121 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170154    2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 21 14:32:22.294121 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170158    2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 21 14:32:22.294121 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170161    2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 21 14:32:22.294121 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170164    2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 21 14:32:22.294121 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170167    2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 21 14:32:22.294121 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170170    2579 flags.go:64] FLAG: --cgroup-root=""
Jul 21 14:32:22.294121 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170173    2579 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 21 14:32:22.294121 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170179    2579 flags.go:64] FLAG: --client-ca-file=""
Jul 21 14:32:22.294121 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170183    2579 flags.go:64] FLAG: --cloud-provider="external"
Jul 21 14:32:22.294121 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170185    2579 flags.go:64] FLAG: --cluster-dns="[]"
Jul 21 14:32:22.294855 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170190    2579 flags.go:64] FLAG: --cluster-domain=""
Jul 21 14:32:22.294855 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170193    2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 21 14:32:22.294855 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170196    2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 21 14:32:22.294855 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170199    2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 21 14:32:22.294855 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170202    2579 flags.go:64] FLAG: --container-log-max-files="5"
Jul 21 14:32:22.294855 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170205    2579 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 21 14:32:22.294855 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170208    2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 21 14:32:22.294855 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170211    2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 21 14:32:22.294855 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170215    2579 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 21 14:32:22.294855 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170218    2579 flags.go:64] FLAG: --contention-profiling="false"
Jul 21 14:32:22.294855 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170220    2579 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 21 14:32:22.294855 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170223    2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 21 14:32:22.294855 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170226    2579 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 21 14:32:22.294855 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170229    2579 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 21 14:32:22.294855 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170233    2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 21 14:32:22.294855 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170236    2579 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 21 14:32:22.294855 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170239    2579 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 21 14:32:22.294855 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170243    2579 flags.go:64] FLAG: --enable-load-reader="false"
Jul 21 14:32:22.294855 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170246    2579 flags.go:64] FLAG: --enable-server="true"
Jul 21 14:32:22.294855 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170248    2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 21 14:32:22.294855 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170252    2579 flags.go:64] FLAG: --event-burst="100"
Jul 21 14:32:22.294855 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170255    2579 flags.go:64] FLAG: --event-qps="50"
Jul 21 14:32:22.294855 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170258    2579 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 21 14:32:22.294855 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170261    2579 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 21 14:32:22.295512 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170263    2579 flags.go:64] FLAG: --eviction-hard=""
Jul 21 14:32:22.295512 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170267    2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 21 14:32:22.295512 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170270    2579 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 21 14:32:22.295512 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170273    2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 21 14:32:22.295512 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170276    2579 flags.go:64] FLAG: --eviction-soft=""
Jul 21 14:32:22.295512 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170280    2579 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 21 14:32:22.295512 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170283    2579 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 21 14:32:22.295512 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170286    2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 21 14:32:22.295512 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170289    2579 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 21 14:32:22.295512 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170292    2579 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 21 14:32:22.295512 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170295    2579 flags.go:64] FLAG: --fail-swap-on="true"
Jul 21 14:32:22.295512 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170298    2579 flags.go:64] FLAG: --feature-gates=""
Jul 21 14:32:22.295512 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170301    2579 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 21 14:32:22.295512 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170304    2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 21 14:32:22.295512 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170307    2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 21 14:32:22.295512 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170310    2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 21 14:32:22.295512 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170313    2579 flags.go:64] FLAG: --healthz-port="10248"
Jul 21 14:32:22.295512 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170316    2579 flags.go:64] FLAG: --help="false"
Jul 21 14:32:22.295512 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170319    2579 flags.go:64] FLAG: --hostname-override="ip-10-0-130-83.ec2.internal"
Jul 21 14:32:22.295512 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170322    2579 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 21 14:32:22.295512 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170325    2579 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 21 14:32:22.295512 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170328    2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 21 14:32:22.295512 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170331    2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 21 14:32:22.295512 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170335    2579 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 21 14:32:22.296207 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170338    2579 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 21 14:32:22.296207 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170342    2579 flags.go:64] FLAG: --image-service-endpoint=""
Jul 21 14:32:22.296207 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170345    2579 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 21 14:32:22.296207 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170348    2579 flags.go:64] FLAG: --kube-api-burst="100"
Jul 21 14:32:22.296207 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170351    2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 21 14:32:22.296207 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170354    2579 flags.go:64] FLAG: --kube-api-qps="50"
Jul 21 14:32:22.296207 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170356    2579 flags.go:64] FLAG: --kube-reserved=""
Jul 21 14:32:22.296207 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170359    2579 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 21 14:32:22.296207 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170362    2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 21 14:32:22.296207 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170365    2579 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 21 14:32:22.296207 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170368    2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 21 14:32:22.296207 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170370    2579 flags.go:64] FLAG: --lock-file=""
Jul 21 14:32:22.296207 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170373    2579 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 21 14:32:22.296207 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170377    2579 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 21 14:32:22.296207 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170380    2579 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 21 14:32:22.296207 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170385    2579 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 21 14:32:22.296207 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170388    2579 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 21 14:32:22.296207 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170391    2579 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 21 14:32:22.296207 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170394    2579 flags.go:64] FLAG: --logging-format="text"
Jul 21 14:32:22.296207 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170397    2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 21 14:32:22.296207 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170400    2579 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 21 14:32:22.296207 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170403    2579 flags.go:64] FLAG: --manifest-url=""
Jul 21 14:32:22.296207 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170405    2579 flags.go:64] FLAG: --manifest-url-header=""
Jul 21 14:32:22.296207 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170409    2579 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 21 14:32:22.296207 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170412    2579 flags.go:64] FLAG: --max-open-files="1000000"
Jul 21 14:32:22.297117 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170421    2579 flags.go:64] FLAG: --max-pods="110"
Jul 21 14:32:22.297117 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170424    2579 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 21 14:32:22.297117 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170427    2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 21 14:32:22.297117 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170430    2579 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 21 14:32:22.297117 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170432    2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 21 14:32:22.297117 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170435    2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 21 14:32:22.297117 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170438    2579 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 21 14:32:22.297117 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170457    2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 21 14:32:22.297117 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170467    2579 flags.go:64] FLAG: --node-status-max-images="50"
Jul 21 14:32:22.297117 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170470    2579 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 21 14:32:22.297117 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170473    2579 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 21 14:32:22.297117 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170476    2579 flags.go:64] FLAG: --pod-cidr=""
Jul 21 14:32:22.297117 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170478    2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 21 14:32:22.297117 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170483    2579 flags.go:64] FLAG: --pod-manifest-path=""
Jul 21 14:32:22.297117 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170486    2579 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 21 14:32:22.297117 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170489    2579 flags.go:64] FLAG: --pods-per-core="0"
Jul 21 14:32:22.297117 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170492    2579 flags.go:64] FLAG: --port="10250"
Jul 21 14:32:22.297117 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170495    2579 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 21 14:32:22.297117 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170498    2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-028572b0ff368039f"
Jul 21 14:32:22.297117 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170501    2579 flags.go:64] FLAG: --qos-reserved=""
Jul 21 14:32:22.297117 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170504    2579 flags.go:64] FLAG: --read-only-port="10255"
Jul 21 14:32:22.297117 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170508    2579 flags.go:64] FLAG: --register-node="true"
Jul 21 14:32:22.297117 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170511    2579 flags.go:64] FLAG: --register-with-taints=""
Jul 21 14:32:22.297117 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170515    2579 flags.go:64] FLAG: --registry-burst="10"
Jul 21 14:32:22.297796 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170521    2579 flags.go:64] FLAG: --registry-qps="5"
Jul 21 14:32:22.297796 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170524    2579 flags.go:64] FLAG: --reserved-cpus=""
Jul 21 14:32:22.297796 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170527    2579 flags.go:64] FLAG: --reserved-memory=""
Jul 21 14:32:22.297796 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170530    2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 21 14:32:22.297796 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170533    2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 21 14:32:22.297796 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170537    2579 flags.go:64] FLAG: --rotate-certificates="false"
Jul 21 14:32:22.297796 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170539    2579 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 21 14:32:22.297796 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170542    2579 flags.go:64] FLAG: --runonce="false"
Jul 21 14:32:22.297796 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170545    2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 21 14:32:22.297796 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170548    2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 21 14:32:22.297796 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170551    2579 flags.go:64] FLAG: --seccomp-default="false"
Jul 21 14:32:22.297796 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170554    2579 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 21 14:32:22.297796 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170557    2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 21 14:32:22.297796 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170560    2579 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 21 14:32:22.297796 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170563    2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 21 14:32:22.297796 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170565    2579 flags.go:64] FLAG: --storage-driver-password="root"
Jul 21 14:32:22.297796 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170569    2579 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 21 14:32:22.297796 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170572    2579 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 21 14:32:22.297796 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170575    2579 flags.go:64] FLAG: --storage-driver-user="root"
Jul 21 14:32:22.297796 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170578    2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 21 14:32:22.297796 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170581    2579 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 21 14:32:22.297796 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170584    2579 flags.go:64] FLAG: --system-cgroups=""
Jul 21 14:32:22.297796 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170586    2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 21 14:32:22.297796 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170592    2579 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 21 14:32:22.297796 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170595    2579 flags.go:64] FLAG: --tls-cert-file=""
Jul 21 14:32:22.298559 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170598    2579 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 21 14:32:22.298559 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170610    2579 flags.go:64] FLAG: --tls-min-version=""
Jul 21 14:32:22.298559 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170612    2579 flags.go:64] FLAG: --tls-private-key-file=""
Jul 21 14:32:22.298559 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170615    2579 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 21 14:32:22.298559 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170619    2579 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 21 14:32:22.298559 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170622    2579 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 21 14:32:22.298559 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170624    2579 flags.go:64] FLAG: --v="2"
Jul 21 14:32:22.298559 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170632    2579 flags.go:64] FLAG: --version="false"
Jul 21 14:32:22.298559 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170637    2579 flags.go:64] FLAG: --vmodule=""
Jul 21 14:32:22.298559 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170641    2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 21 14:32:22.298559 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.170644    2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 21 14:32:22.298559 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170754    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 14:32:22.298559 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170758    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 14:32:22.298559 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170761    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 14:32:22.298559 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170764    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 14:32:22.298559 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170766    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 14:32:22.298559 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170769    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 14:32:22.298559 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170780    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 14:32:22.298559 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170783    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 14:32:22.298559 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170786    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 14:32:22.298559 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170789    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 14:32:22.298559 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170791    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 14:32:22.298559 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170794    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 14:32:22.302408 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170798    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 14:32:22.302408 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170800    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 14:32:22.302408 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170803    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 14:32:22.302408 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170805    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 14:32:22.302408 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170808    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 14:32:22.302408 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170811    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 14:32:22.302408 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170813    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 14:32:22.302408 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170816    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 14:32:22.302408 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170818    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 14:32:22.302408 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170821    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 14:32:22.302408 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170823    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 14:32:22.302408 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170826    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 14:32:22.302408 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170828    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 14:32:22.302408 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170832    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 14:32:22.302408 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170837    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 14:32:22.302408 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170840    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 14:32:22.302408 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170843    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 14:32:22.302408 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170845    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 14:32:22.302408 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170849    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 14:32:22.303214 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170851    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 14:32:22.303214 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170854    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 14:32:22.303214 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170857    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 14:32:22.303214 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170859    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 14:32:22.303214 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170862    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 14:32:22.303214 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170864    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 14:32:22.303214 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170867    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 14:32:22.303214 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170869    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 14:32:22.303214 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170871    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 14:32:22.303214 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170874    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 14:32:22.303214 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170877    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 14:32:22.303214 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170879    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 14:32:22.303214 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170882    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 14:32:22.303214 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170885    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 14:32:22.303214 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170888    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 14:32:22.303214 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170890    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 14:32:22.303214 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170893    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 14:32:22.303214 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170895    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 14:32:22.303214 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170899    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 14:32:22.303214 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170902    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 14:32:22.303763 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170905    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 14:32:22.303763 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170907    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 14:32:22.303763 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170910    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 14:32:22.303763 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170912    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 14:32:22.303763 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170915    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 14:32:22.303763 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170917    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 14:32:22.303763 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170921    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 14:32:22.303763 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170924    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 14:32:22.303763 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170926    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 14:32:22.303763 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170929    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 14:32:22.303763 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170931    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 14:32:22.303763 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170934    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 14:32:22.303763 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170937    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 14:32:22.303763 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170939    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 14:32:22.303763 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170942    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 14:32:22.303763 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170945    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 14:32:22.303763 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170947    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 14:32:22.303763 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170950    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 14:32:22.303763 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170952    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 14:32:22.303763 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170955    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 14:32:22.511913 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170957    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 14:32:22.511913 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170960    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 14:32:22.511913 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170962    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 14:32:22.511913 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170965    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 14:32:22.511913 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170967    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 14:32:22.511913 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170971    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 14:32:22.511913 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170974    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 14:32:22.511913 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170976    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 14:32:22.511913 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170979    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 14:32:22.511913 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170981    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 14:32:22.511913 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170984    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 14:32:22.511913 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170986    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 14:32:22.511913 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170989    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 14:32:22.511913 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170991    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 14:32:22.511913 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170994    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 14:32:22.511913 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170996    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 14:32:22.511913 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.170999    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 14:32:22.511913 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.171001    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 14:32:22.511913 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.171005    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 14:32:22.511913 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.171008    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 14:32:22.527551 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.171010    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 14:32:22.527551 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.171013    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 14:32:22.527551 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.171015    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 14:32:22.527551 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.171019    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 14:32:22.527551 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.171023    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 14:32:22.527551 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.171027    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 14:32:22.527551 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.171029    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 14:32:22.527551 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.171032    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 14:32:22.527551 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.171782    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 21 14:32:22.527551 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.178459    2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 21 14:32:22.527551 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.178475    2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 21 14:32:22.527551 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.179226    2579 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 21 14:32:22.527551 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.179235    2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 21 14:32:22.527551 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179310    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 14:32:22.528103 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179317    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 14:32:22.528103 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179321    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 14:32:22.528103 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179326    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 14:32:22.528103 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179330    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 14:32:22.528103 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179335    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 14:32:22.528103 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179338    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 14:32:22.528103 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179343    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 14:32:22.528103 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179347    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 14:32:22.528103 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179351    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 14:32:22.528103 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179355    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 14:32:22.528103 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179359    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 14:32:22.528103 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179362    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 14:32:22.528103 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179366    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 14:32:22.528103 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179370    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 14:32:22.528103 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179374    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 14:32:22.528103 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179377    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 14:32:22.528103 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179381    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 14:32:22.528103 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179385    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 14:32:22.528103 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179388    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 14:32:22.528103 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179392    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 14:32:22.528742 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179396    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 14:32:22.528742 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179401    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 14:32:22.528742 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179405    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 14:32:22.528742 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179409    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 14:32:22.528742 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179413    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 14:32:22.528742 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179416    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 14:32:22.528742 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179420    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 14:32:22.528742 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179424    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 14:32:22.528742 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179429    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 14:32:22.528742 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179433    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 14:32:22.528742 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179437    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 14:32:22.528742 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179456    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 14:32:22.528742 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179461    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 14:32:22.528742 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179465    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 14:32:22.528742 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179469    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 14:32:22.528742 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179472    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 14:32:22.528742 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179476    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 14:32:22.528742 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179480    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 14:32:22.528742 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179484    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 14:32:22.528742 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179487    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 14:32:22.529366 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179491    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 14:32:22.529366 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179495    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 14:32:22.529366 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179498    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 14:32:22.529366 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179502    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 14:32:22.529366 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179506    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 14:32:22.529366 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179510    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 14:32:22.529366 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179517    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 14:32:22.529366 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179521    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 14:32:22.529366 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179524    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 14:32:22.529366 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179528    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 14:32:22.529366 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179534    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 14:32:22.529366 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179539    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 14:32:22.529366 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179545    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 14:32:22.529366 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179553    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 14:32:22.529366 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179559    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 14:32:22.529366 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179564    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 14:32:22.529366 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179568    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 14:32:22.529366 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179571    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 14:32:22.530344 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179575    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 14:32:22.530344 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179579    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 14:32:22.530344 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179583    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 14:32:22.530344 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179587    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 14:32:22.530344 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179591    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 14:32:22.530344 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179595    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 14:32:22.530344 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179599    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 14:32:22.530344 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179603    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 14:32:22.530344 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179607    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 14:32:22.530344 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179610    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 14:32:22.530344 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179614    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 14:32:22.530344 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179618    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 14:32:22.530344 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179621    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 14:32:22.530344 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179625    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 14:32:22.530344 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179629    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 14:32:22.530344 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179632    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 14:32:22.530344 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179636    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 14:32:22.530344 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179640    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 14:32:22.530344 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179643    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 14:32:22.530344 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179647    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 14:32:22.530968 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179652    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 14:32:22.530968 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179656    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 14:32:22.530968 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179659    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 14:32:22.530968 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179663    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 14:32:22.530968 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179666    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 14:32:22.530968 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179670    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 14:32:22.530968 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179673    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 14:32:22.530968 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179679    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 14:32:22.530968 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179682    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 14:32:22.530968 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179687    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 14:32:22.530968 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179690    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 14:32:22.530968 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179694    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 14:32:22.530968 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179698    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 14:32:22.530968 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179702    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 14:32:22.530968 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179706    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 14:32:22.530968 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179710    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 14:32:22.530968 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179715    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 14:32:22.530968 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179719    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 14:32:22.530968 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179724    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 14:32:22.531542 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179728    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 14:32:22.531542 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.179734    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 21 14:32:22.531542 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179889    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 14:32:22.531542 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179895    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 14:32:22.531542 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179899    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 14:32:22.531542 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179903    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 14:32:22.531542 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179907    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 14:32:22.531542 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179911    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 14:32:22.531542 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179914    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 14:32:22.531542 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179918    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 14:32:22.531542 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179922    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 14:32:22.531542 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179925    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 14:32:22.531542 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179930    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 14:32:22.531542 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179936    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 14:32:22.532036 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179941    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 14:32:22.532036 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179946    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 14:32:22.532036 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179949    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 14:32:22.532036 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179953    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 14:32:22.532036 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179957    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 14:32:22.532036 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179961    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 14:32:22.532036 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179964    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 14:32:22.532036 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179969    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 14:32:22.532036 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179973    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 14:32:22.532036 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179976    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 14:32:22.532036 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179980    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 14:32:22.532036 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179984    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 14:32:22.532036 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179987    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 14:32:22.532036 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179991    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 14:32:22.532036 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179995    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 14:32:22.532036 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.179998    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 14:32:22.532036 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180003    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 14:32:22.532036 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180006    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 14:32:22.532036 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180010    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 14:32:22.532036 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180014    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 14:32:22.532626 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180017    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 14:32:22.532626 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180021    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 14:32:22.532626 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180025    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 14:32:22.532626 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180029    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 14:32:22.532626 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180032    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 14:32:22.532626 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180036    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 14:32:22.532626 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180039    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 14:32:22.532626 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180043    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 14:32:22.532626 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180047    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 14:32:22.532626 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180050    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 14:32:22.532626 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180054    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 14:32:22.532626 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180059    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 14:32:22.532626 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180062    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 14:32:22.532626 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180066    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 14:32:22.532626 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180070    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 14:32:22.532626 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180073    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 14:32:22.532626 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180077    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 14:32:22.532626 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180081    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 14:32:22.532626 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180084    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 14:32:22.532626 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180089    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 14:32:22.533194 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180093    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 14:32:22.533194 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180097    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 14:32:22.533194 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180100    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 14:32:22.533194 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180104    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 14:32:22.533194 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180108    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 14:32:22.533194 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180112    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 14:32:22.533194 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180117    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 14:32:22.533194 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180123    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 14:32:22.533194 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180127    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 14:32:22.533194 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180132    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 14:32:22.533194 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180136    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 14:32:22.533194 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180141    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 14:32:22.533194 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180145    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 14:32:22.533194 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180148    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 14:32:22.533194 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180152    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 14:32:22.533194 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180156    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 14:32:22.533194 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180160    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 14:32:22.533194 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180164    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 14:32:22.533194 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180168    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 14:32:24.534184 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180171    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 14:32:24.534184 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180175    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 14:32:24.534184 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180179    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 14:32:24.534184 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180183    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 14:32:24.534184 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180188    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 14:32:24.534184 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180191    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 14:32:24.534184 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180197    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 14:32:24.534184 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180202    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 14:32:24.534184 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180206    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 14:32:24.534184 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180211    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 14:32:24.534184 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180214    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 14:32:24.534184 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180218    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 14:32:24.534184 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180224    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 14:32:24.534184 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180227    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 14:32:24.534184 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180231    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 14:32:24.534184 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180235    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 14:32:24.534184 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180238    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 14:32:24.534184 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180242    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 14:32:24.534184 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180246    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 14:32:24.535175 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180250    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 14:32:24.535175 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180254    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 14:32:24.535175 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180257    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 14:32:24.535175 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180261    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 14:32:24.535175 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180265    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 14:32:24.535175 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180269    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 14:32:24.535175 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180273    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 14:32:24.535175 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180277    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 14:32:24.535175 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:22.180281    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 14:32:24.535175 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.180288    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 21 14:32:24.535175 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.180407    2579 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 21 14:32:24.535175 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.182801    2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 21 14:32:24.535175 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.184193    2579 server.go:1019] "Starting client certificate rotation"
Jul 21 14:32:24.535175 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.184298    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 21 14:32:24.535795 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.184334    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 21 14:32:24.535795 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.187119    2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 21 14:32:24.535795 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.192005    2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 21 14:32:24.535795 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.204791    2579 log.go:25] "Validated CRI v1 runtime API"
Jul 21 14:32:24.535795 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.207218    2579 log.go:25] "Validated CRI v1 image API"
Jul 21 14:32:24.535795 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.208064    2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 21 14:32:24.535795 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.210383    2579 fs.go:135] Filesystem UUIDs: map[298c104d-adad-4a75-a038-f1c63d2f70f5:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 bebdce1b-36e9-4f4d-96d9-2748d089cf4d:/dev/nvme0n1p4]
Jul 21 14:32:24.535795 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.210410    2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 21 14:32:24.536281 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.215057    2579 manager.go:216] Machine: {Timestamp:2026-07-21 14:32:22.213998374 +0000 UTC m=+0.361568715 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100480 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2304d32658b6178b526badc10d5e8e SystemUUID:ec2304d3-2658-b617-8b52-6badc10d5e8e BootID:9e534f60-df6d-4493-af1f-57d11396b7cc Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:16:1e:cc:ad:53 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:16:1e:cc:ad:53 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ee:0d:07:7b:ed:8f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 21 14:32:24.536281 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.215158    2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 21 14:32:24.536281 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.215231    2579 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 21 14:32:24.536281 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.216154    2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 21 14:32:24.536281 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.216178    2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-83.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 21 14:32:24.536281 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.216597    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 21 14:32:24.536281 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.216878    2579 topology_manager.go:138] "Creating topology manager with none policy"
Jul 21 14:32:24.536281 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.216891    2579 container_manager_linux.go:309] "Creating device plugin manager"
Jul 21 14:32:24.536281 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.216898    2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 21 14:32:24.536281 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.217502    2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 21 14:32:24.536281 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.218665    2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 21 14:32:24.536281 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.221712    2579 state_mem.go:36] "Initialized new in-memory state store"
Jul 21 14:32:24.536281 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.221825    2579 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 21 14:32:24.536281 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.221852    2579 kubelet.go:486] "Attempting to sync node with API server"
Jul 21 14:32:24.536281 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.221862    2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 21 14:32:24.536281 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.221877    2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 21 14:32:24.536281 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.221885    2579 kubelet.go:398] "Adding apiserver pod source"
Jul 21 14:32:24.536829 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.221893    2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 21 14:32:24.536829 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.222266    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 21 14:32:24.536829 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.222284    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 21 14:32:24.536829 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.224938    2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 21 14:32:24.536829 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.226788    2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 21 14:32:24.536829 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.226820    2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 21 14:32:24.536829 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.227802    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 21 14:32:24.536829 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.227814    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 21 14:32:24.536829 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.227820    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 21 14:32:24.536829 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.227826    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 21 14:32:24.536829 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.227838    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 21 14:32:24.536829 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.227847    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 21 14:32:24.536829 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.227859    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 21 14:32:24.536829 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.227867    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 21 14:32:24.536829 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.227884    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 21 14:32:24.536829 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.227892    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 21 14:32:24.536829 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.227904    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 21 14:32:24.536829 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.227913    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 21 14:32:24.536829 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.228456    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 21 14:32:24.536829 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.228466    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 21 14:32:24.536829 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.231674    2579 server.go:1268] "Started kubelet"
Jul 21 14:32:24.536829 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.235395    2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 21 14:32:24.536829 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.235464    2579 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 21 14:32:24.536829 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.235462    2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 21 14:32:24.536829 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.236963    2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-83.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 21 14:32:24.536829 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:22.236985    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 21 14:32:24.536829 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.237198    2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 21 14:32:24.536829 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:22.237197    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-83.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 21 14:32:24.536829 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.238382    2579 server.go:310] "Adding debug handlers to kubelet server"
Jul 21 14:32:24.536829 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.239613    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6qhv2"
Jul 21 14:32:24.536829 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:22.241583    2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 21 14:32:24.536829 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:22.241787    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-83.ec2.internal.18c45492e520c92b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-83.ec2.internal,UID:ip-10-0-130-83.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-83.ec2.internal,},FirstTimestamp:2026-07-21 14:32:22.231648555 +0000 UTC m=+0.379218954,LastTimestamp:2026-07-21 14:32:22.231648555 +0000 UTC m=+0.379218954,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-83.ec2.internal,}"
Jul 21 14:32:24.536829 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.243691    2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 21 14:32:24.536829 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.243700    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 21 14:32:24.536829 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:22.243779    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-83.ec2.internal\" not found"
Jul 21 14:32:24.536829 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.244324    2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 21 14:32:24.536829 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.244602    2579 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 21 14:32:24.536829 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.244614    2579 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 21 14:32:24.536829 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.244730    2579 reconstruct.go:97] "Volume reconstruction finished"
Jul 21 14:32:24.536829 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.244738    2579 reconciler.go:29] "Reconciler: start to sync state"
Jul 21 14:32:24.537911 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.245372    2579 factory.go:153] Registering CRI-O factory
Jul 21 14:32:24.537911 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.245386    2579 factory.go:223] Registration of the crio container factory successfully
Jul 21 14:32:24.537911 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.245464    2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 21 14:32:24.537911 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.245473    2579 factory.go:55] Registering systemd factory
Jul 21 14:32:24.537911 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.245480    2579 factory.go:223] Registration of the systemd container factory successfully
Jul 21 14:32:24.537911 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.245498    2579 factory.go:103] Registering Raw factory
Jul 21 14:32:24.537911 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.245510    2579 manager.go:1196] Started watching for new ooms in manager
Jul 21 14:32:24.537911 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.246380    2579 manager.go:349] Starting recovery of all containers
Jul 21 14:32:24.537911 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:22.247872    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 21 14:32:24.537911 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:22.247909    2579 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-83.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 21 14:32:24.537911 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.255777    2579 manager.go:354] Recovery completed
Jul 21 14:32:24.537911 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.260565    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 14:32:24.537911 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.262241    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-83.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 14:32:24.537911 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.262263    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-83.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 14:32:24.537911 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.262276    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-83.ec2.internal" event="NodeHasSufficientPID"
Jul 21 14:32:24.537911 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.263246    2579 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 21 14:32:24.537911 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.263254    2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 21 14:32:24.537911 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.263271    2579 state_mem.go:36] "Initialized new in-memory state store"
Jul 21 14:32:24.537911 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.264338    2579 policy_none.go:49] "None policy: Start"
Jul 21 14:32:24.537911 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.264355    2579 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 21 14:32:24.537911 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.264368    2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 21 14:32:24.537911 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.265377    2579 policy_none.go:47] "Start"
Jul 21 14:32:24.537911 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.266090    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 14:32:24.537911 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:22.266368    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-83.ec2.internal.18c45492e6f3c761  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-83.ec2.internal,UID:ip-10-0-130-83.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-83.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-83.ec2.internal,},FirstTimestamp:2026-07-21 14:32:22.262253409 +0000 UTC m=+0.409823751,LastTimestamp:2026-07-21 14:32:22.262253409 +0000 UTC m=+0.409823751,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-83.ec2.internal,}"
Jul 21 14:32:24.537911 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.266740    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-83.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 14:32:24.537911 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.266776    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-83.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 14:32:24.537911 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.266789    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-83.ec2.internal" event="NodeHasSufficientPID"
Jul 21 14:32:24.537911 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.268046    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6qhv2"
Jul 21 14:32:24.537911 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.308037    2579 manager.go:337] "Starting Device Plugin manager"
Jul 21 14:32:24.537911 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:22.308175    2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 21 14:32:24.537911 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.308188    2579 server.go:87] "Starting device plugin registration server"
Jul 21 14:32:24.537911 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.308425    2579 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 21 14:32:24.537911 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.308435    2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 21 14:32:24.537911 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.308546    2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 21 14:32:24.537911 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.308615    2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 21 14:32:24.537911 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.308650    2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 21 14:32:24.537911 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:22.308963    2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 21 14:32:24.537911 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:22.308997    2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-83.ec2.internal\" not found"
Jul 21 14:32:24.537911 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.365547    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 21 14:32:24.537911 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.367791    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 21 14:32:24.537911 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.367823    2579 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 21 14:32:24.539041 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.367841    2579 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 21 14:32:24.539041 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:22.367870    2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 21 14:32:24.539041 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.370358    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 14:32:24.539041 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.409326    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 14:32:24.539041 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.409985    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-83.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 14:32:24.539041 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.410006    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-83.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 14:32:24.539041 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.410016    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-83.ec2.internal" event="NodeHasSufficientPID"
Jul 21 14:32:24.539041 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.410035    2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-83.ec2.internal"
Jul 21 14:32:24.539041 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.417963    2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-83.ec2.internal"
Jul 21 14:32:24.539041 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:22.417978    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-83.ec2.internal\": node \"ip-10-0-130-83.ec2.internal\" not found"
Jul 21 14:32:24.539041 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:22.433050    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-83.ec2.internal\" not found"
Jul 21 14:32:24.539041 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.468147    2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-83.ec2.internal"]
Jul 21 14:32:24.539041 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.468192    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 14:32:24.539041 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.469516    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-83.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 14:32:24.539041 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.469537    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-83.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 14:32:24.539041 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.469548    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-83.ec2.internal" event="NodeHasSufficientPID"
Jul 21 14:32:24.539041 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.470702    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 14:32:24.539041 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.470814    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.ec2.internal"
Jul 21 14:32:24.539041 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.470837    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 14:32:24.539041 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.471300    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-83.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 14:32:24.539041 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.471323    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-83.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 14:32:24.539041 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.471329    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-83.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 14:32:24.539041 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.471335    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-83.ec2.internal" event="NodeHasSufficientPID"
Jul 21 14:32:24.539041 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.471352    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-83.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 14:32:24.539041 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.471362    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-83.ec2.internal" event="NodeHasSufficientPID"
Jul 21 14:32:24.539041 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.472491    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-83.ec2.internal"
Jul 21 14:32:24.539041 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.472520    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 14:32:24.539041 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.473148    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-83.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 14:32:24.539041 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.473169    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-83.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 14:32:24.539041 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.473183    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-83.ec2.internal" event="NodeHasSufficientPID"
Jul 21 14:32:24.539041 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:22.505996    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-83.ec2.internal\" not found" node="ip-10-0-130-83.ec2.internal"
Jul 21 14:32:24.539041 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:22.510230    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-83.ec2.internal\" not found" node="ip-10-0-130-83.ec2.internal"
Jul 21 14:32:24.539041 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:22.533782    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-83.ec2.internal\" not found"
Jul 21 14:32:24.539041 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.547315    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d46085a6c06a58d348208916c9b4bb78-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-83.ec2.internal\" (UID: \"d46085a6c06a58d348208916c9b4bb78\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.ec2.internal"
Jul 21 14:32:24.539041 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.547334    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d46085a6c06a58d348208916c9b4bb78-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-83.ec2.internal\" (UID: \"d46085a6c06a58d348208916c9b4bb78\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.ec2.internal"
Jul 21 14:32:24.539985 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.547350    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ada9959260df8e5e8a349b59b83b3eed-config\") pod \"kube-apiserver-proxy-ip-10-0-130-83.ec2.internal\" (UID: \"ada9959260df8e5e8a349b59b83b3eed\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-83.ec2.internal"
Jul 21 14:32:24.539985 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:22.634405    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-83.ec2.internal\" not found"
Jul 21 14:32:24.539985 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.647745    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d46085a6c06a58d348208916c9b4bb78-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-83.ec2.internal\" (UID: \"d46085a6c06a58d348208916c9b4bb78\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.ec2.internal"
Jul 21 14:32:24.539985 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.647773    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d46085a6c06a58d348208916c9b4bb78-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-83.ec2.internal\" (UID: \"d46085a6c06a58d348208916c9b4bb78\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.ec2.internal"
Jul 21 14:32:24.539985 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.647795    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ada9959260df8e5e8a349b59b83b3eed-config\") pod \"kube-apiserver-proxy-ip-10-0-130-83.ec2.internal\" (UID: \"ada9959260df8e5e8a349b59b83b3eed\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-83.ec2.internal"
Jul 21 14:32:24.539985 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.647849    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d46085a6c06a58d348208916c9b4bb78-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-83.ec2.internal\" (UID: \"d46085a6c06a58d348208916c9b4bb78\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.ec2.internal"
Jul 21 14:32:24.539985 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.647854    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ada9959260df8e5e8a349b59b83b3eed-config\") pod \"kube-apiserver-proxy-ip-10-0-130-83.ec2.internal\" (UID: \"ada9959260df8e5e8a349b59b83b3eed\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-83.ec2.internal"
Jul 21 14:32:24.539985 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:22.647895    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d46085a6c06a58d348208916c9b4bb78-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-83.ec2.internal\" (UID: \"d46085a6c06a58d348208916c9b4bb78\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.ec2.internal"
Jul 21 14:32:24.539985 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:24.532999    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-83.ec2.internal\" not found"
Jul 21 14:32:24.692899 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:24.606479    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.ec2.internal"
Jul 21 14:32:24.692899 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:24.611189    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-83.ec2.internal"
Jul 21 14:32:24.692899 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:24.633809    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-83.ec2.internal\" not found"
Jul 21 14:32:24.886085 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:24.734402    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-83.ec2.internal\" not found"
Jul 21 14:32:24.886085 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:24.834847    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-83.ec2.internal\" not found"
Jul 21 14:32:25.063792 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:24.935464    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-83.ec2.internal\" not found"
Jul 21 14:32:25.063792 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:24.981880    2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 21 14:32:25.063792 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:24.981992    2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 21 14:32:25.063792 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:25.035509    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-83.ec2.internal\" not found"
Jul 21 14:32:25.225007 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:25.069198    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-20 14:27:24 +0000 UTC" deadline="2028-04-12 20:30:58.91804765 +0000 UTC"
Jul 21 14:32:25.225007 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:25.069218    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15149h58m33.848832771s"
Jul 21 14:32:25.225007 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:25.136489    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-83.ec2.internal\" not found"
Jul 21 14:32:25.352890 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:25.236710    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-83.ec2.internal\" not found"
Jul 21 14:32:25.352890 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:25.242162    2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 14:32:25.352890 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:25.337344    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-83.ec2.internal\" not found"
Jul 21 14:32:25.466038 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:25.437868    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-83.ec2.internal\" not found"
Jul 21 14:32:25.601905 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:25.474033    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podada9959260df8e5e8a349b59b83b3eed.slice/crio-db8bc3d864bd0bd6b5bda65342d08b5016c315c9e65b3db6e0afe8f9298e6593 WatchSource:0}: Error finding container db8bc3d864bd0bd6b5bda65342d08b5016c315c9e65b3db6e0afe8f9298e6593: Status 404 returned error can't find the container with id db8bc3d864bd0bd6b5bda65342d08b5016c315c9e65b3db6e0afe8f9298e6593
Jul 21 14:32:25.601905 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:25.478338    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 14:32:25.601905 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:25.499714    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd46085a6c06a58d348208916c9b4bb78.slice/crio-0a01d7c4495a026fe5d4695e31df5c673b9e1f758f1abcf197d419b828c67ed0 WatchSource:0}: Error finding container 0a01d7c4495a026fe5d4695e31df5c673b9e1f758f1abcf197d419b828c67ed0: Status 404 returned error can't find the container with id 0a01d7c4495a026fe5d4695e31df5c673b9e1f758f1abcf197d419b828c67ed0
Jul 21 14:32:25.601905 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:25.506882    2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 14:32:25.601905 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:25.541687    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.ec2.internal"
Jul 21 14:32:25.601905 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:25.552898    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 21 14:32:25.601905 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:25.553920    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-83.ec2.internal"
Jul 21 14:32:25.601905 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:25.558772    2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 14:32:25.601905 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:25.560195    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 21 14:32:26.020351 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.020329    2579 apiserver.go:52] "Watching apiserver"
Jul 21 14:32:26.029629 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.029605    2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 21 14:32:26.029936 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.029915    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-vhndm","openshift-cluster-node-tuning-operator/tuned-k4mlq","openshift-dns/node-resolver-gbn95","openshift-image-registry/node-ca-j25df","openshift-multus/multus-additional-cni-plugins-gxsg8","openshift-multus/network-metrics-daemon-2zm5j","openshift-network-diagnostics/network-check-target-q7h42","kube-system/kube-apiserver-proxy-ip-10-0-130-83.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44xjw","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.ec2.internal","openshift-multus/multus-d2vn8","openshift-network-operator/iptables-alerter-8ff6d","openshift-ovn-kubernetes/ovnkube-node-mv82n","kube-system/global-pull-secret-syncer-78qqf"]
Jul 21 14:32:26.032215 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.032201    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q7h42"
Jul 21 14:32:26.032291 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:26.032259    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q7h42" podUID="206dbc0b-6708-45a2-bcd9-50b72fd5e6b5"
Jul 21 14:32:26.034505 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.034490    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.036328 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.036296    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gbn95"
Jul 21 14:32:26.037670 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.037645    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 21 14:32:26.037670 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.037662    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 21 14:32:26.037821 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.037668    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 21 14:32:26.037821 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.037650    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 21 14:32:26.037821 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.037692    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 21 14:32:26.037821 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.037645    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jvq4p\""
Jul 21 14:32:26.037821 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.037654    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 21 14:32:26.038160 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.038144    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-j25df"
Jul 21 14:32:26.038549 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.038533    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-dgwn4\""
Jul 21 14:32:26.038634 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.038626    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 21 14:32:26.038779 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.038765    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 21 14:32:26.040131 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.040109    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gxsg8"
Jul 21 14:32:26.040925 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.040908    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 21 14:32:26.040925 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.040929    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 21 14:32:26.041078 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.040940    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 21 14:32:26.041078 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.040968    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-87lhf\""
Jul 21 14:32:26.041915 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.041902    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 21 14:32:26.042174 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.042153    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zm5j"
Jul 21 14:32:26.042255 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:26.042218    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zm5j" podUID="13a6c11d-b9ba-4c12-836f-2172b4d168ce"
Jul 21 14:32:26.042335 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.042296    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 21 14:32:26.042388 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.042336    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 21 14:32:26.042646 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.042632    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 21 14:32:26.042872 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.042852    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-z4bf8\""
Jul 21 14:32:26.042872 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.042862    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 21 14:32:26.043018 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.042861    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 21 14:32:26.044372 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.044354    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44xjw"
Jul 21 14:32:26.046548 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.046500    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 21 14:32:26.046911 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.046613    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 21 14:32:26.046911 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.046636    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 21 14:32:26.046911 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.046661    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vhndm"
Jul 21 14:32:26.046911 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.046727    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-n45rr\""
Jul 21 14:32:26.048929 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.048848    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.049291 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.049254    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 21 14:32:26.049394 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.049355    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-s7l79\""
Jul 21 14:32:26.049620 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.049604    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 21 14:32:26.051062 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.051045    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78qqf"
Jul 21 14:32:26.051147 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:26.051101    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78qqf" podUID="80d084f2-ce50-4ce6-9a6f-76e108e0215e"
Jul 21 14:32:26.051216 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.051151    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 21 14:32:26.051216 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.051169    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zgbbv\""
Jul 21 14:32:26.051592 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.051578    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 21 14:32:26.054497 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.054134    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 21 14:32:26.054497 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.054449    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e131f9f0-09bd-4d0a-b61b-e13d4efda407-sys-fs\") pod \"aws-ebs-csi-driver-node-44xjw\" (UID: \"e131f9f0-09bd-4d0a-b61b-e13d4efda407\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44xjw"
Jul 21 14:32:26.054604 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.054507    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/37083d35-d10f-4542-871a-a16c12c6bc7f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gxsg8\" (UID: \"37083d35-d10f-4542-871a-a16c12c6bc7f\") " pod="openshift-multus/multus-additional-cni-plugins-gxsg8"
Jul 21 14:32:26.054666 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.054650    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-run-ovn\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.054718 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.054703    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e131f9f0-09bd-4d0a-b61b-e13d4efda407-registration-dir\") pod \"aws-ebs-csi-driver-node-44xjw\" (UID: \"e131f9f0-09bd-4d0a-b61b-e13d4efda407\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44xjw"
Jul 21 14:32:26.054768 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.054737    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qkkp\" (UniqueName: \"kubernetes.io/projected/7845eb46-a515-4a7b-9c2b-88649a1c8ed2-kube-api-access-4qkkp\") pod \"node-ca-j25df\" (UID: \"7845eb46-a515-4a7b-9c2b-88649a1c8ed2\") " pod="openshift-image-registry/node-ca-j25df"
Jul 21 14:32:26.054841 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.054825    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.054893 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.054823    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/37083d35-d10f-4542-871a-a16c12c6bc7f-cni-binary-copy\") pod \"multus-additional-cni-plugins-gxsg8\" (UID: \"37083d35-d10f-4542-871a-a16c12c6bc7f\") " pod="openshift-multus/multus-additional-cni-plugins-gxsg8"
Jul 21 14:32:26.054949 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.054904    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/37083d35-d10f-4542-871a-a16c12c6bc7f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gxsg8\" (UID: \"37083d35-d10f-4542-871a-a16c12c6bc7f\") " pod="openshift-multus/multus-additional-cni-plugins-gxsg8"
Jul 21 14:32:26.054949 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.054940    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpct5\" (UniqueName: \"kubernetes.io/projected/13a6c11d-b9ba-4c12-836f-2172b4d168ce-kube-api-access-fpct5\") pod \"network-metrics-daemon-2zm5j\" (UID: \"13a6c11d-b9ba-4c12-836f-2172b4d168ce\") " pod="openshift-multus/network-metrics-daemon-2zm5j"
Jul 21 14:32:26.055032 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.054975    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-systemd-units\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.055099 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.055082    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-log-socket\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.055410 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.055388    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a93f1733-b8cf-40a0-a492-9d6f01457c06-ovn-node-metrics-cert\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.055485 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.055426    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a93f1733-b8cf-40a0-a492-9d6f01457c06-ovnkube-script-lib\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.055485 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.055451    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b81ae5a9-ca37-4ddd-be8a-3a732b73eb43-tmp-dir\") pod \"node-resolver-gbn95\" (UID: \"b81ae5a9-ca37-4ddd-be8a-3a732b73eb43\") " pod="openshift-dns/node-resolver-gbn95"
Jul 21 14:32:26.055485 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.055474    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7845eb46-a515-4a7b-9c2b-88649a1c8ed2-host\") pod \"node-ca-j25df\" (UID: \"7845eb46-a515-4a7b-9c2b-88649a1c8ed2\") " pod="openshift-image-registry/node-ca-j25df"
Jul 21 14:32:26.055591 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.055496    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7845eb46-a515-4a7b-9c2b-88649a1c8ed2-serviceca\") pod \"node-ca-j25df\" (UID: \"7845eb46-a515-4a7b-9c2b-88649a1c8ed2\") " pod="openshift-image-registry/node-ca-j25df"
Jul 21 14:32:26.055591 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.055519    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13a6c11d-b9ba-4c12-836f-2172b4d168ce-metrics-certs\") pod \"network-metrics-daemon-2zm5j\" (UID: \"13a6c11d-b9ba-4c12-836f-2172b4d168ce\") " pod="openshift-multus/network-metrics-daemon-2zm5j"
Jul 21 14:32:26.055591 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.055540    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-host-run-netns\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.055591 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.055578    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-node-log\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.055719 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.055623    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-host-run-ovn-kubernetes\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.055719 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.055651    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-host-cni-netd\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.055719 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.055676    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/37083d35-d10f-4542-871a-a16c12c6bc7f-cnibin\") pod \"multus-additional-cni-plugins-gxsg8\" (UID: \"37083d35-d10f-4542-871a-a16c12c6bc7f\") " pod="openshift-multus/multus-additional-cni-plugins-gxsg8"
Jul 21 14:32:26.055719 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.055699    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/37083d35-d10f-4542-871a-a16c12c6bc7f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gxsg8\" (UID: \"37083d35-d10f-4542-871a-a16c12c6bc7f\") " pod="openshift-multus/multus-additional-cni-plugins-gxsg8"
Jul 21 14:32:26.055866 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.055722    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbw6k\" (UniqueName: \"kubernetes.io/projected/37083d35-d10f-4542-871a-a16c12c6bc7f-kube-api-access-nbw6k\") pod \"multus-additional-cni-plugins-gxsg8\" (UID: \"37083d35-d10f-4542-871a-a16c12c6bc7f\") " pod="openshift-multus/multus-additional-cni-plugins-gxsg8"
Jul 21 14:32:26.055866 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.055746    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a93f1733-b8cf-40a0-a492-9d6f01457c06-ovnkube-config\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.055866 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.055770    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98pw7\" (UniqueName: \"kubernetes.io/projected/b81ae5a9-ca37-4ddd-be8a-3a732b73eb43-kube-api-access-98pw7\") pod \"node-resolver-gbn95\" (UID: \"b81ae5a9-ca37-4ddd-be8a-3a732b73eb43\") " pod="openshift-dns/node-resolver-gbn95"
Jul 21 14:32:26.055866 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.055794    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e131f9f0-09bd-4d0a-b61b-e13d4efda407-kubelet-dir\") pod \"aws-ebs-csi-driver-node-44xjw\" (UID: \"e131f9f0-09bd-4d0a-b61b-e13d4efda407\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44xjw"
Jul 21 14:32:26.055866 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.055818    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pp42q\" (UniqueName: \"kubernetes.io/projected/e131f9f0-09bd-4d0a-b61b-e13d4efda407-kube-api-access-pp42q\") pod \"aws-ebs-csi-driver-node-44xjw\" (UID: \"e131f9f0-09bd-4d0a-b61b-e13d4efda407\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44xjw"
Jul 21 14:32:26.056089 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.055860    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wl9vm\" (UniqueName: \"kubernetes.io/projected/206dbc0b-6708-45a2-bcd9-50b72fd5e6b5-kube-api-access-wl9vm\") pod \"network-check-target-q7h42\" (UID: \"206dbc0b-6708-45a2-bcd9-50b72fd5e6b5\") " pod="openshift-network-diagnostics/network-check-target-q7h42"
Jul 21 14:32:26.056089 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.055891    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/37083d35-d10f-4542-871a-a16c12c6bc7f-system-cni-dir\") pod \"multus-additional-cni-plugins-gxsg8\" (UID: \"37083d35-d10f-4542-871a-a16c12c6bc7f\") " pod="openshift-multus/multus-additional-cni-plugins-gxsg8"
Jul 21 14:32:26.056089 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.055918    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-host-cni-bin\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.056089 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.055943    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.056089 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.055967    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gn5d5\" (UniqueName: \"kubernetes.io/projected/a93f1733-b8cf-40a0-a492-9d6f01457c06-kube-api-access-gn5d5\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.056089 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.055991    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e131f9f0-09bd-4d0a-b61b-e13d4efda407-device-dir\") pod \"aws-ebs-csi-driver-node-44xjw\" (UID: \"e131f9f0-09bd-4d0a-b61b-e13d4efda407\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44xjw"
Jul 21 14:32:26.056089 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.056013    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e131f9f0-09bd-4d0a-b61b-e13d4efda407-etc-selinux\") pod \"aws-ebs-csi-driver-node-44xjw\" (UID: \"e131f9f0-09bd-4d0a-b61b-e13d4efda407\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44xjw"
Jul 21 14:32:26.056089 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.056035    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-run-systemd\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.056089 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.056058    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-etc-openvswitch\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.056089 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.056080    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-run-openvswitch\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.056429 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.056101    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a93f1733-b8cf-40a0-a492-9d6f01457c06-env-overrides\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.056429 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.056137    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e131f9f0-09bd-4d0a-b61b-e13d4efda407-socket-dir\") pod \"aws-ebs-csi-driver-node-44xjw\" (UID: \"e131f9f0-09bd-4d0a-b61b-e13d4efda407\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44xjw"
Jul 21 14:32:26.056429 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.056157    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/37083d35-d10f-4542-871a-a16c12c6bc7f-os-release\") pod \"multus-additional-cni-plugins-gxsg8\" (UID: \"37083d35-d10f-4542-871a-a16c12c6bc7f\") " pod="openshift-multus/multus-additional-cni-plugins-gxsg8"
Jul 21 14:32:26.056429 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.056171    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-host-kubelet\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.056429 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.056184    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-host-slash\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.056429 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.056204    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-var-lib-openvswitch\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.056429 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.056231    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b81ae5a9-ca37-4ddd-be8a-3a732b73eb43-hosts-file\") pod \"node-resolver-gbn95\" (UID: \"b81ae5a9-ca37-4ddd-be8a-3a732b73eb43\") " pod="openshift-dns/node-resolver-gbn95"
Jul 21 14:32:26.057185 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.057170    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 21 14:32:26.057385 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.057371    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-nnkkc\""
Jul 21 14:32:26.057589 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.057576    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8ff6d"
Jul 21 14:32:26.059689 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.059670    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 21 14:32:26.059784 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.059694    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 21 14:32:26.059784 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.059720    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 21 14:32:26.059784 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.059720    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-9wwbl\""
Jul 21 14:32:26.077892 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.077877    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-vc899"
Jul 21 14:32:26.087896 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.087880    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-vc899"
Jul 21 14:32:26.143370 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.143353    2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 21 14:32:26.156580 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.156561    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-host-run-netns\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.156682 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.156586    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a93f1733-b8cf-40a0-a492-9d6f01457c06-env-overrides\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.156682 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.156609    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/37083d35-d10f-4542-871a-a16c12c6bc7f-os-release\") pod \"multus-additional-cni-plugins-gxsg8\" (UID: \"37083d35-d10f-4542-871a-a16c12c6bc7f\") " pod="openshift-multus/multus-additional-cni-plugins-gxsg8"
Jul 21 14:32:26.156682 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.156624    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6055a183-9e39-4f94-b109-3a62e8c561e7-etc-modprobe-d\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.156682 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.156647    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-etc-kubernetes\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.156682 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.156670    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-host-kubelet\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.156904 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.156693    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e131f9f0-09bd-4d0a-b61b-e13d4efda407-sys-fs\") pod \"aws-ebs-csi-driver-node-44xjw\" (UID: \"e131f9f0-09bd-4d0a-b61b-e13d4efda407\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44xjw"
Jul 21 14:32:26.156904 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.156701    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/37083d35-d10f-4542-871a-a16c12c6bc7f-os-release\") pod \"multus-additional-cni-plugins-gxsg8\" (UID: \"37083d35-d10f-4542-871a-a16c12c6bc7f\") " pod="openshift-multus/multus-additional-cni-plugins-gxsg8"
Jul 21 14:32:26.156904 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.156710    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-host-kubelet\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.156904 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.156733    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6055a183-9e39-4f94-b109-3a62e8c561e7-sys\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.156904 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.156751    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e131f9f0-09bd-4d0a-b61b-e13d4efda407-sys-fs\") pod \"aws-ebs-csi-driver-node-44xjw\" (UID: \"e131f9f0-09bd-4d0a-b61b-e13d4efda407\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44xjw"
Jul 21 14:32:26.156904 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.156783    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6055a183-9e39-4f94-b109-3a62e8c561e7-lib-modules\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.156904 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.156812    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6055a183-9e39-4f94-b109-3a62e8c561e7-var-lib-kubelet\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.156904 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.156835    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwp4j\" (UniqueName: \"kubernetes.io/projected/6055a183-9e39-4f94-b109-3a62e8c561e7-kube-api-access-pwp4j\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.156904 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.156862    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-os-release\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.156904 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.156878    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d1c20830-6622-4c3e-95b3-0cfb33cc1245-host-slash\") pod \"iptables-alerter-8ff6d\" (UID: \"d1c20830-6622-4c3e-95b3-0cfb33cc1245\") " pod="openshift-network-operator/iptables-alerter-8ff6d"
Jul 21 14:32:26.156904 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.156894    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e131f9f0-09bd-4d0a-b61b-e13d4efda407-registration-dir\") pod \"aws-ebs-csi-driver-node-44xjw\" (UID: \"e131f9f0-09bd-4d0a-b61b-e13d4efda407\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44xjw"
Jul 21 14:32:26.157392 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.156909    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpct5\" (UniqueName: \"kubernetes.io/projected/13a6c11d-b9ba-4c12-836f-2172b4d168ce-kube-api-access-fpct5\") pod \"network-metrics-daemon-2zm5j\" (UID: \"13a6c11d-b9ba-4c12-836f-2172b4d168ce\") " pod="openshift-multus/network-metrics-daemon-2zm5j"
Jul 21 14:32:26.157392 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.156942    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-log-socket\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.157392 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.156949    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e131f9f0-09bd-4d0a-b61b-e13d4efda407-registration-dir\") pod \"aws-ebs-csi-driver-node-44xjw\" (UID: \"e131f9f0-09bd-4d0a-b61b-e13d4efda407\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44xjw"
Jul 21 14:32:26.157392 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.156958    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b81ae5a9-ca37-4ddd-be8a-3a732b73eb43-tmp-dir\") pod \"node-resolver-gbn95\" (UID: \"b81ae5a9-ca37-4ddd-be8a-3a732b73eb43\") " pod="openshift-dns/node-resolver-gbn95"
Jul 21 14:32:26.157392 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.156974    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/37083d35-d10f-4542-871a-a16c12c6bc7f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gxsg8\" (UID: \"37083d35-d10f-4542-871a-a16c12c6bc7f\") " pod="openshift-multus/multus-additional-cni-plugins-gxsg8"
Jul 21 14:32:26.157392 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.156977    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-log-socket\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.157392 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.156988    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbw6k\" (UniqueName: \"kubernetes.io/projected/37083d35-d10f-4542-871a-a16c12c6bc7f-kube-api-access-nbw6k\") pod \"multus-additional-cni-plugins-gxsg8\" (UID: \"37083d35-d10f-4542-871a-a16c12c6bc7f\") " pod="openshift-multus/multus-additional-cni-plugins-gxsg8"
Jul 21 14:32:26.157392 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157011    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6055a183-9e39-4f94-b109-3a62e8c561e7-etc-systemd\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.157392 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157049    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a93f1733-b8cf-40a0-a492-9d6f01457c06-env-overrides\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.157392 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157054    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/442fea02-6da4-40ad-9f33-d7824c635d4f-agent-certs\") pod \"konnectivity-agent-vhndm\" (UID: \"442fea02-6da4-40ad-9f33-d7824c635d4f\") " pod="kube-system/konnectivity-agent-vhndm"
Jul 21 14:32:26.157392 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157134    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/80d084f2-ce50-4ce6-9a6f-76e108e0215e-dbus\") pod \"global-pull-secret-syncer-78qqf\" (UID: \"80d084f2-ce50-4ce6-9a6f-76e108e0215e\") " pod="kube-system/global-pull-secret-syncer-78qqf"
Jul 21 14:32:26.157392 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157146    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/37083d35-d10f-4542-871a-a16c12c6bc7f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gxsg8\" (UID: \"37083d35-d10f-4542-871a-a16c12c6bc7f\") " pod="openshift-multus/multus-additional-cni-plugins-gxsg8"
Jul 21 14:32:26.157392 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157157    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-host-run-netns\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.157392 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157172    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6055a183-9e39-4f94-b109-3a62e8c561e7-tmp\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.157392 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157185    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-host-run-netns\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.157392 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157186    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-host-run-k8s-cni-cncf-io\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.157392 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157203    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b81ae5a9-ca37-4ddd-be8a-3a732b73eb43-tmp-dir\") pod \"node-resolver-gbn95\" (UID: \"b81ae5a9-ca37-4ddd-be8a-3a732b73eb43\") " pod="openshift-dns/node-resolver-gbn95"
Jul 21 14:32:26.158000 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157221    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a93f1733-b8cf-40a0-a492-9d6f01457c06-ovnkube-config\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.158000 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157263    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98pw7\" (UniqueName: \"kubernetes.io/projected/b81ae5a9-ca37-4ddd-be8a-3a732b73eb43-kube-api-access-98pw7\") pod \"node-resolver-gbn95\" (UID: \"b81ae5a9-ca37-4ddd-be8a-3a732b73eb43\") " pod="openshift-dns/node-resolver-gbn95"
Jul 21 14:32:26.158000 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157367    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e131f9f0-09bd-4d0a-b61b-e13d4efda407-kubelet-dir\") pod \"aws-ebs-csi-driver-node-44xjw\" (UID: \"e131f9f0-09bd-4d0a-b61b-e13d4efda407\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44xjw"
Jul 21 14:32:26.158000 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157473    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e131f9f0-09bd-4d0a-b61b-e13d4efda407-kubelet-dir\") pod \"aws-ebs-csi-driver-node-44xjw\" (UID: \"e131f9f0-09bd-4d0a-b61b-e13d4efda407\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44xjw"
Jul 21 14:32:26.158000 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157492    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/37083d35-d10f-4542-871a-a16c12c6bc7f-system-cni-dir\") pod \"multus-additional-cni-plugins-gxsg8\" (UID: \"37083d35-d10f-4542-871a-a16c12c6bc7f\") " pod="openshift-multus/multus-additional-cni-plugins-gxsg8"
Jul 21 14:32:26.158000 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157513    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/37083d35-d10f-4542-871a-a16c12c6bc7f-system-cni-dir\") pod \"multus-additional-cni-plugins-gxsg8\" (UID: \"37083d35-d10f-4542-871a-a16c12c6bc7f\") " pod="openshift-multus/multus-additional-cni-plugins-gxsg8"
Jul 21 14:32:26.158000 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157531    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6055a183-9e39-4f94-b109-3a62e8c561e7-etc-sysctl-conf\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.158000 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157569    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-multus-socket-dir-parent\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.158000 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157579    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a93f1733-b8cf-40a0-a492-9d6f01457c06-ovnkube-config\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.158000 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157600    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-host-cni-bin\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.158000 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157626    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.158000 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157644    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e131f9f0-09bd-4d0a-b61b-e13d4efda407-device-dir\") pod \"aws-ebs-csi-driver-node-44xjw\" (UID: \"e131f9f0-09bd-4d0a-b61b-e13d4efda407\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44xjw"
Jul 21 14:32:26.158000 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157652    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-host-cni-bin\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.158000 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157660    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e131f9f0-09bd-4d0a-b61b-e13d4efda407-etc-selinux\") pod \"aws-ebs-csi-driver-node-44xjw\" (UID: \"e131f9f0-09bd-4d0a-b61b-e13d4efda407\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44xjw"
Jul 21 14:32:26.158000 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157665    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.158000 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157682    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/442fea02-6da4-40ad-9f33-d7824c635d4f-konnectivity-ca\") pod \"konnectivity-agent-vhndm\" (UID: \"442fea02-6da4-40ad-9f33-d7824c635d4f\") " pod="kube-system/konnectivity-agent-vhndm"
Jul 21 14:32:26.158601 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157700    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfqv9\" (UniqueName: \"kubernetes.io/projected/d1c20830-6622-4c3e-95b3-0cfb33cc1245-kube-api-access-gfqv9\") pod \"iptables-alerter-8ff6d\" (UID: \"d1c20830-6622-4c3e-95b3-0cfb33cc1245\") " pod="openshift-network-operator/iptables-alerter-8ff6d"
Jul 21 14:32:26.158601 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157704    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e131f9f0-09bd-4d0a-b61b-e13d4efda407-device-dir\") pod \"aws-ebs-csi-driver-node-44xjw\" (UID: \"e131f9f0-09bd-4d0a-b61b-e13d4efda407\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44xjw"
Jul 21 14:32:26.158601 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157715    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-run-openvswitch\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.158601 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157735    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e131f9f0-09bd-4d0a-b61b-e13d4efda407-etc-selinux\") pod \"aws-ebs-csi-driver-node-44xjw\" (UID: \"e131f9f0-09bd-4d0a-b61b-e13d4efda407\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44xjw"
Jul 21 14:32:26.158601 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157741    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e131f9f0-09bd-4d0a-b61b-e13d4efda407-socket-dir\") pod \"aws-ebs-csi-driver-node-44xjw\" (UID: \"e131f9f0-09bd-4d0a-b61b-e13d4efda407\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44xjw"
Jul 21 14:32:26.158601 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157777    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-run-openvswitch\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.158601 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157803    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/212b69da-9153-48f1-94e2-6967b30b699c-multus-daemon-config\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.158601 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157843    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80d084f2-ce50-4ce6-9a6f-76e108e0215e-original-pull-secret\") pod \"global-pull-secret-syncer-78qqf\" (UID: \"80d084f2-ce50-4ce6-9a6f-76e108e0215e\") " pod="kube-system/global-pull-secret-syncer-78qqf"
Jul 21 14:32:26.158601 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157850    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e131f9f0-09bd-4d0a-b61b-e13d4efda407-socket-dir\") pod \"aws-ebs-csi-driver-node-44xjw\" (UID: \"e131f9f0-09bd-4d0a-b61b-e13d4efda407\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44xjw"
Jul 21 14:32:26.158601 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157861    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-host-slash\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.158601 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157889    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-host-slash\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.158601 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157916    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-var-lib-openvswitch\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.158601 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157937    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b81ae5a9-ca37-4ddd-be8a-3a732b73eb43-hosts-file\") pod \"node-resolver-gbn95\" (UID: \"b81ae5a9-ca37-4ddd-be8a-3a732b73eb43\") " pod="openshift-dns/node-resolver-gbn95"
Jul 21 14:32:26.158601 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157949    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-var-lib-openvswitch\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.158601 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157974    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/37083d35-d10f-4542-871a-a16c12c6bc7f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gxsg8\" (UID: \"37083d35-d10f-4542-871a-a16c12c6bc7f\") " pod="openshift-multus/multus-additional-cni-plugins-gxsg8"
Jul 21 14:32:26.158601 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.157990    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6055a183-9e39-4f94-b109-3a62e8c561e7-etc-sysctl-d\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.158601 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158009    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6055a183-9e39-4f94-b109-3a62e8c561e7-run\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.159406 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158032    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6055a183-9e39-4f94-b109-3a62e8c561e7-etc-tuned\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.159406 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158035    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b81ae5a9-ca37-4ddd-be8a-3a732b73eb43-hosts-file\") pod \"node-resolver-gbn95\" (UID: \"b81ae5a9-ca37-4ddd-be8a-3a732b73eb43\") " pod="openshift-dns/node-resolver-gbn95"
Jul 21 14:32:26.159406 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158046    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/212b69da-9153-48f1-94e2-6967b30b699c-cni-binary-copy\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.159406 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158092    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-run-ovn\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.159406 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158112    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qkkp\" (UniqueName: \"kubernetes.io/projected/7845eb46-a515-4a7b-9c2b-88649a1c8ed2-kube-api-access-4qkkp\") pod \"node-ca-j25df\" (UID: \"7845eb46-a515-4a7b-9c2b-88649a1c8ed2\") " pod="openshift-image-registry/node-ca-j25df"
Jul 21 14:32:26.159406 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158125    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-run-ovn\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.159406 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158157    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/37083d35-d10f-4542-871a-a16c12c6bc7f-cni-binary-copy\") pod \"multus-additional-cni-plugins-gxsg8\" (UID: \"37083d35-d10f-4542-871a-a16c12c6bc7f\") " pod="openshift-multus/multus-additional-cni-plugins-gxsg8"
Jul 21 14:32:26.159406 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158174    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/37083d35-d10f-4542-871a-a16c12c6bc7f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gxsg8\" (UID: \"37083d35-d10f-4542-871a-a16c12c6bc7f\") " pod="openshift-multus/multus-additional-cni-plugins-gxsg8"
Jul 21 14:32:26.159406 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158218    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-multus-cni-dir\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.159406 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158250    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-hostroot\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.159406 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158289    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d1c20830-6622-4c3e-95b3-0cfb33cc1245-iptables-alerter-script\") pod \"iptables-alerter-8ff6d\" (UID: \"d1c20830-6622-4c3e-95b3-0cfb33cc1245\") " pod="openshift-network-operator/iptables-alerter-8ff6d"
Jul 21 14:32:26.159406 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158318    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6055a183-9e39-4f94-b109-3a62e8c561e7-etc-kubernetes\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.159406 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158346    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-systemd-units\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.159406 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158387    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a93f1733-b8cf-40a0-a492-9d6f01457c06-ovn-node-metrics-cert\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.159406 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158435    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a93f1733-b8cf-40a0-a492-9d6f01457c06-ovnkube-script-lib\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.159406 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158441    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/37083d35-d10f-4542-871a-a16c12c6bc7f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gxsg8\" (UID: \"37083d35-d10f-4542-871a-a16c12c6bc7f\") " pod="openshift-multus/multus-additional-cni-plugins-gxsg8"
Jul 21 14:32:26.159406 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158472    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-systemd-units\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.159988 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158510    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7845eb46-a515-4a7b-9c2b-88649a1c8ed2-host\") pod \"node-ca-j25df\" (UID: \"7845eb46-a515-4a7b-9c2b-88649a1c8ed2\") " pod="openshift-image-registry/node-ca-j25df"
Jul 21 14:32:26.159988 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158546    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7845eb46-a515-4a7b-9c2b-88649a1c8ed2-serviceca\") pod \"node-ca-j25df\" (UID: \"7845eb46-a515-4a7b-9c2b-88649a1c8ed2\") " pod="openshift-image-registry/node-ca-j25df"
Jul 21 14:32:26.159988 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158572    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-host-var-lib-cni-bin\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.159988 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158591    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7845eb46-a515-4a7b-9c2b-88649a1c8ed2-host\") pod \"node-ca-j25df\" (UID: \"7845eb46-a515-4a7b-9c2b-88649a1c8ed2\") " pod="openshift-image-registry/node-ca-j25df"
Jul 21 14:32:26.159988 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158600    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-host-var-lib-kubelet\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.159988 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158637    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/37083d35-d10f-4542-871a-a16c12c6bc7f-cni-binary-copy\") pod \"multus-additional-cni-plugins-gxsg8\" (UID: \"37083d35-d10f-4542-871a-a16c12c6bc7f\") " pod="openshift-multus/multus-additional-cni-plugins-gxsg8"
Jul 21 14:32:26.159988 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158669    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13a6c11d-b9ba-4c12-836f-2172b4d168ce-metrics-certs\") pod \"network-metrics-daemon-2zm5j\" (UID: \"13a6c11d-b9ba-4c12-836f-2172b4d168ce\") " pod="openshift-multus/network-metrics-daemon-2zm5j"
Jul 21 14:32:26.159988 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158679    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/37083d35-d10f-4542-871a-a16c12c6bc7f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gxsg8\" (UID: \"37083d35-d10f-4542-871a-a16c12c6bc7f\") " pod="openshift-multus/multus-additional-cni-plugins-gxsg8"
Jul 21 14:32:26.159988 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158698    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-node-log\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.159988 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158683    2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 21 14:32:26.159988 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158724    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-host-run-ovn-kubernetes\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.159988 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158745    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-host-cni-netd\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.159988 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:26.158760    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 14:32:26.159988 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158772    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/37083d35-d10f-4542-871a-a16c12c6bc7f-cnibin\") pod \"multus-additional-cni-plugins-gxsg8\" (UID: \"37083d35-d10f-4542-871a-a16c12c6bc7f\") " pod="openshift-multus/multus-additional-cni-plugins-gxsg8"
Jul 21 14:32:26.159988 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158793    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-host-var-lib-cni-multus\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.159988 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158803    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-node-log\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.159988 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:26.158819    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13a6c11d-b9ba-4c12-836f-2172b4d168ce-metrics-certs podName:13a6c11d-b9ba-4c12-836f-2172b4d168ce nodeName:}" failed. No retries permitted until 2026-07-21 14:32:26.658800149 +0000 UTC m=+3.008543562 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13a6c11d-b9ba-4c12-836f-2172b4d168ce-metrics-certs") pod "network-metrics-daemon-2zm5j" (UID: "13a6c11d-b9ba-4c12-836f-2172b4d168ce") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 14:32:26.160470 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158846    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-host-run-ovn-kubernetes\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.160470 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158854    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-host-cni-netd\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.160470 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158898    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pp42q\" (UniqueName: \"kubernetes.io/projected/e131f9f0-09bd-4d0a-b61b-e13d4efda407-kube-api-access-pp42q\") pod \"aws-ebs-csi-driver-node-44xjw\" (UID: \"e131f9f0-09bd-4d0a-b61b-e13d4efda407\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44xjw"
Jul 21 14:32:26.160470 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158910    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/37083d35-d10f-4542-871a-a16c12c6bc7f-cnibin\") pod \"multus-additional-cni-plugins-gxsg8\" (UID: \"37083d35-d10f-4542-871a-a16c12c6bc7f\") " pod="openshift-multus/multus-additional-cni-plugins-gxsg8"
Jul 21 14:32:26.160470 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158924    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl9vm\" (UniqueName: \"kubernetes.io/projected/206dbc0b-6708-45a2-bcd9-50b72fd5e6b5-kube-api-access-wl9vm\") pod \"network-check-target-q7h42\" (UID: \"206dbc0b-6708-45a2-bcd9-50b72fd5e6b5\") " pod="openshift-network-diagnostics/network-check-target-q7h42"
Jul 21 14:32:26.160470 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158949    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6055a183-9e39-4f94-b109-3a62e8c561e7-host\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.160470 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158968    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7845eb46-a515-4a7b-9c2b-88649a1c8ed2-serviceca\") pod \"node-ca-j25df\" (UID: \"7845eb46-a515-4a7b-9c2b-88649a1c8ed2\") " pod="openshift-image-registry/node-ca-j25df"
Jul 21 14:32:26.160470 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158972    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-system-cni-dir\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.160470 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158998    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-multus-conf-dir\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.160470 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.159020    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bw898\" (UniqueName: \"kubernetes.io/projected/212b69da-9153-48f1-94e2-6967b30b699c-kube-api-access-bw898\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.160470 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.159042    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6055a183-9e39-4f94-b109-3a62e8c561e7-etc-sysconfig\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.160470 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.159067    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gn5d5\" (UniqueName: \"kubernetes.io/projected/a93f1733-b8cf-40a0-a492-9d6f01457c06-kube-api-access-gn5d5\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.160470 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.159090    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-host-run-multus-certs\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.160470 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.159115    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/80d084f2-ce50-4ce6-9a6f-76e108e0215e-kubelet-config\") pod \"global-pull-secret-syncer-78qqf\" (UID: \"80d084f2-ce50-4ce6-9a6f-76e108e0215e\") " pod="kube-system/global-pull-secret-syncer-78qqf"
Jul 21 14:32:26.160470 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.159215    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-run-systemd\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.160470 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.159258    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-etc-openvswitch\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.160470 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.159305    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-cnibin\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.160952 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.159359    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-run-systemd\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.160952 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.158903    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a93f1733-b8cf-40a0-a492-9d6f01457c06-ovnkube-script-lib\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.160952 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.159307    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a93f1733-b8cf-40a0-a492-9d6f01457c06-etc-openvswitch\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.161698 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.161683    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a93f1733-b8cf-40a0-a492-9d6f01457c06-ovn-node-metrics-cert\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.165459 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:26.165437    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 14:32:26.165459 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:26.165460    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 14:32:26.165609 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:26.165474    2579 projected.go:196] Error preparing data for projected volume kube-api-access-wl9vm for pod openshift-network-diagnostics/network-check-target-q7h42: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 14:32:26.165609 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:26.165552    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/206dbc0b-6708-45a2-bcd9-50b72fd5e6b5-kube-api-access-wl9vm podName:206dbc0b-6708-45a2-bcd9-50b72fd5e6b5 nodeName:}" failed. No retries permitted until 2026-07-21 14:32:26.665511416 +0000 UTC m=+3.015254815 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wl9vm" (UniqueName: "kubernetes.io/projected/206dbc0b-6708-45a2-bcd9-50b72fd5e6b5-kube-api-access-wl9vm") pod "network-check-target-q7h42" (UID: "206dbc0b-6708-45a2-bcd9-50b72fd5e6b5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 14:32:26.167530 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.167354    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98pw7\" (UniqueName: \"kubernetes.io/projected/b81ae5a9-ca37-4ddd-be8a-3a732b73eb43-kube-api-access-98pw7\") pod \"node-resolver-gbn95\" (UID: \"b81ae5a9-ca37-4ddd-be8a-3a732b73eb43\") " pod="openshift-dns/node-resolver-gbn95"
Jul 21 14:32:26.167530 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.167400    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qkkp\" (UniqueName: \"kubernetes.io/projected/7845eb46-a515-4a7b-9c2b-88649a1c8ed2-kube-api-access-4qkkp\") pod \"node-ca-j25df\" (UID: \"7845eb46-a515-4a7b-9c2b-88649a1c8ed2\") " pod="openshift-image-registry/node-ca-j25df"
Jul 21 14:32:26.169873 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.167914    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbw6k\" (UniqueName: \"kubernetes.io/projected/37083d35-d10f-4542-871a-a16c12c6bc7f-kube-api-access-nbw6k\") pod \"multus-additional-cni-plugins-gxsg8\" (UID: \"37083d35-d10f-4542-871a-a16c12c6bc7f\") " pod="openshift-multus/multus-additional-cni-plugins-gxsg8"
Jul 21 14:32:26.169873 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.168992    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn5d5\" (UniqueName: \"kubernetes.io/projected/a93f1733-b8cf-40a0-a492-9d6f01457c06-kube-api-access-gn5d5\") pod \"ovnkube-node-mv82n\" (UID: \"a93f1733-b8cf-40a0-a492-9d6f01457c06\") " pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.170228 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.170200    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pp42q\" (UniqueName: \"kubernetes.io/projected/e131f9f0-09bd-4d0a-b61b-e13d4efda407-kube-api-access-pp42q\") pod \"aws-ebs-csi-driver-node-44xjw\" (UID: \"e131f9f0-09bd-4d0a-b61b-e13d4efda407\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44xjw"
Jul 21 14:32:26.170983 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.170961    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpct5\" (UniqueName: \"kubernetes.io/projected/13a6c11d-b9ba-4c12-836f-2172b4d168ce-kube-api-access-fpct5\") pod \"network-metrics-daemon-2zm5j\" (UID: \"13a6c11d-b9ba-4c12-836f-2172b4d168ce\") " pod="openshift-multus/network-metrics-daemon-2zm5j"
Jul 21 14:32:26.171759 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.171725    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.ec2.internal" event={"ID":"d46085a6c06a58d348208916c9b4bb78","Type":"ContainerStarted","Data":"0a01d7c4495a026fe5d4695e31df5c673b9e1f758f1abcf197d419b828c67ed0"}
Jul 21 14:32:26.172696 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.172676    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-83.ec2.internal" event={"ID":"ada9959260df8e5e8a349b59b83b3eed","Type":"ContainerStarted","Data":"db8bc3d864bd0bd6b5bda65342d08b5016c315c9e65b3db6e0afe8f9298e6593"}
Jul 21 14:32:26.259636 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.259618    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6055a183-9e39-4f94-b109-3a62e8c561e7-etc-systemd\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.259723 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.259640    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/442fea02-6da4-40ad-9f33-d7824c635d4f-agent-certs\") pod \"konnectivity-agent-vhndm\" (UID: \"442fea02-6da4-40ad-9f33-d7824c635d4f\") " pod="kube-system/konnectivity-agent-vhndm"
Jul 21 14:32:26.259723 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.259654    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/80d084f2-ce50-4ce6-9a6f-76e108e0215e-dbus\") pod \"global-pull-secret-syncer-78qqf\" (UID: \"80d084f2-ce50-4ce6-9a6f-76e108e0215e\") " pod="kube-system/global-pull-secret-syncer-78qqf"
Jul 21 14:32:26.259723 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.259667    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6055a183-9e39-4f94-b109-3a62e8c561e7-tmp\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.259723 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.259683    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-host-run-k8s-cni-cncf-io\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.259723 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.259698    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6055a183-9e39-4f94-b109-3a62e8c561e7-etc-sysctl-conf\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.259723 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.259713    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-multus-socket-dir-parent\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.260051 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.259729    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/442fea02-6da4-40ad-9f33-d7824c635d4f-konnectivity-ca\") pod \"konnectivity-agent-vhndm\" (UID: \"442fea02-6da4-40ad-9f33-d7824c635d4f\") " pod="kube-system/konnectivity-agent-vhndm"
Jul 21 14:32:26.260051 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.259751    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfqv9\" (UniqueName: \"kubernetes.io/projected/d1c20830-6622-4c3e-95b3-0cfb33cc1245-kube-api-access-gfqv9\") pod \"iptables-alerter-8ff6d\" (UID: \"d1c20830-6622-4c3e-95b3-0cfb33cc1245\") " pod="openshift-network-operator/iptables-alerter-8ff6d"
Jul 21 14:32:26.260051 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.259777    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/212b69da-9153-48f1-94e2-6967b30b699c-multus-daemon-config\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.260051 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.259744    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6055a183-9e39-4f94-b109-3a62e8c561e7-etc-systemd\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.260051 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.259807    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-host-run-k8s-cni-cncf-io\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.260051 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.259814    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80d084f2-ce50-4ce6-9a6f-76e108e0215e-original-pull-secret\") pod \"global-pull-secret-syncer-78qqf\" (UID: \"80d084f2-ce50-4ce6-9a6f-76e108e0215e\") " pod="kube-system/global-pull-secret-syncer-78qqf"
Jul 21 14:32:26.260051 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.259853    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-multus-socket-dir-parent\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.260051 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.259866    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6055a183-9e39-4f94-b109-3a62e8c561e7-etc-sysctl-d\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.260051 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.259911    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6055a183-9e39-4f94-b109-3a62e8c561e7-run\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.260051 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.259934    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6055a183-9e39-4f94-b109-3a62e8c561e7-etc-tuned\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.260051 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.259959    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/212b69da-9153-48f1-94e2-6967b30b699c-cni-binary-copy\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.260051 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.259983    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6055a183-9e39-4f94-b109-3a62e8c561e7-etc-sysctl-conf\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.260051 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:26.259888    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 14:32:26.260051 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260008    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-multus-cni-dir\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.260051 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:26.260045    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80d084f2-ce50-4ce6-9a6f-76e108e0215e-original-pull-secret podName:80d084f2-ce50-4ce6-9a6f-76e108e0215e nodeName:}" failed. No retries permitted until 2026-07-21 14:32:26.760030063 +0000 UTC m=+3.109773468 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/80d084f2-ce50-4ce6-9a6f-76e108e0215e-original-pull-secret") pod "global-pull-secret-syncer-78qqf" (UID: "80d084f2-ce50-4ce6-9a6f-76e108e0215e") : object "kube-system"/"original-pull-secret" not registered
Jul 21 14:32:26.260051 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260047    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6055a183-9e39-4f94-b109-3a62e8c561e7-run\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.260826 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260064    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-hostroot\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.260826 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260085    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6055a183-9e39-4f94-b109-3a62e8c561e7-etc-sysctl-d\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.260826 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260090    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d1c20830-6622-4c3e-95b3-0cfb33cc1245-iptables-alerter-script\") pod \"iptables-alerter-8ff6d\" (UID: \"d1c20830-6622-4c3e-95b3-0cfb33cc1245\") " pod="openshift-network-operator/iptables-alerter-8ff6d"
Jul 21 14:32:26.260826 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260131    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-multus-cni-dir\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.260826 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260176    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-hostroot\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.260826 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260188    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6055a183-9e39-4f94-b109-3a62e8c561e7-etc-kubernetes\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.260826 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260138    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6055a183-9e39-4f94-b109-3a62e8c561e7-etc-kubernetes\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.260826 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260231    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-host-var-lib-cni-bin\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.260826 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260255    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-host-var-lib-kubelet\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.260826 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260311    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-host-var-lib-cni-multus\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.260826 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260334    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-host-var-lib-cni-bin\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.260826 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260342    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6055a183-9e39-4f94-b109-3a62e8c561e7-host\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.260826 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260366    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-system-cni-dir\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.260826 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260385    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-host-var-lib-kubelet\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.260826 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260392    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-multus-conf-dir\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.260826 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260415    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bw898\" (UniqueName: \"kubernetes.io/projected/212b69da-9153-48f1-94e2-6967b30b699c-kube-api-access-bw898\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.260826 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260433    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6055a183-9e39-4f94-b109-3a62e8c561e7-host\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.260826 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260437    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6055a183-9e39-4f94-b109-3a62e8c561e7-etc-sysconfig\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.261529 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260471    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-host-run-multus-certs\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.261529 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260481    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6055a183-9e39-4f94-b109-3a62e8c561e7-etc-sysconfig\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.261529 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260496    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/80d084f2-ce50-4ce6-9a6f-76e108e0215e-kubelet-config\") pod \"global-pull-secret-syncer-78qqf\" (UID: \"80d084f2-ce50-4ce6-9a6f-76e108e0215e\") " pod="kube-system/global-pull-secret-syncer-78qqf"
Jul 21 14:32:26.261529 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260521    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-cnibin\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.261529 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260534    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-system-cni-dir\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.261529 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260539    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/212b69da-9153-48f1-94e2-6967b30b699c-multus-daemon-config\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.261529 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260545    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-host-run-netns\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.261529 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260570    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6055a183-9e39-4f94-b109-3a62e8c561e7-etc-modprobe-d\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.261529 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260587    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-multus-conf-dir\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.261529 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260593    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-etc-kubernetes\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.261529 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260633    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6055a183-9e39-4f94-b109-3a62e8c561e7-sys\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.261529 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260638    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-etc-kubernetes\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.261529 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260637    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/212b69da-9153-48f1-94e2-6967b30b699c-cni-binary-copy\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.261529 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260675    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6055a183-9e39-4f94-b109-3a62e8c561e7-sys\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.261529 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260687    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-host-run-multus-certs\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.261529 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260721    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-cnibin\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.261529 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260725    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-host-run-netns\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.261529 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260755    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d1c20830-6622-4c3e-95b3-0cfb33cc1245-iptables-alerter-script\") pod \"iptables-alerter-8ff6d\" (UID: \"d1c20830-6622-4c3e-95b3-0cfb33cc1245\") " pod="openshift-network-operator/iptables-alerter-8ff6d"
Jul 21 14:32:26.262044 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260758    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6055a183-9e39-4f94-b109-3a62e8c561e7-lib-modules\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.262044 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260791    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6055a183-9e39-4f94-b109-3a62e8c561e7-var-lib-kubelet\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.262044 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260801    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6055a183-9e39-4f94-b109-3a62e8c561e7-etc-modprobe-d\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.262044 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260766    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/80d084f2-ce50-4ce6-9a6f-76e108e0215e-kubelet-config\") pod \"global-pull-secret-syncer-78qqf\" (UID: \"80d084f2-ce50-4ce6-9a6f-76e108e0215e\") " pod="kube-system/global-pull-secret-syncer-78qqf"
Jul 21 14:32:26.262044 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260818    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwp4j\" (UniqueName: \"kubernetes.io/projected/6055a183-9e39-4f94-b109-3a62e8c561e7-kube-api-access-pwp4j\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.262044 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260840    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-os-release\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.262044 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260849    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6055a183-9e39-4f94-b109-3a62e8c561e7-lib-modules\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.262044 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260861    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d1c20830-6622-4c3e-95b3-0cfb33cc1245-host-slash\") pod \"iptables-alerter-8ff6d\" (UID: \"d1c20830-6622-4c3e-95b3-0cfb33cc1245\") " pod="openshift-network-operator/iptables-alerter-8ff6d"
Jul 21 14:32:26.262044 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260878    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6055a183-9e39-4f94-b109-3a62e8c561e7-var-lib-kubelet\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.262044 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260928    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d1c20830-6622-4c3e-95b3-0cfb33cc1245-host-slash\") pod \"iptables-alerter-8ff6d\" (UID: \"d1c20830-6622-4c3e-95b3-0cfb33cc1245\") " pod="openshift-network-operator/iptables-alerter-8ff6d"
Jul 21 14:32:26.262044 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260934    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-os-release\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.262044 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.260979    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/212b69da-9153-48f1-94e2-6967b30b699c-host-var-lib-cni-multus\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.262044 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.261074    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/80d084f2-ce50-4ce6-9a6f-76e108e0215e-dbus\") pod \"global-pull-secret-syncer-78qqf\" (UID: \"80d084f2-ce50-4ce6-9a6f-76e108e0215e\") " pod="kube-system/global-pull-secret-syncer-78qqf"
Jul 21 14:32:26.262044 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.261114    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/442fea02-6da4-40ad-9f33-d7824c635d4f-konnectivity-ca\") pod \"konnectivity-agent-vhndm\" (UID: \"442fea02-6da4-40ad-9f33-d7824c635d4f\") " pod="kube-system/konnectivity-agent-vhndm"
Jul 21 14:32:26.262443 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.262184    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6055a183-9e39-4f94-b109-3a62e8c561e7-etc-tuned\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.262840 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.262821    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6055a183-9e39-4f94-b109-3a62e8c561e7-tmp\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.262901 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.262887    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/442fea02-6da4-40ad-9f33-d7824c635d4f-agent-certs\") pod \"konnectivity-agent-vhndm\" (UID: \"442fea02-6da4-40ad-9f33-d7824c635d4f\") " pod="kube-system/konnectivity-agent-vhndm"
Jul 21 14:32:26.269681 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.269658    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfqv9\" (UniqueName: \"kubernetes.io/projected/d1c20830-6622-4c3e-95b3-0cfb33cc1245-kube-api-access-gfqv9\") pod \"iptables-alerter-8ff6d\" (UID: \"d1c20830-6622-4c3e-95b3-0cfb33cc1245\") " pod="openshift-network-operator/iptables-alerter-8ff6d"
Jul 21 14:32:26.270593 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.270578    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwp4j\" (UniqueName: \"kubernetes.io/projected/6055a183-9e39-4f94-b109-3a62e8c561e7-kube-api-access-pwp4j\") pod \"tuned-k4mlq\" (UID: \"6055a183-9e39-4f94-b109-3a62e8c561e7\") " pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.270640 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.270613    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bw898\" (UniqueName: \"kubernetes.io/projected/212b69da-9153-48f1-94e2-6967b30b699c-kube-api-access-bw898\") pod \"multus-d2vn8\" (UID: \"212b69da-9153-48f1-94e2-6967b30b699c\") " pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.344708 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.344691    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:26.349595 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.349574    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-gbn95"
Jul 21 14:32:26.354062 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:26.354040    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda93f1733_b8cf_40a0_a492_9d6f01457c06.slice/crio-e678a3197f7144e757b0d15b7e24ce2f49d747903df980826b3b14512bbd6ddd WatchSource:0}: Error finding container e678a3197f7144e757b0d15b7e24ce2f49d747903df980826b3b14512bbd6ddd: Status 404 returned error can't find the container with id e678a3197f7144e757b0d15b7e24ce2f49d747903df980826b3b14512bbd6ddd
Jul 21 14:32:26.357698 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.357678    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-j25df"
Jul 21 14:32:26.357886 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:26.357869    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb81ae5a9_ca37_4ddd_be8a_3a732b73eb43.slice/crio-402294a50da27da3750db15c7b4c035067ea05d04602da336e2854dd5c55e51a WatchSource:0}: Error finding container 402294a50da27da3750db15c7b4c035067ea05d04602da336e2854dd5c55e51a: Status 404 returned error can't find the container with id 402294a50da27da3750db15c7b4c035067ea05d04602da336e2854dd5c55e51a
Jul 21 14:32:26.362250 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.362234    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gxsg8"
Jul 21 14:32:26.367115 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.367020    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44xjw"
Jul 21 14:32:26.367537 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:26.367511    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7845eb46_a515_4a7b_9c2b_88649a1c8ed2.slice/crio-6a77e7df43acd31a8ef94f9159bd414738c7959582a1df487bc497dc4a3339a0 WatchSource:0}: Error finding container 6a77e7df43acd31a8ef94f9159bd414738c7959582a1df487bc497dc4a3339a0: Status 404 returned error can't find the container with id 6a77e7df43acd31a8ef94f9159bd414738c7959582a1df487bc497dc4a3339a0
Jul 21 14:32:26.372986 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:26.372964    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37083d35_d10f_4542_871a_a16c12c6bc7f.slice/crio-5964baa2c425f91777e2bb5266cc9aabb91418acba03e8800dd37d4973194c21 WatchSource:0}: Error finding container 5964baa2c425f91777e2bb5266cc9aabb91418acba03e8800dd37d4973194c21: Status 404 returned error can't find the container with id 5964baa2c425f91777e2bb5266cc9aabb91418acba03e8800dd37d4973194c21
Jul 21 14:32:26.373570 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.373556    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vhndm"
Jul 21 14:32:26.375411 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:26.375392    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode131f9f0_09bd_4d0a_b61b_e13d4efda407.slice/crio-a9b821aaeb9bd7802f199b7af28b92dccd29f430083390e916e3785eb9030660 WatchSource:0}: Error finding container a9b821aaeb9bd7802f199b7af28b92dccd29f430083390e916e3785eb9030660: Status 404 returned error can't find the container with id a9b821aaeb9bd7802f199b7af28b92dccd29f430083390e916e3785eb9030660
Jul 21 14:32:26.379353 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.379337    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-k4mlq"
Jul 21 14:32:26.382353 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:26.382338    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod442fea02_6da4_40ad_9f33_d7824c635d4f.slice/crio-eab2722eaee21c41f3f7befb023aa8fd50bb0bf4ed1a782e1ee00b1643ca6aa4 WatchSource:0}: Error finding container eab2722eaee21c41f3f7befb023aa8fd50bb0bf4ed1a782e1ee00b1643ca6aa4: Status 404 returned error can't find the container with id eab2722eaee21c41f3f7befb023aa8fd50bb0bf4ed1a782e1ee00b1643ca6aa4
Jul 21 14:32:26.383380 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.383361    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d2vn8"
Jul 21 14:32:26.387073 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:26.387055    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6055a183_9e39_4f94_b109_3a62e8c561e7.slice/crio-a761df89af258625bd3b32a9aac351bcc868b10140b07b315945c76e023f1e43 WatchSource:0}: Error finding container a761df89af258625bd3b32a9aac351bcc868b10140b07b315945c76e023f1e43: Status 404 returned error can't find the container with id a761df89af258625bd3b32a9aac351bcc868b10140b07b315945c76e023f1e43
Jul 21 14:32:26.389021 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.389003    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8ff6d"
Jul 21 14:32:26.391838 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:26.391821    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod212b69da_9153_48f1_94e2_6967b30b699c.slice/crio-53eb42f56c65b0e42938a8989f872135231d4591439edb52cf2a0ebe303a4fb2 WatchSource:0}: Error finding container 53eb42f56c65b0e42938a8989f872135231d4591439edb52cf2a0ebe303a4fb2: Status 404 returned error can't find the container with id 53eb42f56c65b0e42938a8989f872135231d4591439edb52cf2a0ebe303a4fb2
Jul 21 14:32:26.397239 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:26.397223    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1c20830_6622_4c3e_95b3_0cfb33cc1245.slice/crio-d24b6ac5ab190e08a7f58af2141a7dfdc0f47f46995aca7ac4966813a16b0d0e WatchSource:0}: Error finding container d24b6ac5ab190e08a7f58af2141a7dfdc0f47f46995aca7ac4966813a16b0d0e: Status 404 returned error can't find the container with id d24b6ac5ab190e08a7f58af2141a7dfdc0f47f46995aca7ac4966813a16b0d0e
Jul 21 14:32:26.514052 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.514011    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 14:32:26.664462 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.664427    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13a6c11d-b9ba-4c12-836f-2172b4d168ce-metrics-certs\") pod \"network-metrics-daemon-2zm5j\" (UID: \"13a6c11d-b9ba-4c12-836f-2172b4d168ce\") " pod="openshift-multus/network-metrics-daemon-2zm5j"
Jul 21 14:32:26.664547 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:26.664541    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 14:32:26.664593 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:26.664582    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13a6c11d-b9ba-4c12-836f-2172b4d168ce-metrics-certs podName:13a6c11d-b9ba-4c12-836f-2172b4d168ce nodeName:}" failed. No retries permitted until 2026-07-21 14:32:27.664570325 +0000 UTC m=+4.014313715 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13a6c11d-b9ba-4c12-836f-2172b4d168ce-metrics-certs") pod "network-metrics-daemon-2zm5j" (UID: "13a6c11d-b9ba-4c12-836f-2172b4d168ce") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 14:32:26.765557 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.765502    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80d084f2-ce50-4ce6-9a6f-76e108e0215e-original-pull-secret\") pod \"global-pull-secret-syncer-78qqf\" (UID: \"80d084f2-ce50-4ce6-9a6f-76e108e0215e\") " pod="kube-system/global-pull-secret-syncer-78qqf"
Jul 21 14:32:26.765664 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:26.765559    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl9vm\" (UniqueName: \"kubernetes.io/projected/206dbc0b-6708-45a2-bcd9-50b72fd5e6b5-kube-api-access-wl9vm\") pod \"network-check-target-q7h42\" (UID: \"206dbc0b-6708-45a2-bcd9-50b72fd5e6b5\") " pod="openshift-network-diagnostics/network-check-target-q7h42"
Jul 21 14:32:26.765664 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:26.765651    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 14:32:26.765749 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:26.765666    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 14:32:26.765749 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:26.765682    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 14:32:26.765749 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:26.765693    2579 projected.go:196] Error preparing data for projected volume kube-api-access-wl9vm for pod openshift-network-diagnostics/network-check-target-q7h42: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 14:32:26.765749 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:26.765711    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80d084f2-ce50-4ce6-9a6f-76e108e0215e-original-pull-secret podName:80d084f2-ce50-4ce6-9a6f-76e108e0215e nodeName:}" failed. No retries permitted until 2026-07-21 14:32:27.765691719 +0000 UTC m=+4.115435123 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/80d084f2-ce50-4ce6-9a6f-76e108e0215e-original-pull-secret") pod "global-pull-secret-syncer-78qqf" (UID: "80d084f2-ce50-4ce6-9a6f-76e108e0215e") : object "kube-system"/"original-pull-secret" not registered
Jul 21 14:32:26.765749 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:26.765729    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/206dbc0b-6708-45a2-bcd9-50b72fd5e6b5-kube-api-access-wl9vm podName:206dbc0b-6708-45a2-bcd9-50b72fd5e6b5 nodeName:}" failed. No retries permitted until 2026-07-21 14:32:27.765717773 +0000 UTC m=+4.115461176 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-wl9vm" (UniqueName: "kubernetes.io/projected/206dbc0b-6708-45a2-bcd9-50b72fd5e6b5-kube-api-access-wl9vm") pod "network-check-target-q7h42" (UID: "206dbc0b-6708-45a2-bcd9-50b72fd5e6b5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 14:32:27.091487 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:27.091404    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 14:27:26 +0000 UTC" deadline="2028-02-04 17:56:20.464651837 +0000 UTC"
Jul 21 14:32:27.091487 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:27.091442    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13515h23m53.373215614s"
Jul 21 14:32:27.175871 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:27.175830    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-j25df" event={"ID":"7845eb46-a515-4a7b-9c2b-88649a1c8ed2","Type":"ContainerStarted","Data":"6a77e7df43acd31a8ef94f9159bd414738c7959582a1df487bc497dc4a3339a0"}
Jul 21 14:32:27.177207 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:27.177179    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gbn95" event={"ID":"b81ae5a9-ca37-4ddd-be8a-3a732b73eb43","Type":"ContainerStarted","Data":"402294a50da27da3750db15c7b4c035067ea05d04602da336e2854dd5c55e51a"}
Jul 21 14:32:27.180072 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:27.180049    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8ff6d" event={"ID":"d1c20830-6622-4c3e-95b3-0cfb33cc1245","Type":"ContainerStarted","Data":"d24b6ac5ab190e08a7f58af2141a7dfdc0f47f46995aca7ac4966813a16b0d0e"}
Jul 21 14:32:27.182485 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:27.182459    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d2vn8" event={"ID":"212b69da-9153-48f1-94e2-6967b30b699c","Type":"ContainerStarted","Data":"53eb42f56c65b0e42938a8989f872135231d4591439edb52cf2a0ebe303a4fb2"}
Jul 21 14:32:27.184033 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:27.184012    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44xjw" event={"ID":"e131f9f0-09bd-4d0a-b61b-e13d4efda407","Type":"ContainerStarted","Data":"a9b821aaeb9bd7802f199b7af28b92dccd29f430083390e916e3785eb9030660"}
Jul 21 14:32:27.185484 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:27.185463    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mv82n" event={"ID":"a93f1733-b8cf-40a0-a492-9d6f01457c06","Type":"ContainerStarted","Data":"e678a3197f7144e757b0d15b7e24ce2f49d747903df980826b3b14512bbd6ddd"}
Jul 21 14:32:27.187238 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:27.187216    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-k4mlq" event={"ID":"6055a183-9e39-4f94-b109-3a62e8c561e7","Type":"ContainerStarted","Data":"a761df89af258625bd3b32a9aac351bcc868b10140b07b315945c76e023f1e43"}
Jul 21 14:32:27.189523 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:27.189501    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vhndm" event={"ID":"442fea02-6da4-40ad-9f33-d7824c635d4f","Type":"ContainerStarted","Data":"eab2722eaee21c41f3f7befb023aa8fd50bb0bf4ed1a782e1ee00b1643ca6aa4"}
Jul 21 14:32:27.191057 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:27.191036    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gxsg8" event={"ID":"37083d35-d10f-4542-871a-a16c12c6bc7f","Type":"ContainerStarted","Data":"5964baa2c425f91777e2bb5266cc9aabb91418acba03e8800dd37d4973194c21"}
Jul 21 14:32:27.674186 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:27.673596    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13a6c11d-b9ba-4c12-836f-2172b4d168ce-metrics-certs\") pod \"network-metrics-daemon-2zm5j\" (UID: \"13a6c11d-b9ba-4c12-836f-2172b4d168ce\") " pod="openshift-multus/network-metrics-daemon-2zm5j"
Jul 21 14:32:27.674186 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:27.673776    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 14:32:27.674186 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:27.673843    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13a6c11d-b9ba-4c12-836f-2172b4d168ce-metrics-certs podName:13a6c11d-b9ba-4c12-836f-2172b4d168ce nodeName:}" failed. No retries permitted until 2026-07-21 14:32:29.673823877 +0000 UTC m=+6.023567288 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13a6c11d-b9ba-4c12-836f-2172b4d168ce-metrics-certs") pod "network-metrics-daemon-2zm5j" (UID: "13a6c11d-b9ba-4c12-836f-2172b4d168ce") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 14:32:27.778287 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:27.774437    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80d084f2-ce50-4ce6-9a6f-76e108e0215e-original-pull-secret\") pod \"global-pull-secret-syncer-78qqf\" (UID: \"80d084f2-ce50-4ce6-9a6f-76e108e0215e\") " pod="kube-system/global-pull-secret-syncer-78qqf"
Jul 21 14:32:27.778287 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:27.774529    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl9vm\" (UniqueName: \"kubernetes.io/projected/206dbc0b-6708-45a2-bcd9-50b72fd5e6b5-kube-api-access-wl9vm\") pod \"network-check-target-q7h42\" (UID: \"206dbc0b-6708-45a2-bcd9-50b72fd5e6b5\") " pod="openshift-network-diagnostics/network-check-target-q7h42"
Jul 21 14:32:27.783288 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:27.778661    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 14:32:27.783288 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:27.778750    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80d084f2-ce50-4ce6-9a6f-76e108e0215e-original-pull-secret podName:80d084f2-ce50-4ce6-9a6f-76e108e0215e nodeName:}" failed. No retries permitted until 2026-07-21 14:32:29.778730239 +0000 UTC m=+6.128473646 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/80d084f2-ce50-4ce6-9a6f-76e108e0215e-original-pull-secret") pod "global-pull-secret-syncer-78qqf" (UID: "80d084f2-ce50-4ce6-9a6f-76e108e0215e") : object "kube-system"/"original-pull-secret" not registered
Jul 21 14:32:27.783288 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:27.779312    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 14:32:27.783288 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:27.779335    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 14:32:27.783288 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:27.779348    2579 projected.go:196] Error preparing data for projected volume kube-api-access-wl9vm for pod openshift-network-diagnostics/network-check-target-q7h42: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 14:32:27.783288 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:27.779409    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/206dbc0b-6708-45a2-bcd9-50b72fd5e6b5-kube-api-access-wl9vm podName:206dbc0b-6708-45a2-bcd9-50b72fd5e6b5 nodeName:}" failed. No retries permitted until 2026-07-21 14:32:29.779395013 +0000 UTC m=+6.129138406 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-wl9vm" (UniqueName: "kubernetes.io/projected/206dbc0b-6708-45a2-bcd9-50b72fd5e6b5-kube-api-access-wl9vm") pod "network-check-target-q7h42" (UID: "206dbc0b-6708-45a2-bcd9-50b72fd5e6b5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 14:32:28.092182 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:28.092057    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 14:27:26 +0000 UTC" deadline="2028-05-03 08:24:18.942220014 +0000 UTC"
Jul 21 14:32:28.092182 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:28.092123    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15641h51m50.850101532s"
Jul 21 14:32:28.169341 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:28.169310    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zm5j"
Jul 21 14:32:28.169507 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:28.169452    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zm5j" podUID="13a6c11d-b9ba-4c12-836f-2172b4d168ce"
Jul 21 14:32:28.169570 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:28.169535    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78qqf"
Jul 21 14:32:28.169626 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:28.169611    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78qqf" podUID="80d084f2-ce50-4ce6-9a6f-76e108e0215e"
Jul 21 14:32:28.169677 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:28.169663    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q7h42"
Jul 21 14:32:28.169745 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:28.169725    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q7h42" podUID="206dbc0b-6708-45a2-bcd9-50b72fd5e6b5"
Jul 21 14:32:29.690090 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:29.690043    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13a6c11d-b9ba-4c12-836f-2172b4d168ce-metrics-certs\") pod \"network-metrics-daemon-2zm5j\" (UID: \"13a6c11d-b9ba-4c12-836f-2172b4d168ce\") " pod="openshift-multus/network-metrics-daemon-2zm5j"
Jul 21 14:32:29.690561 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:29.690257    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 14:32:29.690561 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:29.690343    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13a6c11d-b9ba-4c12-836f-2172b4d168ce-metrics-certs podName:13a6c11d-b9ba-4c12-836f-2172b4d168ce nodeName:}" failed. No retries permitted until 2026-07-21 14:32:33.690321942 +0000 UTC m=+10.040065359 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13a6c11d-b9ba-4c12-836f-2172b4d168ce-metrics-certs") pod "network-metrics-daemon-2zm5j" (UID: "13a6c11d-b9ba-4c12-836f-2172b4d168ce") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 14:32:29.791089 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:29.790919    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80d084f2-ce50-4ce6-9a6f-76e108e0215e-original-pull-secret\") pod \"global-pull-secret-syncer-78qqf\" (UID: \"80d084f2-ce50-4ce6-9a6f-76e108e0215e\") " pod="kube-system/global-pull-secret-syncer-78qqf"
Jul 21 14:32:29.791089 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:29.790985    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl9vm\" (UniqueName: \"kubernetes.io/projected/206dbc0b-6708-45a2-bcd9-50b72fd5e6b5-kube-api-access-wl9vm\") pod \"network-check-target-q7h42\" (UID: \"206dbc0b-6708-45a2-bcd9-50b72fd5e6b5\") " pod="openshift-network-diagnostics/network-check-target-q7h42"
Jul 21 14:32:29.791362 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:29.791124    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 14:32:29.791362 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:29.791143    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 14:32:29.791362 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:29.791155    2579 projected.go:196] Error preparing data for projected volume kube-api-access-wl9vm for pod openshift-network-diagnostics/network-check-target-q7h42: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 14:32:29.791362 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:29.791210    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/206dbc0b-6708-45a2-bcd9-50b72fd5e6b5-kube-api-access-wl9vm podName:206dbc0b-6708-45a2-bcd9-50b72fd5e6b5 nodeName:}" failed. No retries permitted until 2026-07-21 14:32:33.791193412 +0000 UTC m=+10.140936819 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-wl9vm" (UniqueName: "kubernetes.io/projected/206dbc0b-6708-45a2-bcd9-50b72fd5e6b5-kube-api-access-wl9vm") pod "network-check-target-q7h42" (UID: "206dbc0b-6708-45a2-bcd9-50b72fd5e6b5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 14:32:29.791362 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:29.791298    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 14:32:29.791362 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:29.791333    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80d084f2-ce50-4ce6-9a6f-76e108e0215e-original-pull-secret podName:80d084f2-ce50-4ce6-9a6f-76e108e0215e nodeName:}" failed. No retries permitted until 2026-07-21 14:32:33.791322029 +0000 UTC m=+10.141065420 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/80d084f2-ce50-4ce6-9a6f-76e108e0215e-original-pull-secret") pod "global-pull-secret-syncer-78qqf" (UID: "80d084f2-ce50-4ce6-9a6f-76e108e0215e") : object "kube-system"/"original-pull-secret" not registered
Jul 21 14:32:30.171805 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:30.171732    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78qqf"
Jul 21 14:32:30.171966 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:30.171855    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78qqf" podUID="80d084f2-ce50-4ce6-9a6f-76e108e0215e"
Jul 21 14:32:30.171966 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:30.171914    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q7h42"
Jul 21 14:32:30.172080 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:30.171980    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q7h42" podUID="206dbc0b-6708-45a2-bcd9-50b72fd5e6b5"
Jul 21 14:32:30.172080 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:30.172025    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zm5j"
Jul 21 14:32:30.172186 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:30.172103    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zm5j" podUID="13a6c11d-b9ba-4c12-836f-2172b4d168ce"
Jul 21 14:32:32.166234 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:32.166201    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zm5j"
Jul 21 14:32:32.166660 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:32.166356    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zm5j" podUID="13a6c11d-b9ba-4c12-836f-2172b4d168ce"
Jul 21 14:32:32.166660 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:32.166403    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78qqf"
Jul 21 14:32:32.166660 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:32.166520    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78qqf" podUID="80d084f2-ce50-4ce6-9a6f-76e108e0215e"
Jul 21 14:32:32.166660 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:32.166565    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q7h42"
Jul 21 14:32:32.166660 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:32.166639    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q7h42" podUID="206dbc0b-6708-45a2-bcd9-50b72fd5e6b5"
Jul 21 14:32:33.722187 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:33.721581    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13a6c11d-b9ba-4c12-836f-2172b4d168ce-metrics-certs\") pod \"network-metrics-daemon-2zm5j\" (UID: \"13a6c11d-b9ba-4c12-836f-2172b4d168ce\") " pod="openshift-multus/network-metrics-daemon-2zm5j"
Jul 21 14:32:33.722187 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:33.721757    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 14:32:33.722187 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:33.721820    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13a6c11d-b9ba-4c12-836f-2172b4d168ce-metrics-certs podName:13a6c11d-b9ba-4c12-836f-2172b4d168ce nodeName:}" failed. No retries permitted until 2026-07-21 14:32:41.721801771 +0000 UTC m=+18.071545181 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13a6c11d-b9ba-4c12-836f-2172b4d168ce-metrics-certs") pod "network-metrics-daemon-2zm5j" (UID: "13a6c11d-b9ba-4c12-836f-2172b4d168ce") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 14:32:33.822919 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:33.822641    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80d084f2-ce50-4ce6-9a6f-76e108e0215e-original-pull-secret\") pod \"global-pull-secret-syncer-78qqf\" (UID: \"80d084f2-ce50-4ce6-9a6f-76e108e0215e\") " pod="kube-system/global-pull-secret-syncer-78qqf"
Jul 21 14:32:33.822919 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:33.822721    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl9vm\" (UniqueName: \"kubernetes.io/projected/206dbc0b-6708-45a2-bcd9-50b72fd5e6b5-kube-api-access-wl9vm\") pod \"network-check-target-q7h42\" (UID: \"206dbc0b-6708-45a2-bcd9-50b72fd5e6b5\") " pod="openshift-network-diagnostics/network-check-target-q7h42"
Jul 21 14:32:33.822919 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:33.822782    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 14:32:33.822919 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:33.822832    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 14:32:33.822919 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:33.822843    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80d084f2-ce50-4ce6-9a6f-76e108e0215e-original-pull-secret podName:80d084f2-ce50-4ce6-9a6f-76e108e0215e nodeName:}" failed. No retries permitted until 2026-07-21 14:32:41.822825261 +0000 UTC m=+18.172568670 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/80d084f2-ce50-4ce6-9a6f-76e108e0215e-original-pull-secret") pod "global-pull-secret-syncer-78qqf" (UID: "80d084f2-ce50-4ce6-9a6f-76e108e0215e") : object "kube-system"/"original-pull-secret" not registered
Jul 21 14:32:33.822919 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:33.822846    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 14:32:33.822919 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:33.822862    2579 projected.go:196] Error preparing data for projected volume kube-api-access-wl9vm for pod openshift-network-diagnostics/network-check-target-q7h42: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 14:32:33.822919 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:33.822894    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/206dbc0b-6708-45a2-bcd9-50b72fd5e6b5-kube-api-access-wl9vm podName:206dbc0b-6708-45a2-bcd9-50b72fd5e6b5 nodeName:}" failed. No retries permitted until 2026-07-21 14:32:41.822884729 +0000 UTC m=+18.172628125 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-wl9vm" (UniqueName: "kubernetes.io/projected/206dbc0b-6708-45a2-bcd9-50b72fd5e6b5-kube-api-access-wl9vm") pod "network-check-target-q7h42" (UID: "206dbc0b-6708-45a2-bcd9-50b72fd5e6b5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 14:32:34.167628 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:34.167444    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q7h42"
Jul 21 14:32:34.167628 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:34.167486    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zm5j"
Jul 21 14:32:34.167628 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:34.167557    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q7h42" podUID="206dbc0b-6708-45a2-bcd9-50b72fd5e6b5"
Jul 21 14:32:34.167881 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:34.167629    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zm5j" podUID="13a6c11d-b9ba-4c12-836f-2172b4d168ce"
Jul 21 14:32:34.167881 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:34.167692    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78qqf"
Jul 21 14:32:34.167881 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:34.167762    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78qqf" podUID="80d084f2-ce50-4ce6-9a6f-76e108e0215e"
Jul 21 14:32:36.166965 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:36.166878    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q7h42"
Jul 21 14:32:36.167488 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:36.167009    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q7h42" podUID="206dbc0b-6708-45a2-bcd9-50b72fd5e6b5"
Jul 21 14:32:36.167488 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:36.167403    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zm5j"
Jul 21 14:32:36.167610 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:36.167517    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zm5j" podUID="13a6c11d-b9ba-4c12-836f-2172b4d168ce"
Jul 21 14:32:36.167669 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:36.167650    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78qqf"
Jul 21 14:32:36.167778 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:36.167747    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78qqf" podUID="80d084f2-ce50-4ce6-9a6f-76e108e0215e"
Jul 21 14:32:38.166883 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:38.166848    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q7h42"
Jul 21 14:32:38.167296 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:38.166859    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78qqf"
Jul 21 14:32:38.167296 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:38.166975    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zm5j"
Jul 21 14:32:38.167296 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:38.166982    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q7h42" podUID="206dbc0b-6708-45a2-bcd9-50b72fd5e6b5"
Jul 21 14:32:38.167296 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:38.167081    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78qqf" podUID="80d084f2-ce50-4ce6-9a6f-76e108e0215e"
Jul 21 14:32:38.167296 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:38.167182    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zm5j" podUID="13a6c11d-b9ba-4c12-836f-2172b4d168ce"
Jul 21 14:32:40.166083 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:40.165886    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zm5j"
Jul 21 14:32:40.166521 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:40.165889    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78qqf"
Jul 21 14:32:40.166521 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:40.166173    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zm5j" podUID="13a6c11d-b9ba-4c12-836f-2172b4d168ce"
Jul 21 14:32:40.166521 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:40.165920    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q7h42"
Jul 21 14:32:40.166521 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:40.166410    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q7h42" podUID="206dbc0b-6708-45a2-bcd9-50b72fd5e6b5"
Jul 21 14:32:40.166521 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:40.166235    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78qqf" podUID="80d084f2-ce50-4ce6-9a6f-76e108e0215e"
Jul 21 14:32:41.788486 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:41.788459    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13a6c11d-b9ba-4c12-836f-2172b4d168ce-metrics-certs\") pod \"network-metrics-daemon-2zm5j\" (UID: \"13a6c11d-b9ba-4c12-836f-2172b4d168ce\") " pod="openshift-multus/network-metrics-daemon-2zm5j"
Jul 21 14:32:41.788905 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:41.788620    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 14:32:41.788905 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:41.788689    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13a6c11d-b9ba-4c12-836f-2172b4d168ce-metrics-certs podName:13a6c11d-b9ba-4c12-836f-2172b4d168ce nodeName:}" failed. No retries permitted until 2026-07-21 14:32:57.788671967 +0000 UTC m=+34.138415356 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13a6c11d-b9ba-4c12-836f-2172b4d168ce-metrics-certs") pod "network-metrics-daemon-2zm5j" (UID: "13a6c11d-b9ba-4c12-836f-2172b4d168ce") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 14:32:41.889757 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:41.889645    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl9vm\" (UniqueName: \"kubernetes.io/projected/206dbc0b-6708-45a2-bcd9-50b72fd5e6b5-kube-api-access-wl9vm\") pod \"network-check-target-q7h42\" (UID: \"206dbc0b-6708-45a2-bcd9-50b72fd5e6b5\") " pod="openshift-network-diagnostics/network-check-target-q7h42"
Jul 21 14:32:41.889757 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:41.889708    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80d084f2-ce50-4ce6-9a6f-76e108e0215e-original-pull-secret\") pod \"global-pull-secret-syncer-78qqf\" (UID: \"80d084f2-ce50-4ce6-9a6f-76e108e0215e\") " pod="kube-system/global-pull-secret-syncer-78qqf"
Jul 21 14:32:41.889965 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:41.889808    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 14:32:41.889965 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:41.889812    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 14:32:41.889965 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:41.889837    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 14:32:41.889965 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:41.889851    2579 projected.go:196] Error preparing data for projected volume kube-api-access-wl9vm for pod openshift-network-diagnostics/network-check-target-q7h42: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 14:32:41.889965 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:41.889857    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80d084f2-ce50-4ce6-9a6f-76e108e0215e-original-pull-secret podName:80d084f2-ce50-4ce6-9a6f-76e108e0215e nodeName:}" failed. No retries permitted until 2026-07-21 14:32:57.889842945 +0000 UTC m=+34.239586335 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/80d084f2-ce50-4ce6-9a6f-76e108e0215e-original-pull-secret") pod "global-pull-secret-syncer-78qqf" (UID: "80d084f2-ce50-4ce6-9a6f-76e108e0215e") : object "kube-system"/"original-pull-secret" not registered
Jul 21 14:32:41.889965 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:41.889890    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/206dbc0b-6708-45a2-bcd9-50b72fd5e6b5-kube-api-access-wl9vm podName:206dbc0b-6708-45a2-bcd9-50b72fd5e6b5 nodeName:}" failed. No retries permitted until 2026-07-21 14:32:57.889878262 +0000 UTC m=+34.239621651 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-wl9vm" (UniqueName: "kubernetes.io/projected/206dbc0b-6708-45a2-bcd9-50b72fd5e6b5-kube-api-access-wl9vm") pod "network-check-target-q7h42" (UID: "206dbc0b-6708-45a2-bcd9-50b72fd5e6b5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 14:32:42.166134 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:42.166056    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zm5j"
Jul 21 14:32:42.166322 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:42.166056    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78qqf"
Jul 21 14:32:42.166322 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:42.166189    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q7h42"
Jul 21 14:32:42.166322 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:42.166190    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zm5j" podUID="13a6c11d-b9ba-4c12-836f-2172b4d168ce"
Jul 21 14:32:42.166322 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:42.166297    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78qqf" podUID="80d084f2-ce50-4ce6-9a6f-76e108e0215e"
Jul 21 14:32:42.166532 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:42.166367    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q7h42" podUID="206dbc0b-6708-45a2-bcd9-50b72fd5e6b5"
Jul 21 14:32:44.167446 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:44.167410    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q7h42"
Jul 21 14:32:44.167870 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:44.167519    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q7h42" podUID="206dbc0b-6708-45a2-bcd9-50b72fd5e6b5"
Jul 21 14:32:44.167870 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:44.167592    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zm5j"
Jul 21 14:32:44.167870 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:44.167710    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zm5j" podUID="13a6c11d-b9ba-4c12-836f-2172b4d168ce"
Jul 21 14:32:44.167870 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:44.167774    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78qqf"
Jul 21 14:32:44.168094 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:44.167881    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78qqf" podUID="80d084f2-ce50-4ce6-9a6f-76e108e0215e"
Jul 21 14:32:46.166723 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:46.166693    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q7h42"
Jul 21 14:32:46.167084 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:46.166693    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zm5j"
Jul 21 14:32:46.167084 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:46.166811    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q7h42" podUID="206dbc0b-6708-45a2-bcd9-50b72fd5e6b5"
Jul 21 14:32:46.167084 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:46.166699    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78qqf"
Jul 21 14:32:46.167084 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:46.166876    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zm5j" podUID="13a6c11d-b9ba-4c12-836f-2172b4d168ce"
Jul 21 14:32:46.167084 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:46.166939    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78qqf" podUID="80d084f2-ce50-4ce6-9a6f-76e108e0215e"
Jul 21 14:32:47.233812 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:47.233527    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-gbn95" event={"ID":"b81ae5a9-ca37-4ddd-be8a-3a732b73eb43","Type":"ContainerStarted","Data":"7f235d919533e3527b6dbd1d1eed8de26cdc758372f4413b092912fe3f332791"}
Jul 21 14:32:47.235524 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:47.235343    2579 generic.go:356] "Generic (PLEG): container finished" podID="d46085a6c06a58d348208916c9b4bb78" containerID="a505fe398ce6ba56143579e0ffb6f0462a864d70a908e9d7eaf7c9a129cfbf8a" exitCode=0
Jul 21 14:32:47.235644 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:47.235389    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.ec2.internal" event={"ID":"d46085a6c06a58d348208916c9b4bb78","Type":"ContainerDied","Data":"a505fe398ce6ba56143579e0ffb6f0462a864d70a908e9d7eaf7c9a129cfbf8a"}
Jul 21 14:32:47.237294 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:47.237252    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d2vn8" event={"ID":"212b69da-9153-48f1-94e2-6967b30b699c","Type":"ContainerStarted","Data":"669fa2df7483a5dec92343e2b7ac60ed6dfbddc4f1b6d297d21cf84b4d3c195e"}
Jul 21 14:32:47.239159 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:47.239138    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44xjw" event={"ID":"e131f9f0-09bd-4d0a-b61b-e13d4efda407","Type":"ContainerStarted","Data":"8b8b10f39f6cfa4b8f102e2f6e8521f5cc704c1eae88de32b50b5a4f06264ad1"}
Jul 21 14:32:47.242519 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:47.242498    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mv82n" event={"ID":"a93f1733-b8cf-40a0-a492-9d6f01457c06","Type":"ContainerStarted","Data":"5d98af1059b0356ac899493776953fb35af919d390b81b8d92ec458d0f55c532"}
Jul 21 14:32:47.242592 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:47.242567    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mv82n" event={"ID":"a93f1733-b8cf-40a0-a492-9d6f01457c06","Type":"ContainerStarted","Data":"a026667882721144d812a9c0be64dffdf742a2636b5f2ec0266841f66f60aae3"}
Jul 21 14:32:47.242656 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:47.242617    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mv82n" event={"ID":"a93f1733-b8cf-40a0-a492-9d6f01457c06","Type":"ContainerStarted","Data":"221a9fa514ac5fe86b27ee9de7f2a7abe29f5f37264a6c4fb3f7d0edc533cbb9"}
Jul 21 14:32:47.242707 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:47.242665    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mv82n" event={"ID":"a93f1733-b8cf-40a0-a492-9d6f01457c06","Type":"ContainerStarted","Data":"b13e78621a92fe424748397d8d2c053ebf5eb1939017454e1582bf4a9b13f560"}
Jul 21 14:32:47.242764 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:47.242709    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mv82n" event={"ID":"a93f1733-b8cf-40a0-a492-9d6f01457c06","Type":"ContainerStarted","Data":"f4dba05d1c320117d1a8b7aad811175de7fc8b41ebbdb993263fa9f8c167ecbe"}
Jul 21 14:32:47.242764 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:47.242757    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mv82n" event={"ID":"a93f1733-b8cf-40a0-a492-9d6f01457c06","Type":"ContainerStarted","Data":"32d65d19d9b111740e7e175d401e753719951ac3bb19ba9cc017c6b4c99597b8"}
Jul 21 14:32:47.243966 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:47.243946    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-83.ec2.internal" event={"ID":"ada9959260df8e5e8a349b59b83b3eed","Type":"ContainerStarted","Data":"f260bfb6cd6dda5691703d1ad8c2684eb6757d021a907746d0e07b2bd60e12b5"}
Jul 21 14:32:47.245603 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:47.245535    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-k4mlq" event={"ID":"6055a183-9e39-4f94-b109-3a62e8c561e7","Type":"ContainerStarted","Data":"8613e6c8fb08c99e0026492ed387a596fd91dd5c78477f392bd351c7c76dcd64"}
Jul 21 14:32:47.247103 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:47.247080    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vhndm" event={"ID":"442fea02-6da4-40ad-9f33-d7824c635d4f","Type":"ContainerStarted","Data":"49711441d898721ffb4eb179c98a3f397becf0f838b35ec0f1c9cf56fe9681e9"}
Jul 21 14:32:47.248497 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:47.248475    2579 generic.go:356] "Generic (PLEG): container finished" podID="37083d35-d10f-4542-871a-a16c12c6bc7f" containerID="71a8ef8cf3278882f52a61bebc7bdc24a6c9dab43e16261dce0b10007018ae6c" exitCode=0
Jul 21 14:32:47.248578 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:47.248540    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gxsg8" event={"ID":"37083d35-d10f-4542-871a-a16c12c6bc7f","Type":"ContainerDied","Data":"71a8ef8cf3278882f52a61bebc7bdc24a6c9dab43e16261dce0b10007018ae6c"}
Jul 21 14:32:47.250113 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:47.250052    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-gbn95" podStartSLOduration=3.427405382 podStartE2EDuration="23.250039389s" totalImagesPullingTime="19.822634007s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 14:32:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:32:47.249370235 +0000 UTC m=+23.599113647" watchObservedRunningTime="2026-07-21 14:32:47.250039389 +0000 UTC m=+23.599782801"
Jul 21 14:32:47.250188 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:47.250127    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-j25df" event={"ID":"7845eb46-a515-4a7b-9c2b-88649a1c8ed2","Type":"ContainerStarted","Data":"83f1e37d05554ff415213bb569c64702118975da52808df465170246d9f3bffb"}
Jul 21 14:32:47.285016 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:47.284971    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-83.ec2.internal" podStartSLOduration=22.28495557 podStartE2EDuration="22.28495557s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 14:32:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:32:47.263734937 +0000 UTC m=+23.613478350" watchObservedRunningTime="2026-07-21 14:32:47.28495557 +0000 UTC m=+23.634698983"
Jul 21 14:32:47.306602 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:47.306561    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-k4mlq" podStartSLOduration=3.482190174 podStartE2EDuration="23.306543457s" totalImagesPullingTime="19.824353283s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 14:32:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:32:47.30642989 +0000 UTC m=+23.656173320" watchObservedRunningTime="2026-07-21 14:32:47.306543457 +0000 UTC m=+23.656286869"
Jul 21 14:32:47.343894 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:47.343854    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vhndm" podStartSLOduration=3.546024938 podStartE2EDuration="23.343836227s" totalImagesPullingTime="19.797811289s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:32:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:32:47.343353776 +0000 UTC m=+23.693097188" watchObservedRunningTime="2026-07-21 14:32:47.343836227 +0000 UTC m=+23.693579637"
Jul 21 14:32:47.361829 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:47.361787    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-d2vn8" podStartSLOduration=3.509696535 podStartE2EDuration="23.361775916s" totalImagesPullingTime="19.852079381s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 14:32:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:32:47.361593502 +0000 UTC m=+23.711336913" watchObservedRunningTime="2026-07-21 14:32:47.361775916 +0000 UTC m=+23.711519341"
Jul 21 14:32:47.377746 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:47.377699    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-j25df" podStartSLOduration=3.56525019 podStartE2EDuration="23.377683471s" totalImagesPullingTime="19.812433281s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 14:32:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:32:47.376936024 +0000 UTC m=+23.726679436" watchObservedRunningTime="2026-07-21 14:32:47.377683471 +0000 UTC m=+23.727426884"
Jul 21 14:32:47.398537 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:47.397242    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vhndm_442fea02-6da4-40ad-9f33-d7824c635d4f/konnectivity-agent/0.log" limit=1024
Jul 21 14:32:47.485282 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:47.485150    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vhndm_442fea02-6da4-40ad-9f33-d7824c635d4f/konnectivity-agent/0.log" limit=1024
Jul 21 14:32:47.585979 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:47.585920    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vhndm_442fea02-6da4-40ad-9f33-d7824c635d4f/konnectivity-agent/0.log" limit=1024
Jul 21 14:32:47.642578 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:47.642544    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vhndm_442fea02-6da4-40ad-9f33-d7824c635d4f/konnectivity-agent/0.log" limit=1024
Jul 21 14:32:48.050587 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:48.050428    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vhndm"
Jul 21 14:32:48.051009 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:48.050990    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vhndm"
Jul 21 14:32:48.137245 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:48.137222    2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 21 14:32:48.165924 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:48.165901    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q7h42"
Jul 21 14:32:48.166027 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:48.166005    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zm5j"
Jul 21 14:32:48.166083 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:48.166010    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q7h42" podUID="206dbc0b-6708-45a2-bcd9-50b72fd5e6b5"
Jul 21 14:32:48.166083 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:48.166062    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78qqf"
Jul 21 14:32:48.166188 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:48.166160    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zm5j" podUID="13a6c11d-b9ba-4c12-836f-2172b4d168ce"
Jul 21 14:32:48.166298 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:48.166246    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78qqf" podUID="80d084f2-ce50-4ce6-9a6f-76e108e0215e"
Jul 21 14:32:48.253526 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:48.253463    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.ec2.internal" event={"ID":"d46085a6c06a58d348208916c9b4bb78","Type":"ContainerStarted","Data":"59b0ae45aab6a478cb7f1e6eb07a448ead1e13c71829c8c4c9f39634cb890339"}
Jul 21 14:32:48.254813 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:48.254794    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8ff6d" event={"ID":"d1c20830-6622-4c3e-95b3-0cfb33cc1245","Type":"ContainerStarted","Data":"d413d119ae70454be10be54daa90f306efc385e7febe23e43e4cb0d2381e6bec"}
Jul 21 14:32:48.256399 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:48.256370    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44xjw" event={"ID":"e131f9f0-09bd-4d0a-b61b-e13d4efda407","Type":"ContainerStarted","Data":"0dadc8ccc0aa9150c13b500becc3ba4ce32d439d015d1fe5944f6543c11aa4b0"}
Jul 21 14:32:48.256963 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:48.256943    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vhndm"
Jul 21 14:32:48.257221 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:48.257201    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vhndm"
Jul 21 14:32:48.267377 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:48.267329    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-83.ec2.internal" podStartSLOduration=23.267263041 podStartE2EDuration="23.267263041s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 14:32:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:32:48.267063272 +0000 UTC m=+24.616806685" watchObservedRunningTime="2026-07-21 14:32:48.267263041 +0000 UTC m=+24.617006453"
Jul 21 14:32:48.294620 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:48.294573    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8ff6d" podStartSLOduration=4.505323034 podStartE2EDuration="24.294560026s" totalImagesPullingTime="19.789236992s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 14:32:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:32:48.294288814 +0000 UTC m=+24.644032223" watchObservedRunningTime="2026-07-21 14:32:48.294560026 +0000 UTC m=+24.644303438"
Jul 21 14:32:48.326117 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:48.326084    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vhndm_442fea02-6da4-40ad-9f33-d7824c635d4f/konnectivity-agent/0.log" limit=1024
Jul 21 14:32:48.412510 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:48.412478    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vhndm_442fea02-6da4-40ad-9f33-d7824c635d4f/konnectivity-agent/0.log" limit=1024
Jul 21 14:32:49.122878 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:49.122800    2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-21T14:32:48.137239255Z","UUID":"039337af-3e62-4cda-851e-4276454357c2","Handler":null,"Name":"","Endpoint":""}
Jul 21 14:32:49.124790 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:49.124773    2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 21 14:32:49.124895 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:49.124806    2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 21 14:32:49.262938 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:49.262827    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mv82n" event={"ID":"a93f1733-b8cf-40a0-a492-9d6f01457c06","Type":"ContainerStarted","Data":"5a62b10966709d26d15e2b0db30f792e7b25d51a4727a7329de5367fda6d47da"}
Jul 21 14:32:49.415625 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:49.413949    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vhndm_442fea02-6da4-40ad-9f33-d7824c635d4f/konnectivity-agent/0.log" limit=1024
Jul 21 14:32:50.101212 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:50.101176    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vhndm_442fea02-6da4-40ad-9f33-d7824c635d4f/konnectivity-agent/0.log" limit=1024
Jul 21 14:32:50.151755 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:50.151715    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vhndm_442fea02-6da4-40ad-9f33-d7824c635d4f/konnectivity-agent/0.log" limit=1024
Jul 21 14:32:50.166818 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:50.166794    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q7h42"
Jul 21 14:32:50.166982 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:50.166795    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zm5j"
Jul 21 14:32:50.166982 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:50.166913    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q7h42" podUID="206dbc0b-6708-45a2-bcd9-50b72fd5e6b5"
Jul 21 14:32:50.166982 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:50.166956    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78qqf"
Jul 21 14:32:50.167118 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:50.167075    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zm5j" podUID="13a6c11d-b9ba-4c12-836f-2172b4d168ce"
Jul 21 14:32:50.167174 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:50.167154    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78qqf" podUID="80d084f2-ce50-4ce6-9a6f-76e108e0215e"
Jul 21 14:32:50.267769 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:50.267737    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44xjw" event={"ID":"e131f9f0-09bd-4d0a-b61b-e13d4efda407","Type":"ContainerStarted","Data":"72a8afebe4c559bfdfe906301407e2242128eca040345c51086a7537af25eb93"}
Jul 21 14:32:50.285987 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:50.285942    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-44xjw" podStartSLOduration=3.923118887 podStartE2EDuration="26.285928401s" totalImagesPullingTime="22.362809514s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 14:32:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:32:50.285548908 +0000 UTC m=+26.635292343" watchObservedRunningTime="2026-07-21 14:32:50.285928401 +0000 UTC m=+26.635671812"
Jul 21 14:32:51.241285 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:51.241246    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vhndm_442fea02-6da4-40ad-9f33-d7824c635d4f/konnectivity-agent/0.log" limit=1024
Jul 21 14:32:51.300039 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:51.300011    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vhndm_442fea02-6da4-40ad-9f33-d7824c635d4f/konnectivity-agent/0.log" limit=1024
Jul 21 14:32:52.166957 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:52.166787    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q7h42"
Jul 21 14:32:52.167113 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:52.166792    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zm5j"
Jul 21 14:32:52.167113 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:52.167027    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q7h42" podUID="206dbc0b-6708-45a2-bcd9-50b72fd5e6b5"
Jul 21 14:32:52.167113 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:52.166791    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78qqf"
Jul 21 14:32:52.167113 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:52.167103    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zm5j" podUID="13a6c11d-b9ba-4c12-836f-2172b4d168ce"
Jul 21 14:32:52.167321 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:52.167214    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78qqf" podUID="80d084f2-ce50-4ce6-9a6f-76e108e0215e"
Jul 21 14:32:53.274262 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:53.274229    2579 generic.go:356] "Generic (PLEG): container finished" podID="37083d35-d10f-4542-871a-a16c12c6bc7f" containerID="db6010f17796e7a711822abb47fc09d67e88b4aed900b20ea62004878bf3114a" exitCode=0
Jul 21 14:32:53.275116 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:53.274308    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gxsg8" event={"ID":"37083d35-d10f-4542-871a-a16c12c6bc7f","Type":"ContainerDied","Data":"db6010f17796e7a711822abb47fc09d67e88b4aed900b20ea62004878bf3114a"}
Jul 21 14:32:53.278081 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:53.278061    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mv82n" event={"ID":"a93f1733-b8cf-40a0-a492-9d6f01457c06","Type":"ContainerStarted","Data":"acce405515455404477a4f7e4d95daf083484ee337f305899c71097c6d6bb323"}
Jul 21 14:32:53.278306 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:53.278291    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:53.278400 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:53.278331    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:53.294154 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:53.294137    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:53.317961 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:53.317919    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-mv82n" podStartSLOduration=9.423723489 podStartE2EDuration="29.317906699s" totalImagesPullingTime="19.89418321s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 14:32:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:32:53.316229012 +0000 UTC m=+29.665972425" watchObservedRunningTime="2026-07-21 14:32:53.317906699 +0000 UTC m=+29.667650110"
Jul 21 14:32:54.167226 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:54.167041    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q7h42"
Jul 21 14:32:54.167434 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:54.167105    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78qqf"
Jul 21 14:32:54.167434 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:54.167318    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q7h42" podUID="206dbc0b-6708-45a2-bcd9-50b72fd5e6b5"
Jul 21 14:32:54.167434 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:54.167130    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zm5j"
Jul 21 14:32:54.167434 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:54.167375    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78qqf" podUID="80d084f2-ce50-4ce6-9a6f-76e108e0215e"
Jul 21 14:32:54.167563 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:54.167468    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zm5j" podUID="13a6c11d-b9ba-4c12-836f-2172b4d168ce"
Jul 21 14:32:54.280597 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:54.280557    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:54.295707 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:54.295682    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:32:54.549803 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:54.549686    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-q7h42"]
Jul 21 14:32:54.549959 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:54.549849    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q7h42"
Jul 21 14:32:54.549959 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:54.549941    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q7h42" podUID="206dbc0b-6708-45a2-bcd9-50b72fd5e6b5"
Jul 21 14:32:54.553065 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:54.553040    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2zm5j"]
Jul 21 14:32:54.553185 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:54.553155    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zm5j"
Jul 21 14:32:54.553332 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:54.553310    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zm5j" podUID="13a6c11d-b9ba-4c12-836f-2172b4d168ce"
Jul 21 14:32:54.553836 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:54.553806    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-78qqf"]
Jul 21 14:32:54.553920 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:54.553906    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78qqf"
Jul 21 14:32:54.554019 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:54.553999    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78qqf" podUID="80d084f2-ce50-4ce6-9a6f-76e108e0215e"
Jul 21 14:32:56.166736 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:56.166658    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q7h42"
Jul 21 14:32:56.167140 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:56.166666    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zm5j"
Jul 21 14:32:56.167140 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:56.166777    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q7h42" podUID="206dbc0b-6708-45a2-bcd9-50b72fd5e6b5"
Jul 21 14:32:56.167140 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:56.166857    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zm5j" podUID="13a6c11d-b9ba-4c12-836f-2172b4d168ce"
Jul 21 14:32:56.167140 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:56.166895    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78qqf"
Jul 21 14:32:56.167140 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:56.166948    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78qqf" podUID="80d084f2-ce50-4ce6-9a6f-76e108e0215e"
Jul 21 14:32:56.287057 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:56.287026    2579 generic.go:356] "Generic (PLEG): container finished" podID="37083d35-d10f-4542-871a-a16c12c6bc7f" containerID="25dbebaf58c2f8077dc0186fac985f07525da2da932f4892549195c00f6595be" exitCode=0
Jul 21 14:32:56.287231 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:56.287071    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gxsg8" event={"ID":"37083d35-d10f-4542-871a-a16c12c6bc7f","Type":"ContainerDied","Data":"25dbebaf58c2f8077dc0186fac985f07525da2da932f4892549195c00f6595be"}
Jul 21 14:32:57.552402 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:57.552370    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vhndm_442fea02-6da4-40ad-9f33-d7824c635d4f/konnectivity-agent/0.log" limit=1024
Jul 21 14:32:57.652747 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:57.652591    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vhndm_442fea02-6da4-40ad-9f33-d7824c635d4f/konnectivity-agent/0.log" limit=1024
Jul 21 14:32:57.811559 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:57.811490    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13a6c11d-b9ba-4c12-836f-2172b4d168ce-metrics-certs\") pod \"network-metrics-daemon-2zm5j\" (UID: \"13a6c11d-b9ba-4c12-836f-2172b4d168ce\") " pod="openshift-multus/network-metrics-daemon-2zm5j"
Jul 21 14:32:57.811684 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:57.811600    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 14:32:57.811684 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:57.811648    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13a6c11d-b9ba-4c12-836f-2172b4d168ce-metrics-certs podName:13a6c11d-b9ba-4c12-836f-2172b4d168ce nodeName:}" failed. No retries permitted until 2026-07-21 14:33:29.811633403 +0000 UTC m=+66.161376793 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13a6c11d-b9ba-4c12-836f-2172b4d168ce-metrics-certs") pod "network-metrics-daemon-2zm5j" (UID: "13a6c11d-b9ba-4c12-836f-2172b4d168ce") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 14:32:57.838516 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:57.838491    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vhndm_442fea02-6da4-40ad-9f33-d7824c635d4f/konnectivity-agent/0.log" limit=1024
Jul 21 14:32:57.896745 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:57.896722    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vhndm_442fea02-6da4-40ad-9f33-d7824c635d4f/konnectivity-agent/0.log" limit=1024
Jul 21 14:32:57.911695 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:57.911673    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80d084f2-ce50-4ce6-9a6f-76e108e0215e-original-pull-secret\") pod \"global-pull-secret-syncer-78qqf\" (UID: \"80d084f2-ce50-4ce6-9a6f-76e108e0215e\") " pod="kube-system/global-pull-secret-syncer-78qqf"
Jul 21 14:32:57.911781 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:57.911714    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl9vm\" (UniqueName: \"kubernetes.io/projected/206dbc0b-6708-45a2-bcd9-50b72fd5e6b5-kube-api-access-wl9vm\") pod \"network-check-target-q7h42\" (UID: \"206dbc0b-6708-45a2-bcd9-50b72fd5e6b5\") " pod="openshift-network-diagnostics/network-check-target-q7h42"
Jul 21 14:32:57.911822 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:57.911799    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 14:32:57.911822 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:57.911806    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 14:32:57.911822 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:57.911820    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 14:32:57.911912 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:57.911828    2579 projected.go:196] Error preparing data for projected volume kube-api-access-wl9vm for pod openshift-network-diagnostics/network-check-target-q7h42: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 14:32:57.911912 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:57.911854    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80d084f2-ce50-4ce6-9a6f-76e108e0215e-original-pull-secret podName:80d084f2-ce50-4ce6-9a6f-76e108e0215e nodeName:}" failed. No retries permitted until 2026-07-21 14:33:29.911835276 +0000 UTC m=+66.261578687 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/80d084f2-ce50-4ce6-9a6f-76e108e0215e-original-pull-secret") pod "global-pull-secret-syncer-78qqf" (UID: "80d084f2-ce50-4ce6-9a6f-76e108e0215e") : object "kube-system"/"original-pull-secret" not registered
Jul 21 14:32:57.911912 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:57.911869    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/206dbc0b-6708-45a2-bcd9-50b72fd5e6b5-kube-api-access-wl9vm podName:206dbc0b-6708-45a2-bcd9-50b72fd5e6b5 nodeName:}" failed. No retries permitted until 2026-07-21 14:33:29.911862396 +0000 UTC m=+66.261605785 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-wl9vm" (UniqueName: "kubernetes.io/projected/206dbc0b-6708-45a2-bcd9-50b72fd5e6b5-kube-api-access-wl9vm") pod "network-check-target-q7h42" (UID: "206dbc0b-6708-45a2-bcd9-50b72fd5e6b5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 14:32:58.166099 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:58.166078    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q7h42"
Jul 21 14:32:58.166304 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:58.166106    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zm5j"
Jul 21 14:32:58.166304 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:58.166168    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q7h42" podUID="206dbc0b-6708-45a2-bcd9-50b72fd5e6b5"
Jul 21 14:32:58.166304 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:58.166216    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2zm5j" podUID="13a6c11d-b9ba-4c12-836f-2172b4d168ce"
Jul 21 14:32:58.166304 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:58.166264    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78qqf"
Jul 21 14:32:58.166517 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:58.166348    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-78qqf" podUID="80d084f2-ce50-4ce6-9a6f-76e108e0215e"
Jul 21 14:32:58.391447 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:58.391412    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vhndm_442fea02-6da4-40ad-9f33-d7824c635d4f/konnectivity-agent/0.log" limit=1024
Jul 21 14:32:58.526204 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:58.526134    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vhndm_442fea02-6da4-40ad-9f33-d7824c635d4f/konnectivity-agent/0.log" limit=1024
Jul 21 14:32:58.617807 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:58.617779    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vhndm_442fea02-6da4-40ad-9f33-d7824c635d4f/konnectivity-agent/0.log" limit=1024
Jul 21 14:32:58.960328 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:58.960302    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vhndm_442fea02-6da4-40ad-9f33-d7824c635d4f/konnectivity-agent/0.log" limit=1024
Jul 21 14:32:58.968692 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:58.968675    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-83.ec2.internal" event="NodeReady"
Jul 21 14:32:58.968768 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:58.968759    2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 21 14:32:59.004222 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.004195    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7888877dcd-b9b2z"]
Jul 21 14:32:59.006720 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.006703    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7888877dcd-b9b2z"
Jul 21 14:32:59.009303 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.009264    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-gcwlj\""
Jul 21 14:32:59.009404 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.009307    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 21 14:32:59.009404 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.009321    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 21 14:32:59.009404 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.009330    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 21 14:32:59.014573 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.014551    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 21 14:32:59.015519 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.015499    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-sxjrb"]
Jul 21 14:32:59.020129 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.020112    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-sxjrb"
Jul 21 14:32:59.021503 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.021485    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7888877dcd-b9b2z"]
Jul 21 14:32:59.021967 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.021952    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-qpw79"]
Jul 21 14:32:59.022707 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.022514    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 21 14:32:59.022707 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.022554    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-prcnc\""
Jul 21 14:32:59.022707 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.022613    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 21 14:32:59.022892 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.022712    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 21 14:32:59.026092 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.026066    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qpw79"
Jul 21 14:32:59.026581 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.026562    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-sxjrb"]
Jul 21 14:32:59.028338 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.028321    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 21 14:32:59.028522 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.028507    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 21 14:32:59.028606 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.028539    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 21 14:32:59.028606 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.028590    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wkr9m\""
Jul 21 14:32:59.028606 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.028603    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 21 14:32:59.037816 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.037800    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-s5kpj"]
Jul 21 14:32:59.047778 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.047759    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qpw79"]
Jul 21 14:32:59.047854 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.047835    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-s5kpj"
Jul 21 14:32:59.050145 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.049994    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 21 14:32:59.050145 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.050076    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 21 14:32:59.050346 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.050206    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-t2qjq\""
Jul 21 14:32:59.050973 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.050957    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-s5kpj"]
Jul 21 14:32:59.120863 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.120845    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9bdt\" (UniqueName: \"kubernetes.io/projected/f7f9d541-2a52-4303-819e-858ccc2fc64f-kube-api-access-v9bdt\") pod \"image-registry-7888877dcd-b9b2z\" (UID: \"f7f9d541-2a52-4303-819e-858ccc2fc64f\") " pod="openshift-image-registry/image-registry-7888877dcd-b9b2z"
Jul 21 14:32:59.120969 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.120868    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e5ed87de-30a9-4804-a29c-f60695debb7a-data-volume\") pod \"insights-runtime-extractor-qpw79\" (UID: \"e5ed87de-30a9-4804-a29c-f60695debb7a\") " pod="openshift-insights/insights-runtime-extractor-qpw79"
Jul 21 14:32:59.120969 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.120900    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f7f9d541-2a52-4303-819e-858ccc2fc64f-registry-tls\") pod \"image-registry-7888877dcd-b9b2z\" (UID: \"f7f9d541-2a52-4303-819e-858ccc2fc64f\") " pod="openshift-image-registry/image-registry-7888877dcd-b9b2z"
Jul 21 14:32:59.120969 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.120916    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f7f9d541-2a52-4303-819e-858ccc2fc64f-bound-sa-token\") pod \"image-registry-7888877dcd-b9b2z\" (UID: \"f7f9d541-2a52-4303-819e-858ccc2fc64f\") " pod="openshift-image-registry/image-registry-7888877dcd-b9b2z"
Jul 21 14:32:59.121065 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.120984    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f7f9d541-2a52-4303-819e-858ccc2fc64f-image-registry-private-configuration\") pod \"image-registry-7888877dcd-b9b2z\" (UID: \"f7f9d541-2a52-4303-819e-858ccc2fc64f\") " pod="openshift-image-registry/image-registry-7888877dcd-b9b2z"
Jul 21 14:32:59.121065 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.121006    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f7f9d541-2a52-4303-819e-858ccc2fc64f-ca-trust-extracted\") pod \"image-registry-7888877dcd-b9b2z\" (UID: \"f7f9d541-2a52-4303-819e-858ccc2fc64f\") " pod="openshift-image-registry/image-registry-7888877dcd-b9b2z"
Jul 21 14:32:59.121065 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.121025    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e5ed87de-30a9-4804-a29c-f60695debb7a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qpw79\" (UID: \"e5ed87de-30a9-4804-a29c-f60695debb7a\") " pod="openshift-insights/insights-runtime-extractor-qpw79"
Jul 21 14:32:59.121065 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.121058    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f7f9d541-2a52-4303-819e-858ccc2fc64f-trusted-ca\") pod \"image-registry-7888877dcd-b9b2z\" (UID: \"f7f9d541-2a52-4303-819e-858ccc2fc64f\") " pod="openshift-image-registry/image-registry-7888877dcd-b9b2z"
Jul 21 14:32:59.121180 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.121072    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aca305cc-e88b-477b-828f-371769c6bb26-cert\") pod \"ingress-canary-sxjrb\" (UID: \"aca305cc-e88b-477b-828f-371769c6bb26\") " pod="openshift-ingress-canary/ingress-canary-sxjrb"
Jul 21 14:32:59.121180 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.121105    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e5ed87de-30a9-4804-a29c-f60695debb7a-crio-socket\") pod \"insights-runtime-extractor-qpw79\" (UID: \"e5ed87de-30a9-4804-a29c-f60695debb7a\") " pod="openshift-insights/insights-runtime-extractor-qpw79"
Jul 21 14:32:59.121180 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.121120    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dnzc\" (UniqueName: \"kubernetes.io/projected/e5ed87de-30a9-4804-a29c-f60695debb7a-kube-api-access-8dnzc\") pod \"insights-runtime-extractor-qpw79\" (UID: \"e5ed87de-30a9-4804-a29c-f60695debb7a\") " pod="openshift-insights/insights-runtime-extractor-qpw79"
Jul 21 14:32:59.121180 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.121141    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e5ed87de-30a9-4804-a29c-f60695debb7a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qpw79\" (UID: \"e5ed87de-30a9-4804-a29c-f60695debb7a\") " pod="openshift-insights/insights-runtime-extractor-qpw79"
Jul 21 14:32:59.121180 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.121164    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f7f9d541-2a52-4303-819e-858ccc2fc64f-installation-pull-secrets\") pod \"image-registry-7888877dcd-b9b2z\" (UID: \"f7f9d541-2a52-4303-819e-858ccc2fc64f\") " pod="openshift-image-registry/image-registry-7888877dcd-b9b2z"
Jul 21 14:32:59.121352 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.121184    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f7f9d541-2a52-4303-819e-858ccc2fc64f-registry-certificates\") pod \"image-registry-7888877dcd-b9b2z\" (UID: \"f7f9d541-2a52-4303-819e-858ccc2fc64f\") " pod="openshift-image-registry/image-registry-7888877dcd-b9b2z"
Jul 21 14:32:59.121352 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.121204    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbcw9\" (UniqueName: \"kubernetes.io/projected/aca305cc-e88b-477b-828f-371769c6bb26-kube-api-access-jbcw9\") pod \"ingress-canary-sxjrb\" (UID: \"aca305cc-e88b-477b-828f-371769c6bb26\") " pod="openshift-ingress-canary/ingress-canary-sxjrb"
Jul 21 14:32:59.222462 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.222407    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9bdt\" (UniqueName: \"kubernetes.io/projected/f7f9d541-2a52-4303-819e-858ccc2fc64f-kube-api-access-v9bdt\") pod \"image-registry-7888877dcd-b9b2z\" (UID: \"f7f9d541-2a52-4303-819e-858ccc2fc64f\") " pod="openshift-image-registry/image-registry-7888877dcd-b9b2z"
Jul 21 14:32:59.222462 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.222431    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e5ed87de-30a9-4804-a29c-f60695debb7a-data-volume\") pod \"insights-runtime-extractor-qpw79\" (UID: \"e5ed87de-30a9-4804-a29c-f60695debb7a\") " pod="openshift-insights/insights-runtime-extractor-qpw79"
Jul 21 14:32:59.222462 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.222456    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f7f9d541-2a52-4303-819e-858ccc2fc64f-registry-tls\") pod \"image-registry-7888877dcd-b9b2z\" (UID: \"f7f9d541-2a52-4303-819e-858ccc2fc64f\") " pod="openshift-image-registry/image-registry-7888877dcd-b9b2z"
Jul 21 14:32:59.222664 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.222470    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f7f9d541-2a52-4303-819e-858ccc2fc64f-bound-sa-token\") pod \"image-registry-7888877dcd-b9b2z\" (UID: \"f7f9d541-2a52-4303-819e-858ccc2fc64f\") " pod="openshift-image-registry/image-registry-7888877dcd-b9b2z"
Jul 21 14:32:59.222664 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.222489    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7xsq\" (UniqueName: \"kubernetes.io/projected/eee39cdb-3b6f-423b-b442-ef9153eabb26-kube-api-access-x7xsq\") pod \"dns-default-s5kpj\" (UID: \"eee39cdb-3b6f-423b-b442-ef9153eabb26\") " pod="openshift-dns/dns-default-s5kpj"
Jul 21 14:32:59.222664 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:59.222569    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 21 14:32:59.222664 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:59.222587    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7888877dcd-b9b2z: secret "image-registry-tls" not found
Jul 21 14:32:59.222664 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.222594    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f7f9d541-2a52-4303-819e-858ccc2fc64f-image-registry-private-configuration\") pod \"image-registry-7888877dcd-b9b2z\" (UID: \"f7f9d541-2a52-4303-819e-858ccc2fc64f\") " pod="openshift-image-registry/image-registry-7888877dcd-b9b2z"
Jul 21 14:32:59.222664 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.222621    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f7f9d541-2a52-4303-819e-858ccc2fc64f-ca-trust-extracted\") pod \"image-registry-7888877dcd-b9b2z\" (UID: \"f7f9d541-2a52-4303-819e-858ccc2fc64f\") " pod="openshift-image-registry/image-registry-7888877dcd-b9b2z"
Jul 21 14:32:59.222664 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:59.222637    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f7f9d541-2a52-4303-819e-858ccc2fc64f-registry-tls podName:f7f9d541-2a52-4303-819e-858ccc2fc64f nodeName:}" failed. No retries permitted until 2026-07-21 14:32:59.722618454 +0000 UTC m=+36.072361851 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f7f9d541-2a52-4303-819e-858ccc2fc64f-registry-tls") pod "image-registry-7888877dcd-b9b2z" (UID: "f7f9d541-2a52-4303-819e-858ccc2fc64f") : secret "image-registry-tls" not found
Jul 21 14:32:59.222944 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.222669    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e5ed87de-30a9-4804-a29c-f60695debb7a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qpw79\" (UID: \"e5ed87de-30a9-4804-a29c-f60695debb7a\") " pod="openshift-insights/insights-runtime-extractor-qpw79"
Jul 21 14:32:59.222944 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.222700    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eee39cdb-3b6f-423b-b442-ef9153eabb26-metrics-tls\") pod \"dns-default-s5kpj\" (UID: \"eee39cdb-3b6f-423b-b442-ef9153eabb26\") " pod="openshift-dns/dns-default-s5kpj"
Jul 21 14:32:59.222944 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.222726    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f7f9d541-2a52-4303-819e-858ccc2fc64f-trusted-ca\") pod \"image-registry-7888877dcd-b9b2z\" (UID: \"f7f9d541-2a52-4303-819e-858ccc2fc64f\") " pod="openshift-image-registry/image-registry-7888877dcd-b9b2z"
Jul 21 14:32:59.222944 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.222748    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aca305cc-e88b-477b-828f-371769c6bb26-cert\") pod \"ingress-canary-sxjrb\" (UID: \"aca305cc-e88b-477b-828f-371769c6bb26\") " pod="openshift-ingress-canary/ingress-canary-sxjrb"
Jul 21 14:32:59.222944 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.222770    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eee39cdb-3b6f-423b-b442-ef9153eabb26-tmp-dir\") pod \"dns-default-s5kpj\" (UID: \"eee39cdb-3b6f-423b-b442-ef9153eabb26\") " pod="openshift-dns/dns-default-s5kpj"
Jul 21 14:32:59.222944 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:59.222818    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 21 14:32:59.222944 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:59.222903    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e5ed87de-30a9-4804-a29c-f60695debb7a-kube-rbac-proxy-cm podName:e5ed87de-30a9-4804-a29c-f60695debb7a nodeName:}" failed. No retries permitted until 2026-07-21 14:32:59.722886273 +0000 UTC m=+36.072629680 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e5ed87de-30a9-4804-a29c-f60695debb7a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qpw79" (UID: "e5ed87de-30a9-4804-a29c-f60695debb7a") : configmap "kube-rbac-proxy" not found
Jul 21 14:32:59.223306 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.222940    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e5ed87de-30a9-4804-a29c-f60695debb7a-crio-socket\") pod \"insights-runtime-extractor-qpw79\" (UID: \"e5ed87de-30a9-4804-a29c-f60695debb7a\") " pod="openshift-insights/insights-runtime-extractor-qpw79"
Jul 21 14:32:59.223306 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.222979    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dnzc\" (UniqueName: \"kubernetes.io/projected/e5ed87de-30a9-4804-a29c-f60695debb7a-kube-api-access-8dnzc\") pod \"insights-runtime-extractor-qpw79\" (UID: \"e5ed87de-30a9-4804-a29c-f60695debb7a\") " pod="openshift-insights/insights-runtime-extractor-qpw79"
Jul 21 14:32:59.223306 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.223017    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e5ed87de-30a9-4804-a29c-f60695debb7a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qpw79\" (UID: \"e5ed87de-30a9-4804-a29c-f60695debb7a\") " pod="openshift-insights/insights-runtime-extractor-qpw79"
Jul 21 14:32:59.223306 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.223058    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/eee39cdb-3b6f-423b-b442-ef9153eabb26-config-volume\") pod \"dns-default-s5kpj\" (UID: \"eee39cdb-3b6f-423b-b442-ef9153eabb26\") " pod="openshift-dns/dns-default-s5kpj"
Jul 21 14:32:59.223306 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.223063    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e5ed87de-30a9-4804-a29c-f60695debb7a-data-volume\") pod \"insights-runtime-extractor-qpw79\" (UID: \"e5ed87de-30a9-4804-a29c-f60695debb7a\") " pod="openshift-insights/insights-runtime-extractor-qpw79"
Jul 21 14:32:59.223306 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.223087    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f7f9d541-2a52-4303-819e-858ccc2fc64f-installation-pull-secrets\") pod \"image-registry-7888877dcd-b9b2z\" (UID: \"f7f9d541-2a52-4303-819e-858ccc2fc64f\") " pod="openshift-image-registry/image-registry-7888877dcd-b9b2z"
Jul 21 14:32:59.223306 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.223124    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f7f9d541-2a52-4303-819e-858ccc2fc64f-ca-trust-extracted\") pod \"image-registry-7888877dcd-b9b2z\" (UID: \"f7f9d541-2a52-4303-819e-858ccc2fc64f\") " pod="openshift-image-registry/image-registry-7888877dcd-b9b2z"
Jul 21 14:32:59.223306 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:59.223208    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 14:32:59.223306 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:59.223257    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5ed87de-30a9-4804-a29c-f60695debb7a-insights-runtime-extractor-tls podName:e5ed87de-30a9-4804-a29c-f60695debb7a nodeName:}" failed. No retries permitted until 2026-07-21 14:32:59.723244441 +0000 UTC m=+36.072987830 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e5ed87de-30a9-4804-a29c-f60695debb7a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qpw79" (UID: "e5ed87de-30a9-4804-a29c-f60695debb7a") : secret "insights-runtime-extractor-tls" not found
Jul 21 14:32:59.223797 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.223331    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f7f9d541-2a52-4303-819e-858ccc2fc64f-registry-certificates\") pod \"image-registry-7888877dcd-b9b2z\" (UID: \"f7f9d541-2a52-4303-819e-858ccc2fc64f\") " pod="openshift-image-registry/image-registry-7888877dcd-b9b2z"
Jul 21 14:32:59.223797 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.223367    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbcw9\" (UniqueName: \"kubernetes.io/projected/aca305cc-e88b-477b-828f-371769c6bb26-kube-api-access-jbcw9\") pod \"ingress-canary-sxjrb\" (UID: \"aca305cc-e88b-477b-828f-371769c6bb26\") " pod="openshift-ingress-canary/ingress-canary-sxjrb"
Jul 21 14:32:59.223797 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.223435    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e5ed87de-30a9-4804-a29c-f60695debb7a-crio-socket\") pod \"insights-runtime-extractor-qpw79\" (UID: \"e5ed87de-30a9-4804-a29c-f60695debb7a\") " pod="openshift-insights/insights-runtime-extractor-qpw79"
Jul 21 14:32:59.223797 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.223576    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f7f9d541-2a52-4303-819e-858ccc2fc64f-trusted-ca\") pod \"image-registry-7888877dcd-b9b2z\" (UID: \"f7f9d541-2a52-4303-819e-858ccc2fc64f\") " pod="openshift-image-registry/image-registry-7888877dcd-b9b2z"
Jul 21 14:32:59.223797 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.223776    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f7f9d541-2a52-4303-819e-858ccc2fc64f-registry-certificates\") pod \"image-registry-7888877dcd-b9b2z\" (UID: \"f7f9d541-2a52-4303-819e-858ccc2fc64f\") " pod="openshift-image-registry/image-registry-7888877dcd-b9b2z"
Jul 21 14:32:59.226619 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.226600    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f7f9d541-2a52-4303-819e-858ccc2fc64f-installation-pull-secrets\") pod \"image-registry-7888877dcd-b9b2z\" (UID: \"f7f9d541-2a52-4303-819e-858ccc2fc64f\") " pod="openshift-image-registry/image-registry-7888877dcd-b9b2z"
Jul 21 14:32:59.226711 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.226631    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f7f9d541-2a52-4303-819e-858ccc2fc64f-image-registry-private-configuration\") pod \"image-registry-7888877dcd-b9b2z\" (UID: \"f7f9d541-2a52-4303-819e-858ccc2fc64f\") " pod="openshift-image-registry/image-registry-7888877dcd-b9b2z"
Jul 21 14:32:59.226711 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.226680    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/aca305cc-e88b-477b-828f-371769c6bb26-cert\") pod \"ingress-canary-sxjrb\" (UID: \"aca305cc-e88b-477b-828f-371769c6bb26\") " pod="openshift-ingress-canary/ingress-canary-sxjrb"
Jul 21 14:32:59.238045 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.238021    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dnzc\" (UniqueName: \"kubernetes.io/projected/e5ed87de-30a9-4804-a29c-f60695debb7a-kube-api-access-8dnzc\") pod \"insights-runtime-extractor-qpw79\" (UID: \"e5ed87de-30a9-4804-a29c-f60695debb7a\") " pod="openshift-insights/insights-runtime-extractor-qpw79"
Jul 21 14:32:59.239234 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.239205    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9bdt\" (UniqueName: \"kubernetes.io/projected/f7f9d541-2a52-4303-819e-858ccc2fc64f-kube-api-access-v9bdt\") pod \"image-registry-7888877dcd-b9b2z\" (UID: \"f7f9d541-2a52-4303-819e-858ccc2fc64f\") " pod="openshift-image-registry/image-registry-7888877dcd-b9b2z"
Jul 21 14:32:59.239357 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.239342    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbcw9\" (UniqueName: \"kubernetes.io/projected/aca305cc-e88b-477b-828f-371769c6bb26-kube-api-access-jbcw9\") pod \"ingress-canary-sxjrb\" (UID: \"aca305cc-e88b-477b-828f-371769c6bb26\") " pod="openshift-ingress-canary/ingress-canary-sxjrb"
Jul 21 14:32:59.239625 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.239610    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f7f9d541-2a52-4303-819e-858ccc2fc64f-bound-sa-token\") pod \"image-registry-7888877dcd-b9b2z\" (UID: \"f7f9d541-2a52-4303-819e-858ccc2fc64f\") " pod="openshift-image-registry/image-registry-7888877dcd-b9b2z"
Jul 21 14:32:59.290323 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.290299    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gbn95_b81ae5a9-ca37-4ddd-be8a-3a732b73eb43/dns-node-resolver/0.log"
Jul 21 14:32:59.324217 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.324189    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7xsq\" (UniqueName: \"kubernetes.io/projected/eee39cdb-3b6f-423b-b442-ef9153eabb26-kube-api-access-x7xsq\") pod \"dns-default-s5kpj\" (UID: \"eee39cdb-3b6f-423b-b442-ef9153eabb26\") " pod="openshift-dns/dns-default-s5kpj"
Jul 21 14:32:59.324379 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.324229    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eee39cdb-3b6f-423b-b442-ef9153eabb26-metrics-tls\") pod \"dns-default-s5kpj\" (UID: \"eee39cdb-3b6f-423b-b442-ef9153eabb26\") " pod="openshift-dns/dns-default-s5kpj"
Jul 21 14:32:59.324379 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.324254    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eee39cdb-3b6f-423b-b442-ef9153eabb26-tmp-dir\") pod \"dns-default-s5kpj\" (UID: \"eee39cdb-3b6f-423b-b442-ef9153eabb26\") " pod="openshift-dns/dns-default-s5kpj"
Jul 21 14:32:59.324379 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.324317    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/eee39cdb-3b6f-423b-b442-ef9153eabb26-config-volume\") pod \"dns-default-s5kpj\" (UID: \"eee39cdb-3b6f-423b-b442-ef9153eabb26\") " pod="openshift-dns/dns-default-s5kpj"
Jul 21 14:32:59.324539 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:59.324378    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 14:32:59.324539 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:59.324457    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eee39cdb-3b6f-423b-b442-ef9153eabb26-metrics-tls podName:eee39cdb-3b6f-423b-b442-ef9153eabb26 nodeName:}" failed. No retries permitted until 2026-07-21 14:32:59.8244358 +0000 UTC m=+36.174179193 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eee39cdb-3b6f-423b-b442-ef9153eabb26-metrics-tls") pod "dns-default-s5kpj" (UID: "eee39cdb-3b6f-423b-b442-ef9153eabb26") : secret "dns-default-metrics-tls" not found
Jul 21 14:32:59.324727 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.324699    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eee39cdb-3b6f-423b-b442-ef9153eabb26-tmp-dir\") pod \"dns-default-s5kpj\" (UID: \"eee39cdb-3b6f-423b-b442-ef9153eabb26\") " pod="openshift-dns/dns-default-s5kpj"
Jul 21 14:32:59.324989 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.324969    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/eee39cdb-3b6f-423b-b442-ef9153eabb26-config-volume\") pod \"dns-default-s5kpj\" (UID: \"eee39cdb-3b6f-423b-b442-ef9153eabb26\") " pod="openshift-dns/dns-default-s5kpj"
Jul 21 14:32:59.330482 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.330462    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-sxjrb"
Jul 21 14:32:59.332940 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.332917    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7xsq\" (UniqueName: \"kubernetes.io/projected/eee39cdb-3b6f-423b-b442-ef9153eabb26-kube-api-access-x7xsq\") pod \"dns-default-s5kpj\" (UID: \"eee39cdb-3b6f-423b-b442-ef9153eabb26\") " pod="openshift-dns/dns-default-s5kpj"
Jul 21 14:32:59.353915 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.353875    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vhndm_442fea02-6da4-40ad-9f33-d7824c635d4f/konnectivity-agent/0.log" limit=1024
Jul 21 14:32:59.466858 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.466828    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-sxjrb"]
Jul 21 14:32:59.471391 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:32:59.471366    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaca305cc_e88b_477b_828f_371769c6bb26.slice/crio-df58c51f8660ed4c3c33ece3dfb3b0c533949957e100d15c063c8fd842eb503e WatchSource:0}: Error finding container df58c51f8660ed4c3c33ece3dfb3b0c533949957e100d15c063c8fd842eb503e: Status 404 returned error can't find the container with id df58c51f8660ed4c3c33ece3dfb3b0c533949957e100d15c063c8fd842eb503e
Jul 21 14:32:59.728373 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.728110    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e5ed87de-30a9-4804-a29c-f60695debb7a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qpw79\" (UID: \"e5ed87de-30a9-4804-a29c-f60695debb7a\") " pod="openshift-insights/insights-runtime-extractor-qpw79"
Jul 21 14:32:59.728373 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:59.728263    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 14:32:59.728373 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.728346    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f7f9d541-2a52-4303-819e-858ccc2fc64f-registry-tls\") pod \"image-registry-7888877dcd-b9b2z\" (UID: \"f7f9d541-2a52-4303-819e-858ccc2fc64f\") " pod="openshift-image-registry/image-registry-7888877dcd-b9b2z"
Jul 21 14:32:59.728373 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.728372    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e5ed87de-30a9-4804-a29c-f60695debb7a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qpw79\" (UID: \"e5ed87de-30a9-4804-a29c-f60695debb7a\") " pod="openshift-insights/insights-runtime-extractor-qpw79"
Jul 21 14:32:59.729168 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:59.728448    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5ed87de-30a9-4804-a29c-f60695debb7a-insights-runtime-extractor-tls podName:e5ed87de-30a9-4804-a29c-f60695debb7a nodeName:}" failed. No retries permitted until 2026-07-21 14:33:00.728424293 +0000 UTC m=+37.078167686 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e5ed87de-30a9-4804-a29c-f60695debb7a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qpw79" (UID: "e5ed87de-30a9-4804-a29c-f60695debb7a") : secret "insights-runtime-extractor-tls" not found
Jul 21 14:32:59.729168 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:59.728452    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 21 14:32:59.729168 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:59.728474    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 21 14:32:59.729168 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:59.728489    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7888877dcd-b9b2z: secret "image-registry-tls" not found
Jul 21 14:32:59.729168 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:59.728515    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e5ed87de-30a9-4804-a29c-f60695debb7a-kube-rbac-proxy-cm podName:e5ed87de-30a9-4804-a29c-f60695debb7a nodeName:}" failed. No retries permitted until 2026-07-21 14:33:00.728506904 +0000 UTC m=+37.078250294 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e5ed87de-30a9-4804-a29c-f60695debb7a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qpw79" (UID: "e5ed87de-30a9-4804-a29c-f60695debb7a") : configmap "kube-rbac-proxy" not found
Jul 21 14:32:59.729168 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:59.728527    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f7f9d541-2a52-4303-819e-858ccc2fc64f-registry-tls podName:f7f9d541-2a52-4303-819e-858ccc2fc64f nodeName:}" failed. No retries permitted until 2026-07-21 14:33:00.728522048 +0000 UTC m=+37.078265438 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f7f9d541-2a52-4303-819e-858ccc2fc64f-registry-tls") pod "image-registry-7888877dcd-b9b2z" (UID: "f7f9d541-2a52-4303-819e-858ccc2fc64f") : secret "image-registry-tls" not found
Jul 21 14:32:59.829347 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:32:59.829314    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eee39cdb-3b6f-423b-b442-ef9153eabb26-metrics-tls\") pod \"dns-default-s5kpj\" (UID: \"eee39cdb-3b6f-423b-b442-ef9153eabb26\") " pod="openshift-dns/dns-default-s5kpj"
Jul 21 14:32:59.829503 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:59.829445    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 14:32:59.829503 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:32:59.829492    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eee39cdb-3b6f-423b-b442-ef9153eabb26-metrics-tls podName:eee39cdb-3b6f-423b-b442-ef9153eabb26 nodeName:}" failed. No retries permitted until 2026-07-21 14:33:00.829479677 +0000 UTC m=+37.179223068 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eee39cdb-3b6f-423b-b442-ef9153eabb26-metrics-tls") pod "dns-default-s5kpj" (UID: "eee39cdb-3b6f-423b-b442-ef9153eabb26") : secret "dns-default-metrics-tls" not found
Jul 21 14:33:00.095937 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:00.095861    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-j25df_7845eb46-a515-4a7b-9c2b-88649a1c8ed2/node-ca/0.log"
Jul 21 14:33:00.166666 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:00.166638    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zm5j"
Jul 21 14:33:00.166809 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:00.166638    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q7h42"
Jul 21 14:33:00.166809 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:00.166780    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78qqf"
Jul 21 14:33:00.169534 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:00.169357    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 21 14:33:00.169534 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:00.169409    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 21 14:33:00.169534 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:00.169419    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-c99qd\""
Jul 21 14:33:00.169534 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:00.169424    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 21 14:33:00.169534 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:00.169413    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 21 14:33:00.169898 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:00.169880    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cprh8\""
Jul 21 14:33:00.297944 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:00.297909    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-sxjrb" event={"ID":"aca305cc-e88b-477b-828f-371769c6bb26","Type":"ContainerStarted","Data":"df58c51f8660ed4c3c33ece3dfb3b0c533949957e100d15c063c8fd842eb503e"}
Jul 21 14:33:00.300447 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:00.300411    2579 generic.go:356] "Generic (PLEG): container finished" podID="37083d35-d10f-4542-871a-a16c12c6bc7f" containerID="d51dc40f4103f68aa3d700b906a88950f89b11135d8bd5d11008ddb391365a74" exitCode=0
Jul 21 14:33:00.300447 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:00.300449    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gxsg8" event={"ID":"37083d35-d10f-4542-871a-a16c12c6bc7f","Type":"ContainerDied","Data":"d51dc40f4103f68aa3d700b906a88950f89b11135d8bd5d11008ddb391365a74"}
Jul 21 14:33:00.693128 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:00.693091    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-tp2h9"]
Jul 21 14:33:00.695280 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:00.695251    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-tp2h9"
Jul 21 14:33:00.697877 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:00.697859    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 21 14:33:00.698030 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:00.698008    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 21 14:33:00.698124 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:00.698072    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 21 14:33:00.698301 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:00.698264    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 21 14:33:00.698392 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:00.698324    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-t454x\""
Jul 21 14:33:00.702646 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:00.702623    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-tp2h9"]
Jul 21 14:33:00.736684 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:00.736662    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f7f9d541-2a52-4303-819e-858ccc2fc64f-registry-tls\") pod \"image-registry-7888877dcd-b9b2z\" (UID: \"f7f9d541-2a52-4303-819e-858ccc2fc64f\") " pod="openshift-image-registry/image-registry-7888877dcd-b9b2z"
Jul 21 14:33:00.737033 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:00.736695    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e5ed87de-30a9-4804-a29c-f60695debb7a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qpw79\" (UID: \"e5ed87de-30a9-4804-a29c-f60695debb7a\") " pod="openshift-insights/insights-runtime-extractor-qpw79"
Jul 21 14:33:00.737033 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:00.736745    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e5ed87de-30a9-4804-a29c-f60695debb7a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qpw79\" (UID: \"e5ed87de-30a9-4804-a29c-f60695debb7a\") " pod="openshift-insights/insights-runtime-extractor-qpw79"
Jul 21 14:33:00.737033 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:33:00.736818    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 21 14:33:00.737033 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:33:00.736834    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 14:33:00.737033 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:33:00.736843    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 21 14:33:00.737033 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:33:00.736861    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7888877dcd-b9b2z: secret "image-registry-tls" not found
Jul 21 14:33:00.737033 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:33:00.736889    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e5ed87de-30a9-4804-a29c-f60695debb7a-kube-rbac-proxy-cm podName:e5ed87de-30a9-4804-a29c-f60695debb7a nodeName:}" failed. No retries permitted until 2026-07-21 14:33:02.736870355 +0000 UTC m=+39.086613764 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e5ed87de-30a9-4804-a29c-f60695debb7a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qpw79" (UID: "e5ed87de-30a9-4804-a29c-f60695debb7a") : configmap "kube-rbac-proxy" not found
Jul 21 14:33:00.737033 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:33:00.736908    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5ed87de-30a9-4804-a29c-f60695debb7a-insights-runtime-extractor-tls podName:e5ed87de-30a9-4804-a29c-f60695debb7a nodeName:}" failed. No retries permitted until 2026-07-21 14:33:02.736898653 +0000 UTC m=+39.086642046 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e5ed87de-30a9-4804-a29c-f60695debb7a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qpw79" (UID: "e5ed87de-30a9-4804-a29c-f60695debb7a") : secret "insights-runtime-extractor-tls" not found
Jul 21 14:33:00.737033 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:33:00.736921    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f7f9d541-2a52-4303-819e-858ccc2fc64f-registry-tls podName:f7f9d541-2a52-4303-819e-858ccc2fc64f nodeName:}" failed. No retries permitted until 2026-07-21 14:33:02.736915778 +0000 UTC m=+39.086659168 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f7f9d541-2a52-4303-819e-858ccc2fc64f-registry-tls") pod "image-registry-7888877dcd-b9b2z" (UID: "f7f9d541-2a52-4303-819e-858ccc2fc64f") : secret "image-registry-tls" not found
Jul 21 14:33:00.837456 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:00.837430    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eee39cdb-3b6f-423b-b442-ef9153eabb26-metrics-tls\") pod \"dns-default-s5kpj\" (UID: \"eee39cdb-3b6f-423b-b442-ef9153eabb26\") " pod="openshift-dns/dns-default-s5kpj"
Jul 21 14:33:00.837593 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:00.837464    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/541cc8c3-f492-49ed-98b8-64aa260db7aa-signing-cabundle\") pod \"service-ca-58598695f7-tp2h9\" (UID: \"541cc8c3-f492-49ed-98b8-64aa260db7aa\") " pod="openshift-service-ca/service-ca-58598695f7-tp2h9"
Jul 21 14:33:00.837593 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:00.837504    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/541cc8c3-f492-49ed-98b8-64aa260db7aa-signing-key\") pod \"service-ca-58598695f7-tp2h9\" (UID: \"541cc8c3-f492-49ed-98b8-64aa260db7aa\") " pod="openshift-service-ca/service-ca-58598695f7-tp2h9"
Jul 21 14:33:00.837593 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:00.837549    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fm2v\" (UniqueName: \"kubernetes.io/projected/541cc8c3-f492-49ed-98b8-64aa260db7aa-kube-api-access-6fm2v\") pod \"service-ca-58598695f7-tp2h9\" (UID: \"541cc8c3-f492-49ed-98b8-64aa260db7aa\") " pod="openshift-service-ca/service-ca-58598695f7-tp2h9"
Jul 21 14:33:00.837593 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:33:00.837582    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 14:33:00.837759 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:33:00.837686    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eee39cdb-3b6f-423b-b442-ef9153eabb26-metrics-tls podName:eee39cdb-3b6f-423b-b442-ef9153eabb26 nodeName:}" failed. No retries permitted until 2026-07-21 14:33:02.83766778 +0000 UTC m=+39.187411187 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eee39cdb-3b6f-423b-b442-ef9153eabb26-metrics-tls") pod "dns-default-s5kpj" (UID: "eee39cdb-3b6f-423b-b442-ef9153eabb26") : secret "dns-default-metrics-tls" not found
Jul 21 14:33:00.938107 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:00.938076    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/541cc8c3-f492-49ed-98b8-64aa260db7aa-signing-key\") pod \"service-ca-58598695f7-tp2h9\" (UID: \"541cc8c3-f492-49ed-98b8-64aa260db7aa\") " pod="openshift-service-ca/service-ca-58598695f7-tp2h9"
Jul 21 14:33:00.938250 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:00.938129    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fm2v\" (UniqueName: \"kubernetes.io/projected/541cc8c3-f492-49ed-98b8-64aa260db7aa-kube-api-access-6fm2v\") pod \"service-ca-58598695f7-tp2h9\" (UID: \"541cc8c3-f492-49ed-98b8-64aa260db7aa\") " pod="openshift-service-ca/service-ca-58598695f7-tp2h9"
Jul 21 14:33:00.938250 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:00.938240    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/541cc8c3-f492-49ed-98b8-64aa260db7aa-signing-cabundle\") pod \"service-ca-58598695f7-tp2h9\" (UID: \"541cc8c3-f492-49ed-98b8-64aa260db7aa\") " pod="openshift-service-ca/service-ca-58598695f7-tp2h9"
Jul 21 14:33:00.939019 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:00.938972    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/541cc8c3-f492-49ed-98b8-64aa260db7aa-signing-cabundle\") pod \"service-ca-58598695f7-tp2h9\" (UID: \"541cc8c3-f492-49ed-98b8-64aa260db7aa\") " pod="openshift-service-ca/service-ca-58598695f7-tp2h9"
Jul 21 14:33:00.942455 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:00.942430    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/541cc8c3-f492-49ed-98b8-64aa260db7aa-signing-key\") pod \"service-ca-58598695f7-tp2h9\" (UID: \"541cc8c3-f492-49ed-98b8-64aa260db7aa\") " pod="openshift-service-ca/service-ca-58598695f7-tp2h9"
Jul 21 14:33:00.947947 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:00.947872    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fm2v\" (UniqueName: \"kubernetes.io/projected/541cc8c3-f492-49ed-98b8-64aa260db7aa-kube-api-access-6fm2v\") pod \"service-ca-58598695f7-tp2h9\" (UID: \"541cc8c3-f492-49ed-98b8-64aa260db7aa\") " pod="openshift-service-ca/service-ca-58598695f7-tp2h9"
Jul 21 14:33:01.006770 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:01.006748    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-tp2h9"
Jul 21 14:33:01.151211 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:01.151167    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-tp2h9"]
Jul 21 14:33:01.156614 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:33:01.156534    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod541cc8c3_f492_49ed_98b8_64aa260db7aa.slice/crio-d27fc2b4fa60e9239d10dc29f9b1b25ed8551fe656f22eb9362f7bee8a3803e3 WatchSource:0}: Error finding container d27fc2b4fa60e9239d10dc29f9b1b25ed8551fe656f22eb9362f7bee8a3803e3: Status 404 returned error can't find the container with id d27fc2b4fa60e9239d10dc29f9b1b25ed8551fe656f22eb9362f7bee8a3803e3
Jul 21 14:33:01.304354 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:01.304256    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-tp2h9" event={"ID":"541cc8c3-f492-49ed-98b8-64aa260db7aa","Type":"ContainerStarted","Data":"d27fc2b4fa60e9239d10dc29f9b1b25ed8551fe656f22eb9362f7bee8a3803e3"}
Jul 21 14:33:02.756816 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:02.756144    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f7f9d541-2a52-4303-819e-858ccc2fc64f-registry-tls\") pod \"image-registry-7888877dcd-b9b2z\" (UID: \"f7f9d541-2a52-4303-819e-858ccc2fc64f\") " pod="openshift-image-registry/image-registry-7888877dcd-b9b2z"
Jul 21 14:33:02.756816 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:02.756192    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e5ed87de-30a9-4804-a29c-f60695debb7a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qpw79\" (UID: \"e5ed87de-30a9-4804-a29c-f60695debb7a\") " pod="openshift-insights/insights-runtime-extractor-qpw79"
Jul 21 14:33:02.756816 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:02.756247    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e5ed87de-30a9-4804-a29c-f60695debb7a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qpw79\" (UID: \"e5ed87de-30a9-4804-a29c-f60695debb7a\") " pod="openshift-insights/insights-runtime-extractor-qpw79"
Jul 21 14:33:02.756816 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:33:02.756405    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 14:33:02.756816 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:33:02.756469    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5ed87de-30a9-4804-a29c-f60695debb7a-insights-runtime-extractor-tls podName:e5ed87de-30a9-4804-a29c-f60695debb7a nodeName:}" failed. No retries permitted until 2026-07-21 14:33:06.756449272 +0000 UTC m=+43.106192665 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e5ed87de-30a9-4804-a29c-f60695debb7a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qpw79" (UID: "e5ed87de-30a9-4804-a29c-f60695debb7a") : secret "insights-runtime-extractor-tls" not found
Jul 21 14:33:02.756816 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:33:02.756538    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 21 14:33:02.756816 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:33:02.756559    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7888877dcd-b9b2z: secret "image-registry-tls" not found
Jul 21 14:33:02.756816 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:33:02.756620    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f7f9d541-2a52-4303-819e-858ccc2fc64f-registry-tls podName:f7f9d541-2a52-4303-819e-858ccc2fc64f nodeName:}" failed. No retries permitted until 2026-07-21 14:33:06.756604437 +0000 UTC m=+43.106347847 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f7f9d541-2a52-4303-819e-858ccc2fc64f-registry-tls") pod "image-registry-7888877dcd-b9b2z" (UID: "f7f9d541-2a52-4303-819e-858ccc2fc64f") : secret "image-registry-tls" not found
Jul 21 14:33:02.757592 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:02.757048    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e5ed87de-30a9-4804-a29c-f60695debb7a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qpw79\" (UID: \"e5ed87de-30a9-4804-a29c-f60695debb7a\") " pod="openshift-insights/insights-runtime-extractor-qpw79"
Jul 21 14:33:02.857108 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:02.857074    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eee39cdb-3b6f-423b-b442-ef9153eabb26-metrics-tls\") pod \"dns-default-s5kpj\" (UID: \"eee39cdb-3b6f-423b-b442-ef9153eabb26\") " pod="openshift-dns/dns-default-s5kpj"
Jul 21 14:33:02.857289 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:33:02.857219    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 14:33:02.857347 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:33:02.857308    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eee39cdb-3b6f-423b-b442-ef9153eabb26-metrics-tls podName:eee39cdb-3b6f-423b-b442-ef9153eabb26 nodeName:}" failed. No retries permitted until 2026-07-21 14:33:06.85728689 +0000 UTC m=+43.207030298 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eee39cdb-3b6f-423b-b442-ef9153eabb26-metrics-tls") pod "dns-default-s5kpj" (UID: "eee39cdb-3b6f-423b-b442-ef9153eabb26") : secret "dns-default-metrics-tls" not found
Jul 21 14:33:03.311019 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:03.310839    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-sxjrb" event={"ID":"aca305cc-e88b-477b-828f-371769c6bb26","Type":"ContainerStarted","Data":"0a30628a2362634c7e2647ef1d8f267cb3e479ecce133bf2a839d31a043daddd"}
Jul 21 14:33:03.328638 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:03.328542    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-sxjrb" podStartSLOduration=2.293937909 podStartE2EDuration="5.328528033s" totalImagesPullingTime="3.034590124s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:32:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:33:03.327946706 +0000 UTC m=+39.677690143" watchObservedRunningTime="2026-07-21 14:33:03.328528033 +0000 UTC m=+39.678271445"
Jul 21 14:33:04.315512 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:04.315465    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-tp2h9" event={"ID":"541cc8c3-f492-49ed-98b8-64aa260db7aa","Type":"ContainerStarted","Data":"041c0dc44ff9e5ce2a210b9f990be28dfee5fb55b8ffeafab1481086a0565dd4"}
Jul 21 14:33:04.331838 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:04.331782    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-tp2h9" podStartSLOduration=2.2270668000000002 podStartE2EDuration="4.331765854s" totalImagesPullingTime="2.104699054s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:33:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:33:04.331342008 +0000 UTC m=+40.681085424" watchObservedRunningTime="2026-07-21 14:33:04.331765854 +0000 UTC m=+40.681509267"
Jul 21 14:33:06.323108 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:06.323078    2579 generic.go:356] "Generic (PLEG): container finished" podID="37083d35-d10f-4542-871a-a16c12c6bc7f" containerID="344c4017ae5397ee92bdc79b5dc36ac55971559293a91b664514454f65dab7d7" exitCode=0
Jul 21 14:33:06.323504 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:06.323121    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gxsg8" event={"ID":"37083d35-d10f-4542-871a-a16c12c6bc7f","Type":"ContainerDied","Data":"344c4017ae5397ee92bdc79b5dc36ac55971559293a91b664514454f65dab7d7"}
Jul 21 14:33:06.790420 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:06.790395    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e5ed87de-30a9-4804-a29c-f60695debb7a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qpw79\" (UID: \"e5ed87de-30a9-4804-a29c-f60695debb7a\") " pod="openshift-insights/insights-runtime-extractor-qpw79"
Jul 21 14:33:06.790520 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:06.790476    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f7f9d541-2a52-4303-819e-858ccc2fc64f-registry-tls\") pod \"image-registry-7888877dcd-b9b2z\" (UID: \"f7f9d541-2a52-4303-819e-858ccc2fc64f\") " pod="openshift-image-registry/image-registry-7888877dcd-b9b2z"
Jul 21 14:33:06.790565 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:33:06.790551    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 14:33:06.790597 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:33:06.790570    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 21 14:33:06.790597 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:33:06.790580    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7888877dcd-b9b2z: secret "image-registry-tls" not found
Jul 21 14:33:06.790657 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:33:06.790619    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5ed87de-30a9-4804-a29c-f60695debb7a-insights-runtime-extractor-tls podName:e5ed87de-30a9-4804-a29c-f60695debb7a nodeName:}" failed. No retries permitted until 2026-07-21 14:33:14.790599997 +0000 UTC m=+51.140343401 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e5ed87de-30a9-4804-a29c-f60695debb7a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qpw79" (UID: "e5ed87de-30a9-4804-a29c-f60695debb7a") : secret "insights-runtime-extractor-tls" not found
Jul 21 14:33:06.790657 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:33:06.790633    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f7f9d541-2a52-4303-819e-858ccc2fc64f-registry-tls podName:f7f9d541-2a52-4303-819e-858ccc2fc64f nodeName:}" failed. No retries permitted until 2026-07-21 14:33:14.790627214 +0000 UTC m=+51.140370603 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f7f9d541-2a52-4303-819e-858ccc2fc64f-registry-tls") pod "image-registry-7888877dcd-b9b2z" (UID: "f7f9d541-2a52-4303-819e-858ccc2fc64f") : secret "image-registry-tls" not found
Jul 21 14:33:06.891395 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:06.891369    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eee39cdb-3b6f-423b-b442-ef9153eabb26-metrics-tls\") pod \"dns-default-s5kpj\" (UID: \"eee39cdb-3b6f-423b-b442-ef9153eabb26\") " pod="openshift-dns/dns-default-s5kpj"
Jul 21 14:33:06.891498 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:33:06.891485    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 14:33:06.891541 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:33:06.891531    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eee39cdb-3b6f-423b-b442-ef9153eabb26-metrics-tls podName:eee39cdb-3b6f-423b-b442-ef9153eabb26 nodeName:}" failed. No retries permitted until 2026-07-21 14:33:14.89151935 +0000 UTC m=+51.241262741 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eee39cdb-3b6f-423b-b442-ef9153eabb26-metrics-tls") pod "dns-default-s5kpj" (UID: "eee39cdb-3b6f-423b-b442-ef9153eabb26") : secret "dns-default-metrics-tls" not found
Jul 21 14:33:07.328715 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:07.328681    2579 generic.go:356] "Generic (PLEG): container finished" podID="37083d35-d10f-4542-871a-a16c12c6bc7f" containerID="6c04bd068318e5c5cc75692c661e2e6c6834e267b2d1fd4565449eadf7ac0cfd" exitCode=0
Jul 21 14:33:07.329048 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:07.328727    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gxsg8" event={"ID":"37083d35-d10f-4542-871a-a16c12c6bc7f","Type":"ContainerDied","Data":"6c04bd068318e5c5cc75692c661e2e6c6834e267b2d1fd4565449eadf7ac0cfd"}
Jul 21 14:33:08.333758 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:08.333722    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gxsg8" event={"ID":"37083d35-d10f-4542-871a-a16c12c6bc7f","Type":"ContainerStarted","Data":"3abda977be8fbf7a4dd538001bb370addd9d57df7b74d9f7dae6a113361fab3f"}
Jul 21 14:33:08.357151 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:08.357102    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-gxsg8" podStartSLOduration=8.460660284 podStartE2EDuration="44.357087329s" totalImagesPullingTime="35.896427045s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 14:32:24 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:33:08.355427667 +0000 UTC m=+44.705171079" watchObservedRunningTime="2026-07-21 14:33:08.357087329 +0000 UTC m=+44.706830741"
Jul 21 14:33:14.847052 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:14.847022    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f7f9d541-2a52-4303-819e-858ccc2fc64f-registry-tls\") pod \"image-registry-7888877dcd-b9b2z\" (UID: \"f7f9d541-2a52-4303-819e-858ccc2fc64f\") " pod="openshift-image-registry/image-registry-7888877dcd-b9b2z"
Jul 21 14:33:14.847526 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:14.847074    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e5ed87de-30a9-4804-a29c-f60695debb7a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qpw79\" (UID: \"e5ed87de-30a9-4804-a29c-f60695debb7a\") " pod="openshift-insights/insights-runtime-extractor-qpw79"
Jul 21 14:33:14.850419 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:14.850384    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e5ed87de-30a9-4804-a29c-f60695debb7a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qpw79\" (UID: \"e5ed87de-30a9-4804-a29c-f60695debb7a\") " pod="openshift-insights/insights-runtime-extractor-qpw79"
Jul 21 14:33:14.850539 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:14.850388    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f7f9d541-2a52-4303-819e-858ccc2fc64f-registry-tls\") pod \"image-registry-7888877dcd-b9b2z\" (UID: \"f7f9d541-2a52-4303-819e-858ccc2fc64f\") " pod="openshift-image-registry/image-registry-7888877dcd-b9b2z"
Jul 21 14:33:14.916809 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:14.916787    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7888877dcd-b9b2z"
Jul 21 14:33:14.935913 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:14.935894    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qpw79"
Jul 21 14:33:14.947793 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:14.947771    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eee39cdb-3b6f-423b-b442-ef9153eabb26-metrics-tls\") pod \"dns-default-s5kpj\" (UID: \"eee39cdb-3b6f-423b-b442-ef9153eabb26\") " pod="openshift-dns/dns-default-s5kpj"
Jul 21 14:33:14.949739 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:14.949721    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eee39cdb-3b6f-423b-b442-ef9153eabb26-metrics-tls\") pod \"dns-default-s5kpj\" (UID: \"eee39cdb-3b6f-423b-b442-ef9153eabb26\") " pod="openshift-dns/dns-default-s5kpj"
Jul 21 14:33:14.957028 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:14.957012    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-s5kpj"
Jul 21 14:33:15.075796 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:15.075767    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7888877dcd-b9b2z"]
Jul 21 14:33:15.079156 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:33:15.079127    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7f9d541_2a52_4303_819e_858ccc2fc64f.slice/crio-866bc62cd615836d873fe750ab734f164b19f540e6392627b1a1a0251df5275a WatchSource:0}: Error finding container 866bc62cd615836d873fe750ab734f164b19f540e6392627b1a1a0251df5275a: Status 404 returned error can't find the container with id 866bc62cd615836d873fe750ab734f164b19f540e6392627b1a1a0251df5275a
Jul 21 14:33:15.092724 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:15.092701    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qpw79"]
Jul 21 14:33:15.095582 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:33:15.095560    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5ed87de_30a9_4804_a29c_f60695debb7a.slice/crio-e9215f640a806c494c9d66097101f36727038957badeeda7f27c7d3bc2a9e8b5 WatchSource:0}: Error finding container e9215f640a806c494c9d66097101f36727038957badeeda7f27c7d3bc2a9e8b5: Status 404 returned error can't find the container with id e9215f640a806c494c9d66097101f36727038957badeeda7f27c7d3bc2a9e8b5
Jul 21 14:33:15.116124 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:15.116101    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-s5kpj"]
Jul 21 14:33:15.119385 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:33:15.119358    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeee39cdb_3b6f_423b_b442_ef9153eabb26.slice/crio-9e010aeb9aa1eb429290d10e79649dcf5375f4ef0dbf1a7d343b87abb4f78f04 WatchSource:0}: Error finding container 9e010aeb9aa1eb429290d10e79649dcf5375f4ef0dbf1a7d343b87abb4f78f04: Status 404 returned error can't find the container with id 9e010aeb9aa1eb429290d10e79649dcf5375f4ef0dbf1a7d343b87abb4f78f04
Jul 21 14:33:15.350839 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:15.350765    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qpw79" event={"ID":"e5ed87de-30a9-4804-a29c-f60695debb7a","Type":"ContainerStarted","Data":"e9215f640a806c494c9d66097101f36727038957badeeda7f27c7d3bc2a9e8b5"}
Jul 21 14:33:15.352148 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:15.352115    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7888877dcd-b9b2z" event={"ID":"f7f9d541-2a52-4303-819e-858ccc2fc64f","Type":"ContainerStarted","Data":"b7785f008f01dd7d5c9b07142194bffe6ebb46fcfe2c219568b1bd28b5b88563"}
Jul 21 14:33:15.352326 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:15.352163    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7888877dcd-b9b2z" event={"ID":"f7f9d541-2a52-4303-819e-858ccc2fc64f","Type":"ContainerStarted","Data":"866bc62cd615836d873fe750ab734f164b19f540e6392627b1a1a0251df5275a"}
Jul 21 14:33:15.352326 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:15.352190    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7888877dcd-b9b2z"
Jul 21 14:33:15.353307 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:15.353283    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-s5kpj" event={"ID":"eee39cdb-3b6f-423b-b442-ef9153eabb26","Type":"ContainerStarted","Data":"9e010aeb9aa1eb429290d10e79649dcf5375f4ef0dbf1a7d343b87abb4f78f04"}
Jul 21 14:33:15.371112 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:15.371068    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7888877dcd-b9b2z" podStartSLOduration=33.371045271 podStartE2EDuration="33.371045271s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:32:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:33:15.370236127 +0000 UTC m=+51.719979539" watchObservedRunningTime="2026-07-21 14:33:15.371045271 +0000 UTC m=+51.720798256"
Jul 21 14:33:18.363491 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:18.363416    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-s5kpj" event={"ID":"eee39cdb-3b6f-423b-b442-ef9153eabb26","Type":"ContainerStarted","Data":"a696d7d54788119ac6ea83bc3d04f2165c5aea6718df765599a8e3086faf4d5c"}
Jul 21 14:33:18.363491 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:18.363467    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-s5kpj" event={"ID":"eee39cdb-3b6f-423b-b442-ef9153eabb26","Type":"ContainerStarted","Data":"1d0966e5f010c8013ff6d1acfe1a75dc0771d2ecc3809c6ad256c1e030359d6c"}
Jul 21 14:33:18.363948 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:18.363552    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-s5kpj"
Jul 21 14:33:18.380801 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:18.380755    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-s5kpj" podStartSLOduration=16.775605382 podStartE2EDuration="19.380741936s" totalImagesPullingTime="2.605136554s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:32:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:33:18.379674503 +0000 UTC m=+54.729417937" watchObservedRunningTime="2026-07-21 14:33:18.380741936 +0000 UTC m=+54.730485347"
Jul 21 14:33:21.916606 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:21.916566    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7888877dcd-b9b2z"]
Jul 21 14:33:21.948732 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:21.948703    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-74df7f78d8-tc8sc"]
Jul 21 14:33:21.950790 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:21.950770    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck"]
Jul 21 14:33:21.950934 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:21.950912    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74df7f78d8-tc8sc"
Jul 21 14:33:21.952780 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:21.952759    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck"
Jul 21 14:33:21.953532 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:21.953507    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 21 14:33:21.956223 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:21.956191    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 21 14:33:21.956334 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:21.956222    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 21 14:33:21.956452 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:21.956345    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 21 14:33:21.956452 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:21.956387    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 21 14:33:21.956695 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:21.956677    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 21 14:33:21.956769 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:21.956717    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 21 14:33:21.956825 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:21.956722    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 21 14:33:21.976469 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:21.976442    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-74df7f78d8-tc8sc"]
Jul 21 14:33:21.977185 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:21.977166    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck"]
Jul 21 14:33:22.001420 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.001397    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrcn8\" (UniqueName: \"kubernetes.io/projected/388d14d1-066a-4836-90ec-e19acd84741c-kube-api-access-vrcn8\") pod \"klusterlet-addon-workmgr-74df7f78d8-tc8sc\" (UID: \"388d14d1-066a-4836-90ec-e19acd84741c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74df7f78d8-tc8sc"
Jul 21 14:33:22.001506 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.001424    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/388d14d1-066a-4836-90ec-e19acd84741c-tmp\") pod \"klusterlet-addon-workmgr-74df7f78d8-tc8sc\" (UID: \"388d14d1-066a-4836-90ec-e19acd84741c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74df7f78d8-tc8sc"
Jul 21 14:33:22.001506 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.001460    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/308b65b5-da2a-456e-b554-69cdc6462a47-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck\" (UID: \"308b65b5-da2a-456e-b554-69cdc6462a47\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck"
Jul 21 14:33:22.001605 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.001563    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/388d14d1-066a-4836-90ec-e19acd84741c-klusterlet-config\") pod \"klusterlet-addon-workmgr-74df7f78d8-tc8sc\" (UID: \"388d14d1-066a-4836-90ec-e19acd84741c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74df7f78d8-tc8sc"
Jul 21 14:33:22.001605 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.001592    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/308b65b5-da2a-456e-b554-69cdc6462a47-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck\" (UID: \"308b65b5-da2a-456e-b554-69cdc6462a47\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck"
Jul 21 14:33:22.001677 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.001630    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/308b65b5-da2a-456e-b554-69cdc6462a47-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck\" (UID: \"308b65b5-da2a-456e-b554-69cdc6462a47\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck"
Jul 21 14:33:22.001732 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.001673    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxmnz\" (UniqueName: \"kubernetes.io/projected/308b65b5-da2a-456e-b554-69cdc6462a47-kube-api-access-sxmnz\") pod \"cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck\" (UID: \"308b65b5-da2a-456e-b554-69cdc6462a47\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck"
Jul 21 14:33:22.001773 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.001748    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/308b65b5-da2a-456e-b554-69cdc6462a47-hub\") pod \"cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck\" (UID: \"308b65b5-da2a-456e-b554-69cdc6462a47\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck"
Jul 21 14:33:22.001804 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.001777    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/308b65b5-da2a-456e-b554-69cdc6462a47-ca\") pod \"cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck\" (UID: \"308b65b5-da2a-456e-b554-69cdc6462a47\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck"
Jul 21 14:33:22.048899 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.048871    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6fd455ff8b-m2b7t"]
Jul 21 14:33:22.050787 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.050770    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6fd455ff8b-m2b7t"
Jul 21 14:33:22.077448 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.077420    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6fd455ff8b-m2b7t"]
Jul 21 14:33:22.102948 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.102916    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/308b65b5-da2a-456e-b554-69cdc6462a47-hub\") pod \"cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck\" (UID: \"308b65b5-da2a-456e-b554-69cdc6462a47\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck"
Jul 21 14:33:22.103051 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.102956    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/308b65b5-da2a-456e-b554-69cdc6462a47-ca\") pod \"cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck\" (UID: \"308b65b5-da2a-456e-b554-69cdc6462a47\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck"
Jul 21 14:33:22.103051 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.102984    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a9bde86f-cc13-46cf-a337-9fc75c370062-bound-sa-token\") pod \"image-registry-6fd455ff8b-m2b7t\" (UID: \"a9bde86f-cc13-46cf-a337-9fc75c370062\") " pod="openshift-image-registry/image-registry-6fd455ff8b-m2b7t"
Jul 21 14:33:22.103051 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.103010    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a9bde86f-cc13-46cf-a337-9fc75c370062-image-registry-private-configuration\") pod \"image-registry-6fd455ff8b-m2b7t\" (UID: \"a9bde86f-cc13-46cf-a337-9fc75c370062\") " pod="openshift-image-registry/image-registry-6fd455ff8b-m2b7t"
Jul 21 14:33:22.103051 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.103038    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a9bde86f-cc13-46cf-a337-9fc75c370062-registry-tls\") pod \"image-registry-6fd455ff8b-m2b7t\" (UID: \"a9bde86f-cc13-46cf-a337-9fc75c370062\") " pod="openshift-image-registry/image-registry-6fd455ff8b-m2b7t"
Jul 21 14:33:22.103254 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.103097    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a9bde86f-cc13-46cf-a337-9fc75c370062-ca-trust-extracted\") pod \"image-registry-6fd455ff8b-m2b7t\" (UID: \"a9bde86f-cc13-46cf-a337-9fc75c370062\") " pod="openshift-image-registry/image-registry-6fd455ff8b-m2b7t"
Jul 21 14:33:22.103254 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.103139    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrcn8\" (UniqueName: \"kubernetes.io/projected/388d14d1-066a-4836-90ec-e19acd84741c-kube-api-access-vrcn8\") pod \"klusterlet-addon-workmgr-74df7f78d8-tc8sc\" (UID: \"388d14d1-066a-4836-90ec-e19acd84741c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74df7f78d8-tc8sc"
Jul 21 14:33:22.103254 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.103176    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/388d14d1-066a-4836-90ec-e19acd84741c-tmp\") pod \"klusterlet-addon-workmgr-74df7f78d8-tc8sc\" (UID: \"388d14d1-066a-4836-90ec-e19acd84741c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74df7f78d8-tc8sc"
Jul 21 14:33:22.103254 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.103220    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/308b65b5-da2a-456e-b554-69cdc6462a47-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck\" (UID: \"308b65b5-da2a-456e-b554-69cdc6462a47\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck"
Jul 21 14:33:22.103254 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.103247    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a9bde86f-cc13-46cf-a337-9fc75c370062-trusted-ca\") pod \"image-registry-6fd455ff8b-m2b7t\" (UID: \"a9bde86f-cc13-46cf-a337-9fc75c370062\") " pod="openshift-image-registry/image-registry-6fd455ff8b-m2b7t"
Jul 21 14:33:22.103526 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.103330    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/388d14d1-066a-4836-90ec-e19acd84741c-klusterlet-config\") pod \"klusterlet-addon-workmgr-74df7f78d8-tc8sc\" (UID: \"388d14d1-066a-4836-90ec-e19acd84741c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74df7f78d8-tc8sc"
Jul 21 14:33:22.103526 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.103358    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/308b65b5-da2a-456e-b554-69cdc6462a47-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck\" (UID: \"308b65b5-da2a-456e-b554-69cdc6462a47\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck"
Jul 21 14:33:22.103526 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.103403    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/308b65b5-da2a-456e-b554-69cdc6462a47-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck\" (UID: \"308b65b5-da2a-456e-b554-69cdc6462a47\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck"
Jul 21 14:33:22.103526 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.103430    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxmnz\" (UniqueName: \"kubernetes.io/projected/308b65b5-da2a-456e-b554-69cdc6462a47-kube-api-access-sxmnz\") pod \"cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck\" (UID: \"308b65b5-da2a-456e-b554-69cdc6462a47\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck"
Jul 21 14:33:22.103526 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.103457    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a9bde86f-cc13-46cf-a337-9fc75c370062-registry-certificates\") pod \"image-registry-6fd455ff8b-m2b7t\" (UID: \"a9bde86f-cc13-46cf-a337-9fc75c370062\") " pod="openshift-image-registry/image-registry-6fd455ff8b-m2b7t"
Jul 21 14:33:22.103526 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.103481    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mlkx\" (UniqueName: \"kubernetes.io/projected/a9bde86f-cc13-46cf-a337-9fc75c370062-kube-api-access-8mlkx\") pod \"image-registry-6fd455ff8b-m2b7t\" (UID: \"a9bde86f-cc13-46cf-a337-9fc75c370062\") " pod="openshift-image-registry/image-registry-6fd455ff8b-m2b7t"
Jul 21 14:33:22.103526 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.103513    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a9bde86f-cc13-46cf-a337-9fc75c370062-installation-pull-secrets\") pod \"image-registry-6fd455ff8b-m2b7t\" (UID: \"a9bde86f-cc13-46cf-a337-9fc75c370062\") " pod="openshift-image-registry/image-registry-6fd455ff8b-m2b7t"
Jul 21 14:33:22.103916 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.103606    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/388d14d1-066a-4836-90ec-e19acd84741c-tmp\") pod \"klusterlet-addon-workmgr-74df7f78d8-tc8sc\" (UID: \"388d14d1-066a-4836-90ec-e19acd84741c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74df7f78d8-tc8sc"
Jul 21 14:33:22.104235 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.104192    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/308b65b5-da2a-456e-b554-69cdc6462a47-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck\" (UID: \"308b65b5-da2a-456e-b554-69cdc6462a47\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck"
Jul 21 14:33:22.106256 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.106231    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/308b65b5-da2a-456e-b554-69cdc6462a47-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck\" (UID: \"308b65b5-da2a-456e-b554-69cdc6462a47\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck"
Jul 21 14:33:22.106378 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.106243    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/388d14d1-066a-4836-90ec-e19acd84741c-klusterlet-config\") pod \"klusterlet-addon-workmgr-74df7f78d8-tc8sc\" (UID: \"388d14d1-066a-4836-90ec-e19acd84741c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74df7f78d8-tc8sc"
Jul 21 14:33:22.106553 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.106518    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/308b65b5-da2a-456e-b554-69cdc6462a47-ca\") pod \"cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck\" (UID: \"308b65b5-da2a-456e-b554-69cdc6462a47\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck"
Jul 21 14:33:22.106631 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.106619    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/308b65b5-da2a-456e-b554-69cdc6462a47-hub\") pod \"cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck\" (UID: \"308b65b5-da2a-456e-b554-69cdc6462a47\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck"
Jul 21 14:33:22.106724 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.106704    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/308b65b5-da2a-456e-b554-69cdc6462a47-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck\" (UID: \"308b65b5-da2a-456e-b554-69cdc6462a47\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck"
Jul 21 14:33:22.127627 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.127604    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrcn8\" (UniqueName: \"kubernetes.io/projected/388d14d1-066a-4836-90ec-e19acd84741c-kube-api-access-vrcn8\") pod \"klusterlet-addon-workmgr-74df7f78d8-tc8sc\" (UID: \"388d14d1-066a-4836-90ec-e19acd84741c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74df7f78d8-tc8sc"
Jul 21 14:33:22.129982 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.129961    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxmnz\" (UniqueName: \"kubernetes.io/projected/308b65b5-da2a-456e-b554-69cdc6462a47-kube-api-access-sxmnz\") pod \"cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck\" (UID: \"308b65b5-da2a-456e-b554-69cdc6462a47\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck"
Jul 21 14:33:22.204747 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.204675    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a9bde86f-cc13-46cf-a337-9fc75c370062-registry-certificates\") pod \"image-registry-6fd455ff8b-m2b7t\" (UID: \"a9bde86f-cc13-46cf-a337-9fc75c370062\") " pod="openshift-image-registry/image-registry-6fd455ff8b-m2b7t"
Jul 21 14:33:22.204747 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.204712    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mlkx\" (UniqueName: \"kubernetes.io/projected/a9bde86f-cc13-46cf-a337-9fc75c370062-kube-api-access-8mlkx\") pod \"image-registry-6fd455ff8b-m2b7t\" (UID: \"a9bde86f-cc13-46cf-a337-9fc75c370062\") " pod="openshift-image-registry/image-registry-6fd455ff8b-m2b7t"
Jul 21 14:33:22.204747 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.204742    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a9bde86f-cc13-46cf-a337-9fc75c370062-installation-pull-secrets\") pod \"image-registry-6fd455ff8b-m2b7t\" (UID: \"a9bde86f-cc13-46cf-a337-9fc75c370062\") " pod="openshift-image-registry/image-registry-6fd455ff8b-m2b7t"
Jul 21 14:33:22.204975 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.204946    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a9bde86f-cc13-46cf-a337-9fc75c370062-bound-sa-token\") pod \"image-registry-6fd455ff8b-m2b7t\" (UID: \"a9bde86f-cc13-46cf-a337-9fc75c370062\") " pod="openshift-image-registry/image-registry-6fd455ff8b-m2b7t"
Jul 21 14:33:22.205034 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.204981    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a9bde86f-cc13-46cf-a337-9fc75c370062-image-registry-private-configuration\") pod \"image-registry-6fd455ff8b-m2b7t\" (UID: \"a9bde86f-cc13-46cf-a337-9fc75c370062\") " pod="openshift-image-registry/image-registry-6fd455ff8b-m2b7t"
Jul 21 14:33:22.205034 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.205015    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a9bde86f-cc13-46cf-a337-9fc75c370062-registry-tls\") pod \"image-registry-6fd455ff8b-m2b7t\" (UID: \"a9bde86f-cc13-46cf-a337-9fc75c370062\") " pod="openshift-image-registry/image-registry-6fd455ff8b-m2b7t"
Jul 21 14:33:22.205136 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.205042    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a9bde86f-cc13-46cf-a337-9fc75c370062-ca-trust-extracted\") pod \"image-registry-6fd455ff8b-m2b7t\" (UID: \"a9bde86f-cc13-46cf-a337-9fc75c370062\") " pod="openshift-image-registry/image-registry-6fd455ff8b-m2b7t"
Jul 21 14:33:22.205136 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.205096    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a9bde86f-cc13-46cf-a337-9fc75c370062-trusted-ca\") pod \"image-registry-6fd455ff8b-m2b7t\" (UID: \"a9bde86f-cc13-46cf-a337-9fc75c370062\") " pod="openshift-image-registry/image-registry-6fd455ff8b-m2b7t"
Jul 21 14:33:22.205544 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.205516    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a9bde86f-cc13-46cf-a337-9fc75c370062-ca-trust-extracted\") pod \"image-registry-6fd455ff8b-m2b7t\" (UID: \"a9bde86f-cc13-46cf-a337-9fc75c370062\") " pod="openshift-image-registry/image-registry-6fd455ff8b-m2b7t"
Jul 21 14:33:22.205544 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.205538    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a9bde86f-cc13-46cf-a337-9fc75c370062-registry-certificates\") pod \"image-registry-6fd455ff8b-m2b7t\" (UID: \"a9bde86f-cc13-46cf-a337-9fc75c370062\") " pod="openshift-image-registry/image-registry-6fd455ff8b-m2b7t"
Jul 21 14:33:22.206163 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.206139    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a9bde86f-cc13-46cf-a337-9fc75c370062-trusted-ca\") pod \"image-registry-6fd455ff8b-m2b7t\" (UID: \"a9bde86f-cc13-46cf-a337-9fc75c370062\") " pod="openshift-image-registry/image-registry-6fd455ff8b-m2b7t"
Jul 21 14:33:22.207558 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.207537    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a9bde86f-cc13-46cf-a337-9fc75c370062-installation-pull-secrets\") pod \"image-registry-6fd455ff8b-m2b7t\" (UID: \"a9bde86f-cc13-46cf-a337-9fc75c370062\") " pod="openshift-image-registry/image-registry-6fd455ff8b-m2b7t"
Jul 21 14:33:22.207901 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.207880    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a9bde86f-cc13-46cf-a337-9fc75c370062-image-registry-private-configuration\") pod \"image-registry-6fd455ff8b-m2b7t\" (UID: \"a9bde86f-cc13-46cf-a337-9fc75c370062\") " pod="openshift-image-registry/image-registry-6fd455ff8b-m2b7t"
Jul 21 14:33:22.208038 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.207990    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a9bde86f-cc13-46cf-a337-9fc75c370062-registry-tls\") pod \"image-registry-6fd455ff8b-m2b7t\" (UID: \"a9bde86f-cc13-46cf-a337-9fc75c370062\") " pod="openshift-image-registry/image-registry-6fd455ff8b-m2b7t"
Jul 21 14:33:22.215429 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.215405    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mlkx\" (UniqueName: \"kubernetes.io/projected/a9bde86f-cc13-46cf-a337-9fc75c370062-kube-api-access-8mlkx\") pod \"image-registry-6fd455ff8b-m2b7t\" (UID: \"a9bde86f-cc13-46cf-a337-9fc75c370062\") " pod="openshift-image-registry/image-registry-6fd455ff8b-m2b7t"
Jul 21 14:33:22.215631 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.215602    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a9bde86f-cc13-46cf-a337-9fc75c370062-bound-sa-token\") pod \"image-registry-6fd455ff8b-m2b7t\" (UID: \"a9bde86f-cc13-46cf-a337-9fc75c370062\") " pod="openshift-image-registry/image-registry-6fd455ff8b-m2b7t"
Jul 21 14:33:22.264358 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.264329    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74df7f78d8-tc8sc"
Jul 21 14:33:22.279443 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.279425    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck"
Jul 21 14:33:22.360251 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.360143    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6fd455ff8b-m2b7t"
Jul 21 14:33:22.377514 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.377478    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qpw79" event={"ID":"e5ed87de-30a9-4804-a29c-f60695debb7a","Type":"ContainerStarted","Data":"4baec2c5ae4ec6d07d82741c3919634d156291c1352854bd467afd410d8b572c"}
Jul 21 14:33:22.411529 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.411501    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-74df7f78d8-tc8sc"]
Jul 21 14:33:22.415653 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:33:22.415361    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod388d14d1_066a_4836_90ec_e19acd84741c.slice/crio-37cebca70bd37b48d8d48f3ca61e0ef9a27b853fcfd67692a9d0b1ab94dca448 WatchSource:0}: Error finding container 37cebca70bd37b48d8d48f3ca61e0ef9a27b853fcfd67692a9d0b1ab94dca448: Status 404 returned error can't find the container with id 37cebca70bd37b48d8d48f3ca61e0ef9a27b853fcfd67692a9d0b1ab94dca448
Jul 21 14:33:22.432392 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.432358    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck"]
Jul 21 14:33:22.434857 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:33:22.434828    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod308b65b5_da2a_456e_b554_69cdc6462a47.slice/crio-7ec2070460bbc3a495b84a03e306de3148fc1054f73aabd32a7cc2577e9bea3f WatchSource:0}: Error finding container 7ec2070460bbc3a495b84a03e306de3148fc1054f73aabd32a7cc2577e9bea3f: Status 404 returned error can't find the container with id 7ec2070460bbc3a495b84a03e306de3148fc1054f73aabd32a7cc2577e9bea3f
Jul 21 14:33:22.491215 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:22.491153    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6fd455ff8b-m2b7t"]
Jul 21 14:33:22.494066 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:33:22.494040    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9bde86f_cc13_46cf_a337_9fc75c370062.slice/crio-69e2018c92ded28a93e96c20352090347dc471c99b2ccafa87249887a24fc281 WatchSource:0}: Error finding container 69e2018c92ded28a93e96c20352090347dc471c99b2ccafa87249887a24fc281: Status 404 returned error can't find the container with id 69e2018c92ded28a93e96c20352090347dc471c99b2ccafa87249887a24fc281
Jul 21 14:33:23.392013 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:23.391963    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6fd455ff8b-m2b7t" event={"ID":"a9bde86f-cc13-46cf-a337-9fc75c370062","Type":"ContainerStarted","Data":"96080144e7e0611a726b4f2fc1805099537bfdffd0873b000bd23e8cc1b635de"}
Jul 21 14:33:23.392466 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:23.392066    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6fd455ff8b-m2b7t" event={"ID":"a9bde86f-cc13-46cf-a337-9fc75c370062","Type":"ContainerStarted","Data":"69e2018c92ded28a93e96c20352090347dc471c99b2ccafa87249887a24fc281"}
Jul 21 14:33:23.392854 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:23.392830    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6fd455ff8b-m2b7t"
Jul 21 14:33:23.398891 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:23.398838    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck" event={"ID":"308b65b5-da2a-456e-b554-69cdc6462a47","Type":"ContainerStarted","Data":"7ec2070460bbc3a495b84a03e306de3148fc1054f73aabd32a7cc2577e9bea3f"}
Jul 21 14:33:23.400414 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:23.400361    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74df7f78d8-tc8sc" event={"ID":"388d14d1-066a-4836-90ec-e19acd84741c","Type":"ContainerStarted","Data":"37cebca70bd37b48d8d48f3ca61e0ef9a27b853fcfd67692a9d0b1ab94dca448"}
Jul 21 14:33:23.417781 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:23.417701    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6fd455ff8b-m2b7t" podStartSLOduration=1.41768442 podStartE2EDuration="1.41768442s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:33:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:33:23.417311438 +0000 UTC m=+59.767054851" watchObservedRunningTime="2026-07-21 14:33:23.41768442 +0000 UTC m=+59.767427832"
Jul 21 14:33:26.306856 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:26.306826    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mv82n"
Jul 21 14:33:28.368898 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:28.368867    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-s5kpj"
Jul 21 14:33:29.877552 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:29.877526    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13a6c11d-b9ba-4c12-836f-2172b4d168ce-metrics-certs\") pod \"network-metrics-daemon-2zm5j\" (UID: \"13a6c11d-b9ba-4c12-836f-2172b4d168ce\") " pod="openshift-multus/network-metrics-daemon-2zm5j"
Jul 21 14:33:29.880171 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:29.880151    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 21 14:33:29.890128 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:29.890111    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13a6c11d-b9ba-4c12-836f-2172b4d168ce-metrics-certs\") pod \"network-metrics-daemon-2zm5j\" (UID: \"13a6c11d-b9ba-4c12-836f-2172b4d168ce\") " pod="openshift-multus/network-metrics-daemon-2zm5j"
Jul 21 14:33:29.978083 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:29.978034    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl9vm\" (UniqueName: \"kubernetes.io/projected/206dbc0b-6708-45a2-bcd9-50b72fd5e6b5-kube-api-access-wl9vm\") pod \"network-check-target-q7h42\" (UID: \"206dbc0b-6708-45a2-bcd9-50b72fd5e6b5\") " pod="openshift-network-diagnostics/network-check-target-q7h42"
Jul 21 14:33:29.978169 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:29.978105    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80d084f2-ce50-4ce6-9a6f-76e108e0215e-original-pull-secret\") pod \"global-pull-secret-syncer-78qqf\" (UID: \"80d084f2-ce50-4ce6-9a6f-76e108e0215e\") " pod="kube-system/global-pull-secret-syncer-78qqf"
Jul 21 14:33:29.980967 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:29.980947    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 21 14:33:29.981063 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:29.981035    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 21 14:33:29.990873 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:29.990847    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/80d084f2-ce50-4ce6-9a6f-76e108e0215e-original-pull-secret\") pod \"global-pull-secret-syncer-78qqf\" (UID: \"80d084f2-ce50-4ce6-9a6f-76e108e0215e\") " pod="kube-system/global-pull-secret-syncer-78qqf"
Jul 21 14:33:29.991691 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:29.991677    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 21 14:33:30.001872 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:30.001855    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wl9vm\" (UniqueName: \"kubernetes.io/projected/206dbc0b-6708-45a2-bcd9-50b72fd5e6b5-kube-api-access-wl9vm\") pod \"network-check-target-q7h42\" (UID: \"206dbc0b-6708-45a2-bcd9-50b72fd5e6b5\") " pod="openshift-network-diagnostics/network-check-target-q7h42"
Jul 21 14:33:30.180536 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:30.180513    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cprh8\""
Jul 21 14:33:30.187696 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:30.187678    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-c99qd\""
Jul 21 14:33:30.188717 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:30.188702    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2zm5j"
Jul 21 14:33:30.191625 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:30.191609    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-78qqf"
Jul 21 14:33:30.196288 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:30.196257    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q7h42"
Jul 21 14:33:30.338380 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:30.338305    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-78qqf"]
Jul 21 14:33:30.342621 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:33:30.342591    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80d084f2_ce50_4ce6_9a6f_76e108e0215e.slice/crio-f3d970b6c84e53507d47f51b94562e3dc09d2b39341a0041df9ccd9422bfd51f WatchSource:0}: Error finding container f3d970b6c84e53507d47f51b94562e3dc09d2b39341a0041df9ccd9422bfd51f: Status 404 returned error can't find the container with id f3d970b6c84e53507d47f51b94562e3dc09d2b39341a0041df9ccd9422bfd51f
Jul 21 14:33:30.429045 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:30.429008    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qpw79" event={"ID":"e5ed87de-30a9-4804-a29c-f60695debb7a","Type":"ContainerStarted","Data":"cd2d110f5a8cfe39079febd5f6a0db1217a88237073056a711b56b80978d93c6"}
Jul 21 14:33:30.430729 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:30.430698    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-78qqf" event={"ID":"80d084f2-ce50-4ce6-9a6f-76e108e0215e","Type":"ContainerStarted","Data":"f3d970b6c84e53507d47f51b94562e3dc09d2b39341a0041df9ccd9422bfd51f"}
Jul 21 14:33:30.432980 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:30.432953    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck" event={"ID":"308b65b5-da2a-456e-b554-69cdc6462a47","Type":"ContainerStarted","Data":"d833751a885f52ab9ec41c6e86cfeda1bb8f48ab3810bab71d8ce67a26885fa2"}
Jul 21 14:33:30.433096 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:30.432992    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck" event={"ID":"308b65b5-da2a-456e-b554-69cdc6462a47","Type":"ContainerStarted","Data":"8af771be988c7e3d7361e3277ea3f246643c39a7b5b4b5ffd299596e0e15dee4"}
Jul 21 14:33:30.433096 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:30.433011    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck" event={"ID":"308b65b5-da2a-456e-b554-69cdc6462a47","Type":"ContainerStarted","Data":"841a0c7504e7f90b326548d8d59774b44555186a973aa79e02436dcbcd85f96a"}
Jul 21 14:33:30.434571 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:30.434549    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74df7f78d8-tc8sc" event={"ID":"388d14d1-066a-4836-90ec-e19acd84741c","Type":"ContainerStarted","Data":"62d97bebb318ed98db7cf6019fb8a182f4003e0781350aa1e9444dc7beb461f0"}
Jul 21 14:33:30.434837 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:30.434818    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74df7f78d8-tc8sc"
Jul 21 14:33:30.436532 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:30.436506    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74df7f78d8-tc8sc"
Jul 21 14:33:30.453726 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:30.453684    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b4d5cbf6f-9k5ck" podStartSLOduration=2.104056699 podStartE2EDuration="9.453670385s" totalImagesPullingTime="7.349613686s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:33:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:33:30.452204493 +0000 UTC m=+66.801947982" watchObservedRunningTime="2026-07-21 14:33:30.453670385 +0000 UTC m=+66.803413793"
Jul 21 14:33:30.468518 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:30.468450    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74df7f78d8-tc8sc" podStartSLOduration=2.099683232 podStartE2EDuration="9.468431893s" totalImagesPullingTime="7.368748661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:33:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:33:30.46763188 +0000 UTC m=+66.817375330" watchObservedRunningTime="2026-07-21 14:33:30.468431893 +0000 UTC m=+66.818175306"
Jul 21 14:33:30.561211 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:30.561129    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2zm5j"]
Jul 21 14:33:30.563998 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:30.563974    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-q7h42"]
Jul 21 14:33:30.566841 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:33:30.566813    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13a6c11d_b9ba_4c12_836f_2172b4d168ce.slice/crio-53aa40da5c37719f43e8c6ced770804a9dbd6c0368ca3afdff0fc5f0329b6d89 WatchSource:0}: Error finding container 53aa40da5c37719f43e8c6ced770804a9dbd6c0368ca3afdff0fc5f0329b6d89: Status 404 returned error can't find the container with id 53aa40da5c37719f43e8c6ced770804a9dbd6c0368ca3afdff0fc5f0329b6d89
Jul 21 14:33:30.567554 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:33:30.567533    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod206dbc0b_6708_45a2_bcd9_50b72fd5e6b5.slice/crio-e9776442673b39a67d357ab1e7c6cae2e7011c51049d1c468b14635c8526833e WatchSource:0}: Error finding container e9776442673b39a67d357ab1e7c6cae2e7011c51049d1c468b14635c8526833e: Status 404 returned error can't find the container with id e9776442673b39a67d357ab1e7c6cae2e7011c51049d1c468b14635c8526833e
Jul 21 14:33:31.440839 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:31.440797    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2zm5j" event={"ID":"13a6c11d-b9ba-4c12-836f-2172b4d168ce","Type":"ContainerStarted","Data":"53aa40da5c37719f43e8c6ced770804a9dbd6c0368ca3afdff0fc5f0329b6d89"}
Jul 21 14:33:31.443461 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:31.442944    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-q7h42" event={"ID":"206dbc0b-6708-45a2-bcd9-50b72fd5e6b5","Type":"ContainerStarted","Data":"e9776442673b39a67d357ab1e7c6cae2e7011c51049d1c468b14635c8526833e"}
Jul 21 14:33:31.925778 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:31.925744    2579 patch_prober.go:28] interesting pod/image-registry-7888877dcd-b9b2z container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 21 14:33:31.925900 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:31.925829    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7888877dcd-b9b2z" podUID="f7f9d541-2a52-4303-819e-858ccc2fc64f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 21 14:33:32.449926 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:32.449882    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2zm5j" event={"ID":"13a6c11d-b9ba-4c12-836f-2172b4d168ce","Type":"ContainerStarted","Data":"e8d51611894dc9866882e6990d74737816a1889b9001219e5714e33eab926905"}
Jul 21 14:33:37.470401 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:37.470366    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qpw79" event={"ID":"e5ed87de-30a9-4804-a29c-f60695debb7a","Type":"ContainerStarted","Data":"c037677153cfe22c5e50c343e5bef4535411d752d2a657297057f3396d77b6ce"}
Jul 21 14:33:37.471889 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:37.471865    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-78qqf" event={"ID":"80d084f2-ce50-4ce6-9a6f-76e108e0215e","Type":"ContainerStarted","Data":"d8206cc21ed595ec4dd8538576237da42f8f685f89190d050c879fff0072f9f3"}
Jul 21 14:33:37.473725 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:37.473577    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2zm5j" event={"ID":"13a6c11d-b9ba-4c12-836f-2172b4d168ce","Type":"ContainerStarted","Data":"80e157c4afc4ed0c33667cfb0149fc7274d864d919a21a11c37d24bc6120c289"}
Jul 21 14:33:37.475096 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:37.475075    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-q7h42" event={"ID":"206dbc0b-6708-45a2-bcd9-50b72fd5e6b5","Type":"ContainerStarted","Data":"fadec86a1e529bc69f4b3b0db12dec0c741a35ac43280b6af69facb3c7ed8bf7"}
Jul 21 14:33:37.475218 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:37.475204    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-q7h42"
Jul 21 14:33:37.501340 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:37.501299    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-qpw79" podStartSLOduration=17.663870143 podStartE2EDuration="39.501265444s" totalImagesPullingTime="21.837395301s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 14:32:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:33:37.500353315 +0000 UTC m=+73.850096728" watchObservedRunningTime="2026-07-21 14:33:37.501265444 +0000 UTC m=+73.851008855"
Jul 21 14:33:37.517018 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:37.516979    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-2zm5j" podStartSLOduration=72.212798264 podStartE2EDuration="1m13.51696964s" totalImagesPullingTime="1.304171376s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:32:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:33:37.516352473 +0000 UTC m=+73.866095886" watchObservedRunningTime="2026-07-21 14:33:37.51696964 +0000 UTC m=+73.866713051"
Jul 21 14:33:37.536020 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:37.535981    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-78qqf" podStartSLOduration=66.8356054 podStartE2EDuration="1m13.53596922s" totalImagesPullingTime="6.70036382s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 14:32:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:33:37.535143322 +0000 UTC m=+73.884886735" watchObservedRunningTime="2026-07-21 14:33:37.53596922 +0000 UTC m=+73.885712614"
Jul 21 14:33:37.559675 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:37.559636    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-q7h42" podStartSLOduration=67.072086915 podStartE2EDuration="1m13.5596247s" totalImagesPullingTime="6.487537785s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:32:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:33:37.558906159 +0000 UTC m=+73.908649562" watchObservedRunningTime="2026-07-21 14:33:37.5596247 +0000 UTC m=+73.909368111"
Jul 21 14:33:41.921362 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:41.921332    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7888877dcd-b9b2z"
Jul 21 14:33:45.389803 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:45.389770    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-sct6c"]
Jul 21 14:33:45.392255 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:45.392232    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sct6c"
Jul 21 14:33:45.395212 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:45.395186    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 21 14:33:45.395212 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:45.395201    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-b2phk\""
Jul 21 14:33:45.395401 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:45.395361    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 21 14:33:45.396134 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:45.396108    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 21 14:33:45.396251 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:45.396151    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 21 14:33:45.396251 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:45.396194    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 21 14:33:45.396398 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:45.396315    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 21 14:33:45.416663 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:45.416643    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6fd455ff8b-m2b7t"
Jul 21 14:33:45.501875 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:45.501851    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fbd4dc34-49eb-4fab-95c0-8867f8fec989-root\") pod \"node-exporter-sct6c\" (UID: \"fbd4dc34-49eb-4fab-95c0-8867f8fec989\") " pod="openshift-monitoring/node-exporter-sct6c"
Jul 21 14:33:45.502014 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:45.501879    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fbd4dc34-49eb-4fab-95c0-8867f8fec989-node-exporter-accelerators-collector-config\") pod \"node-exporter-sct6c\" (UID: \"fbd4dc34-49eb-4fab-95c0-8867f8fec989\") " pod="openshift-monitoring/node-exporter-sct6c"
Jul 21 14:33:45.502014 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:45.501911    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fbd4dc34-49eb-4fab-95c0-8867f8fec989-sys\") pod \"node-exporter-sct6c\" (UID: \"fbd4dc34-49eb-4fab-95c0-8867f8fec989\") " pod="openshift-monitoring/node-exporter-sct6c"
Jul 21 14:33:45.502014 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:45.501992    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fbd4dc34-49eb-4fab-95c0-8867f8fec989-node-exporter-textfile\") pod \"node-exporter-sct6c\" (UID: \"fbd4dc34-49eb-4fab-95c0-8867f8fec989\") " pod="openshift-monitoring/node-exporter-sct6c"
Jul 21 14:33:45.502193 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:45.502025    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fbd4dc34-49eb-4fab-95c0-8867f8fec989-node-exporter-tls\") pod \"node-exporter-sct6c\" (UID: \"fbd4dc34-49eb-4fab-95c0-8867f8fec989\") " pod="openshift-monitoring/node-exporter-sct6c"
Jul 21 14:33:45.502193 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:45.502050    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fbd4dc34-49eb-4fab-95c0-8867f8fec989-node-exporter-wtmp\") pod \"node-exporter-sct6c\" (UID: \"fbd4dc34-49eb-4fab-95c0-8867f8fec989\") " pod="openshift-monitoring/node-exporter-sct6c"
Jul 21 14:33:45.502193 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:45.502074    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q26lb\" (UniqueName: \"kubernetes.io/projected/fbd4dc34-49eb-4fab-95c0-8867f8fec989-kube-api-access-q26lb\") pod \"node-exporter-sct6c\" (UID: \"fbd4dc34-49eb-4fab-95c0-8867f8fec989\") " pod="openshift-monitoring/node-exporter-sct6c"
Jul 21 14:33:45.502193 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:45.502153    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fbd4dc34-49eb-4fab-95c0-8867f8fec989-metrics-client-ca\") pod \"node-exporter-sct6c\" (UID: \"fbd4dc34-49eb-4fab-95c0-8867f8fec989\") " pod="openshift-monitoring/node-exporter-sct6c"
Jul 21 14:33:45.502193 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:45.502178    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fbd4dc34-49eb-4fab-95c0-8867f8fec989-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sct6c\" (UID: \"fbd4dc34-49eb-4fab-95c0-8867f8fec989\") " pod="openshift-monitoring/node-exporter-sct6c"
Jul 21 14:33:45.602749 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:45.602705    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fbd4dc34-49eb-4fab-95c0-8867f8fec989-root\") pod \"node-exporter-sct6c\" (UID: \"fbd4dc34-49eb-4fab-95c0-8867f8fec989\") " pod="openshift-monitoring/node-exporter-sct6c"
Jul 21 14:33:45.603144 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:45.602749    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fbd4dc34-49eb-4fab-95c0-8867f8fec989-node-exporter-accelerators-collector-config\") pod \"node-exporter-sct6c\" (UID: \"fbd4dc34-49eb-4fab-95c0-8867f8fec989\") " pod="openshift-monitoring/node-exporter-sct6c"
Jul 21 14:33:45.603144 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:45.602793    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fbd4dc34-49eb-4fab-95c0-8867f8fec989-sys\") pod \"node-exporter-sct6c\" (UID: \"fbd4dc34-49eb-4fab-95c0-8867f8fec989\") " pod="openshift-monitoring/node-exporter-sct6c"
Jul 21 14:33:45.603144 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:45.602803    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fbd4dc34-49eb-4fab-95c0-8867f8fec989-root\") pod \"node-exporter-sct6c\" (UID: \"fbd4dc34-49eb-4fab-95c0-8867f8fec989\") " pod="openshift-monitoring/node-exporter-sct6c"
Jul 21 14:33:45.603144 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:45.602832    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fbd4dc34-49eb-4fab-95c0-8867f8fec989-node-exporter-textfile\") pod \"node-exporter-sct6c\" (UID: \"fbd4dc34-49eb-4fab-95c0-8867f8fec989\") " pod="openshift-monitoring/node-exporter-sct6c"
Jul 21 14:33:45.603144 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:45.602847    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fbd4dc34-49eb-4fab-95c0-8867f8fec989-sys\") pod \"node-exporter-sct6c\" (UID: \"fbd4dc34-49eb-4fab-95c0-8867f8fec989\") " pod="openshift-monitoring/node-exporter-sct6c"
Jul 21 14:33:45.603144 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:45.602865    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fbd4dc34-49eb-4fab-95c0-8867f8fec989-node-exporter-tls\") pod \"node-exporter-sct6c\" (UID: \"fbd4dc34-49eb-4fab-95c0-8867f8fec989\") " pod="openshift-monitoring/node-exporter-sct6c"
Jul 21 14:33:45.603144 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:45.602889    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fbd4dc34-49eb-4fab-95c0-8867f8fec989-node-exporter-wtmp\") pod \"node-exporter-sct6c\" (UID: \"fbd4dc34-49eb-4fab-95c0-8867f8fec989\") " pod="openshift-monitoring/node-exporter-sct6c"
Jul 21 14:33:45.603144 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:45.602915    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q26lb\" (UniqueName: \"kubernetes.io/projected/fbd4dc34-49eb-4fab-95c0-8867f8fec989-kube-api-access-q26lb\") pod \"node-exporter-sct6c\" (UID: \"fbd4dc34-49eb-4fab-95c0-8867f8fec989\") " pod="openshift-monitoring/node-exporter-sct6c"
Jul 21 14:33:45.603144 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:45.602955    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fbd4dc34-49eb-4fab-95c0-8867f8fec989-metrics-client-ca\") pod \"node-exporter-sct6c\" (UID: \"fbd4dc34-49eb-4fab-95c0-8867f8fec989\") " pod="openshift-monitoring/node-exporter-sct6c"
Jul 21 14:33:45.603144 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:45.602979    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fbd4dc34-49eb-4fab-95c0-8867f8fec989-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sct6c\" (UID: \"fbd4dc34-49eb-4fab-95c0-8867f8fec989\") " pod="openshift-monitoring/node-exporter-sct6c"
Jul 21 14:33:45.603144 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:45.603069    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fbd4dc34-49eb-4fab-95c0-8867f8fec989-node-exporter-wtmp\") pod \"node-exporter-sct6c\" (UID: \"fbd4dc34-49eb-4fab-95c0-8867f8fec989\") " pod="openshift-monitoring/node-exporter-sct6c"
Jul 21 14:33:45.603144 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:45.603136    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fbd4dc34-49eb-4fab-95c0-8867f8fec989-node-exporter-textfile\") pod \"node-exporter-sct6c\" (UID: \"fbd4dc34-49eb-4fab-95c0-8867f8fec989\") " pod="openshift-monitoring/node-exporter-sct6c"
Jul 21 14:33:45.603688 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:45.603376    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fbd4dc34-49eb-4fab-95c0-8867f8fec989-node-exporter-accelerators-collector-config\") pod \"node-exporter-sct6c\" (UID: \"fbd4dc34-49eb-4fab-95c0-8867f8fec989\") " pod="openshift-monitoring/node-exporter-sct6c"
Jul 21 14:33:45.603688 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:45.603635    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fbd4dc34-49eb-4fab-95c0-8867f8fec989-metrics-client-ca\") pod \"node-exporter-sct6c\" (UID: \"fbd4dc34-49eb-4fab-95c0-8867f8fec989\") " pod="openshift-monitoring/node-exporter-sct6c"
Jul 21 14:33:45.605358 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:45.605337    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fbd4dc34-49eb-4fab-95c0-8867f8fec989-node-exporter-tls\") pod \"node-exporter-sct6c\" (UID: \"fbd4dc34-49eb-4fab-95c0-8867f8fec989\") " pod="openshift-monitoring/node-exporter-sct6c"
Jul 21 14:33:45.605650 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:45.605632    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fbd4dc34-49eb-4fab-95c0-8867f8fec989-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sct6c\" (UID: \"fbd4dc34-49eb-4fab-95c0-8867f8fec989\") " pod="openshift-monitoring/node-exporter-sct6c"
Jul 21 14:33:45.614255 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:45.614236    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q26lb\" (UniqueName: \"kubernetes.io/projected/fbd4dc34-49eb-4fab-95c0-8867f8fec989-kube-api-access-q26lb\") pod \"node-exporter-sct6c\" (UID: \"fbd4dc34-49eb-4fab-95c0-8867f8fec989\") " pod="openshift-monitoring/node-exporter-sct6c"
Jul 21 14:33:45.703334 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:45.703241    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sct6c"
Jul 21 14:33:45.714150 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:33:45.714119    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbd4dc34_49eb_4fab_95c0_8867f8fec989.slice/crio-be937b444126b2a1f7d297a0e560e6861264b69018ef78e9263f635207292886 WatchSource:0}: Error finding container be937b444126b2a1f7d297a0e560e6861264b69018ef78e9263f635207292886: Status 404 returned error can't find the container with id be937b444126b2a1f7d297a0e560e6861264b69018ef78e9263f635207292886
Jul 21 14:33:46.508327 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:46.508300    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sct6c" event={"ID":"fbd4dc34-49eb-4fab-95c0-8867f8fec989","Type":"ContainerStarted","Data":"be937b444126b2a1f7d297a0e560e6861264b69018ef78e9263f635207292886"}
Jul 21 14:33:46.937208 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:46.937177    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7888877dcd-b9b2z" podUID="f7f9d541-2a52-4303-819e-858ccc2fc64f" containerName="registry" containerID="cri-o://b7785f008f01dd7d5c9b07142194bffe6ebb46fcfe2c219568b1bd28b5b88563" gracePeriod=30
Jul 21 14:33:47.169388 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:47.169363    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7888877dcd-b9b2z"
Jul 21 14:33:47.314114 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:47.314059    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f7f9d541-2a52-4303-819e-858ccc2fc64f-bound-sa-token\") pod \"f7f9d541-2a52-4303-819e-858ccc2fc64f\" (UID: \"f7f9d541-2a52-4303-819e-858ccc2fc64f\") "
Jul 21 14:33:47.314114 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:47.314111    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f7f9d541-2a52-4303-819e-858ccc2fc64f-image-registry-private-configuration\") pod \"f7f9d541-2a52-4303-819e-858ccc2fc64f\" (UID: \"f7f9d541-2a52-4303-819e-858ccc2fc64f\") "
Jul 21 14:33:47.314244 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:47.314141    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v9bdt\" (UniqueName: \"kubernetes.io/projected/f7f9d541-2a52-4303-819e-858ccc2fc64f-kube-api-access-v9bdt\") pod \"f7f9d541-2a52-4303-819e-858ccc2fc64f\" (UID: \"f7f9d541-2a52-4303-819e-858ccc2fc64f\") "
Jul 21 14:33:47.314244 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:47.314171    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f7f9d541-2a52-4303-819e-858ccc2fc64f-ca-trust-extracted\") pod \"f7f9d541-2a52-4303-819e-858ccc2fc64f\" (UID: \"f7f9d541-2a52-4303-819e-858ccc2fc64f\") "
Jul 21 14:33:47.314359 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:47.314237    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f7f9d541-2a52-4303-819e-858ccc2fc64f-installation-pull-secrets\") pod \"f7f9d541-2a52-4303-819e-858ccc2fc64f\" (UID: \"f7f9d541-2a52-4303-819e-858ccc2fc64f\") "
Jul 21 14:33:47.314359 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:47.314333    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f7f9d541-2a52-4303-819e-858ccc2fc64f-registry-certificates\") pod \"f7f9d541-2a52-4303-819e-858ccc2fc64f\" (UID: \"f7f9d541-2a52-4303-819e-858ccc2fc64f\") "
Jul 21 14:33:47.314449 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:47.314373    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f7f9d541-2a52-4303-819e-858ccc2fc64f-trusted-ca\") pod \"f7f9d541-2a52-4303-819e-858ccc2fc64f\" (UID: \"f7f9d541-2a52-4303-819e-858ccc2fc64f\") "
Jul 21 14:33:47.314449 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:47.314415    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f7f9d541-2a52-4303-819e-858ccc2fc64f-registry-tls\") pod \"f7f9d541-2a52-4303-819e-858ccc2fc64f\" (UID: \"f7f9d541-2a52-4303-819e-858ccc2fc64f\") "
Jul 21 14:33:47.314834 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:47.314803    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7f9d541-2a52-4303-819e-858ccc2fc64f-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f7f9d541-2a52-4303-819e-858ccc2fc64f" (UID: "f7f9d541-2a52-4303-819e-858ccc2fc64f"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:33:47.314952 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:47.314901    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7f9d541-2a52-4303-819e-858ccc2fc64f-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f7f9d541-2a52-4303-819e-858ccc2fc64f" (UID: "f7f9d541-2a52-4303-819e-858ccc2fc64f"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:33:47.316774 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:47.316722    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7f9d541-2a52-4303-819e-858ccc2fc64f-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f7f9d541-2a52-4303-819e-858ccc2fc64f" (UID: "f7f9d541-2a52-4303-819e-858ccc2fc64f"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:33:47.316892 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:47.316847    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7f9d541-2a52-4303-819e-858ccc2fc64f-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f7f9d541-2a52-4303-819e-858ccc2fc64f" (UID: "f7f9d541-2a52-4303-819e-858ccc2fc64f"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:33:47.316892 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:47.316851    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7f9d541-2a52-4303-819e-858ccc2fc64f-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "f7f9d541-2a52-4303-819e-858ccc2fc64f" (UID: "f7f9d541-2a52-4303-819e-858ccc2fc64f"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:33:47.317013 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:47.316902    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7f9d541-2a52-4303-819e-858ccc2fc64f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f7f9d541-2a52-4303-819e-858ccc2fc64f" (UID: "f7f9d541-2a52-4303-819e-858ccc2fc64f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:33:47.317013 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:47.317002    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7f9d541-2a52-4303-819e-858ccc2fc64f-kube-api-access-v9bdt" (OuterVolumeSpecName: "kube-api-access-v9bdt") pod "f7f9d541-2a52-4303-819e-858ccc2fc64f" (UID: "f7f9d541-2a52-4303-819e-858ccc2fc64f"). InnerVolumeSpecName "kube-api-access-v9bdt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:33:47.322903 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:47.322880    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7f9d541-2a52-4303-819e-858ccc2fc64f-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f7f9d541-2a52-4303-819e-858ccc2fc64f" (UID: "f7f9d541-2a52-4303-819e-858ccc2fc64f"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 14:33:47.415503 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:47.415478    2579 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f7f9d541-2a52-4303-819e-858ccc2fc64f-image-registry-private-configuration\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:33:47.415503 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:47.415502    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9bdt\" (UniqueName: \"kubernetes.io/projected/f7f9d541-2a52-4303-819e-858ccc2fc64f-kube-api-access-v9bdt\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:33:47.415653 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:47.415517    2579 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f7f9d541-2a52-4303-819e-858ccc2fc64f-ca-trust-extracted\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:33:47.415653 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:47.415529    2579 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f7f9d541-2a52-4303-819e-858ccc2fc64f-installation-pull-secrets\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:33:47.415653 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:47.415542    2579 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f7f9d541-2a52-4303-819e-858ccc2fc64f-registry-certificates\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:33:47.415653 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:47.415555    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f7f9d541-2a52-4303-819e-858ccc2fc64f-trusted-ca\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:33:47.415653 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:47.415569    2579 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f7f9d541-2a52-4303-819e-858ccc2fc64f-registry-tls\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:33:47.415653 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:47.415580    2579 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f7f9d541-2a52-4303-819e-858ccc2fc64f-bound-sa-token\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:33:47.512299 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:47.512255    2579 generic.go:356] "Generic (PLEG): container finished" podID="f7f9d541-2a52-4303-819e-858ccc2fc64f" containerID="b7785f008f01dd7d5c9b07142194bffe6ebb46fcfe2c219568b1bd28b5b88563" exitCode=0
Jul 21 14:33:47.512678 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:47.512294    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7888877dcd-b9b2z" event={"ID":"f7f9d541-2a52-4303-819e-858ccc2fc64f","Type":"ContainerDied","Data":"b7785f008f01dd7d5c9b07142194bffe6ebb46fcfe2c219568b1bd28b5b88563"}
Jul 21 14:33:47.512678 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:47.512311    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7888877dcd-b9b2z"
Jul 21 14:33:47.512678 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:47.512362    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7888877dcd-b9b2z" event={"ID":"f7f9d541-2a52-4303-819e-858ccc2fc64f","Type":"ContainerDied","Data":"866bc62cd615836d873fe750ab734f164b19f540e6392627b1a1a0251df5275a"}
Jul 21 14:33:47.512678 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:47.512382    2579 scope.go:117] "RemoveContainer" containerID="b7785f008f01dd7d5c9b07142194bffe6ebb46fcfe2c219568b1bd28b5b88563"
Jul 21 14:33:47.513988 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:47.513963    2579 generic.go:356] "Generic (PLEG): container finished" podID="fbd4dc34-49eb-4fab-95c0-8867f8fec989" containerID="a869fefb518e4629e3ed1b3bc5d20a86c4fca7a7d0b3794ab988b7b9fd44cdd8" exitCode=0
Jul 21 14:33:47.514085 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:47.514031    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sct6c" event={"ID":"fbd4dc34-49eb-4fab-95c0-8867f8fec989","Type":"ContainerDied","Data":"a869fefb518e4629e3ed1b3bc5d20a86c4fca7a7d0b3794ab988b7b9fd44cdd8"}
Jul 21 14:33:47.520684 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:47.520661    2579 scope.go:117] "RemoveContainer" containerID="b7785f008f01dd7d5c9b07142194bffe6ebb46fcfe2c219568b1bd28b5b88563"
Jul 21 14:33:47.520940 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:33:47.520921    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7785f008f01dd7d5c9b07142194bffe6ebb46fcfe2c219568b1bd28b5b88563\": container with ID starting with b7785f008f01dd7d5c9b07142194bffe6ebb46fcfe2c219568b1bd28b5b88563 not found: ID does not exist" containerID="b7785f008f01dd7d5c9b07142194bffe6ebb46fcfe2c219568b1bd28b5b88563"
Jul 21 14:33:47.521008 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:47.520952    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7785f008f01dd7d5c9b07142194bffe6ebb46fcfe2c219568b1bd28b5b88563"} err="failed to get container status \"b7785f008f01dd7d5c9b07142194bffe6ebb46fcfe2c219568b1bd28b5b88563\": rpc error: code = NotFound desc = could not find container \"b7785f008f01dd7d5c9b07142194bffe6ebb46fcfe2c219568b1bd28b5b88563\": container with ID starting with b7785f008f01dd7d5c9b07142194bffe6ebb46fcfe2c219568b1bd28b5b88563 not found: ID does not exist"
Jul 21 14:33:47.551227 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:47.551204    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7888877dcd-b9b2z"]
Jul 21 14:33:47.554861 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:47.554842    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7888877dcd-b9b2z"]
Jul 21 14:33:48.171025 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:48.170991    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7f9d541-2a52-4303-819e-858ccc2fc64f" path="/var/lib/kubelet/pods/f7f9d541-2a52-4303-819e-858ccc2fc64f/volumes"
Jul 21 14:33:48.520609 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:48.520525    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sct6c" event={"ID":"fbd4dc34-49eb-4fab-95c0-8867f8fec989","Type":"ContainerStarted","Data":"aae42681f76680807d3cdf6ae5aafc9201bd8ec949ee906ec75cde0d5a98f100"}
Jul 21 14:33:48.520609 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:48.520587    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sct6c" event={"ID":"fbd4dc34-49eb-4fab-95c0-8867f8fec989","Type":"ContainerStarted","Data":"f90871e65dcd08ac8f09d11ba8dad197de743864a11afba39520e7603e2ba45d"}
Jul 21 14:33:48.542437 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:48.542390    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-sct6c" podStartSLOduration=2.483181209 podStartE2EDuration="3.542374668s" totalImagesPullingTime="996.395858ms" totalInitContainerRuntime="62.797601ms" isStatefulPod=true podCreationTimestamp="2026-07-21 14:33:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:33:48.541051267 +0000 UTC m=+84.890794698" watchObservedRunningTime="2026-07-21 14:33:48.542374668 +0000 UTC m=+84.892118079"
Jul 21 14:33:59.275783 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:59.275746    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-68ffff969-9ldkn"]
Jul 21 14:33:59.276226 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:59.275969    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7f9d541-2a52-4303-819e-858ccc2fc64f" containerName="registry"
Jul 21 14:33:59.276226 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:59.275980    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7f9d541-2a52-4303-819e-858ccc2fc64f" containerName="registry"
Jul 21 14:33:59.276226 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:59.276024    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7f9d541-2a52-4303-819e-858ccc2fc64f" containerName="registry"
Jul 21 14:33:59.531339 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:59.531204    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68ffff969-9ldkn"]
Jul 21 14:33:59.531482 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:59.531346    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68ffff969-9ldkn"
Jul 21 14:33:59.534106 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:59.534084    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 21 14:33:59.535070 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:59.535047    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 21 14:33:59.535070 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:59.535057    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 21 14:33:59.535239 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:59.535058    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 21 14:33:59.535239 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:59.535138    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 21 14:33:59.535448 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:59.535432    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 21 14:33:59.535448 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:59.535442    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-fknrw\""
Jul 21 14:33:59.535545 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:59.535489    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 21 14:33:59.539944 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:59.539927    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 21 14:33:59.708242 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:59.708215    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5122a0fa-4500-40e2-85d4-738dad277264-oauth-serving-cert\") pod \"console-68ffff969-9ldkn\" (UID: \"5122a0fa-4500-40e2-85d4-738dad277264\") " pod="openshift-console/console-68ffff969-9ldkn"
Jul 21 14:33:59.708404 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:59.708253    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5122a0fa-4500-40e2-85d4-738dad277264-console-serving-cert\") pod \"console-68ffff969-9ldkn\" (UID: \"5122a0fa-4500-40e2-85d4-738dad277264\") " pod="openshift-console/console-68ffff969-9ldkn"
Jul 21 14:33:59.708404 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:59.708290    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5122a0fa-4500-40e2-85d4-738dad277264-console-config\") pod \"console-68ffff969-9ldkn\" (UID: \"5122a0fa-4500-40e2-85d4-738dad277264\") " pod="openshift-console/console-68ffff969-9ldkn"
Jul 21 14:33:59.708404 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:59.708331    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5122a0fa-4500-40e2-85d4-738dad277264-service-ca\") pod \"console-68ffff969-9ldkn\" (UID: \"5122a0fa-4500-40e2-85d4-738dad277264\") " pod="openshift-console/console-68ffff969-9ldkn"
Jul 21 14:33:59.708404 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:59.708361    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2xz5\" (UniqueName: \"kubernetes.io/projected/5122a0fa-4500-40e2-85d4-738dad277264-kube-api-access-p2xz5\") pod \"console-68ffff969-9ldkn\" (UID: \"5122a0fa-4500-40e2-85d4-738dad277264\") " pod="openshift-console/console-68ffff969-9ldkn"
Jul 21 14:33:59.708404 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:59.708380    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5122a0fa-4500-40e2-85d4-738dad277264-trusted-ca-bundle\") pod \"console-68ffff969-9ldkn\" (UID: \"5122a0fa-4500-40e2-85d4-738dad277264\") " pod="openshift-console/console-68ffff969-9ldkn"
Jul 21 14:33:59.708404 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:59.708406    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5122a0fa-4500-40e2-85d4-738dad277264-console-oauth-config\") pod \"console-68ffff969-9ldkn\" (UID: \"5122a0fa-4500-40e2-85d4-738dad277264\") " pod="openshift-console/console-68ffff969-9ldkn"
Jul 21 14:33:59.809567 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:59.809503    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5122a0fa-4500-40e2-85d4-738dad277264-service-ca\") pod \"console-68ffff969-9ldkn\" (UID: \"5122a0fa-4500-40e2-85d4-738dad277264\") " pod="openshift-console/console-68ffff969-9ldkn"
Jul 21 14:33:59.809567 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:59.809535    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2xz5\" (UniqueName: \"kubernetes.io/projected/5122a0fa-4500-40e2-85d4-738dad277264-kube-api-access-p2xz5\") pod \"console-68ffff969-9ldkn\" (UID: \"5122a0fa-4500-40e2-85d4-738dad277264\") " pod="openshift-console/console-68ffff969-9ldkn"
Jul 21 14:33:59.809567 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:59.809563    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5122a0fa-4500-40e2-85d4-738dad277264-trusted-ca-bundle\") pod \"console-68ffff969-9ldkn\" (UID: \"5122a0fa-4500-40e2-85d4-738dad277264\") " pod="openshift-console/console-68ffff969-9ldkn"
Jul 21 14:33:59.809781 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:59.809591    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5122a0fa-4500-40e2-85d4-738dad277264-console-oauth-config\") pod \"console-68ffff969-9ldkn\" (UID: \"5122a0fa-4500-40e2-85d4-738dad277264\") " pod="openshift-console/console-68ffff969-9ldkn"
Jul 21 14:33:59.809781 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:59.809633    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5122a0fa-4500-40e2-85d4-738dad277264-oauth-serving-cert\") pod \"console-68ffff969-9ldkn\" (UID: \"5122a0fa-4500-40e2-85d4-738dad277264\") " pod="openshift-console/console-68ffff969-9ldkn"
Jul 21 14:33:59.809781 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:59.809692    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5122a0fa-4500-40e2-85d4-738dad277264-console-serving-cert\") pod \"console-68ffff969-9ldkn\" (UID: \"5122a0fa-4500-40e2-85d4-738dad277264\") " pod="openshift-console/console-68ffff969-9ldkn"
Jul 21 14:33:59.809781 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:59.809726    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5122a0fa-4500-40e2-85d4-738dad277264-console-config\") pod \"console-68ffff969-9ldkn\" (UID: \"5122a0fa-4500-40e2-85d4-738dad277264\") " pod="openshift-console/console-68ffff969-9ldkn"
Jul 21 14:33:59.810376 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:59.810355    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5122a0fa-4500-40e2-85d4-738dad277264-console-config\") pod \"console-68ffff969-9ldkn\" (UID: \"5122a0fa-4500-40e2-85d4-738dad277264\") " pod="openshift-console/console-68ffff969-9ldkn"
Jul 21 14:33:59.810463 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:59.810373    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5122a0fa-4500-40e2-85d4-738dad277264-service-ca\") pod \"console-68ffff969-9ldkn\" (UID: \"5122a0fa-4500-40e2-85d4-738dad277264\") " pod="openshift-console/console-68ffff969-9ldkn"
Jul 21 14:33:59.810463 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:59.810395    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5122a0fa-4500-40e2-85d4-738dad277264-oauth-serving-cert\") pod \"console-68ffff969-9ldkn\" (UID: \"5122a0fa-4500-40e2-85d4-738dad277264\") " pod="openshift-console/console-68ffff969-9ldkn"
Jul 21 14:33:59.810541 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:59.810526    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5122a0fa-4500-40e2-85d4-738dad277264-trusted-ca-bundle\") pod \"console-68ffff969-9ldkn\" (UID: \"5122a0fa-4500-40e2-85d4-738dad277264\") " pod="openshift-console/console-68ffff969-9ldkn"
Jul 21 14:33:59.812052 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:59.812032    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5122a0fa-4500-40e2-85d4-738dad277264-console-serving-cert\") pod \"console-68ffff969-9ldkn\" (UID: \"5122a0fa-4500-40e2-85d4-738dad277264\") " pod="openshift-console/console-68ffff969-9ldkn"
Jul 21 14:33:59.812136 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:59.812095    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5122a0fa-4500-40e2-85d4-738dad277264-console-oauth-config\") pod \"console-68ffff969-9ldkn\" (UID: \"5122a0fa-4500-40e2-85d4-738dad277264\") " pod="openshift-console/console-68ffff969-9ldkn"
Jul 21 14:33:59.816448 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:59.816420    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2xz5\" (UniqueName: \"kubernetes.io/projected/5122a0fa-4500-40e2-85d4-738dad277264-kube-api-access-p2xz5\") pod \"console-68ffff969-9ldkn\" (UID: \"5122a0fa-4500-40e2-85d4-738dad277264\") " pod="openshift-console/console-68ffff969-9ldkn"
Jul 21 14:33:59.840427 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:59.840407    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68ffff969-9ldkn"
Jul 21 14:33:59.984038 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:33:59.984010    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68ffff969-9ldkn"]
Jul 21 14:33:59.988000 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:33:59.987975    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5122a0fa_4500_40e2_85d4_738dad277264.slice/crio-600339f5def9c929487f98aa14997939e3e820b7cda9771b15c8d35f94ce5091 WatchSource:0}: Error finding container 600339f5def9c929487f98aa14997939e3e820b7cda9771b15c8d35f94ce5091: Status 404 returned error can't find the container with id 600339f5def9c929487f98aa14997939e3e820b7cda9771b15c8d35f94ce5091
Jul 21 14:34:00.559638 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:34:00.559603    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68ffff969-9ldkn" event={"ID":"5122a0fa-4500-40e2-85d4-738dad277264","Type":"ContainerStarted","Data":"600339f5def9c929487f98aa14997939e3e820b7cda9771b15c8d35f94ce5091"}
Jul 21 14:34:03.573088 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:34:03.573015    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68ffff969-9ldkn" event={"ID":"5122a0fa-4500-40e2-85d4-738dad277264","Type":"ContainerStarted","Data":"babfafa4221b6054b5c4b4de96a2297bd2397bee4d9a335bf4fd83783620b63a"}
Jul 21 14:34:03.591945 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:34:03.591899    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-68ffff969-9ldkn" podStartSLOduration=1.2622566 podStartE2EDuration="4.591885939s" totalImagesPullingTime="3.329629339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:33:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:34:03.590385808 +0000 UTC m=+99.940129248" watchObservedRunningTime="2026-07-21 14:34:03.591885939 +0000 UTC m=+99.941629350"
Jul 21 14:34:08.481122 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:34:08.481089    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-q7h42"
Jul 21 14:34:09.841420 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:34:09.841387    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-68ffff969-9ldkn"
Jul 21 14:34:09.841772 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:34:09.841474    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-68ffff969-9ldkn"
Jul 21 14:34:09.845883 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:34:09.845863    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-68ffff969-9ldkn"
Jul 21 14:34:10.599170 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:34:10.599146    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-68ffff969-9ldkn"
Jul 21 14:35:03.844808 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:03.844769    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7d8c6997bb-8dt2h"]
Jul 21 14:35:03.847991 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:03.847965    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d8c6997bb-8dt2h"
Jul 21 14:35:03.859545 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:03.859520    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d8c6997bb-8dt2h"]
Jul 21 14:35:03.964431 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:03.964390    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/37b95e9b-ff18-406e-9f5b-840b9a507b43-service-ca\") pod \"console-7d8c6997bb-8dt2h\" (UID: \"37b95e9b-ff18-406e-9f5b-840b9a507b43\") " pod="openshift-console/console-7d8c6997bb-8dt2h"
Jul 21 14:35:03.964431 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:03.964431    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/37b95e9b-ff18-406e-9f5b-840b9a507b43-oauth-serving-cert\") pod \"console-7d8c6997bb-8dt2h\" (UID: \"37b95e9b-ff18-406e-9f5b-840b9a507b43\") " pod="openshift-console/console-7d8c6997bb-8dt2h"
Jul 21 14:35:03.964641 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:03.964453    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xphzg\" (UniqueName: \"kubernetes.io/projected/37b95e9b-ff18-406e-9f5b-840b9a507b43-kube-api-access-xphzg\") pod \"console-7d8c6997bb-8dt2h\" (UID: \"37b95e9b-ff18-406e-9f5b-840b9a507b43\") " pod="openshift-console/console-7d8c6997bb-8dt2h"
Jul 21 14:35:03.964641 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:03.964491    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/37b95e9b-ff18-406e-9f5b-840b9a507b43-console-oauth-config\") pod \"console-7d8c6997bb-8dt2h\" (UID: \"37b95e9b-ff18-406e-9f5b-840b9a507b43\") " pod="openshift-console/console-7d8c6997bb-8dt2h"
Jul 21 14:35:03.964641 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:03.964518    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/37b95e9b-ff18-406e-9f5b-840b9a507b43-console-config\") pod \"console-7d8c6997bb-8dt2h\" (UID: \"37b95e9b-ff18-406e-9f5b-840b9a507b43\") " pod="openshift-console/console-7d8c6997bb-8dt2h"
Jul 21 14:35:03.964641 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:03.964534    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37b95e9b-ff18-406e-9f5b-840b9a507b43-trusted-ca-bundle\") pod \"console-7d8c6997bb-8dt2h\" (UID: \"37b95e9b-ff18-406e-9f5b-840b9a507b43\") " pod="openshift-console/console-7d8c6997bb-8dt2h"
Jul 21 14:35:03.964641 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:03.964592    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/37b95e9b-ff18-406e-9f5b-840b9a507b43-console-serving-cert\") pod \"console-7d8c6997bb-8dt2h\" (UID: \"37b95e9b-ff18-406e-9f5b-840b9a507b43\") " pod="openshift-console/console-7d8c6997bb-8dt2h"
Jul 21 14:35:04.065411 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:04.065367    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/37b95e9b-ff18-406e-9f5b-840b9a507b43-console-config\") pod \"console-7d8c6997bb-8dt2h\" (UID: \"37b95e9b-ff18-406e-9f5b-840b9a507b43\") " pod="openshift-console/console-7d8c6997bb-8dt2h"
Jul 21 14:35:04.065411 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:04.065413    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37b95e9b-ff18-406e-9f5b-840b9a507b43-trusted-ca-bundle\") pod \"console-7d8c6997bb-8dt2h\" (UID: \"37b95e9b-ff18-406e-9f5b-840b9a507b43\") " pod="openshift-console/console-7d8c6997bb-8dt2h"
Jul 21 14:35:04.065649 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:04.065540    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/37b95e9b-ff18-406e-9f5b-840b9a507b43-console-serving-cert\") pod \"console-7d8c6997bb-8dt2h\" (UID: \"37b95e9b-ff18-406e-9f5b-840b9a507b43\") " pod="openshift-console/console-7d8c6997bb-8dt2h"
Jul 21 14:35:04.065707 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:04.065669    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/37b95e9b-ff18-406e-9f5b-840b9a507b43-service-ca\") pod \"console-7d8c6997bb-8dt2h\" (UID: \"37b95e9b-ff18-406e-9f5b-840b9a507b43\") " pod="openshift-console/console-7d8c6997bb-8dt2h"
Jul 21 14:35:04.065707 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:04.065693    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/37b95e9b-ff18-406e-9f5b-840b9a507b43-oauth-serving-cert\") pod \"console-7d8c6997bb-8dt2h\" (UID: \"37b95e9b-ff18-406e-9f5b-840b9a507b43\") " pod="openshift-console/console-7d8c6997bb-8dt2h"
Jul 21 14:35:04.065807 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:04.065712    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xphzg\" (UniqueName: \"kubernetes.io/projected/37b95e9b-ff18-406e-9f5b-840b9a507b43-kube-api-access-xphzg\") pod \"console-7d8c6997bb-8dt2h\" (UID: \"37b95e9b-ff18-406e-9f5b-840b9a507b43\") " pod="openshift-console/console-7d8c6997bb-8dt2h"
Jul 21 14:35:04.065807 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:04.065736    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/37b95e9b-ff18-406e-9f5b-840b9a507b43-console-oauth-config\") pod \"console-7d8c6997bb-8dt2h\" (UID: \"37b95e9b-ff18-406e-9f5b-840b9a507b43\") " pod="openshift-console/console-7d8c6997bb-8dt2h"
Jul 21 14:35:04.066140 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:04.066108    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/37b95e9b-ff18-406e-9f5b-840b9a507b43-console-config\") pod \"console-7d8c6997bb-8dt2h\" (UID: \"37b95e9b-ff18-406e-9f5b-840b9a507b43\") " pod="openshift-console/console-7d8c6997bb-8dt2h"
Jul 21 14:35:04.066311 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:04.066294    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37b95e9b-ff18-406e-9f5b-840b9a507b43-trusted-ca-bundle\") pod \"console-7d8c6997bb-8dt2h\" (UID: \"37b95e9b-ff18-406e-9f5b-840b9a507b43\") " pod="openshift-console/console-7d8c6997bb-8dt2h"
Jul 21 14:35:04.066408 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:04.066385    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/37b95e9b-ff18-406e-9f5b-840b9a507b43-service-ca\") pod \"console-7d8c6997bb-8dt2h\" (UID: \"37b95e9b-ff18-406e-9f5b-840b9a507b43\") " pod="openshift-console/console-7d8c6997bb-8dt2h"
Jul 21 14:35:04.066465 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:04.066412    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/37b95e9b-ff18-406e-9f5b-840b9a507b43-oauth-serving-cert\") pod \"console-7d8c6997bb-8dt2h\" (UID: \"37b95e9b-ff18-406e-9f5b-840b9a507b43\") " pod="openshift-console/console-7d8c6997bb-8dt2h"
Jul 21 14:35:04.068048 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:04.068028    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/37b95e9b-ff18-406e-9f5b-840b9a507b43-console-serving-cert\") pod \"console-7d8c6997bb-8dt2h\" (UID: \"37b95e9b-ff18-406e-9f5b-840b9a507b43\") " pod="openshift-console/console-7d8c6997bb-8dt2h"
Jul 21 14:35:04.068129 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:04.068048    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/37b95e9b-ff18-406e-9f5b-840b9a507b43-console-oauth-config\") pod \"console-7d8c6997bb-8dt2h\" (UID: \"37b95e9b-ff18-406e-9f5b-840b9a507b43\") " pod="openshift-console/console-7d8c6997bb-8dt2h"
Jul 21 14:35:04.074126 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:04.074101    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xphzg\" (UniqueName: \"kubernetes.io/projected/37b95e9b-ff18-406e-9f5b-840b9a507b43-kube-api-access-xphzg\") pod \"console-7d8c6997bb-8dt2h\" (UID: \"37b95e9b-ff18-406e-9f5b-840b9a507b43\") " pod="openshift-console/console-7d8c6997bb-8dt2h"
Jul 21 14:35:04.156691 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:04.156659    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d8c6997bb-8dt2h"
Jul 21 14:35:04.289298 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:04.289262    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d8c6997bb-8dt2h"]
Jul 21 14:35:04.291560 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:35:04.291526    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37b95e9b_ff18_406e_9f5b_840b9a507b43.slice/crio-5aeaf37224855e6c2b03c1142473dc3cd081497b56247bbbaa39f125eaab8802 WatchSource:0}: Error finding container 5aeaf37224855e6c2b03c1142473dc3cd081497b56247bbbaa39f125eaab8802: Status 404 returned error can't find the container with id 5aeaf37224855e6c2b03c1142473dc3cd081497b56247bbbaa39f125eaab8802
Jul 21 14:35:04.772186 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:04.772152    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d8c6997bb-8dt2h" event={"ID":"37b95e9b-ff18-406e-9f5b-840b9a507b43","Type":"ContainerStarted","Data":"1b1213625f58bb56f33e5b3e0fbf1dff7cad2f44777643f7a291f94385ce9d7e"}
Jul 21 14:35:04.772341 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:04.772199    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d8c6997bb-8dt2h" event={"ID":"37b95e9b-ff18-406e-9f5b-840b9a507b43","Type":"ContainerStarted","Data":"5aeaf37224855e6c2b03c1142473dc3cd081497b56247bbbaa39f125eaab8802"}
Jul 21 14:35:04.790682 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:04.790630    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7d8c6997bb-8dt2h" podStartSLOduration=1.790615829 podStartE2EDuration="1.790615829s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:35:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:35:04.789624226 +0000 UTC m=+161.139367638" watchObservedRunningTime="2026-07-21 14:35:04.790615829 +0000 UTC m=+161.140359240"
Jul 21 14:35:14.157141 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:14.157112    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7d8c6997bb-8dt2h"
Jul 21 14:35:14.157618 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:14.157188    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7d8c6997bb-8dt2h"
Jul 21 14:35:14.161851 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:14.161832    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7d8c6997bb-8dt2h"
Jul 21 14:35:14.809368 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:14.809343    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7d8c6997bb-8dt2h"
Jul 21 14:35:14.858774 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:14.858740    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68ffff969-9ldkn"]
Jul 21 14:35:39.883681 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:39.883566    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-68ffff969-9ldkn" podUID="5122a0fa-4500-40e2-85d4-738dad277264" containerName="console" containerID="cri-o://babfafa4221b6054b5c4b4de96a2297bd2397bee4d9a335bf4fd83783620b63a" gracePeriod=15
Jul 21 14:35:40.125804 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:40.125783    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68ffff969-9ldkn"
Jul 21 14:35:40.219387 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:40.219325    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p2xz5\" (UniqueName: \"kubernetes.io/projected/5122a0fa-4500-40e2-85d4-738dad277264-kube-api-access-p2xz5\") pod \"5122a0fa-4500-40e2-85d4-738dad277264\" (UID: \"5122a0fa-4500-40e2-85d4-738dad277264\") "
Jul 21 14:35:40.219387 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:40.219376    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5122a0fa-4500-40e2-85d4-738dad277264-console-serving-cert\") pod \"5122a0fa-4500-40e2-85d4-738dad277264\" (UID: \"5122a0fa-4500-40e2-85d4-738dad277264\") "
Jul 21 14:35:40.219586 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:40.219395    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5122a0fa-4500-40e2-85d4-738dad277264-service-ca\") pod \"5122a0fa-4500-40e2-85d4-738dad277264\" (UID: \"5122a0fa-4500-40e2-85d4-738dad277264\") "
Jul 21 14:35:40.219586 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:40.219516    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5122a0fa-4500-40e2-85d4-738dad277264-oauth-serving-cert\") pod \"5122a0fa-4500-40e2-85d4-738dad277264\" (UID: \"5122a0fa-4500-40e2-85d4-738dad277264\") "
Jul 21 14:35:40.219586 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:40.219552    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5122a0fa-4500-40e2-85d4-738dad277264-console-config\") pod \"5122a0fa-4500-40e2-85d4-738dad277264\" (UID: \"5122a0fa-4500-40e2-85d4-738dad277264\") "
Jul 21 14:35:40.219737 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:40.219674    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5122a0fa-4500-40e2-85d4-738dad277264-console-oauth-config\") pod \"5122a0fa-4500-40e2-85d4-738dad277264\" (UID: \"5122a0fa-4500-40e2-85d4-738dad277264\") "
Jul 21 14:35:40.219737 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:40.219722    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5122a0fa-4500-40e2-85d4-738dad277264-trusted-ca-bundle\") pod \"5122a0fa-4500-40e2-85d4-738dad277264\" (UID: \"5122a0fa-4500-40e2-85d4-738dad277264\") "
Jul 21 14:35:40.219846 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:40.219821    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5122a0fa-4500-40e2-85d4-738dad277264-service-ca" (OuterVolumeSpecName: "service-ca") pod "5122a0fa-4500-40e2-85d4-738dad277264" (UID: "5122a0fa-4500-40e2-85d4-738dad277264"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:35:40.219975 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:40.219945    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5122a0fa-4500-40e2-85d4-738dad277264-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5122a0fa-4500-40e2-85d4-738dad277264" (UID: "5122a0fa-4500-40e2-85d4-738dad277264"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:35:40.220078 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:40.219961    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5122a0fa-4500-40e2-85d4-738dad277264-console-config" (OuterVolumeSpecName: "console-config") pod "5122a0fa-4500-40e2-85d4-738dad277264" (UID: "5122a0fa-4500-40e2-85d4-738dad277264"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:35:40.220137 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:40.219964    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5122a0fa-4500-40e2-85d4-738dad277264-service-ca\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:35:40.220137 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:40.220128    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5122a0fa-4500-40e2-85d4-738dad277264-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5122a0fa-4500-40e2-85d4-738dad277264" (UID: "5122a0fa-4500-40e2-85d4-738dad277264"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:35:40.221573 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:40.221543    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5122a0fa-4500-40e2-85d4-738dad277264-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5122a0fa-4500-40e2-85d4-738dad277264" (UID: "5122a0fa-4500-40e2-85d4-738dad277264"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:35:40.221573 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:40.221561    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5122a0fa-4500-40e2-85d4-738dad277264-kube-api-access-p2xz5" (OuterVolumeSpecName: "kube-api-access-p2xz5") pod "5122a0fa-4500-40e2-85d4-738dad277264" (UID: "5122a0fa-4500-40e2-85d4-738dad277264"). InnerVolumeSpecName "kube-api-access-p2xz5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:35:40.221694 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:40.221607    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5122a0fa-4500-40e2-85d4-738dad277264-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5122a0fa-4500-40e2-85d4-738dad277264" (UID: "5122a0fa-4500-40e2-85d4-738dad277264"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:35:40.321001 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:40.320981    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5122a0fa-4500-40e2-85d4-738dad277264-console-serving-cert\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:35:40.321001 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:40.321000    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5122a0fa-4500-40e2-85d4-738dad277264-oauth-serving-cert\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:35:40.321117 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:40.321009    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5122a0fa-4500-40e2-85d4-738dad277264-console-config\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:35:40.321117 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:40.321017    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5122a0fa-4500-40e2-85d4-738dad277264-console-oauth-config\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:35:40.321117 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:40.321026    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5122a0fa-4500-40e2-85d4-738dad277264-trusted-ca-bundle\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:35:40.321117 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:40.321034    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2xz5\" (UniqueName: \"kubernetes.io/projected/5122a0fa-4500-40e2-85d4-738dad277264-kube-api-access-p2xz5\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:35:40.893028 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:40.892996    2579 generic.go:356] "Generic (PLEG): container finished" podID="5122a0fa-4500-40e2-85d4-738dad277264" containerID="babfafa4221b6054b5c4b4de96a2297bd2397bee4d9a335bf4fd83783620b63a" exitCode=0
Jul 21 14:35:40.893500 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:40.893041    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68ffff969-9ldkn"
Jul 21 14:35:40.893500 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:40.893071    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68ffff969-9ldkn" event={"ID":"5122a0fa-4500-40e2-85d4-738dad277264","Type":"ContainerDied","Data":"babfafa4221b6054b5c4b4de96a2297bd2397bee4d9a335bf4fd83783620b63a"}
Jul 21 14:35:40.893500 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:40.893118    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68ffff969-9ldkn" event={"ID":"5122a0fa-4500-40e2-85d4-738dad277264","Type":"ContainerDied","Data":"600339f5def9c929487f98aa14997939e3e820b7cda9771b15c8d35f94ce5091"}
Jul 21 14:35:40.893500 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:40.893149    2579 scope.go:117] "RemoveContainer" containerID="babfafa4221b6054b5c4b4de96a2297bd2397bee4d9a335bf4fd83783620b63a"
Jul 21 14:35:40.901224 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:40.901206    2579 scope.go:117] "RemoveContainer" containerID="babfafa4221b6054b5c4b4de96a2297bd2397bee4d9a335bf4fd83783620b63a"
Jul 21 14:35:40.901488 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:35:40.901469    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"babfafa4221b6054b5c4b4de96a2297bd2397bee4d9a335bf4fd83783620b63a\": container with ID starting with babfafa4221b6054b5c4b4de96a2297bd2397bee4d9a335bf4fd83783620b63a not found: ID does not exist" containerID="babfafa4221b6054b5c4b4de96a2297bd2397bee4d9a335bf4fd83783620b63a"
Jul 21 14:35:40.901550 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:40.901499    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"babfafa4221b6054b5c4b4de96a2297bd2397bee4d9a335bf4fd83783620b63a"} err="failed to get container status \"babfafa4221b6054b5c4b4de96a2297bd2397bee4d9a335bf4fd83783620b63a\": rpc error: code = NotFound desc = could not find container \"babfafa4221b6054b5c4b4de96a2297bd2397bee4d9a335bf4fd83783620b63a\": container with ID starting with babfafa4221b6054b5c4b4de96a2297bd2397bee4d9a335bf4fd83783620b63a not found: ID does not exist"
Jul 21 14:35:40.912643 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:40.912618    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68ffff969-9ldkn"]
Jul 21 14:35:40.915618 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:40.915599    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-68ffff969-9ldkn"]
Jul 21 14:35:42.171094 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:35:42.171061    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5122a0fa-4500-40e2-85d4-738dad277264" path="/var/lib/kubelet/pods/5122a0fa-4500-40e2-85d4-738dad277264/volumes"
Jul 21 14:36:52.748090 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:36:52.748005    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jrbcp"]
Jul 21 14:36:52.748659 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:36:52.748359    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5122a0fa-4500-40e2-85d4-738dad277264" containerName="console"
Jul 21 14:36:52.748659 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:36:52.748378    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5122a0fa-4500-40e2-85d4-738dad277264" containerName="console"
Jul 21 14:36:52.748659 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:36:52.748443    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5122a0fa-4500-40e2-85d4-738dad277264" containerName="console"
Jul 21 14:36:52.751253 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:36:52.751231    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jrbcp"
Jul 21 14:36:52.753720 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:36:52.753702    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-qj6pp\""
Jul 21 14:36:52.753901 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:36:52.753884    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 21 14:36:52.753976 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:36:52.753961    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 21 14:36:52.766622 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:36:52.766600    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jrbcp"]
Jul 21 14:36:52.795256 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:36:52.795228    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/407e94e0-9f9b-4479-a8cf-b9103b915724-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jrbcp\" (UID: \"407e94e0-9f9b-4479-a8cf-b9103b915724\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jrbcp"
Jul 21 14:36:52.795363 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:36:52.795262    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vngp7\" (UniqueName: \"kubernetes.io/projected/407e94e0-9f9b-4479-a8cf-b9103b915724-kube-api-access-vngp7\") pod \"cert-manager-operator-controller-manager-55c444bf86-jrbcp\" (UID: \"407e94e0-9f9b-4479-a8cf-b9103b915724\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jrbcp"
Jul 21 14:36:52.896509 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:36:52.896487    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/407e94e0-9f9b-4479-a8cf-b9103b915724-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jrbcp\" (UID: \"407e94e0-9f9b-4479-a8cf-b9103b915724\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jrbcp"
Jul 21 14:36:52.896590 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:36:52.896515    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vngp7\" (UniqueName: \"kubernetes.io/projected/407e94e0-9f9b-4479-a8cf-b9103b915724-kube-api-access-vngp7\") pod \"cert-manager-operator-controller-manager-55c444bf86-jrbcp\" (UID: \"407e94e0-9f9b-4479-a8cf-b9103b915724\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jrbcp"
Jul 21 14:36:52.896835 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:36:52.896818    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/407e94e0-9f9b-4479-a8cf-b9103b915724-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jrbcp\" (UID: \"407e94e0-9f9b-4479-a8cf-b9103b915724\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jrbcp"
Jul 21 14:36:52.905465 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:36:52.905447    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vngp7\" (UniqueName: \"kubernetes.io/projected/407e94e0-9f9b-4479-a8cf-b9103b915724-kube-api-access-vngp7\") pod \"cert-manager-operator-controller-manager-55c444bf86-jrbcp\" (UID: \"407e94e0-9f9b-4479-a8cf-b9103b915724\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jrbcp"
Jul 21 14:36:53.060040 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:36:53.059991    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jrbcp"
Jul 21 14:36:53.186639 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:36:53.186612    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jrbcp"]
Jul 21 14:36:53.190309 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:36:53.190264    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod407e94e0_9f9b_4479_a8cf_b9103b915724.slice/crio-46bf71cf0150474b3f35f8b1a02fdb89590c19b0a0b4012283fbecadd324ede6 WatchSource:0}: Error finding container 46bf71cf0150474b3f35f8b1a02fdb89590c19b0a0b4012283fbecadd324ede6: Status 404 returned error can't find the container with id 46bf71cf0150474b3f35f8b1a02fdb89590c19b0a0b4012283fbecadd324ede6
Jul 21 14:36:54.129740 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:36:54.129705    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jrbcp" event={"ID":"407e94e0-9f9b-4479-a8cf-b9103b915724","Type":"ContainerStarted","Data":"46bf71cf0150474b3f35f8b1a02fdb89590c19b0a0b4012283fbecadd324ede6"}
Jul 21 14:36:57.143959 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:36:57.143912    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jrbcp" event={"ID":"407e94e0-9f9b-4479-a8cf-b9103b915724","Type":"ContainerStarted","Data":"fe53f75f426ced5e75fc3e2d160fee35e2b0aa60ba729b96754ee6c2710cc831"}
Jul 21 14:36:57.165898 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:36:57.165841    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jrbcp" podStartSLOduration=1.6712901260000002 podStartE2EDuration="5.165824022s" totalImagesPullingTime="3.494533896s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:36:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:36:57.164848909 +0000 UTC m=+273.514592321" watchObservedRunningTime="2026-07-21 14:36:57.165824022 +0000 UTC m=+273.515567433"
Jul 21 14:37:24.059984 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:24.059958    2579 kubelet.go:1648] "Image garbage collection succeeded"
Jul 21 14:37:36.209625 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:36.209591    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b978d8c-gpnqz"]
Jul 21 14:37:36.212766 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:36.212751    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-b978d8c-gpnqz"
Jul 21 14:37:36.216495 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:36.216469    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 21 14:37:36.216628 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:36.216492    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 21 14:37:36.217023 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:36.217003    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 21 14:37:36.217204 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:36.217190    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-2tzgd\""
Jul 21 14:37:36.217245 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:36.217209    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 21 14:37:36.256304 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:36.256285    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b978d8c-gpnqz"]
Jul 21 14:37:36.319311 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:36.319285    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rn9vv\" (UniqueName: \"kubernetes.io/projected/0e657220-fcc4-4498-b47b-c757abba31c7-kube-api-access-rn9vv\") pod \"opendatahub-operator-controller-manager-b978d8c-gpnqz\" (UID: \"0e657220-fcc4-4498-b47b-c757abba31c7\") " pod="opendatahub/opendatahub-operator-controller-manager-b978d8c-gpnqz"
Jul 21 14:37:36.319418 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:36.319361    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0e657220-fcc4-4498-b47b-c757abba31c7-tmp\") pod \"opendatahub-operator-controller-manager-b978d8c-gpnqz\" (UID: \"0e657220-fcc4-4498-b47b-c757abba31c7\") " pod="opendatahub/opendatahub-operator-controller-manager-b978d8c-gpnqz"
Jul 21 14:37:36.319418 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:36.319389    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0e657220-fcc4-4498-b47b-c757abba31c7-webhook-cert\") pod \"opendatahub-operator-controller-manager-b978d8c-gpnqz\" (UID: \"0e657220-fcc4-4498-b47b-c757abba31c7\") " pod="opendatahub/opendatahub-operator-controller-manager-b978d8c-gpnqz"
Jul 21 14:37:36.319418 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:36.319413    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0e657220-fcc4-4498-b47b-c757abba31c7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b978d8c-gpnqz\" (UID: \"0e657220-fcc4-4498-b47b-c757abba31c7\") " pod="opendatahub/opendatahub-operator-controller-manager-b978d8c-gpnqz"
Jul 21 14:37:36.319567 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:36.319469    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0e657220-fcc4-4498-b47b-c757abba31c7-manifests\") pod \"opendatahub-operator-controller-manager-b978d8c-gpnqz\" (UID: \"0e657220-fcc4-4498-b47b-c757abba31c7\") " pod="opendatahub/opendatahub-operator-controller-manager-b978d8c-gpnqz"
Jul 21 14:37:36.420214 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:36.420188    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rn9vv\" (UniqueName: \"kubernetes.io/projected/0e657220-fcc4-4498-b47b-c757abba31c7-kube-api-access-rn9vv\") pod \"opendatahub-operator-controller-manager-b978d8c-gpnqz\" (UID: \"0e657220-fcc4-4498-b47b-c757abba31c7\") " pod="opendatahub/opendatahub-operator-controller-manager-b978d8c-gpnqz"
Jul 21 14:37:36.420332 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:36.420241    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0e657220-fcc4-4498-b47b-c757abba31c7-tmp\") pod \"opendatahub-operator-controller-manager-b978d8c-gpnqz\" (UID: \"0e657220-fcc4-4498-b47b-c757abba31c7\") " pod="opendatahub/opendatahub-operator-controller-manager-b978d8c-gpnqz"
Jul 21 14:37:36.420405 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:36.420349    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0e657220-fcc4-4498-b47b-c757abba31c7-webhook-cert\") pod \"opendatahub-operator-controller-manager-b978d8c-gpnqz\" (UID: \"0e657220-fcc4-4498-b47b-c757abba31c7\") " pod="opendatahub/opendatahub-operator-controller-manager-b978d8c-gpnqz"
Jul 21 14:37:36.420405 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:36.420390    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0e657220-fcc4-4498-b47b-c757abba31c7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b978d8c-gpnqz\" (UID: \"0e657220-fcc4-4498-b47b-c757abba31c7\") " pod="opendatahub/opendatahub-operator-controller-manager-b978d8c-gpnqz"
Jul 21 14:37:36.420501 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:36.420421    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0e657220-fcc4-4498-b47b-c757abba31c7-manifests\") pod \"opendatahub-operator-controller-manager-b978d8c-gpnqz\" (UID: \"0e657220-fcc4-4498-b47b-c757abba31c7\") " pod="opendatahub/opendatahub-operator-controller-manager-b978d8c-gpnqz"
Jul 21 14:37:36.420740 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:36.420713    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0e657220-fcc4-4498-b47b-c757abba31c7-manifests\") pod \"opendatahub-operator-controller-manager-b978d8c-gpnqz\" (UID: \"0e657220-fcc4-4498-b47b-c757abba31c7\") " pod="opendatahub/opendatahub-operator-controller-manager-b978d8c-gpnqz"
Jul 21 14:37:36.422295 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:36.422260    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0e657220-fcc4-4498-b47b-c757abba31c7-tmp\") pod \"opendatahub-operator-controller-manager-b978d8c-gpnqz\" (UID: \"0e657220-fcc4-4498-b47b-c757abba31c7\") " pod="opendatahub/opendatahub-operator-controller-manager-b978d8c-gpnqz"
Jul 21 14:37:36.422643 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:36.422624    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0e657220-fcc4-4498-b47b-c757abba31c7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b978d8c-gpnqz\" (UID: \"0e657220-fcc4-4498-b47b-c757abba31c7\") " pod="opendatahub/opendatahub-operator-controller-manager-b978d8c-gpnqz"
Jul 21 14:37:36.422707 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:36.422677    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0e657220-fcc4-4498-b47b-c757abba31c7-webhook-cert\") pod \"opendatahub-operator-controller-manager-b978d8c-gpnqz\" (UID: \"0e657220-fcc4-4498-b47b-c757abba31c7\") " pod="opendatahub/opendatahub-operator-controller-manager-b978d8c-gpnqz"
Jul 21 14:37:36.428910 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:36.428890    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rn9vv\" (UniqueName: \"kubernetes.io/projected/0e657220-fcc4-4498-b47b-c757abba31c7-kube-api-access-rn9vv\") pod \"opendatahub-operator-controller-manager-b978d8c-gpnqz\" (UID: \"0e657220-fcc4-4498-b47b-c757abba31c7\") " pod="opendatahub/opendatahub-operator-controller-manager-b978d8c-gpnqz"
Jul 21 14:37:36.522439 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:36.522392    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-b978d8c-gpnqz"
Jul 21 14:37:36.650680 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:36.650654    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b978d8c-gpnqz"]
Jul 21 14:37:36.654309 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:37:36.654279    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e657220_fcc4_4498_b47b_c757abba31c7.slice/crio-fa94e03b3a4ecf8202ac7749ac030e3b9f7fa91e95eb2e8c8d492a87a18f40e0 WatchSource:0}: Error finding container fa94e03b3a4ecf8202ac7749ac030e3b9f7fa91e95eb2e8c8d492a87a18f40e0: Status 404 returned error can't find the container with id fa94e03b3a4ecf8202ac7749ac030e3b9f7fa91e95eb2e8c8d492a87a18f40e0
Jul 21 14:37:36.656113 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:36.656098    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 14:37:37.273337 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:37.273303    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b978d8c-gpnqz" event={"ID":"0e657220-fcc4-4498-b47b-c757abba31c7","Type":"ContainerStarted","Data":"fa94e03b3a4ecf8202ac7749ac030e3b9f7fa91e95eb2e8c8d492a87a18f40e0"}
Jul 21 14:37:40.287455 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:40.287423    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b978d8c-gpnqz" event={"ID":"0e657220-fcc4-4498-b47b-c757abba31c7","Type":"ContainerStarted","Data":"eb593df446028039f32769f3be0fe031a1bedc04b36cbdfe8d5e906a94d6123c"}
Jul 21 14:37:41.292311 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:41.292254    2579 generic.go:356] "Generic (PLEG): container finished" podID="0e657220-fcc4-4498-b47b-c757abba31c7" containerID="eb593df446028039f32769f3be0fe031a1bedc04b36cbdfe8d5e906a94d6123c" exitCode=0
Jul 21 14:37:41.292708 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:41.292320    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b978d8c-gpnqz" event={"ID":"0e657220-fcc4-4498-b47b-c757abba31c7","Type":"ContainerDied","Data":"eb593df446028039f32769f3be0fe031a1bedc04b36cbdfe8d5e906a94d6123c"}
Jul 21 14:37:42.298005 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:42.297972    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b978d8c-gpnqz" event={"ID":"0e657220-fcc4-4498-b47b-c757abba31c7","Type":"ContainerStarted","Data":"489eb886ff74137aaba1cbebbf9bb449c5fafc060a1f5e98139b5d5bf8f04049"}
Jul 21 14:37:42.298402 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:42.298104    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-b978d8c-gpnqz"
Jul 21 14:37:42.325674 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:42.325627    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-b978d8c-gpnqz" podStartSLOduration=1.874634613 podStartE2EDuration="6.325614909s" totalImagesPullingTime="3.419041977s" totalInitContainerRuntime="1.031938319s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:37:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:37:42.324165918 +0000 UTC m=+318.673909341" watchObservedRunningTime="2026-07-21 14:37:42.325614909 +0000 UTC m=+318.675358321"
Jul 21 14:37:53.303197 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:53.303165    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-b978d8c-gpnqz"
Jul 21 14:37:56.320288 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:56.320245    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-76j8p"]
Jul 21 14:37:56.326937 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:56.326915    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-76j8p"
Jul 21 14:37:56.329933 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:56.329912    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 21 14:37:56.330054 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:56.329980    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-9dnv5\""
Jul 21 14:37:56.336879 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:56.336811    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-76j8p"]
Jul 21 14:37:56.354549 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:56.354522    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fae5f2ef-b1c4-44a8-9016-b74864a3dc50-cert\") pod \"odh-model-controller-7998c67dc4-76j8p\" (UID: \"fae5f2ef-b1c4-44a8-9016-b74864a3dc50\") " pod="opendatahub/odh-model-controller-7998c67dc4-76j8p"
Jul 21 14:37:56.354789 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:56.354771    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzkzq\" (UniqueName: \"kubernetes.io/projected/fae5f2ef-b1c4-44a8-9016-b74864a3dc50-kube-api-access-nzkzq\") pod \"odh-model-controller-7998c67dc4-76j8p\" (UID: \"fae5f2ef-b1c4-44a8-9016-b74864a3dc50\") " pod="opendatahub/odh-model-controller-7998c67dc4-76j8p"
Jul 21 14:37:56.456169 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:56.456135    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fae5f2ef-b1c4-44a8-9016-b74864a3dc50-cert\") pod \"odh-model-controller-7998c67dc4-76j8p\" (UID: \"fae5f2ef-b1c4-44a8-9016-b74864a3dc50\") " pod="opendatahub/odh-model-controller-7998c67dc4-76j8p"
Jul 21 14:37:56.456354 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:56.456196    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzkzq\" (UniqueName: \"kubernetes.io/projected/fae5f2ef-b1c4-44a8-9016-b74864a3dc50-kube-api-access-nzkzq\") pod \"odh-model-controller-7998c67dc4-76j8p\" (UID: \"fae5f2ef-b1c4-44a8-9016-b74864a3dc50\") " pod="opendatahub/odh-model-controller-7998c67dc4-76j8p"
Jul 21 14:37:56.456354 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:37:56.456331    2579 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 21 14:37:56.456454 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:37:56.456409    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fae5f2ef-b1c4-44a8-9016-b74864a3dc50-cert podName:fae5f2ef-b1c4-44a8-9016-b74864a3dc50 nodeName:}" failed. No retries permitted until 2026-07-21 14:37:56.956387773 +0000 UTC m=+333.306131167 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/fae5f2ef-b1c4-44a8-9016-b74864a3dc50-cert") pod "odh-model-controller-7998c67dc4-76j8p" (UID: "fae5f2ef-b1c4-44a8-9016-b74864a3dc50") : secret "odh-model-controller-webhook-cert" not found
Jul 21 14:37:56.465765 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:56.465734    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzkzq\" (UniqueName: \"kubernetes.io/projected/fae5f2ef-b1c4-44a8-9016-b74864a3dc50-kube-api-access-nzkzq\") pod \"odh-model-controller-7998c67dc4-76j8p\" (UID: \"fae5f2ef-b1c4-44a8-9016-b74864a3dc50\") " pod="opendatahub/odh-model-controller-7998c67dc4-76j8p"
Jul 21 14:37:56.960642 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:56.960605    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fae5f2ef-b1c4-44a8-9016-b74864a3dc50-cert\") pod \"odh-model-controller-7998c67dc4-76j8p\" (UID: \"fae5f2ef-b1c4-44a8-9016-b74864a3dc50\") " pod="opendatahub/odh-model-controller-7998c67dc4-76j8p"
Jul 21 14:37:56.963133 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:56.963097    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fae5f2ef-b1c4-44a8-9016-b74864a3dc50-cert\") pod \"odh-model-controller-7998c67dc4-76j8p\" (UID: \"fae5f2ef-b1c4-44a8-9016-b74864a3dc50\") " pod="opendatahub/odh-model-controller-7998c67dc4-76j8p"
Jul 21 14:37:57.238664 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:57.238580    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-76j8p"
Jul 21 14:37:57.371842 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:57.371818    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-76j8p"]
Jul 21 14:37:57.373813 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:37:57.373784    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfae5f2ef_b1c4_44a8_9016_b74864a3dc50.slice/crio-b033494f758663294de89f67fa4a5d9aa41cbf30ad29165c6b19d0d3b2562b7f WatchSource:0}: Error finding container b033494f758663294de89f67fa4a5d9aa41cbf30ad29165c6b19d0d3b2562b7f: Status 404 returned error can't find the container with id b033494f758663294de89f67fa4a5d9aa41cbf30ad29165c6b19d0d3b2562b7f
Jul 21 14:37:58.363834 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:37:58.363794    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-76j8p" event={"ID":"fae5f2ef-b1c4-44a8-9016-b74864a3dc50","Type":"ContainerStarted","Data":"b033494f758663294de89f67fa4a5d9aa41cbf30ad29165c6b19d0d3b2562b7f"}
Jul 21 14:38:02.380548 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:02.380508    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-76j8p" event={"ID":"fae5f2ef-b1c4-44a8-9016-b74864a3dc50","Type":"ContainerStarted","Data":"86a78d1de82fa949b98d32b748bca1609b1f3ec7e123f86c406f5a1117a19031"}
Jul 21 14:38:02.380923 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:02.380573    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-76j8p"
Jul 21 14:38:02.398343 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:02.398295    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-76j8p" podStartSLOduration=2.469566192 podStartE2EDuration="6.398259101s" totalImagesPullingTime="3.928692909s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:37:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:38:02.396524752 +0000 UTC m=+338.746268163" watchObservedRunningTime="2026-07-21 14:38:02.398259101 +0000 UTC m=+338.748002561"
Jul 21 14:38:05.958634 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:05.958599    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-24blz"]
Jul 21 14:38:05.961538 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:05.961522    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-24blz"
Jul 21 14:38:05.964140 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:05.964120    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 21 14:38:05.964280 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:05.964154    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-cjxpv\""
Jul 21 14:38:05.972131 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:05.972109    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7psmb"]
Jul 21 14:38:05.976837 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:05.976786    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7psmb"
Jul 21 14:38:05.977807 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:05.977783    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-24blz"]
Jul 21 14:38:05.979506 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:05.979473    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-cq4l8\""
Jul 21 14:38:05.979725 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:05.979697    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 21 14:38:05.985062 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:05.985022    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7psmb"]
Jul 21 14:38:06.028946 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:06.028913    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86ee645c-7f41-4aac-afaf-285233965458-cert\") pod \"kserve-controller-manager-d6f4b65b8-24blz\" (UID: \"86ee645c-7f41-4aac-afaf-285233965458\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-24blz"
Jul 21 14:38:06.029127 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:06.028953    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6q6f\" (UniqueName: \"kubernetes.io/projected/86ee645c-7f41-4aac-afaf-285233965458-kube-api-access-h6q6f\") pod \"kserve-controller-manager-d6f4b65b8-24blz\" (UID: \"86ee645c-7f41-4aac-afaf-285233965458\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-24blz"
Jul 21 14:38:06.029127 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:06.029038    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48gj2\" (UniqueName: \"kubernetes.io/projected/2cac96bd-ceca-4fc0-b229-601105cb5647-kube-api-access-48gj2\") pod \"llmisvc-controller-manager-777c6bd7d5-7psmb\" (UID: \"2cac96bd-ceca-4fc0-b229-601105cb5647\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7psmb"
Jul 21 14:38:06.029127 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:06.029077    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2cac96bd-ceca-4fc0-b229-601105cb5647-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7psmb\" (UID: \"2cac96bd-ceca-4fc0-b229-601105cb5647\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7psmb"
Jul 21 14:38:06.130541 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:06.130504    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86ee645c-7f41-4aac-afaf-285233965458-cert\") pod \"kserve-controller-manager-d6f4b65b8-24blz\" (UID: \"86ee645c-7f41-4aac-afaf-285233965458\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-24blz"
Jul 21 14:38:06.130741 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:06.130553    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6q6f\" (UniqueName: \"kubernetes.io/projected/86ee645c-7f41-4aac-afaf-285233965458-kube-api-access-h6q6f\") pod \"kserve-controller-manager-d6f4b65b8-24blz\" (UID: \"86ee645c-7f41-4aac-afaf-285233965458\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-24blz"
Jul 21 14:38:06.130741 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:06.130599    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48gj2\" (UniqueName: \"kubernetes.io/projected/2cac96bd-ceca-4fc0-b229-601105cb5647-kube-api-access-48gj2\") pod \"llmisvc-controller-manager-777c6bd7d5-7psmb\" (UID: \"2cac96bd-ceca-4fc0-b229-601105cb5647\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7psmb"
Jul 21 14:38:06.130741 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:06.130637    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2cac96bd-ceca-4fc0-b229-601105cb5647-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7psmb\" (UID: \"2cac96bd-ceca-4fc0-b229-601105cb5647\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7psmb"
Jul 21 14:38:06.130912 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:38:06.130776    2579 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 21 14:38:06.130912 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:38:06.130838    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cac96bd-ceca-4fc0-b229-601105cb5647-cert podName:2cac96bd-ceca-4fc0-b229-601105cb5647 nodeName:}" failed. No retries permitted until 2026-07-21 14:38:06.630818069 +0000 UTC m=+342.980561467 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/2cac96bd-ceca-4fc0-b229-601105cb5647-cert") pod "llmisvc-controller-manager-777c6bd7d5-7psmb" (UID: "2cac96bd-ceca-4fc0-b229-601105cb5647") : secret "llmisvc-webhook-server-cert" not found
Jul 21 14:38:06.133594 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:06.133565    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86ee645c-7f41-4aac-afaf-285233965458-cert\") pod \"kserve-controller-manager-d6f4b65b8-24blz\" (UID: \"86ee645c-7f41-4aac-afaf-285233965458\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-24blz"
Jul 21 14:38:06.144079 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:06.144053    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48gj2\" (UniqueName: \"kubernetes.io/projected/2cac96bd-ceca-4fc0-b229-601105cb5647-kube-api-access-48gj2\") pod \"llmisvc-controller-manager-777c6bd7d5-7psmb\" (UID: \"2cac96bd-ceca-4fc0-b229-601105cb5647\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7psmb"
Jul 21 14:38:06.145302 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:06.145260    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6q6f\" (UniqueName: \"kubernetes.io/projected/86ee645c-7f41-4aac-afaf-285233965458-kube-api-access-h6q6f\") pod \"kserve-controller-manager-d6f4b65b8-24blz\" (UID: \"86ee645c-7f41-4aac-afaf-285233965458\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-24blz"
Jul 21 14:38:06.273627 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:06.273536    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-24blz"
Jul 21 14:38:06.400378 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:06.400351    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-24blz"]
Jul 21 14:38:06.402886 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:38:06.402858    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86ee645c_7f41_4aac_afaf_285233965458.slice/crio-9f435a229de5fd88264977d0f548b6af415c32fdb7cabdb70ebf4e93e41b3dc8 WatchSource:0}: Error finding container 9f435a229de5fd88264977d0f548b6af415c32fdb7cabdb70ebf4e93e41b3dc8: Status 404 returned error can't find the container with id 9f435a229de5fd88264977d0f548b6af415c32fdb7cabdb70ebf4e93e41b3dc8
Jul 21 14:38:06.612448 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:06.612366    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-74d9d9b8c-dnpv2"]
Jul 21 14:38:06.616815 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:06.616796    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-74d9d9b8c-dnpv2"
Jul 21 14:38:06.619407 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:06.619378    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 21 14:38:06.619545 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:06.619418    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 21 14:38:06.619628 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:06.619552    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 21 14:38:06.619628 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:06.619614    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 21 14:38:06.630983 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:06.630951    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-74d9d9b8c-dnpv2"]
Jul 21 14:38:06.636198 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:06.636157    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k92l8\" (UniqueName: \"kubernetes.io/projected/d1c73e43-f9e9-49f3-af4b-e411a0a21f1b-kube-api-access-k92l8\") pod \"kube-auth-proxy-74d9d9b8c-dnpv2\" (UID: \"d1c73e43-f9e9-49f3-af4b-e411a0a21f1b\") " pod="openshift-ingress/kube-auth-proxy-74d9d9b8c-dnpv2"
Jul 21 14:38:06.636331 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:06.636219    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2cac96bd-ceca-4fc0-b229-601105cb5647-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7psmb\" (UID: \"2cac96bd-ceca-4fc0-b229-601105cb5647\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7psmb"
Jul 21 14:38:06.636331 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:06.636286    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1c73e43-f9e9-49f3-af4b-e411a0a21f1b-tls-certs\") pod \"kube-auth-proxy-74d9d9b8c-dnpv2\" (UID: \"d1c73e43-f9e9-49f3-af4b-e411a0a21f1b\") " pod="openshift-ingress/kube-auth-proxy-74d9d9b8c-dnpv2"
Jul 21 14:38:06.636448 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:06.636390    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d1c73e43-f9e9-49f3-af4b-e411a0a21f1b-tmp\") pod \"kube-auth-proxy-74d9d9b8c-dnpv2\" (UID: \"d1c73e43-f9e9-49f3-af4b-e411a0a21f1b\") " pod="openshift-ingress/kube-auth-proxy-74d9d9b8c-dnpv2"
Jul 21 14:38:06.638880 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:06.638858    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2cac96bd-ceca-4fc0-b229-601105cb5647-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7psmb\" (UID: \"2cac96bd-ceca-4fc0-b229-601105cb5647\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7psmb"
Jul 21 14:38:06.737291 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:06.737225    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k92l8\" (UniqueName: \"kubernetes.io/projected/d1c73e43-f9e9-49f3-af4b-e411a0a21f1b-kube-api-access-k92l8\") pod \"kube-auth-proxy-74d9d9b8c-dnpv2\" (UID: \"d1c73e43-f9e9-49f3-af4b-e411a0a21f1b\") " pod="openshift-ingress/kube-auth-proxy-74d9d9b8c-dnpv2"
Jul 21 14:38:06.737498 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:06.737300    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1c73e43-f9e9-49f3-af4b-e411a0a21f1b-tls-certs\") pod \"kube-auth-proxy-74d9d9b8c-dnpv2\" (UID: \"d1c73e43-f9e9-49f3-af4b-e411a0a21f1b\") " pod="openshift-ingress/kube-auth-proxy-74d9d9b8c-dnpv2"
Jul 21 14:38:06.737498 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:06.737363    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d1c73e43-f9e9-49f3-af4b-e411a0a21f1b-tmp\") pod \"kube-auth-proxy-74d9d9b8c-dnpv2\" (UID: \"d1c73e43-f9e9-49f3-af4b-e411a0a21f1b\") " pod="openshift-ingress/kube-auth-proxy-74d9d9b8c-dnpv2"
Jul 21 14:38:06.737498 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:38:06.737406    2579 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 21 14:38:06.737498 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:38:06.737497    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1c73e43-f9e9-49f3-af4b-e411a0a21f1b-tls-certs podName:d1c73e43-f9e9-49f3-af4b-e411a0a21f1b nodeName:}" failed. No retries permitted until 2026-07-21 14:38:07.237475355 +0000 UTC m=+343.587218744 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/d1c73e43-f9e9-49f3-af4b-e411a0a21f1b-tls-certs") pod "kube-auth-proxy-74d9d9b8c-dnpv2" (UID: "d1c73e43-f9e9-49f3-af4b-e411a0a21f1b") : secret "kube-auth-proxy-tls" not found
Jul 21 14:38:06.739659 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:06.739634    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d1c73e43-f9e9-49f3-af4b-e411a0a21f1b-tmp\") pod \"kube-auth-proxy-74d9d9b8c-dnpv2\" (UID: \"d1c73e43-f9e9-49f3-af4b-e411a0a21f1b\") " pod="openshift-ingress/kube-auth-proxy-74d9d9b8c-dnpv2"
Jul 21 14:38:06.752874 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:06.752840    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k92l8\" (UniqueName: \"kubernetes.io/projected/d1c73e43-f9e9-49f3-af4b-e411a0a21f1b-kube-api-access-k92l8\") pod \"kube-auth-proxy-74d9d9b8c-dnpv2\" (UID: \"d1c73e43-f9e9-49f3-af4b-e411a0a21f1b\") " pod="openshift-ingress/kube-auth-proxy-74d9d9b8c-dnpv2"
Jul 21 14:38:06.888770 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:06.888738    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7psmb"
Jul 21 14:38:07.018663 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:07.018640    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7psmb"]
Jul 21 14:38:07.020754 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:38:07.020721    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2cac96bd_ceca_4fc0_b229_601105cb5647.slice/crio-bdea4636745c367de99349b041dbb9d5e333576bfc5a851735949df00a11cfc8 WatchSource:0}: Error finding container bdea4636745c367de99349b041dbb9d5e333576bfc5a851735949df00a11cfc8: Status 404 returned error can't find the container with id bdea4636745c367de99349b041dbb9d5e333576bfc5a851735949df00a11cfc8
Jul 21 14:38:07.242609 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:07.242514    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1c73e43-f9e9-49f3-af4b-e411a0a21f1b-tls-certs\") pod \"kube-auth-proxy-74d9d9b8c-dnpv2\" (UID: \"d1c73e43-f9e9-49f3-af4b-e411a0a21f1b\") " pod="openshift-ingress/kube-auth-proxy-74d9d9b8c-dnpv2"
Jul 21 14:38:07.244929 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:07.244906    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1c73e43-f9e9-49f3-af4b-e411a0a21f1b-tls-certs\") pod \"kube-auth-proxy-74d9d9b8c-dnpv2\" (UID: \"d1c73e43-f9e9-49f3-af4b-e411a0a21f1b\") " pod="openshift-ingress/kube-auth-proxy-74d9d9b8c-dnpv2"
Jul 21 14:38:07.400506 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:07.400468    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7psmb" event={"ID":"2cac96bd-ceca-4fc0-b229-601105cb5647","Type":"ContainerStarted","Data":"bdea4636745c367de99349b041dbb9d5e333576bfc5a851735949df00a11cfc8"}
Jul 21 14:38:07.401429 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:07.401409    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-24blz" event={"ID":"86ee645c-7f41-4aac-afaf-285233965458","Type":"ContainerStarted","Data":"9f435a229de5fd88264977d0f548b6af415c32fdb7cabdb70ebf4e93e41b3dc8"}
Jul 21 14:38:07.530945 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:07.530847    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-74d9d9b8c-dnpv2"
Jul 21 14:38:07.701360 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:07.701240    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-74d9d9b8c-dnpv2"]
Jul 21 14:38:07.704412 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:38:07.704383    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1c73e43_f9e9_49f3_af4b_e411a0a21f1b.slice/crio-a22d33ea6d2f5409bb9d81a281acb63a21f90b3014f238b3c5c8479ae7393561 WatchSource:0}: Error finding container a22d33ea6d2f5409bb9d81a281acb63a21f90b3014f238b3c5c8479ae7393561: Status 404 returned error can't find the container with id a22d33ea6d2f5409bb9d81a281acb63a21f90b3014f238b3c5c8479ae7393561
Jul 21 14:38:08.408392 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:08.408304    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-74d9d9b8c-dnpv2" event={"ID":"d1c73e43-f9e9-49f3-af4b-e411a0a21f1b","Type":"ContainerStarted","Data":"a22d33ea6d2f5409bb9d81a281acb63a21f90b3014f238b3c5c8479ae7393561"}
Jul 21 14:38:08.601689 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:08.601634    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b22v6"]
Jul 21 14:38:08.605050 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:08.605025    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b22v6"
Jul 21 14:38:08.607876 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:08.607847    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 21 14:38:08.608567 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:08.608191    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 21 14:38:08.608567 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:08.608334    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-lt6n5\""
Jul 21 14:38:08.608567 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:08.608440    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 21 14:38:08.608567 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:08.608527    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 21 14:38:08.623064 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:08.623029    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b22v6"]
Jul 21 14:38:08.656306 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:08.656247    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/88af516f-4d3f-43f0-8701-4dd909af8e39-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-b22v6\" (UID: \"88af516f-4d3f-43f0-8701-4dd909af8e39\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b22v6"
Jul 21 14:38:08.656533 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:08.656334    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwbbj\" (UniqueName: \"kubernetes.io/projected/88af516f-4d3f-43f0-8701-4dd909af8e39-kube-api-access-vwbbj\") pod \"istiod-openshift-gateway-94bb8fbfd-b22v6\" (UID: \"88af516f-4d3f-43f0-8701-4dd909af8e39\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b22v6"
Jul 21 14:38:08.656533 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:08.656371    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88af516f-4d3f-43f0-8701-4dd909af8e39-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-b22v6\" (UID: \"88af516f-4d3f-43f0-8701-4dd909af8e39\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b22v6"
Jul 21 14:38:08.656533 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:08.656436    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/88af516f-4d3f-43f0-8701-4dd909af8e39-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-b22v6\" (UID: \"88af516f-4d3f-43f0-8701-4dd909af8e39\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b22v6"
Jul 21 14:38:08.656533 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:08.656469    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/88af516f-4d3f-43f0-8701-4dd909af8e39-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-b22v6\" (UID: \"88af516f-4d3f-43f0-8701-4dd909af8e39\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b22v6"
Jul 21 14:38:08.656533 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:08.656523    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/88af516f-4d3f-43f0-8701-4dd909af8e39-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-b22v6\" (UID: \"88af516f-4d3f-43f0-8701-4dd909af8e39\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b22v6"
Jul 21 14:38:08.656819 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:08.656561    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/88af516f-4d3f-43f0-8701-4dd909af8e39-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-b22v6\" (UID: \"88af516f-4d3f-43f0-8701-4dd909af8e39\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b22v6"
Jul 21 14:38:08.757938 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:08.757396    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/88af516f-4d3f-43f0-8701-4dd909af8e39-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-b22v6\" (UID: \"88af516f-4d3f-43f0-8701-4dd909af8e39\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b22v6"
Jul 21 14:38:08.757938 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:08.757467    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwbbj\" (UniqueName: \"kubernetes.io/projected/88af516f-4d3f-43f0-8701-4dd909af8e39-kube-api-access-vwbbj\") pod \"istiod-openshift-gateway-94bb8fbfd-b22v6\" (UID: \"88af516f-4d3f-43f0-8701-4dd909af8e39\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b22v6"
Jul 21 14:38:08.757938 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:08.757502    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88af516f-4d3f-43f0-8701-4dd909af8e39-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-b22v6\" (UID: \"88af516f-4d3f-43f0-8701-4dd909af8e39\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b22v6"
Jul 21 14:38:08.757938 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:08.757562    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/88af516f-4d3f-43f0-8701-4dd909af8e39-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-b22v6\" (UID: \"88af516f-4d3f-43f0-8701-4dd909af8e39\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b22v6"
Jul 21 14:38:08.757938 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:08.757592    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/88af516f-4d3f-43f0-8701-4dd909af8e39-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-b22v6\" (UID: \"88af516f-4d3f-43f0-8701-4dd909af8e39\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b22v6"
Jul 21 14:38:08.757938 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:08.757619    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/88af516f-4d3f-43f0-8701-4dd909af8e39-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-b22v6\" (UID: \"88af516f-4d3f-43f0-8701-4dd909af8e39\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b22v6"
Jul 21 14:38:08.757938 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:08.757643    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/88af516f-4d3f-43f0-8701-4dd909af8e39-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-b22v6\" (UID: \"88af516f-4d3f-43f0-8701-4dd909af8e39\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b22v6"
Jul 21 14:38:08.759041 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:08.758720    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/88af516f-4d3f-43f0-8701-4dd909af8e39-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-b22v6\" (UID: \"88af516f-4d3f-43f0-8701-4dd909af8e39\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b22v6"
Jul 21 14:38:08.761063 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:08.761016    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/88af516f-4d3f-43f0-8701-4dd909af8e39-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-b22v6\" (UID: \"88af516f-4d3f-43f0-8701-4dd909af8e39\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b22v6"
Jul 21 14:38:08.761644 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:08.761592    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/88af516f-4d3f-43f0-8701-4dd909af8e39-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-b22v6\" (UID: \"88af516f-4d3f-43f0-8701-4dd909af8e39\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b22v6"
Jul 21 14:38:08.762398 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:08.762376    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/88af516f-4d3f-43f0-8701-4dd909af8e39-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-b22v6\" (UID: \"88af516f-4d3f-43f0-8701-4dd909af8e39\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b22v6"
Jul 21 14:38:08.762636 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:08.762618    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/88af516f-4d3f-43f0-8701-4dd909af8e39-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-b22v6\" (UID: \"88af516f-4d3f-43f0-8701-4dd909af8e39\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b22v6"
Jul 21 14:38:08.768873 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:08.768814    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwbbj\" (UniqueName: \"kubernetes.io/projected/88af516f-4d3f-43f0-8701-4dd909af8e39-kube-api-access-vwbbj\") pod \"istiod-openshift-gateway-94bb8fbfd-b22v6\" (UID: \"88af516f-4d3f-43f0-8701-4dd909af8e39\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b22v6"
Jul 21 14:38:08.769064 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:08.769044    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/88af516f-4d3f-43f0-8701-4dd909af8e39-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-b22v6\" (UID: \"88af516f-4d3f-43f0-8701-4dd909af8e39\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b22v6"
Jul 21 14:38:08.918014 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:08.917978    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b22v6"
Jul 21 14:38:09.724711 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:09.724680    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b22v6"]
Jul 21 14:38:10.323127 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:38:10.323092    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88af516f_4d3f_43f0_8701_4dd909af8e39.slice/crio-46920f51c4e730940bb7c960e4c5b0a66880cb97730169ec635f55bc03510cf1 WatchSource:0}: Error finding container 46920f51c4e730940bb7c960e4c5b0a66880cb97730169ec635f55bc03510cf1: Status 404 returned error can't find the container with id 46920f51c4e730940bb7c960e4c5b0a66880cb97730169ec635f55bc03510cf1
Jul 21 14:38:10.419398 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:10.419365    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b22v6" event={"ID":"88af516f-4d3f-43f0-8701-4dd909af8e39","Type":"ContainerStarted","Data":"46920f51c4e730940bb7c960e4c5b0a66880cb97730169ec635f55bc03510cf1"}
Jul 21 14:38:11.427324 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:11.427209    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7psmb" event={"ID":"2cac96bd-ceca-4fc0-b229-601105cb5647","Type":"ContainerStarted","Data":"33530bdd7802811b5149d24bab2080025a943c07c9afd8fecc97b0d9e5306a6d"}
Jul 21 14:38:11.427808 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:11.427476    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7psmb"
Jul 21 14:38:11.429719 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:11.429689    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-24blz" event={"ID":"86ee645c-7f41-4aac-afaf-285233965458","Type":"ContainerStarted","Data":"3aa172c91bfe40d6a829c0dca9157c460b61c0ffda13f6c2902762f0725a7f91"}
Jul 21 14:38:11.430175 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:11.430130    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-24blz"
Jul 21 14:38:11.432333 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:11.432308    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-74d9d9b8c-dnpv2" event={"ID":"d1c73e43-f9e9-49f3-af4b-e411a0a21f1b","Type":"ContainerStarted","Data":"c777f533683df5e7f75261c4b721792c48b95cba4bb6b2709a9e5a4e47cf27ee"}
Jul 21 14:38:11.455290 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:11.455225    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7psmb" podStartSLOduration=2.462861393 podStartE2EDuration="6.455209862s" totalImagesPullingTime="3.992348469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:38:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:38:11.454223558 +0000 UTC m=+347.803966983" watchObservedRunningTime="2026-07-21 14:38:11.455209862 +0000 UTC m=+347.804953274"
Jul 21 14:38:11.475294 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:11.474109    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-74d9d9b8c-dnpv2" podStartSLOduration=2.163175309 podStartE2EDuration="5.47409065s" totalImagesPullingTime="3.310915341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:38:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:38:11.473807444 +0000 UTC m=+347.823550857" watchObservedRunningTime="2026-07-21 14:38:11.47409065 +0000 UTC m=+347.823834060"
Jul 21 14:38:11.491423 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:11.490097    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-24blz" podStartSLOduration=3.282914454 podStartE2EDuration="6.490081621s" totalImagesPullingTime="3.207167167s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:38:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:38:11.489886354 +0000 UTC m=+347.839629811" watchObservedRunningTime="2026-07-21 14:38:11.490081621 +0000 UTC m=+347.839825035"
Jul 21 14:38:13.145842 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:13.145807    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 21 14:38:13.146102 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:13.145874    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 21 14:38:13.386989 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:13.386953    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-76j8p"
Jul 21 14:38:13.445211 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:13.445077    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b22v6" event={"ID":"88af516f-4d3f-43f0-8701-4dd909af8e39","Type":"ContainerStarted","Data":"c0455d0363167f91bea9bb778fffa1716af60c92b4fa909f8b1cec76dd9ecc58"}
Jul 21 14:38:13.470978 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:13.470921    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b22v6" podStartSLOduration=2.6505902409999997 podStartE2EDuration="5.470900704s" totalImagesPullingTime="2.820310463s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:38:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:38:13.468687704 +0000 UTC m=+349.818431119" watchObservedRunningTime="2026-07-21 14:38:13.470900704 +0000 UTC m=+349.820644119"
Jul 21 14:38:14.449037 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:14.449009    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b22v6"
Jul 21 14:38:14.450451 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:14.450423    2579 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-b22v6 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 21 14:38:14.450556 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:14.450481    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b22v6" podUID="88af516f-4d3f-43f0-8701-4dd909af8e39" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 21 14:38:15.454356 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:15.454329    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-b22v6"
Jul 21 14:38:42.439432 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:42.439360    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7psmb"
Jul 21 14:38:43.450514 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:43.450486    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-24blz"
Jul 21 14:38:57.257180 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:57.257146    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-96xvz"]
Jul 21 14:38:57.259463 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:57.259435    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-96xvz"
Jul 21 14:38:57.262179 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:57.262156    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-jhd9k\""
Jul 21 14:38:57.262349 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:57.262227    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 21 14:38:57.263247 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:57.263224    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 21 14:38:57.271731 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:57.271705    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-96xvz"]
Jul 21 14:38:57.438351 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:57.438314    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsvm9\" (UniqueName: \"kubernetes.io/projected/46a38fb7-8e8a-4fd5-a640-69197310826d-kube-api-access-tsvm9\") pod \"kuadrant-operator-catalog-96xvz\" (UID: \"46a38fb7-8e8a-4fd5-a640-69197310826d\") " pod="kuadrant-system/kuadrant-operator-catalog-96xvz"
Jul 21 14:38:57.539213 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:57.539121    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsvm9\" (UniqueName: \"kubernetes.io/projected/46a38fb7-8e8a-4fd5-a640-69197310826d-kube-api-access-tsvm9\") pod \"kuadrant-operator-catalog-96xvz\" (UID: \"46a38fb7-8e8a-4fd5-a640-69197310826d\") " pod="kuadrant-system/kuadrant-operator-catalog-96xvz"
Jul 21 14:38:57.547716 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:57.547684    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsvm9\" (UniqueName: \"kubernetes.io/projected/46a38fb7-8e8a-4fd5-a640-69197310826d-kube-api-access-tsvm9\") pod \"kuadrant-operator-catalog-96xvz\" (UID: \"46a38fb7-8e8a-4fd5-a640-69197310826d\") " pod="kuadrant-system/kuadrant-operator-catalog-96xvz"
Jul 21 14:38:57.570606 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:57.570578    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-96xvz"
Jul 21 14:38:57.627288 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:57.623431    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-96xvz"]
Jul 21 14:38:57.717502 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:57.717321    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-96xvz"]
Jul 21 14:38:57.720039 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:38:57.720012    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46a38fb7_8e8a_4fd5_a640_69197310826d.slice/crio-8fb43a4a01c2f3f98d859cebf3cf3c8772356f166c4bdc45feda9298fafa12e1 WatchSource:0}: Error finding container 8fb43a4a01c2f3f98d859cebf3cf3c8772356f166c4bdc45feda9298fafa12e1: Status 404 returned error can't find the container with id 8fb43a4a01c2f3f98d859cebf3cf3c8772356f166c4bdc45feda9298fafa12e1
Jul 21 14:38:57.831171 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:57.831090    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ksq67"]
Jul 21 14:38:57.833996 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:57.833977    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ksq67"
Jul 21 14:38:57.844977 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:57.844942    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ksq67"]
Jul 21 14:38:57.943936 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:57.943895    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4zj4\" (UniqueName: \"kubernetes.io/projected/6e2fa36d-c716-4d5f-b9d6-222eed421ac9-kube-api-access-d4zj4\") pod \"kuadrant-operator-catalog-ksq67\" (UID: \"6e2fa36d-c716-4d5f-b9d6-222eed421ac9\") " pod="kuadrant-system/kuadrant-operator-catalog-ksq67"
Jul 21 14:38:58.044473 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:58.044435    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4zj4\" (UniqueName: \"kubernetes.io/projected/6e2fa36d-c716-4d5f-b9d6-222eed421ac9-kube-api-access-d4zj4\") pod \"kuadrant-operator-catalog-ksq67\" (UID: \"6e2fa36d-c716-4d5f-b9d6-222eed421ac9\") " pod="kuadrant-system/kuadrant-operator-catalog-ksq67"
Jul 21 14:38:58.052680 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:58.052651    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4zj4\" (UniqueName: \"kubernetes.io/projected/6e2fa36d-c716-4d5f-b9d6-222eed421ac9-kube-api-access-d4zj4\") pod \"kuadrant-operator-catalog-ksq67\" (UID: \"6e2fa36d-c716-4d5f-b9d6-222eed421ac9\") " pod="kuadrant-system/kuadrant-operator-catalog-ksq67"
Jul 21 14:38:58.148576 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:58.148542    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ksq67"
Jul 21 14:38:58.282716 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:58.282575    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ksq67"]
Jul 21 14:38:58.285294 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:38:58.285244    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e2fa36d_c716_4d5f_b9d6_222eed421ac9.slice/crio-8604105a305237dcb72e4483bbecb400fa6d5a7f7a14b1d96eb45e5345063f18 WatchSource:0}: Error finding container 8604105a305237dcb72e4483bbecb400fa6d5a7f7a14b1d96eb45e5345063f18: Status 404 returned error can't find the container with id 8604105a305237dcb72e4483bbecb400fa6d5a7f7a14b1d96eb45e5345063f18
Jul 21 14:38:58.622579 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:58.622476    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-96xvz" event={"ID":"46a38fb7-8e8a-4fd5-a640-69197310826d","Type":"ContainerStarted","Data":"8fb43a4a01c2f3f98d859cebf3cf3c8772356f166c4bdc45feda9298fafa12e1"}
Jul 21 14:38:58.623834 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:38:58.623807    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ksq67" event={"ID":"6e2fa36d-c716-4d5f-b9d6-222eed421ac9","Type":"ContainerStarted","Data":"8604105a305237dcb72e4483bbecb400fa6d5a7f7a14b1d96eb45e5345063f18"}
Jul 21 14:39:00.370745 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:00.370710    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-98b5cf9c5-5c64f"]
Jul 21 14:39:00.372972 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:00.372956    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-98b5cf9c5-5c64f"
Jul 21 14:39:00.385197 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:00.385174    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-98b5cf9c5-5c64f"]
Jul 21 14:39:00.469247 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:00.469212    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7132eb81-7f77-487d-8b0c-c964da260e10-console-oauth-config\") pod \"console-98b5cf9c5-5c64f\" (UID: \"7132eb81-7f77-487d-8b0c-c964da260e10\") " pod="openshift-console/console-98b5cf9c5-5c64f"
Jul 21 14:39:00.469247 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:00.469245    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rmxl\" (UniqueName: \"kubernetes.io/projected/7132eb81-7f77-487d-8b0c-c964da260e10-kube-api-access-6rmxl\") pod \"console-98b5cf9c5-5c64f\" (UID: \"7132eb81-7f77-487d-8b0c-c964da260e10\") " pod="openshift-console/console-98b5cf9c5-5c64f"
Jul 21 14:39:00.469473 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:00.469285    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7132eb81-7f77-487d-8b0c-c964da260e10-service-ca\") pod \"console-98b5cf9c5-5c64f\" (UID: \"7132eb81-7f77-487d-8b0c-c964da260e10\") " pod="openshift-console/console-98b5cf9c5-5c64f"
Jul 21 14:39:00.469473 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:00.469337    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7132eb81-7f77-487d-8b0c-c964da260e10-trusted-ca-bundle\") pod \"console-98b5cf9c5-5c64f\" (UID: \"7132eb81-7f77-487d-8b0c-c964da260e10\") " pod="openshift-console/console-98b5cf9c5-5c64f"
Jul 21 14:39:00.469473 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:00.469357    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7132eb81-7f77-487d-8b0c-c964da260e10-console-serving-cert\") pod \"console-98b5cf9c5-5c64f\" (UID: \"7132eb81-7f77-487d-8b0c-c964da260e10\") " pod="openshift-console/console-98b5cf9c5-5c64f"
Jul 21 14:39:00.469473 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:00.469418    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7132eb81-7f77-487d-8b0c-c964da260e10-oauth-serving-cert\") pod \"console-98b5cf9c5-5c64f\" (UID: \"7132eb81-7f77-487d-8b0c-c964da260e10\") " pod="openshift-console/console-98b5cf9c5-5c64f"
Jul 21 14:39:00.469473 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:00.469448    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7132eb81-7f77-487d-8b0c-c964da260e10-console-config\") pod \"console-98b5cf9c5-5c64f\" (UID: \"7132eb81-7f77-487d-8b0c-c964da260e10\") " pod="openshift-console/console-98b5cf9c5-5c64f"
Jul 21 14:39:00.570132 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:00.570098    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7132eb81-7f77-487d-8b0c-c964da260e10-console-oauth-config\") pod \"console-98b5cf9c5-5c64f\" (UID: \"7132eb81-7f77-487d-8b0c-c964da260e10\") " pod="openshift-console/console-98b5cf9c5-5c64f"
Jul 21 14:39:00.570132 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:00.570133    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rmxl\" (UniqueName: \"kubernetes.io/projected/7132eb81-7f77-487d-8b0c-c964da260e10-kube-api-access-6rmxl\") pod \"console-98b5cf9c5-5c64f\" (UID: \"7132eb81-7f77-487d-8b0c-c964da260e10\") " pod="openshift-console/console-98b5cf9c5-5c64f"
Jul 21 14:39:00.570376 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:00.570163    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7132eb81-7f77-487d-8b0c-c964da260e10-service-ca\") pod \"console-98b5cf9c5-5c64f\" (UID: \"7132eb81-7f77-487d-8b0c-c964da260e10\") " pod="openshift-console/console-98b5cf9c5-5c64f"
Jul 21 14:39:00.570376 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:00.570342    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7132eb81-7f77-487d-8b0c-c964da260e10-trusted-ca-bundle\") pod \"console-98b5cf9c5-5c64f\" (UID: \"7132eb81-7f77-487d-8b0c-c964da260e10\") " pod="openshift-console/console-98b5cf9c5-5c64f"
Jul 21 14:39:00.570462 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:00.570376    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7132eb81-7f77-487d-8b0c-c964da260e10-console-serving-cert\") pod \"console-98b5cf9c5-5c64f\" (UID: \"7132eb81-7f77-487d-8b0c-c964da260e10\") " pod="openshift-console/console-98b5cf9c5-5c64f"
Jul 21 14:39:00.570462 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:00.570424    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7132eb81-7f77-487d-8b0c-c964da260e10-oauth-serving-cert\") pod \"console-98b5cf9c5-5c64f\" (UID: \"7132eb81-7f77-487d-8b0c-c964da260e10\") " pod="openshift-console/console-98b5cf9c5-5c64f"
Jul 21 14:39:00.570555 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:00.570475    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7132eb81-7f77-487d-8b0c-c964da260e10-console-config\") pod \"console-98b5cf9c5-5c64f\" (UID: \"7132eb81-7f77-487d-8b0c-c964da260e10\") " pod="openshift-console/console-98b5cf9c5-5c64f"
Jul 21 14:39:00.570969 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:00.570943    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7132eb81-7f77-487d-8b0c-c964da260e10-service-ca\") pod \"console-98b5cf9c5-5c64f\" (UID: \"7132eb81-7f77-487d-8b0c-c964da260e10\") " pod="openshift-console/console-98b5cf9c5-5c64f"
Jul 21 14:39:00.571173 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:00.571148    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7132eb81-7f77-487d-8b0c-c964da260e10-console-config\") pod \"console-98b5cf9c5-5c64f\" (UID: \"7132eb81-7f77-487d-8b0c-c964da260e10\") " pod="openshift-console/console-98b5cf9c5-5c64f"
Jul 21 14:39:00.571257 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:00.571239    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7132eb81-7f77-487d-8b0c-c964da260e10-oauth-serving-cert\") pod \"console-98b5cf9c5-5c64f\" (UID: \"7132eb81-7f77-487d-8b0c-c964da260e10\") " pod="openshift-console/console-98b5cf9c5-5c64f"
Jul 21 14:39:00.571394 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:00.571375    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7132eb81-7f77-487d-8b0c-c964da260e10-trusted-ca-bundle\") pod \"console-98b5cf9c5-5c64f\" (UID: \"7132eb81-7f77-487d-8b0c-c964da260e10\") " pod="openshift-console/console-98b5cf9c5-5c64f"
Jul 21 14:39:00.572803 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:00.572778    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7132eb81-7f77-487d-8b0c-c964da260e10-console-serving-cert\") pod \"console-98b5cf9c5-5c64f\" (UID: \"7132eb81-7f77-487d-8b0c-c964da260e10\") " pod="openshift-console/console-98b5cf9c5-5c64f"
Jul 21 14:39:00.572888 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:00.572841    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7132eb81-7f77-487d-8b0c-c964da260e10-console-oauth-config\") pod \"console-98b5cf9c5-5c64f\" (UID: \"7132eb81-7f77-487d-8b0c-c964da260e10\") " pod="openshift-console/console-98b5cf9c5-5c64f"
Jul 21 14:39:00.577944 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:00.577928    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rmxl\" (UniqueName: \"kubernetes.io/projected/7132eb81-7f77-487d-8b0c-c964da260e10-kube-api-access-6rmxl\") pod \"console-98b5cf9c5-5c64f\" (UID: \"7132eb81-7f77-487d-8b0c-c964da260e10\") " pod="openshift-console/console-98b5cf9c5-5c64f"
Jul 21 14:39:00.634617 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:00.634582    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ksq67" event={"ID":"6e2fa36d-c716-4d5f-b9d6-222eed421ac9","Type":"ContainerStarted","Data":"8813ba64efc103b2b4aac1eec0f3c1ec40bf1ed7afe46bbf7ae500ab402608f3"}
Jul 21 14:39:00.636046 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:00.636023    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-96xvz" event={"ID":"46a38fb7-8e8a-4fd5-a640-69197310826d","Type":"ContainerStarted","Data":"840422b27b90a2de7dd4b39f58b3e12eda583f7a4eb4dbf2fdeb9b9098c3ee81"}
Jul 21 14:39:00.636175 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:00.636054    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-96xvz" podUID="46a38fb7-8e8a-4fd5-a640-69197310826d" containerName="registry-server" containerID="cri-o://840422b27b90a2de7dd4b39f58b3e12eda583f7a4eb4dbf2fdeb9b9098c3ee81" gracePeriod=2
Jul 21 14:39:00.652922 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:00.652878    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-ksq67" podStartSLOduration=2.182472889 podStartE2EDuration="3.652862533s" totalImagesPullingTime="1.470389644s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:38:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:39:00.651748182 +0000 UTC m=+397.001491613" watchObservedRunningTime="2026-07-21 14:39:00.652862533 +0000 UTC m=+397.002605923"
Jul 21 14:39:00.667371 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:00.667320    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-96xvz" podStartSLOduration=1.632126446 podStartE2EDuration="3.66730756s" totalImagesPullingTime="2.035181114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:38:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:39:00.666054515 +0000 UTC m=+397.015797927" watchObservedRunningTime="2026-07-21 14:39:00.66730756 +0000 UTC m=+397.017050972"
Jul 21 14:39:00.681656 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:00.681620    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-98b5cf9c5-5c64f"
Jul 21 14:39:00.814967 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:00.814939    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-98b5cf9c5-5c64f"]
Jul 21 14:39:00.818552 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:39:00.818509    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7132eb81_7f77_487d_8b0c_c964da260e10.slice/crio-1f9f5643d12d94e6ebbfe08f64485f7b3cce10f33b88b5518b0a72ebb44210d5 WatchSource:0}: Error finding container 1f9f5643d12d94e6ebbfe08f64485f7b3cce10f33b88b5518b0a72ebb44210d5: Status 404 returned error can't find the container with id 1f9f5643d12d94e6ebbfe08f64485f7b3cce10f33b88b5518b0a72ebb44210d5
Jul 21 14:39:00.893934 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:00.893912    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-96xvz"
Jul 21 14:39:01.075110 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:01.075077    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tsvm9\" (UniqueName: \"kubernetes.io/projected/46a38fb7-8e8a-4fd5-a640-69197310826d-kube-api-access-tsvm9\") pod \"46a38fb7-8e8a-4fd5-a640-69197310826d\" (UID: \"46a38fb7-8e8a-4fd5-a640-69197310826d\") "
Jul 21 14:39:01.077227 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:01.077206    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46a38fb7-8e8a-4fd5-a640-69197310826d-kube-api-access-tsvm9" (OuterVolumeSpecName: "kube-api-access-tsvm9") pod "46a38fb7-8e8a-4fd5-a640-69197310826d" (UID: "46a38fb7-8e8a-4fd5-a640-69197310826d"). InnerVolumeSpecName "kube-api-access-tsvm9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:39:01.175890 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:01.175818    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tsvm9\" (UniqueName: \"kubernetes.io/projected/46a38fb7-8e8a-4fd5-a640-69197310826d-kube-api-access-tsvm9\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:39:01.641782 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:01.641747    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-98b5cf9c5-5c64f" event={"ID":"7132eb81-7f77-487d-8b0c-c964da260e10","Type":"ContainerStarted","Data":"d53aaae9f9445914b033bf6c96a6ff54531dbddfa1b1d7945ec156fb44d90e63"}
Jul 21 14:39:01.642241 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:01.641811    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-98b5cf9c5-5c64f" event={"ID":"7132eb81-7f77-487d-8b0c-c964da260e10","Type":"ContainerStarted","Data":"1f9f5643d12d94e6ebbfe08f64485f7b3cce10f33b88b5518b0a72ebb44210d5"}
Jul 21 14:39:01.642940 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:01.642914    2579 generic.go:356] "Generic (PLEG): container finished" podID="46a38fb7-8e8a-4fd5-a640-69197310826d" containerID="840422b27b90a2de7dd4b39f58b3e12eda583f7a4eb4dbf2fdeb9b9098c3ee81" exitCode=0
Jul 21 14:39:01.643031 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:01.642945    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-96xvz"
Jul 21 14:39:01.643031 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:01.642986    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-96xvz" event={"ID":"46a38fb7-8e8a-4fd5-a640-69197310826d","Type":"ContainerDied","Data":"840422b27b90a2de7dd4b39f58b3e12eda583f7a4eb4dbf2fdeb9b9098c3ee81"}
Jul 21 14:39:01.643101 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:01.643033    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-96xvz" event={"ID":"46a38fb7-8e8a-4fd5-a640-69197310826d","Type":"ContainerDied","Data":"8fb43a4a01c2f3f98d859cebf3cf3c8772356f166c4bdc45feda9298fafa12e1"}
Jul 21 14:39:01.643101 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:01.643056    2579 scope.go:117] "RemoveContainer" containerID="840422b27b90a2de7dd4b39f58b3e12eda583f7a4eb4dbf2fdeb9b9098c3ee81"
Jul 21 14:39:01.651299 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:01.651279    2579 scope.go:117] "RemoveContainer" containerID="840422b27b90a2de7dd4b39f58b3e12eda583f7a4eb4dbf2fdeb9b9098c3ee81"
Jul 21 14:39:01.651560 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:39:01.651544    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"840422b27b90a2de7dd4b39f58b3e12eda583f7a4eb4dbf2fdeb9b9098c3ee81\": container with ID starting with 840422b27b90a2de7dd4b39f58b3e12eda583f7a4eb4dbf2fdeb9b9098c3ee81 not found: ID does not exist" containerID="840422b27b90a2de7dd4b39f58b3e12eda583f7a4eb4dbf2fdeb9b9098c3ee81"
Jul 21 14:39:01.651614 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:01.651568    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"840422b27b90a2de7dd4b39f58b3e12eda583f7a4eb4dbf2fdeb9b9098c3ee81"} err="failed to get container status \"840422b27b90a2de7dd4b39f58b3e12eda583f7a4eb4dbf2fdeb9b9098c3ee81\": rpc error: code = NotFound desc = could not find container \"840422b27b90a2de7dd4b39f58b3e12eda583f7a4eb4dbf2fdeb9b9098c3ee81\": container with ID starting with 840422b27b90a2de7dd4b39f58b3e12eda583f7a4eb4dbf2fdeb9b9098c3ee81 not found: ID does not exist"
Jul 21 14:39:01.661147 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:01.661103    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-98b5cf9c5-5c64f" podStartSLOduration=1.6610899030000001 podStartE2EDuration="1.661089903s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:39:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:39:01.660117968 +0000 UTC m=+398.009861401" watchObservedRunningTime="2026-07-21 14:39:01.661089903 +0000 UTC m=+398.010833315"
Jul 21 14:39:01.672995 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:01.672975    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-96xvz"]
Jul 21 14:39:01.676639 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:01.676608    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-96xvz"]
Jul 21 14:39:02.171952 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:02.171919    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46a38fb7-8e8a-4fd5-a640-69197310826d" path="/var/lib/kubelet/pods/46a38fb7-8e8a-4fd5-a640-69197310826d/volumes"
Jul 21 14:39:08.149393 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:08.149360    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-ksq67"
Jul 21 14:39:08.149800 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:08.149417    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-ksq67"
Jul 21 14:39:08.171797 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:08.171774    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-ksq67"
Jul 21 14:39:08.695840 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:08.695815    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-ksq67"
Jul 21 14:39:10.681778 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:10.681747    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-98b5cf9c5-5c64f"
Jul 21 14:39:10.682164 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:10.681832    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-98b5cf9c5-5c64f"
Jul 21 14:39:10.686673 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:10.686653    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-98b5cf9c5-5c64f"
Jul 21 14:39:11.689356 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:11.689331    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-98b5cf9c5-5c64f"
Jul 21 14:39:11.732524 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:11.732489    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d8c6997bb-8dt2h"]
Jul 21 14:39:34.353669 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:34.353639    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cgcmh"]
Jul 21 14:39:34.354041 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:34.353920    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46a38fb7-8e8a-4fd5-a640-69197310826d" containerName="registry-server"
Jul 21 14:39:34.354041 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:34.353930    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="46a38fb7-8e8a-4fd5-a640-69197310826d" containerName="registry-server"
Jul 21 14:39:34.354041 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:34.353985    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="46a38fb7-8e8a-4fd5-a640-69197310826d" containerName="registry-server"
Jul 21 14:39:34.361042 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:34.361021    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cgcmh"
Jul 21 14:39:34.363766 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:34.363745    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-pbvb5\""
Jul 21 14:39:34.367970 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:34.367945    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cgcmh"]
Jul 21 14:39:34.432650 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:34.432618    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnd4g\" (UniqueName: \"kubernetes.io/projected/bf927633-7540-474c-8c7f-097a5bdaf75c-kube-api-access-qnd4g\") pod \"limitador-operator-controller-manager-859d7b78bd-cgcmh\" (UID: \"bf927633-7540-474c-8c7f-097a5bdaf75c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cgcmh"
Jul 21 14:39:34.533422 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:34.533387    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnd4g\" (UniqueName: \"kubernetes.io/projected/bf927633-7540-474c-8c7f-097a5bdaf75c-kube-api-access-qnd4g\") pod \"limitador-operator-controller-manager-859d7b78bd-cgcmh\" (UID: \"bf927633-7540-474c-8c7f-097a5bdaf75c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cgcmh"
Jul 21 14:39:34.550729 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:34.550706    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnd4g\" (UniqueName: \"kubernetes.io/projected/bf927633-7540-474c-8c7f-097a5bdaf75c-kube-api-access-qnd4g\") pod \"limitador-operator-controller-manager-859d7b78bd-cgcmh\" (UID: \"bf927633-7540-474c-8c7f-097a5bdaf75c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cgcmh"
Jul 21 14:39:34.671544 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:34.671519    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cgcmh"
Jul 21 14:39:34.797314 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:34.797293    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cgcmh"]
Jul 21 14:39:34.799581 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:39:34.799553    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf927633_7540_474c_8c7f_097a5bdaf75c.slice/crio-a35b7712110a40b1bc0219a3076a4b9aa7e489d13d67ff84af94a56482034277 WatchSource:0}: Error finding container a35b7712110a40b1bc0219a3076a4b9aa7e489d13d67ff84af94a56482034277: Status 404 returned error can't find the container with id a35b7712110a40b1bc0219a3076a4b9aa7e489d13d67ff84af94a56482034277
Jul 21 14:39:35.785968 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:35.785934    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cgcmh" event={"ID":"bf927633-7540-474c-8c7f-097a5bdaf75c","Type":"ContainerStarted","Data":"a35b7712110a40b1bc0219a3076a4b9aa7e489d13d67ff84af94a56482034277"}
Jul 21 14:39:36.757790 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:36.757756    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7d8c6997bb-8dt2h" podUID="37b95e9b-ff18-406e-9f5b-840b9a507b43" containerName="console" containerID="cri-o://1b1213625f58bb56f33e5b3e0fbf1dff7cad2f44777643f7a291f94385ce9d7e" gracePeriod=15
Jul 21 14:39:36.791078 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:36.791049    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cgcmh" event={"ID":"bf927633-7540-474c-8c7f-097a5bdaf75c","Type":"ContainerStarted","Data":"2da0212e902a116b998ec856a212d222cc275a782d75eb2e0785ec8add36f3dd"}
Jul 21 14:39:36.791366 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:36.791144    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cgcmh"
Jul 21 14:39:36.815367 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:36.815327    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cgcmh" podStartSLOduration=0.985584601 podStartE2EDuration="2.815313431s" totalImagesPullingTime="1.82972883s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:39:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:39:36.814228413 +0000 UTC m=+433.163971854" watchObservedRunningTime="2026-07-21 14:39:36.815313431 +0000 UTC m=+433.165056842"
Jul 21 14:39:37.000303 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:37.000282    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d8c6997bb-8dt2h"
Jul 21 14:39:37.153866 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:37.153839    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/37b95e9b-ff18-406e-9f5b-840b9a507b43-console-serving-cert\") pod \"37b95e9b-ff18-406e-9f5b-840b9a507b43\" (UID: \"37b95e9b-ff18-406e-9f5b-840b9a507b43\") "
Jul 21 14:39:37.154013 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:37.153874    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/37b95e9b-ff18-406e-9f5b-840b9a507b43-service-ca\") pod \"37b95e9b-ff18-406e-9f5b-840b9a507b43\" (UID: \"37b95e9b-ff18-406e-9f5b-840b9a507b43\") "
Jul 21 14:39:37.154013 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:37.153902    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/37b95e9b-ff18-406e-9f5b-840b9a507b43-oauth-serving-cert\") pod \"37b95e9b-ff18-406e-9f5b-840b9a507b43\" (UID: \"37b95e9b-ff18-406e-9f5b-840b9a507b43\") "
Jul 21 14:39:37.154013 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:37.153923    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37b95e9b-ff18-406e-9f5b-840b9a507b43-trusted-ca-bundle\") pod \"37b95e9b-ff18-406e-9f5b-840b9a507b43\" (UID: \"37b95e9b-ff18-406e-9f5b-840b9a507b43\") "
Jul 21 14:39:37.154013 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:37.153957    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/37b95e9b-ff18-406e-9f5b-840b9a507b43-console-oauth-config\") pod \"37b95e9b-ff18-406e-9f5b-840b9a507b43\" (UID: \"37b95e9b-ff18-406e-9f5b-840b9a507b43\") "
Jul 21 14:39:37.154221 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:37.154072    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xphzg\" (UniqueName: \"kubernetes.io/projected/37b95e9b-ff18-406e-9f5b-840b9a507b43-kube-api-access-xphzg\") pod \"37b95e9b-ff18-406e-9f5b-840b9a507b43\" (UID: \"37b95e9b-ff18-406e-9f5b-840b9a507b43\") "
Jul 21 14:39:37.154221 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:37.154119    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/37b95e9b-ff18-406e-9f5b-840b9a507b43-console-config\") pod \"37b95e9b-ff18-406e-9f5b-840b9a507b43\" (UID: \"37b95e9b-ff18-406e-9f5b-840b9a507b43\") "
Jul 21 14:39:37.154371 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:37.154330    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37b95e9b-ff18-406e-9f5b-840b9a507b43-service-ca" (OuterVolumeSpecName: "service-ca") pod "37b95e9b-ff18-406e-9f5b-840b9a507b43" (UID: "37b95e9b-ff18-406e-9f5b-840b9a507b43"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:39:37.154371 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:37.154335    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37b95e9b-ff18-406e-9f5b-840b9a507b43-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "37b95e9b-ff18-406e-9f5b-840b9a507b43" (UID: "37b95e9b-ff18-406e-9f5b-840b9a507b43"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:39:37.154557 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:37.154433    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37b95e9b-ff18-406e-9f5b-840b9a507b43-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "37b95e9b-ff18-406e-9f5b-840b9a507b43" (UID: "37b95e9b-ff18-406e-9f5b-840b9a507b43"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:39:37.154645 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:37.154539    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/37b95e9b-ff18-406e-9f5b-840b9a507b43-service-ca\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:39:37.154645 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:37.154587    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/37b95e9b-ff18-406e-9f5b-840b9a507b43-oauth-serving-cert\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:39:37.154645 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:37.154619    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37b95e9b-ff18-406e-9f5b-840b9a507b43-console-config" (OuterVolumeSpecName: "console-config") pod "37b95e9b-ff18-406e-9f5b-840b9a507b43" (UID: "37b95e9b-ff18-406e-9f5b-840b9a507b43"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:39:37.156074 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:37.156049    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37b95e9b-ff18-406e-9f5b-840b9a507b43-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "37b95e9b-ff18-406e-9f5b-840b9a507b43" (UID: "37b95e9b-ff18-406e-9f5b-840b9a507b43"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:39:37.156074 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:37.156071    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37b95e9b-ff18-406e-9f5b-840b9a507b43-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "37b95e9b-ff18-406e-9f5b-840b9a507b43" (UID: "37b95e9b-ff18-406e-9f5b-840b9a507b43"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:39:37.156237 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:37.156154    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37b95e9b-ff18-406e-9f5b-840b9a507b43-kube-api-access-xphzg" (OuterVolumeSpecName: "kube-api-access-xphzg") pod "37b95e9b-ff18-406e-9f5b-840b9a507b43" (UID: "37b95e9b-ff18-406e-9f5b-840b9a507b43"). InnerVolumeSpecName "kube-api-access-xphzg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:39:37.255497 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:37.255449    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37b95e9b-ff18-406e-9f5b-840b9a507b43-trusted-ca-bundle\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:39:37.255497 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:37.255467    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/37b95e9b-ff18-406e-9f5b-840b9a507b43-console-oauth-config\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:39:37.255497 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:37.255477    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xphzg\" (UniqueName: \"kubernetes.io/projected/37b95e9b-ff18-406e-9f5b-840b9a507b43-kube-api-access-xphzg\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:39:37.255497 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:37.255486    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/37b95e9b-ff18-406e-9f5b-840b9a507b43-console-config\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:39:37.255497 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:37.255495    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/37b95e9b-ff18-406e-9f5b-840b9a507b43-console-serving-cert\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:39:37.797143 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:37.797109    2579 generic.go:356] "Generic (PLEG): container finished" podID="37b95e9b-ff18-406e-9f5b-840b9a507b43" containerID="1b1213625f58bb56f33e5b3e0fbf1dff7cad2f44777643f7a291f94385ce9d7e" exitCode=0
Jul 21 14:39:37.797587 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:37.797156    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d8c6997bb-8dt2h"
Jul 21 14:39:37.797587 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:37.797191    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d8c6997bb-8dt2h" event={"ID":"37b95e9b-ff18-406e-9f5b-840b9a507b43","Type":"ContainerDied","Data":"1b1213625f58bb56f33e5b3e0fbf1dff7cad2f44777643f7a291f94385ce9d7e"}
Jul 21 14:39:37.797587 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:37.797241    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d8c6997bb-8dt2h" event={"ID":"37b95e9b-ff18-406e-9f5b-840b9a507b43","Type":"ContainerDied","Data":"5aeaf37224855e6c2b03c1142473dc3cd081497b56247bbbaa39f125eaab8802"}
Jul 21 14:39:37.797587 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:37.797261    2579 scope.go:117] "RemoveContainer" containerID="1b1213625f58bb56f33e5b3e0fbf1dff7cad2f44777643f7a291f94385ce9d7e"
Jul 21 14:39:37.806314 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:37.806293    2579 scope.go:117] "RemoveContainer" containerID="1b1213625f58bb56f33e5b3e0fbf1dff7cad2f44777643f7a291f94385ce9d7e"
Jul 21 14:39:37.806565 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:39:37.806548    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b1213625f58bb56f33e5b3e0fbf1dff7cad2f44777643f7a291f94385ce9d7e\": container with ID starting with 1b1213625f58bb56f33e5b3e0fbf1dff7cad2f44777643f7a291f94385ce9d7e not found: ID does not exist" containerID="1b1213625f58bb56f33e5b3e0fbf1dff7cad2f44777643f7a291f94385ce9d7e"
Jul 21 14:39:37.806611 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:37.806573    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b1213625f58bb56f33e5b3e0fbf1dff7cad2f44777643f7a291f94385ce9d7e"} err="failed to get container status \"1b1213625f58bb56f33e5b3e0fbf1dff7cad2f44777643f7a291f94385ce9d7e\": rpc error: code = NotFound desc = could not find container \"1b1213625f58bb56f33e5b3e0fbf1dff7cad2f44777643f7a291f94385ce9d7e\": container with ID starting with 1b1213625f58bb56f33e5b3e0fbf1dff7cad2f44777643f7a291f94385ce9d7e not found: ID does not exist"
Jul 21 14:39:37.817166 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:37.817144    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d8c6997bb-8dt2h"]
Jul 21 14:39:37.821227 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:37.821209    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7d8c6997bb-8dt2h"]
Jul 21 14:39:38.172399 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:38.172368    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37b95e9b-ff18-406e-9f5b-840b9a507b43" path="/var/lib/kubelet/pods/37b95e9b-ff18-406e-9f5b-840b9a507b43/volumes"
Jul 21 14:39:46.688716 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:46.688638    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-xvpks"]
Jul 21 14:39:46.689086 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:46.688928    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37b95e9b-ff18-406e-9f5b-840b9a507b43" containerName="console"
Jul 21 14:39:46.689086 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:46.688938    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="37b95e9b-ff18-406e-9f5b-840b9a507b43" containerName="console"
Jul 21 14:39:46.689086 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:46.688995    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="37b95e9b-ff18-406e-9f5b-840b9a507b43" containerName="console"
Jul 21 14:39:46.691658 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:46.691642    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-xvpks"
Jul 21 14:39:46.694515 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:46.694495    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 21 14:39:46.694606 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:46.694495    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-z5vqs\""
Jul 21 14:39:46.702834 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:46.702812    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-xvpks"]
Jul 21 14:39:46.833095 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:46.833064    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzv6k\" (UniqueName: \"kubernetes.io/projected/1bd08cca-a503-4f55-9557-40d4eab4116d-kube-api-access-jzv6k\") pod \"dns-operator-controller-manager-864dd78c4c-xvpks\" (UID: \"1bd08cca-a503-4f55-9557-40d4eab4116d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-xvpks"
Jul 21 14:39:46.934518 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:46.934480    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzv6k\" (UniqueName: \"kubernetes.io/projected/1bd08cca-a503-4f55-9557-40d4eab4116d-kube-api-access-jzv6k\") pod \"dns-operator-controller-manager-864dd78c4c-xvpks\" (UID: \"1bd08cca-a503-4f55-9557-40d4eab4116d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-xvpks"
Jul 21 14:39:46.943947 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:46.943891    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzv6k\" (UniqueName: \"kubernetes.io/projected/1bd08cca-a503-4f55-9557-40d4eab4116d-kube-api-access-jzv6k\") pod \"dns-operator-controller-manager-864dd78c4c-xvpks\" (UID: \"1bd08cca-a503-4f55-9557-40d4eab4116d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-xvpks"
Jul 21 14:39:47.001987 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:47.001958    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-xvpks"
Jul 21 14:39:47.131759 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:47.131733    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-xvpks"]
Jul 21 14:39:47.134145 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:39:47.134118    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bd08cca_a503_4f55_9557_40d4eab4116d.slice/crio-ebdbd60817d361506879a5a0aa99bba536e59bd86c89b64cfe5a7d35acb05019 WatchSource:0}: Error finding container ebdbd60817d361506879a5a0aa99bba536e59bd86c89b64cfe5a7d35acb05019: Status 404 returned error can't find the container with id ebdbd60817d361506879a5a0aa99bba536e59bd86c89b64cfe5a7d35acb05019
Jul 21 14:39:47.800467 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:47.800437    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cgcmh"
Jul 21 14:39:47.842476 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:47.842433    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-xvpks" event={"ID":"1bd08cca-a503-4f55-9557-40d4eab4116d","Type":"ContainerStarted","Data":"ebdbd60817d361506879a5a0aa99bba536e59bd86c89b64cfe5a7d35acb05019"}
Jul 21 14:39:49.854841 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:49.854800    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-xvpks" event={"ID":"1bd08cca-a503-4f55-9557-40d4eab4116d","Type":"ContainerStarted","Data":"3799e524f73416f08c4f6acb7d9b5b10503a1ef70e08733ec7b7251365d0ec40"}
Jul 21 14:39:49.855218 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:49.854892    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-xvpks"
Jul 21 14:39:49.878390 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:49.878323    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-xvpks" podStartSLOduration=2.058224761 podStartE2EDuration="3.878301525s" totalImagesPullingTime="1.820076764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:39:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:39:49.8782621 +0000 UTC m=+446.228005513" watchObservedRunningTime="2026-07-21 14:39:49.878301525 +0000 UTC m=+446.228044941"
Jul 21 14:39:58.568825 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:58.568789    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z799w"]
Jul 21 14:39:58.571996 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:58.571978    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z799w"
Jul 21 14:39:58.575218 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:58.575198    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-lwd2v\""
Jul 21 14:39:58.589732 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:58.589709    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z799w"]
Jul 21 14:39:58.639254 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:58.639222    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mt692\" (UniqueName: \"kubernetes.io/projected/e0840d52-b972-42ef-9e4c-17addec286c9-kube-api-access-mt692\") pod \"kuadrant-operator-controller-manager-55467d9f7-z799w\" (UID: \"e0840d52-b972-42ef-9e4c-17addec286c9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z799w"
Jul 21 14:39:58.639411 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:58.639326    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e0840d52-b972-42ef-9e4c-17addec286c9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-z799w\" (UID: \"e0840d52-b972-42ef-9e4c-17addec286c9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z799w"
Jul 21 14:39:58.740398 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:58.740355    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mt692\" (UniqueName: \"kubernetes.io/projected/e0840d52-b972-42ef-9e4c-17addec286c9-kube-api-access-mt692\") pod \"kuadrant-operator-controller-manager-55467d9f7-z799w\" (UID: \"e0840d52-b972-42ef-9e4c-17addec286c9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z799w"
Jul 21 14:39:58.740547 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:58.740453    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e0840d52-b972-42ef-9e4c-17addec286c9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-z799w\" (UID: \"e0840d52-b972-42ef-9e4c-17addec286c9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z799w"
Jul 21 14:39:58.740848 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:58.740826    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e0840d52-b972-42ef-9e4c-17addec286c9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-z799w\" (UID: \"e0840d52-b972-42ef-9e4c-17addec286c9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z799w"
Jul 21 14:39:58.749204 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:58.749173    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt692\" (UniqueName: \"kubernetes.io/projected/e0840d52-b972-42ef-9e4c-17addec286c9-kube-api-access-mt692\") pod \"kuadrant-operator-controller-manager-55467d9f7-z799w\" (UID: \"e0840d52-b972-42ef-9e4c-17addec286c9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z799w"
Jul 21 14:39:58.882231 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:58.882173    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z799w"
Jul 21 14:39:59.036167 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.036132    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z799w"]
Jul 21 14:39:59.039239 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:39:59.039211    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0840d52_b972_42ef_9e4c_17addec286c9.slice/crio-c62a5fff43feeaecafb88631b6903b32720bedc38e83f8219727530051f01088 WatchSource:0}: Error finding container c62a5fff43feeaecafb88631b6903b32720bedc38e83f8219727530051f01088: Status 404 returned error can't find the container with id c62a5fff43feeaecafb88631b6903b32720bedc38e83f8219727530051f01088
Jul 21 14:39:59.057923 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.057903    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z799w"]
Jul 21 14:39:59.063357 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.063337    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z799w"]
Jul 21 14:39:59.088002 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.087977    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qdhk8"]
Jul 21 14:39:59.092234 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.092214    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qdhk8"
Jul 21 14:39:59.106939 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.106912    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cgcmh"]
Jul 21 14:39:59.107335 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.107117    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cgcmh" podUID="bf927633-7540-474c-8c7f-097a5bdaf75c" containerName="manager" containerID="cri-o://2da0212e902a116b998ec856a212d222cc275a782d75eb2e0785ec8add36f3dd" gracePeriod=2
Jul 21 14:39:59.108716 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.108696    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qdhk8"]
Jul 21 14:39:59.117210 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.117184    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cgcmh"]
Jul 21 14:39:59.131905 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:39:59.131866    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-cgcmh\" is forbidden: User \"system:node:ip-10-0-130-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-83.ec2.internal' and this object" podUID="bf927633-7540-474c-8c7f-097a5bdaf75c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cgcmh"
Jul 21 14:39:59.142940 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.142921    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvp9p"]
Jul 21 14:39:59.143398 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.143382    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf927633-7540-474c-8c7f-097a5bdaf75c" containerName="manager"
Jul 21 14:39:59.143461 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.143401    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf927633-7540-474c-8c7f-097a5bdaf75c" containerName="manager"
Jul 21 14:39:59.143500 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.143484    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf927633-7540-474c-8c7f-097a5bdaf75c" containerName="manager"
Jul 21 14:39:59.144551 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.144532    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b62n7\" (UniqueName: \"kubernetes.io/projected/ef529aba-cd19-4ad0-812d-6cf20b3cf90a-kube-api-access-b62n7\") pod \"kuadrant-operator-controller-manager-55467d9f7-qdhk8\" (UID: \"ef529aba-cd19-4ad0-812d-6cf20b3cf90a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qdhk8"
Jul 21 14:39:59.144617 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.144597    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ef529aba-cd19-4ad0-812d-6cf20b3cf90a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-qdhk8\" (UID: \"ef529aba-cd19-4ad0-812d-6cf20b3cf90a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qdhk8"
Jul 21 14:39:59.146528 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.146512    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvp9p"
Jul 21 14:39:59.149063 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:39:59.149033    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-cgcmh\" is forbidden: User \"system:node:ip-10-0-130-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-83.ec2.internal' and this object" podUID="bf927633-7540-474c-8c7f-097a5bdaf75c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cgcmh"
Jul 21 14:39:59.163292 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.161700    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvp9p"]
Jul 21 14:39:59.246355 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.245971    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b62n7\" (UniqueName: \"kubernetes.io/projected/ef529aba-cd19-4ad0-812d-6cf20b3cf90a-kube-api-access-b62n7\") pod \"kuadrant-operator-controller-manager-55467d9f7-qdhk8\" (UID: \"ef529aba-cd19-4ad0-812d-6cf20b3cf90a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qdhk8"
Jul 21 14:39:59.246504 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.246394    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6jrd\" (UniqueName: \"kubernetes.io/projected/80b639cf-e471-4034-bbb7-674deed96de1-kube-api-access-d6jrd\") pod \"limitador-operator-controller-manager-859d7b78bd-bvp9p\" (UID: \"80b639cf-e471-4034-bbb7-674deed96de1\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvp9p"
Jul 21 14:39:59.246565 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.246509    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ef529aba-cd19-4ad0-812d-6cf20b3cf90a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-qdhk8\" (UID: \"ef529aba-cd19-4ad0-812d-6cf20b3cf90a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qdhk8"
Jul 21 14:39:59.246797 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.246780    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ef529aba-cd19-4ad0-812d-6cf20b3cf90a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-qdhk8\" (UID: \"ef529aba-cd19-4ad0-812d-6cf20b3cf90a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qdhk8"
Jul 21 14:39:59.255085 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.255053    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b62n7\" (UniqueName: \"kubernetes.io/projected/ef529aba-cd19-4ad0-812d-6cf20b3cf90a-kube-api-access-b62n7\") pod \"kuadrant-operator-controller-manager-55467d9f7-qdhk8\" (UID: \"ef529aba-cd19-4ad0-812d-6cf20b3cf90a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qdhk8"
Jul 21 14:39:59.329008 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.328987    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cgcmh"
Jul 21 14:39:59.331091 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:39:59.331065    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-cgcmh\" is forbidden: User \"system:node:ip-10-0-130-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-83.ec2.internal' and this object" podUID="bf927633-7540-474c-8c7f-097a5bdaf75c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cgcmh"
Jul 21 14:39:59.347500 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.347480    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qnd4g\" (UniqueName: \"kubernetes.io/projected/bf927633-7540-474c-8c7f-097a5bdaf75c-kube-api-access-qnd4g\") pod \"bf927633-7540-474c-8c7f-097a5bdaf75c\" (UID: \"bf927633-7540-474c-8c7f-097a5bdaf75c\") "
Jul 21 14:39:59.347790 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.347770    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6jrd\" (UniqueName: \"kubernetes.io/projected/80b639cf-e471-4034-bbb7-674deed96de1-kube-api-access-d6jrd\") pod \"limitador-operator-controller-manager-859d7b78bd-bvp9p\" (UID: \"80b639cf-e471-4034-bbb7-674deed96de1\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvp9p"
Jul 21 14:39:59.349388 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.349362    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf927633-7540-474c-8c7f-097a5bdaf75c-kube-api-access-qnd4g" (OuterVolumeSpecName: "kube-api-access-qnd4g") pod "bf927633-7540-474c-8c7f-097a5bdaf75c" (UID: "bf927633-7540-474c-8c7f-097a5bdaf75c"). InnerVolumeSpecName "kube-api-access-qnd4g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:39:59.358460 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.358441    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6jrd\" (UniqueName: \"kubernetes.io/projected/80b639cf-e471-4034-bbb7-674deed96de1-kube-api-access-d6jrd\") pod \"limitador-operator-controller-manager-859d7b78bd-bvp9p\" (UID: \"80b639cf-e471-4034-bbb7-674deed96de1\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvp9p"
Jul 21 14:39:59.404984 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.404936    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qdhk8"
Jul 21 14:39:59.448362 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.448337    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qnd4g\" (UniqueName: \"kubernetes.io/projected/bf927633-7540-474c-8c7f-097a5bdaf75c-kube-api-access-qnd4g\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:39:59.479636 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.479610    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvp9p"
Jul 21 14:39:59.545230 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.545204    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qdhk8"]
Jul 21 14:39:59.548318 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:39:59.548257    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef529aba_cd19_4ad0_812d_6cf20b3cf90a.slice/crio-f3ed9d25d6d0bb4ba61c0a8afe3e6a49865b9b75c7a29fee7bb1d85cf460f2fa WatchSource:0}: Error finding container f3ed9d25d6d0bb4ba61c0a8afe3e6a49865b9b75c7a29fee7bb1d85cf460f2fa: Status 404 returned error can't find the container with id f3ed9d25d6d0bb4ba61c0a8afe3e6a49865b9b75c7a29fee7bb1d85cf460f2fa
Jul 21 14:39:59.615198 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.615168    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvp9p"]
Jul 21 14:39:59.618170 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:39:59.618144    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80b639cf_e471_4034_bbb7_674deed96de1.slice/crio-98efbbd3bba44cf3877a382f7bc56aad086ac9091da5f4ecfaceb94e1a7d5251 WatchSource:0}: Error finding container 98efbbd3bba44cf3877a382f7bc56aad086ac9091da5f4ecfaceb94e1a7d5251: Status 404 returned error can't find the container with id 98efbbd3bba44cf3877a382f7bc56aad086ac9091da5f4ecfaceb94e1a7d5251
Jul 21 14:39:59.902787 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.902753    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvp9p" event={"ID":"80b639cf-e471-4034-bbb7-674deed96de1","Type":"ContainerStarted","Data":"9f129f02064254e4d2c799037183371ddeb387da1330d62cb5707543f08dac61"}
Jul 21 14:39:59.902965 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.902820    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvp9p"
Jul 21 14:39:59.902965 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.902844    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvp9p" event={"ID":"80b639cf-e471-4034-bbb7-674deed96de1","Type":"ContainerStarted","Data":"98efbbd3bba44cf3877a382f7bc56aad086ac9091da5f4ecfaceb94e1a7d5251"}
Jul 21 14:39:59.905100 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:39:59.905068    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-cgcmh\" is forbidden: User \"system:node:ip-10-0-130-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-83.ec2.internal' and this object" podUID="bf927633-7540-474c-8c7f-097a5bdaf75c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cgcmh"
Jul 21 14:39:59.905846 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.905819    2579 generic.go:356] "Generic (PLEG): container finished" podID="bf927633-7540-474c-8c7f-097a5bdaf75c" containerID="2da0212e902a116b998ec856a212d222cc275a782d75eb2e0785ec8add36f3dd" exitCode=0
Jul 21 14:39:59.905967 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.905862    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cgcmh"
Jul 21 14:39:59.905967 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.905923    2579 scope.go:117] "RemoveContainer" containerID="2da0212e902a116b998ec856a212d222cc275a782d75eb2e0785ec8add36f3dd"
Jul 21 14:39:59.908028 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.907982    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qdhk8" event={"ID":"ef529aba-cd19-4ad0-812d-6cf20b3cf90a","Type":"ContainerStarted","Data":"f3ed9d25d6d0bb4ba61c0a8afe3e6a49865b9b75c7a29fee7bb1d85cf460f2fa"}
Jul 21 14:39:59.916428 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.916408    2579 scope.go:117] "RemoveContainer" containerID="2da0212e902a116b998ec856a212d222cc275a782d75eb2e0785ec8add36f3dd"
Jul 21 14:39:59.916692 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:39:59.916671    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2da0212e902a116b998ec856a212d222cc275a782d75eb2e0785ec8add36f3dd\": container with ID starting with 2da0212e902a116b998ec856a212d222cc275a782d75eb2e0785ec8add36f3dd not found: ID does not exist" containerID="2da0212e902a116b998ec856a212d222cc275a782d75eb2e0785ec8add36f3dd"
Jul 21 14:39:59.916785 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.916699    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2da0212e902a116b998ec856a212d222cc275a782d75eb2e0785ec8add36f3dd"} err="failed to get container status \"2da0212e902a116b998ec856a212d222cc275a782d75eb2e0785ec8add36f3dd\": rpc error: code = NotFound desc = could not find container \"2da0212e902a116b998ec856a212d222cc275a782d75eb2e0785ec8add36f3dd\": container with ID starting with 2da0212e902a116b998ec856a212d222cc275a782d75eb2e0785ec8add36f3dd not found: ID does not exist"
Jul 21 14:39:59.925947 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:39:59.925910    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvp9p" podStartSLOduration=0.925897828 podStartE2EDuration="925.897828ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:39:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:39:59.924252902 +0000 UTC m=+456.273996314" watchObservedRunningTime="2026-07-21 14:39:59.925897828 +0000 UTC m=+456.275641240"
Jul 21 14:39:59.926310 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:39:59.926284    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-cgcmh\" is forbidden: User \"system:node:ip-10-0-130-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-83.ec2.internal' and this object" podUID="bf927633-7540-474c-8c7f-097a5bdaf75c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cgcmh"
Jul 21 14:40:00.173412 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:00.173337    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf927633-7540-474c-8c7f-097a5bdaf75c" path="/var/lib/kubelet/pods/bf927633-7540-474c-8c7f-097a5bdaf75c/volumes"
Jul 21 14:40:00.862839 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:00.862806    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-xvpks"
Jul 21 14:40:03.936227 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:03.936186    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z799w" podUID="e0840d52-b972-42ef-9e4c-17addec286c9" containerName="manager" containerID="cri-o://9a5d7bc9c6d944b14c41b0a8addba13f94f01129dcdc1ecc029f942aecea064f" gracePeriod=2
Jul 21 14:40:03.938010 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:03.937982    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qdhk8" event={"ID":"ef529aba-cd19-4ad0-812d-6cf20b3cf90a","Type":"ContainerStarted","Data":"df838f59759bba10ce4d0765c5b8e149e4b216f4654bb6d44110297579cec835"}
Jul 21 14:40:03.938142 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:03.938037    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qdhk8"
Jul 21 14:40:03.959263 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:03.959210    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qdhk8" podStartSLOduration=1.244000782 podStartE2EDuration="4.959195158s" totalImagesPullingTime="3.715194376s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:39:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:40:03.958776225 +0000 UTC m=+460.308519637" watchObservedRunningTime="2026-07-21 14:40:03.959195158 +0000 UTC m=+460.308938634"
Jul 21 14:40:04.185857 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:04.185834    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z799w"
Jul 21 14:40:04.289705 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:04.289669    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e0840d52-b972-42ef-9e4c-17addec286c9-extensions-socket-volume\") pod \"e0840d52-b972-42ef-9e4c-17addec286c9\" (UID: \"e0840d52-b972-42ef-9e4c-17addec286c9\") "
Jul 21 14:40:04.289884 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:04.289758    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mt692\" (UniqueName: \"kubernetes.io/projected/e0840d52-b972-42ef-9e4c-17addec286c9-kube-api-access-mt692\") pod \"e0840d52-b972-42ef-9e4c-17addec286c9\" (UID: \"e0840d52-b972-42ef-9e4c-17addec286c9\") "
Jul 21 14:40:04.289997 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:04.289971    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0840d52-b972-42ef-9e4c-17addec286c9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e0840d52-b972-42ef-9e4c-17addec286c9" (UID: "e0840d52-b972-42ef-9e4c-17addec286c9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 14:40:04.291692 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:04.291660    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0840d52-b972-42ef-9e4c-17addec286c9-kube-api-access-mt692" (OuterVolumeSpecName: "kube-api-access-mt692") pod "e0840d52-b972-42ef-9e4c-17addec286c9" (UID: "e0840d52-b972-42ef-9e4c-17addec286c9"). InnerVolumeSpecName "kube-api-access-mt692". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:40:04.390629 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:04.390583    2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e0840d52-b972-42ef-9e4c-17addec286c9-extensions-socket-volume\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:40:04.390629 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:04.390620    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mt692\" (UniqueName: \"kubernetes.io/projected/e0840d52-b972-42ef-9e4c-17addec286c9-kube-api-access-mt692\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:40:04.944887 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:04.944851    2579 generic.go:356] "Generic (PLEG): container finished" podID="e0840d52-b972-42ef-9e4c-17addec286c9" containerID="9a5d7bc9c6d944b14c41b0a8addba13f94f01129dcdc1ecc029f942aecea064f" exitCode=2
Jul 21 14:40:04.945311 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:04.944892    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z799w"
Jul 21 14:40:04.945311 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:04.944947    2579 scope.go:117] "RemoveContainer" containerID="9a5d7bc9c6d944b14c41b0a8addba13f94f01129dcdc1ecc029f942aecea064f"
Jul 21 14:40:04.947398 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:40:04.947362    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-z799w\" is forbidden: User \"system:node:ip-10-0-130-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-83.ec2.internal' and this object" podUID="e0840d52-b972-42ef-9e4c-17addec286c9" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z799w"
Jul 21 14:40:04.954208 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:04.954189    2579 scope.go:117] "RemoveContainer" containerID="9a5d7bc9c6d944b14c41b0a8addba13f94f01129dcdc1ecc029f942aecea064f"
Jul 21 14:40:04.954613 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:40:04.954592    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a5d7bc9c6d944b14c41b0a8addba13f94f01129dcdc1ecc029f942aecea064f\": container with ID starting with 9a5d7bc9c6d944b14c41b0a8addba13f94f01129dcdc1ecc029f942aecea064f not found: ID does not exist" containerID="9a5d7bc9c6d944b14c41b0a8addba13f94f01129dcdc1ecc029f942aecea064f"
Jul 21 14:40:04.954687 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:04.954621    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a5d7bc9c6d944b14c41b0a8addba13f94f01129dcdc1ecc029f942aecea064f"} err="failed to get container status \"9a5d7bc9c6d944b14c41b0a8addba13f94f01129dcdc1ecc029f942aecea064f\": rpc error: code = NotFound desc = could not find container \"9a5d7bc9c6d944b14c41b0a8addba13f94f01129dcdc1ecc029f942aecea064f\": container with ID starting with 9a5d7bc9c6d944b14c41b0a8addba13f94f01129dcdc1ecc029f942aecea064f not found: ID does not exist"
Jul 21 14:40:04.955559 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:40:04.955542    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-z799w\" is forbidden: User \"system:node:ip-10-0-130-83.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-83.ec2.internal' and this object" podUID="e0840d52-b972-42ef-9e4c-17addec286c9" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-z799w"
Jul 21 14:40:06.171822 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:06.171791    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0840d52-b972-42ef-9e4c-17addec286c9" path="/var/lib/kubelet/pods/e0840d52-b972-42ef-9e4c-17addec286c9/volumes"
Jul 21 14:40:10.920717 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:10.920688    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvp9p"
Jul 21 14:40:14.947823 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:14.947788    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qdhk8"
Jul 21 14:40:27.584184 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:27.584144    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qdhk8"]
Jul 21 14:40:27.584685 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:27.584412    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qdhk8" podUID="ef529aba-cd19-4ad0-812d-6cf20b3cf90a" containerName="manager" containerID="cri-o://df838f59759bba10ce4d0765c5b8e149e4b216f4654bb6d44110297579cec835" gracePeriod=10
Jul 21 14:40:27.851552 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:27.851528    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qdhk8"
Jul 21 14:40:27.978117 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:27.978079    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b62n7\" (UniqueName: \"kubernetes.io/projected/ef529aba-cd19-4ad0-812d-6cf20b3cf90a-kube-api-access-b62n7\") pod \"ef529aba-cd19-4ad0-812d-6cf20b3cf90a\" (UID: \"ef529aba-cd19-4ad0-812d-6cf20b3cf90a\") "
Jul 21 14:40:27.978117 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:27.978121    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ef529aba-cd19-4ad0-812d-6cf20b3cf90a-extensions-socket-volume\") pod \"ef529aba-cd19-4ad0-812d-6cf20b3cf90a\" (UID: \"ef529aba-cd19-4ad0-812d-6cf20b3cf90a\") "
Jul 21 14:40:27.978535 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:27.978511    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef529aba-cd19-4ad0-812d-6cf20b3cf90a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ef529aba-cd19-4ad0-812d-6cf20b3cf90a" (UID: "ef529aba-cd19-4ad0-812d-6cf20b3cf90a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 14:40:27.980003 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:27.979979    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef529aba-cd19-4ad0-812d-6cf20b3cf90a-kube-api-access-b62n7" (OuterVolumeSpecName: "kube-api-access-b62n7") pod "ef529aba-cd19-4ad0-812d-6cf20b3cf90a" (UID: "ef529aba-cd19-4ad0-812d-6cf20b3cf90a"). InnerVolumeSpecName "kube-api-access-b62n7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:40:28.052252 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:28.052227    2579 generic.go:356] "Generic (PLEG): container finished" podID="ef529aba-cd19-4ad0-812d-6cf20b3cf90a" containerID="df838f59759bba10ce4d0765c5b8e149e4b216f4654bb6d44110297579cec835" exitCode=0
Jul 21 14:40:28.052363 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:28.052292    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qdhk8"
Jul 21 14:40:28.052363 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:28.052325    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qdhk8" event={"ID":"ef529aba-cd19-4ad0-812d-6cf20b3cf90a","Type":"ContainerDied","Data":"df838f59759bba10ce4d0765c5b8e149e4b216f4654bb6d44110297579cec835"}
Jul 21 14:40:28.052476 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:28.052414    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qdhk8" event={"ID":"ef529aba-cd19-4ad0-812d-6cf20b3cf90a","Type":"ContainerDied","Data":"f3ed9d25d6d0bb4ba61c0a8afe3e6a49865b9b75c7a29fee7bb1d85cf460f2fa"}
Jul 21 14:40:28.052476 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:28.052448    2579 scope.go:117] "RemoveContainer" containerID="df838f59759bba10ce4d0765c5b8e149e4b216f4654bb6d44110297579cec835"
Jul 21 14:40:28.060941 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:28.060924    2579 scope.go:117] "RemoveContainer" containerID="df838f59759bba10ce4d0765c5b8e149e4b216f4654bb6d44110297579cec835"
Jul 21 14:40:28.061174 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:40:28.061156    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df838f59759bba10ce4d0765c5b8e149e4b216f4654bb6d44110297579cec835\": container with ID starting with df838f59759bba10ce4d0765c5b8e149e4b216f4654bb6d44110297579cec835 not found: ID does not exist" containerID="df838f59759bba10ce4d0765c5b8e149e4b216f4654bb6d44110297579cec835"
Jul 21 14:40:28.061215 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:28.061181    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df838f59759bba10ce4d0765c5b8e149e4b216f4654bb6d44110297579cec835"} err="failed to get container status \"df838f59759bba10ce4d0765c5b8e149e4b216f4654bb6d44110297579cec835\": rpc error: code = NotFound desc = could not find container \"df838f59759bba10ce4d0765c5b8e149e4b216f4654bb6d44110297579cec835\": container with ID starting with df838f59759bba10ce4d0765c5b8e149e4b216f4654bb6d44110297579cec835 not found: ID does not exist"
Jul 21 14:40:28.073814 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:28.073792    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qdhk8"]
Jul 21 14:40:28.078687 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:28.078666    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-qdhk8"]
Jul 21 14:40:28.078937 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:28.078921    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b62n7\" (UniqueName: \"kubernetes.io/projected/ef529aba-cd19-4ad0-812d-6cf20b3cf90a-kube-api-access-b62n7\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:40:28.078975 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:28.078941    2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ef529aba-cd19-4ad0-812d-6cf20b3cf90a-extensions-socket-volume\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:40:28.171945 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:28.171923    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef529aba-cd19-4ad0-812d-6cf20b3cf90a" path="/var/lib/kubelet/pods/ef529aba-cd19-4ad0-812d-6cf20b3cf90a/volumes"
Jul 21 14:40:45.785924 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:45.785890    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zndd7"]
Jul 21 14:40:45.786524 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:45.786349    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef529aba-cd19-4ad0-812d-6cf20b3cf90a" containerName="manager"
Jul 21 14:40:45.786524 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:45.786368    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef529aba-cd19-4ad0-812d-6cf20b3cf90a" containerName="manager"
Jul 21 14:40:45.786524 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:45.786384    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0840d52-b972-42ef-9e4c-17addec286c9" containerName="manager"
Jul 21 14:40:45.786524 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:45.786392    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0840d52-b972-42ef-9e4c-17addec286c9" containerName="manager"
Jul 21 14:40:45.786524 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:45.786472    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0840d52-b972-42ef-9e4c-17addec286c9" containerName="manager"
Jul 21 14:40:45.786524 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:45.786485    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef529aba-cd19-4ad0-812d-6cf20b3cf90a" containerName="manager"
Jul 21 14:40:45.789583 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:45.789564    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-zndd7"
Jul 21 14:40:45.791864 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:45.791847    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 21 14:40:45.791940 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:45.791849    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-sftv9\""
Jul 21 14:40:45.795359 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:45.795338    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zndd7"]
Jul 21 14:40:45.885646 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:45.885613    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zndd7"]
Jul 21 14:40:45.907456 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:45.907432    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0c8aaf75-56e8-4c54-8f93-2ae58f6e2184-config-file\") pod \"limitador-limitador-64c94fd64f-zndd7\" (UID: \"0c8aaf75-56e8-4c54-8f93-2ae58f6e2184\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-zndd7"
Jul 21 14:40:45.907578 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:45.907468    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhrhg\" (UniqueName: \"kubernetes.io/projected/0c8aaf75-56e8-4c54-8f93-2ae58f6e2184-kube-api-access-jhrhg\") pod \"limitador-limitador-64c94fd64f-zndd7\" (UID: \"0c8aaf75-56e8-4c54-8f93-2ae58f6e2184\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-zndd7"
Jul 21 14:40:46.008113 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:46.008081    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0c8aaf75-56e8-4c54-8f93-2ae58f6e2184-config-file\") pod \"limitador-limitador-64c94fd64f-zndd7\" (UID: \"0c8aaf75-56e8-4c54-8f93-2ae58f6e2184\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-zndd7"
Jul 21 14:40:46.008264 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:46.008121    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhrhg\" (UniqueName: \"kubernetes.io/projected/0c8aaf75-56e8-4c54-8f93-2ae58f6e2184-kube-api-access-jhrhg\") pod \"limitador-limitador-64c94fd64f-zndd7\" (UID: \"0c8aaf75-56e8-4c54-8f93-2ae58f6e2184\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-zndd7"
Jul 21 14:40:46.008761 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:46.008742    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0c8aaf75-56e8-4c54-8f93-2ae58f6e2184-config-file\") pod \"limitador-limitador-64c94fd64f-zndd7\" (UID: \"0c8aaf75-56e8-4c54-8f93-2ae58f6e2184\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-zndd7"
Jul 21 14:40:46.017101 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:46.017075    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhrhg\" (UniqueName: \"kubernetes.io/projected/0c8aaf75-56e8-4c54-8f93-2ae58f6e2184-kube-api-access-jhrhg\") pod \"limitador-limitador-64c94fd64f-zndd7\" (UID: \"0c8aaf75-56e8-4c54-8f93-2ae58f6e2184\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-zndd7"
Jul 21 14:40:46.101155 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:46.101101    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-zndd7"
Jul 21 14:40:46.228926 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:46.228892    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zndd7"]
Jul 21 14:40:46.232196 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:40:46.232170    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c8aaf75_56e8_4c54_8f93_2ae58f6e2184.slice/crio-d1f25c8039e7dfe6eda47cc6d75f5c00410a1de6d56fd1de809fad4ad6797314 WatchSource:0}: Error finding container d1f25c8039e7dfe6eda47cc6d75f5c00410a1de6d56fd1de809fad4ad6797314: Status 404 returned error can't find the container with id d1f25c8039e7dfe6eda47cc6d75f5c00410a1de6d56fd1de809fad4ad6797314
Jul 21 14:40:47.135937 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:47.135816    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-zndd7" event={"ID":"0c8aaf75-56e8-4c54-8f93-2ae58f6e2184","Type":"ContainerStarted","Data":"d1f25c8039e7dfe6eda47cc6d75f5c00410a1de6d56fd1de809fad4ad6797314"}
Jul 21 14:40:49.145931 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:49.145901    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-zndd7" event={"ID":"0c8aaf75-56e8-4c54-8f93-2ae58f6e2184","Type":"ContainerStarted","Data":"8c2e057029ef461c1886674f1cf9846ececa93d303cec129384509f326686d0c"}
Jul 21 14:40:49.146306 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:49.145996    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-zndd7"
Jul 21 14:40:49.165623 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:40:49.165580    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-zndd7" podStartSLOduration=1.5494371230000001 podStartE2EDuration="4.165567852s" totalImagesPullingTime="2.616130729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:40:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:40:49.164014065 +0000 UTC m=+505.513757480" watchObservedRunningTime="2026-07-21 14:40:49.165567852 +0000 UTC m=+505.515311264"
Jul 21 14:41:00.150431 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:00.150397    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-zndd7"
Jul 21 14:41:02.771507 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:02.771470    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zndd7"]
Jul 21 14:41:02.771873 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:02.771664    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-zndd7" podUID="0c8aaf75-56e8-4c54-8f93-2ae58f6e2184" containerName="limitador" containerID="cri-o://8c2e057029ef461c1886674f1cf9846ececa93d303cec129384509f326686d0c" gracePeriod=30
Jul 21 14:41:03.201643 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:03.201615    2579 generic.go:356] "Generic (PLEG): container finished" podID="0c8aaf75-56e8-4c54-8f93-2ae58f6e2184" containerID="8c2e057029ef461c1886674f1cf9846ececa93d303cec129384509f326686d0c" exitCode=0
Jul 21 14:41:03.201808 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:03.201684    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-zndd7" event={"ID":"0c8aaf75-56e8-4c54-8f93-2ae58f6e2184","Type":"ContainerDied","Data":"8c2e057029ef461c1886674f1cf9846ececa93d303cec129384509f326686d0c"}
Jul 21 14:41:03.310635 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:03.310613    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-zndd7"
Jul 21 14:41:03.342531 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:03.342474    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0c8aaf75-56e8-4c54-8f93-2ae58f6e2184-config-file\") pod \"0c8aaf75-56e8-4c54-8f93-2ae58f6e2184\" (UID: \"0c8aaf75-56e8-4c54-8f93-2ae58f6e2184\") "
Jul 21 14:41:03.342653 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:03.342541    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jhrhg\" (UniqueName: \"kubernetes.io/projected/0c8aaf75-56e8-4c54-8f93-2ae58f6e2184-kube-api-access-jhrhg\") pod \"0c8aaf75-56e8-4c54-8f93-2ae58f6e2184\" (UID: \"0c8aaf75-56e8-4c54-8f93-2ae58f6e2184\") "
Jul 21 14:41:03.342784 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:03.342764    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c8aaf75-56e8-4c54-8f93-2ae58f6e2184-config-file" (OuterVolumeSpecName: "config-file") pod "0c8aaf75-56e8-4c54-8f93-2ae58f6e2184" (UID: "0c8aaf75-56e8-4c54-8f93-2ae58f6e2184"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 14:41:03.344500 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:03.344479    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c8aaf75-56e8-4c54-8f93-2ae58f6e2184-kube-api-access-jhrhg" (OuterVolumeSpecName: "kube-api-access-jhrhg") pod "0c8aaf75-56e8-4c54-8f93-2ae58f6e2184" (UID: "0c8aaf75-56e8-4c54-8f93-2ae58f6e2184"). InnerVolumeSpecName "kube-api-access-jhrhg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:41:03.443861 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:03.443838    2579 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0c8aaf75-56e8-4c54-8f93-2ae58f6e2184-config-file\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:41:03.443861 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:03.443861    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhrhg\" (UniqueName: \"kubernetes.io/projected/0c8aaf75-56e8-4c54-8f93-2ae58f6e2184-kube-api-access-jhrhg\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:41:04.206851 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:04.206825    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-zndd7"
Jul 21 14:41:04.207221 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:04.206859    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-zndd7" event={"ID":"0c8aaf75-56e8-4c54-8f93-2ae58f6e2184","Type":"ContainerDied","Data":"d1f25c8039e7dfe6eda47cc6d75f5c00410a1de6d56fd1de809fad4ad6797314"}
Jul 21 14:41:04.207221 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:04.206919    2579 scope.go:117] "RemoveContainer" containerID="8c2e057029ef461c1886674f1cf9846ececa93d303cec129384509f326686d0c"
Jul 21 14:41:04.222781 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:04.222759    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zndd7"]
Jul 21 14:41:04.226010 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:04.225989    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-zndd7"]
Jul 21 14:41:05.828125 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:05.828091    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-wg2sl"]
Jul 21 14:41:05.828495 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:05.828435    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c8aaf75-56e8-4c54-8f93-2ae58f6e2184" containerName="limitador"
Jul 21 14:41:05.828495 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:05.828448    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c8aaf75-56e8-4c54-8f93-2ae58f6e2184" containerName="limitador"
Jul 21 14:41:05.828579 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:05.828517    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c8aaf75-56e8-4c54-8f93-2ae58f6e2184" containerName="limitador"
Jul 21 14:41:05.832776 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:05.832745    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wg2sl"
Jul 21 14:41:05.835926 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:05.835904    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 21 14:41:05.836153 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:05.836139    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 14:41:05.837182 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:05.837161    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 14:41:05.837182 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:05.837177    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-ppqdb\""
Jul 21 14:41:05.840743 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:05.840725    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-wg2sl"]
Jul 21 14:41:05.863603 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:05.863574    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/bff4c9fc-5579-48fa-bdbf-5efcb4f3c738-data\") pod \"postgres-69fb4594cf-wg2sl\" (UID: \"bff4c9fc-5579-48fa-bdbf-5efcb4f3c738\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wg2sl"
Jul 21 14:41:05.863708 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:05.863610    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7mvt\" (UniqueName: \"kubernetes.io/projected/bff4c9fc-5579-48fa-bdbf-5efcb4f3c738-kube-api-access-z7mvt\") pod \"postgres-69fb4594cf-wg2sl\" (UID: \"bff4c9fc-5579-48fa-bdbf-5efcb4f3c738\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wg2sl"
Jul 21 14:41:05.964802 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:05.964778    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/bff4c9fc-5579-48fa-bdbf-5efcb4f3c738-data\") pod \"postgres-69fb4594cf-wg2sl\" (UID: \"bff4c9fc-5579-48fa-bdbf-5efcb4f3c738\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wg2sl"
Jul 21 14:41:05.964883 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:05.964805    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7mvt\" (UniqueName: \"kubernetes.io/projected/bff4c9fc-5579-48fa-bdbf-5efcb4f3c738-kube-api-access-z7mvt\") pod \"postgres-69fb4594cf-wg2sl\" (UID: \"bff4c9fc-5579-48fa-bdbf-5efcb4f3c738\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wg2sl"
Jul 21 14:41:05.965117 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:05.965098    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/bff4c9fc-5579-48fa-bdbf-5efcb4f3c738-data\") pod \"postgres-69fb4594cf-wg2sl\" (UID: \"bff4c9fc-5579-48fa-bdbf-5efcb4f3c738\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wg2sl"
Jul 21 14:41:05.973318 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:05.973295    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7mvt\" (UniqueName: \"kubernetes.io/projected/bff4c9fc-5579-48fa-bdbf-5efcb4f3c738-kube-api-access-z7mvt\") pod \"postgres-69fb4594cf-wg2sl\" (UID: \"bff4c9fc-5579-48fa-bdbf-5efcb4f3c738\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wg2sl"
Jul 21 14:41:06.143554 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:06.143532    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wg2sl"
Jul 21 14:41:06.172331 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:06.172303    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c8aaf75-56e8-4c54-8f93-2ae58f6e2184" path="/var/lib/kubelet/pods/0c8aaf75-56e8-4c54-8f93-2ae58f6e2184/volumes"
Jul 21 14:41:06.270553 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:06.270480    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-wg2sl"]
Jul 21 14:41:06.272659 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:41:06.272633    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbff4c9fc_5579_48fa_bdbf_5efcb4f3c738.slice/crio-e6cfdd6f93f5c8ade342fb7b9bf13d374fae3893d8dda64b8cede3659e451874 WatchSource:0}: Error finding container e6cfdd6f93f5c8ade342fb7b9bf13d374fae3893d8dda64b8cede3659e451874: Status 404 returned error can't find the container with id e6cfdd6f93f5c8ade342fb7b9bf13d374fae3893d8dda64b8cede3659e451874
Jul 21 14:41:07.224049 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:07.223981    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wg2sl" event={"ID":"bff4c9fc-5579-48fa-bdbf-5efcb4f3c738","Type":"ContainerStarted","Data":"e6cfdd6f93f5c8ade342fb7b9bf13d374fae3893d8dda64b8cede3659e451874"}
Jul 21 14:41:12.250020 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:12.249975    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wg2sl" event={"ID":"bff4c9fc-5579-48fa-bdbf-5efcb4f3c738","Type":"ContainerStarted","Data":"3709bb94b855473dc841bc2bba9b9de3057a1c0ce26d295218c4d1a0c2751140"}
Jul 21 14:41:12.250500 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:12.250087    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wg2sl"
Jul 21 14:41:12.267007 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:12.266962    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wg2sl" podStartSLOduration=2.316574084 podStartE2EDuration="7.266947636s" totalImagesPullingTime="4.950373552s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:41:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:41:12.266698387 +0000 UTC m=+528.616441803" watchObservedRunningTime="2026-07-21 14:41:12.266947636 +0000 UTC m=+528.616691047"
Jul 21 14:41:18.283301 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:18.283213    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wg2sl"
Jul 21 14:41:27.665175 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:27.665133    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6cd5d674d5-bz26w"]
Jul 21 14:41:27.688983 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:27.688943    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cd5d674d5-bz26w"]
Jul 21 14:41:27.689161 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:27.689078    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cd5d674d5-bz26w"
Jul 21 14:41:27.692723 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:27.692698    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-bbcx6\""
Jul 21 14:41:27.692723 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:27.692715    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 21 14:41:27.692910 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:27.692710    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 21 14:41:27.834465 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:27.834435    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nth8p\" (UniqueName: \"kubernetes.io/projected/6a5eb107-a31c-4e18-a09d-fe813a4aa7a7-kube-api-access-nth8p\") pod \"maas-controller-6cd5d674d5-bz26w\" (UID: \"6a5eb107-a31c-4e18-a09d-fe813a4aa7a7\") " pod="opendatahub/maas-controller-6cd5d674d5-bz26w"
Jul 21 14:41:27.834641 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:27.834474    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6a5eb107-a31c-4e18-a09d-fe813a4aa7a7-webhook-cert\") pod \"maas-controller-6cd5d674d5-bz26w\" (UID: \"6a5eb107-a31c-4e18-a09d-fe813a4aa7a7\") " pod="opendatahub/maas-controller-6cd5d674d5-bz26w"
Jul 21 14:41:27.935211 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:27.935128    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nth8p\" (UniqueName: \"kubernetes.io/projected/6a5eb107-a31c-4e18-a09d-fe813a4aa7a7-kube-api-access-nth8p\") pod \"maas-controller-6cd5d674d5-bz26w\" (UID: \"6a5eb107-a31c-4e18-a09d-fe813a4aa7a7\") " pod="opendatahub/maas-controller-6cd5d674d5-bz26w"
Jul 21 14:41:27.935211 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:27.935169    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6a5eb107-a31c-4e18-a09d-fe813a4aa7a7-webhook-cert\") pod \"maas-controller-6cd5d674d5-bz26w\" (UID: \"6a5eb107-a31c-4e18-a09d-fe813a4aa7a7\") " pod="opendatahub/maas-controller-6cd5d674d5-bz26w"
Jul 21 14:41:27.937523 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:27.937495    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6a5eb107-a31c-4e18-a09d-fe813a4aa7a7-webhook-cert\") pod \"maas-controller-6cd5d674d5-bz26w\" (UID: \"6a5eb107-a31c-4e18-a09d-fe813a4aa7a7\") " pod="opendatahub/maas-controller-6cd5d674d5-bz26w"
Jul 21 14:41:27.946118 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:27.946091    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nth8p\" (UniqueName: \"kubernetes.io/projected/6a5eb107-a31c-4e18-a09d-fe813a4aa7a7-kube-api-access-nth8p\") pod \"maas-controller-6cd5d674d5-bz26w\" (UID: \"6a5eb107-a31c-4e18-a09d-fe813a4aa7a7\") " pod="opendatahub/maas-controller-6cd5d674d5-bz26w"
Jul 21 14:41:27.946874 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:27.946856    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7487796bf9-jpsvm"]
Jul 21 14:41:27.949323 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:27.949309    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7487796bf9-jpsvm"
Jul 21 14:41:27.958284 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:27.958240    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7487796bf9-jpsvm"]
Jul 21 14:41:28.000345 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:28.000322    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cd5d674d5-bz26w"
Jul 21 14:41:28.036151 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:28.036122    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a944ad88-63b9-4e0b-ba20-159794e0f55e-webhook-cert\") pod \"maas-controller-7487796bf9-jpsvm\" (UID: \"a944ad88-63b9-4e0b-ba20-159794e0f55e\") " pod="opendatahub/maas-controller-7487796bf9-jpsvm"
Jul 21 14:41:28.036277 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:28.036195    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6897r\" (UniqueName: \"kubernetes.io/projected/a944ad88-63b9-4e0b-ba20-159794e0f55e-kube-api-access-6897r\") pod \"maas-controller-7487796bf9-jpsvm\" (UID: \"a944ad88-63b9-4e0b-ba20-159794e0f55e\") " pod="opendatahub/maas-controller-7487796bf9-jpsvm"
Jul 21 14:41:28.123406 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:28.123384    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cd5d674d5-bz26w"]
Jul 21 14:41:28.125968 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:41:28.125941    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a5eb107_a31c_4e18_a09d_fe813a4aa7a7.slice/crio-eefa535073701f00aa5b81e6ab3e427173b68b4ce9d497f996da6ae70144a063 WatchSource:0}: Error finding container eefa535073701f00aa5b81e6ab3e427173b68b4ce9d497f996da6ae70144a063: Status 404 returned error can't find the container with id eefa535073701f00aa5b81e6ab3e427173b68b4ce9d497f996da6ae70144a063
Jul 21 14:41:28.137256 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:28.137229    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6897r\" (UniqueName: \"kubernetes.io/projected/a944ad88-63b9-4e0b-ba20-159794e0f55e-kube-api-access-6897r\") pod \"maas-controller-7487796bf9-jpsvm\" (UID: \"a944ad88-63b9-4e0b-ba20-159794e0f55e\") " pod="opendatahub/maas-controller-7487796bf9-jpsvm"
Jul 21 14:41:28.137383 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:28.137324    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a944ad88-63b9-4e0b-ba20-159794e0f55e-webhook-cert\") pod \"maas-controller-7487796bf9-jpsvm\" (UID: \"a944ad88-63b9-4e0b-ba20-159794e0f55e\") " pod="opendatahub/maas-controller-7487796bf9-jpsvm"
Jul 21 14:41:28.139496 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:28.139477    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a944ad88-63b9-4e0b-ba20-159794e0f55e-webhook-cert\") pod \"maas-controller-7487796bf9-jpsvm\" (UID: \"a944ad88-63b9-4e0b-ba20-159794e0f55e\") " pod="opendatahub/maas-controller-7487796bf9-jpsvm"
Jul 21 14:41:28.144910 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:28.144887    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6897r\" (UniqueName: \"kubernetes.io/projected/a944ad88-63b9-4e0b-ba20-159794e0f55e-kube-api-access-6897r\") pod \"maas-controller-7487796bf9-jpsvm\" (UID: \"a944ad88-63b9-4e0b-ba20-159794e0f55e\") " pod="opendatahub/maas-controller-7487796bf9-jpsvm"
Jul 21 14:41:28.261637 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:28.261570    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7487796bf9-jpsvm"
Jul 21 14:41:28.314339 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:28.314307    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cd5d674d5-bz26w" event={"ID":"6a5eb107-a31c-4e18-a09d-fe813a4aa7a7","Type":"ContainerStarted","Data":"eefa535073701f00aa5b81e6ab3e427173b68b4ce9d497f996da6ae70144a063"}
Jul 21 14:41:28.382826 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:28.382805    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7487796bf9-jpsvm"]
Jul 21 14:41:28.384890 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:41:28.384860    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda944ad88_63b9_4e0b_ba20_159794e0f55e.slice/crio-873fa09f8b8e5360e6e1ab4cf01076df4298f5fce1f1fc665688bfab02974d86 WatchSource:0}: Error finding container 873fa09f8b8e5360e6e1ab4cf01076df4298f5fce1f1fc665688bfab02974d86: Status 404 returned error can't find the container with id 873fa09f8b8e5360e6e1ab4cf01076df4298f5fce1f1fc665688bfab02974d86
Jul 21 14:41:29.321425 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:29.321385    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7487796bf9-jpsvm" event={"ID":"a944ad88-63b9-4e0b-ba20-159794e0f55e","Type":"ContainerStarted","Data":"873fa09f8b8e5360e6e1ab4cf01076df4298f5fce1f1fc665688bfab02974d86"}
Jul 21 14:41:31.331636 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:31.331598    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7487796bf9-jpsvm" event={"ID":"a944ad88-63b9-4e0b-ba20-159794e0f55e","Type":"ContainerStarted","Data":"1b464fd3bc013e3b883163cb9c4deb6b445359b24b8c496c9f73968116f49f0e"}
Jul 21 14:41:31.332084 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:31.331725    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7487796bf9-jpsvm"
Jul 21 14:41:31.333309 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:31.333283    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cd5d674d5-bz26w" event={"ID":"6a5eb107-a31c-4e18-a09d-fe813a4aa7a7","Type":"ContainerStarted","Data":"563c82e190fa295e64b657cca7edafec74fa4efca9c835217def1f8316396796"}
Jul 21 14:41:31.333462 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:31.333395    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6cd5d674d5-bz26w"
Jul 21 14:41:31.351463 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:31.351423    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7487796bf9-jpsvm" podStartSLOduration=2.237909905 podStartE2EDuration="4.351410351s" totalImagesPullingTime="2.113500446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:41:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:41:31.349967797 +0000 UTC m=+547.699711211" watchObservedRunningTime="2026-07-21 14:41:31.351410351 +0000 UTC m=+547.701153761"
Jul 21 14:41:31.369563 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:31.369525    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6cd5d674d5-bz26w" podStartSLOduration=1.991622497 podStartE2EDuration="4.369512819s" totalImagesPullingTime="2.377890322s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:41:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:41:31.368057262 +0000 UTC m=+547.717800673" watchObservedRunningTime="2026-07-21 14:41:31.369512819 +0000 UTC m=+547.719256228"
Jul 21 14:41:42.339292 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:42.339240    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7487796bf9-jpsvm"
Jul 21 14:41:42.339697 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:42.339412    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6cd5d674d5-bz26w"
Jul 21 14:41:42.402750 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:42.402723    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6cd5d674d5-bz26w"]
Jul 21 14:41:42.402962 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:42.402916    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6cd5d674d5-bz26w" podUID="6a5eb107-a31c-4e18-a09d-fe813a4aa7a7" containerName="manager" containerID="cri-o://563c82e190fa295e64b657cca7edafec74fa4efca9c835217def1f8316396796" gracePeriod=30
Jul 21 14:41:42.641506 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:42.641481    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cd5d674d5-bz26w"
Jul 21 14:41:42.757448 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:42.757423    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6a5eb107-a31c-4e18-a09d-fe813a4aa7a7-webhook-cert\") pod \"6a5eb107-a31c-4e18-a09d-fe813a4aa7a7\" (UID: \"6a5eb107-a31c-4e18-a09d-fe813a4aa7a7\") "
Jul 21 14:41:42.757585 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:42.757475    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nth8p\" (UniqueName: \"kubernetes.io/projected/6a5eb107-a31c-4e18-a09d-fe813a4aa7a7-kube-api-access-nth8p\") pod \"6a5eb107-a31c-4e18-a09d-fe813a4aa7a7\" (UID: \"6a5eb107-a31c-4e18-a09d-fe813a4aa7a7\") "
Jul 21 14:41:42.759448 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:42.759421    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a5eb107-a31c-4e18-a09d-fe813a4aa7a7-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6a5eb107-a31c-4e18-a09d-fe813a4aa7a7" (UID: "6a5eb107-a31c-4e18-a09d-fe813a4aa7a7"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:41:42.759572 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:42.759497    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a5eb107-a31c-4e18-a09d-fe813a4aa7a7-kube-api-access-nth8p" (OuterVolumeSpecName: "kube-api-access-nth8p") pod "6a5eb107-a31c-4e18-a09d-fe813a4aa7a7" (UID: "6a5eb107-a31c-4e18-a09d-fe813a4aa7a7"). InnerVolumeSpecName "kube-api-access-nth8p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:41:42.857950 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:42.857928    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nth8p\" (UniqueName: \"kubernetes.io/projected/6a5eb107-a31c-4e18-a09d-fe813a4aa7a7-kube-api-access-nth8p\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:41:42.857950 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:42.857949    2579 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6a5eb107-a31c-4e18-a09d-fe813a4aa7a7-webhook-cert\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:41:43.382925 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:43.382888    2579 generic.go:356] "Generic (PLEG): container finished" podID="6a5eb107-a31c-4e18-a09d-fe813a4aa7a7" containerID="563c82e190fa295e64b657cca7edafec74fa4efca9c835217def1f8316396796" exitCode=0
Jul 21 14:41:43.382925 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:43.382921    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cd5d674d5-bz26w" event={"ID":"6a5eb107-a31c-4e18-a09d-fe813a4aa7a7","Type":"ContainerDied","Data":"563c82e190fa295e64b657cca7edafec74fa4efca9c835217def1f8316396796"}
Jul 21 14:41:43.382925 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:43.382925    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cd5d674d5-bz26w"
Jul 21 14:41:43.383433 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:43.382956    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cd5d674d5-bz26w" event={"ID":"6a5eb107-a31c-4e18-a09d-fe813a4aa7a7","Type":"ContainerDied","Data":"eefa535073701f00aa5b81e6ab3e427173b68b4ce9d497f996da6ae70144a063"}
Jul 21 14:41:43.383433 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:43.382976    2579 scope.go:117] "RemoveContainer" containerID="563c82e190fa295e64b657cca7edafec74fa4efca9c835217def1f8316396796"
Jul 21 14:41:43.391258 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:43.391244    2579 scope.go:117] "RemoveContainer" containerID="563c82e190fa295e64b657cca7edafec74fa4efca9c835217def1f8316396796"
Jul 21 14:41:43.391513 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:41:43.391496    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"563c82e190fa295e64b657cca7edafec74fa4efca9c835217def1f8316396796\": container with ID starting with 563c82e190fa295e64b657cca7edafec74fa4efca9c835217def1f8316396796 not found: ID does not exist" containerID="563c82e190fa295e64b657cca7edafec74fa4efca9c835217def1f8316396796"
Jul 21 14:41:43.391563 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:43.391520    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"563c82e190fa295e64b657cca7edafec74fa4efca9c835217def1f8316396796"} err="failed to get container status \"563c82e190fa295e64b657cca7edafec74fa4efca9c835217def1f8316396796\": rpc error: code = NotFound desc = could not find container \"563c82e190fa295e64b657cca7edafec74fa4efca9c835217def1f8316396796\": container with ID starting with 563c82e190fa295e64b657cca7edafec74fa4efca9c835217def1f8316396796 not found: ID does not exist"
Jul 21 14:41:43.404404 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:43.404380    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6cd5d674d5-bz26w"]
Jul 21 14:41:43.410411 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:43.410393    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6cd5d674d5-bz26w"]
Jul 21 14:41:44.171584 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:41:44.171556    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a5eb107-a31c-4e18-a09d-fe813a4aa7a7" path="/var/lib/kubelet/pods/6a5eb107-a31c-4e18-a09d-fe813a4aa7a7/volumes"
Jul 21 14:42:00.991467 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:00.991430    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-56b468fd85-pctgt"]
Jul 21 14:42:00.992043 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:00.991761    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a5eb107-a31c-4e18-a09d-fe813a4aa7a7" containerName="manager"
Jul 21 14:42:00.992043 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:00.991772    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a5eb107-a31c-4e18-a09d-fe813a4aa7a7" containerName="manager"
Jul 21 14:42:00.992043 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:00.991838    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a5eb107-a31c-4e18-a09d-fe813a4aa7a7" containerName="manager"
Jul 21 14:42:00.999522 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:00.999499    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-56b468fd85-pctgt"
Jul 21 14:42:01.002232 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:01.002208    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-szwsv\""
Jul 21 14:42:01.002385 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:01.002316    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 21 14:42:01.005428 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:01.005404    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-56b468fd85-pctgt"]
Jul 21 14:42:01.095842 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:01.095805    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmkhw\" (UniqueName: \"kubernetes.io/projected/7710d3a2-0914-427a-aa34-6941e5886b4e-kube-api-access-wmkhw\") pod \"maas-api-56b468fd85-pctgt\" (UID: \"7710d3a2-0914-427a-aa34-6941e5886b4e\") " pod="odh-ai-gateway-infra/maas-api-56b468fd85-pctgt"
Jul 21 14:42:01.096029 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:01.095868    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7710d3a2-0914-427a-aa34-6941e5886b4e-openshift-service-ca\") pod \"maas-api-56b468fd85-pctgt\" (UID: \"7710d3a2-0914-427a-aa34-6941e5886b4e\") " pod="odh-ai-gateway-infra/maas-api-56b468fd85-pctgt"
Jul 21 14:42:01.096029 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:01.095949    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7710d3a2-0914-427a-aa34-6941e5886b4e-maas-api-tls\") pod \"maas-api-56b468fd85-pctgt\" (UID: \"7710d3a2-0914-427a-aa34-6941e5886b4e\") " pod="odh-ai-gateway-infra/maas-api-56b468fd85-pctgt"
Jul 21 14:42:01.196699 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:01.196665    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmkhw\" (UniqueName: \"kubernetes.io/projected/7710d3a2-0914-427a-aa34-6941e5886b4e-kube-api-access-wmkhw\") pod \"maas-api-56b468fd85-pctgt\" (UID: \"7710d3a2-0914-427a-aa34-6941e5886b4e\") " pod="odh-ai-gateway-infra/maas-api-56b468fd85-pctgt"
Jul 21 14:42:01.197009 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:01.196986    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7710d3a2-0914-427a-aa34-6941e5886b4e-openshift-service-ca\") pod \"maas-api-56b468fd85-pctgt\" (UID: \"7710d3a2-0914-427a-aa34-6941e5886b4e\") " pod="odh-ai-gateway-infra/maas-api-56b468fd85-pctgt"
Jul 21 14:42:01.197101 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:01.197048    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7710d3a2-0914-427a-aa34-6941e5886b4e-maas-api-tls\") pod \"maas-api-56b468fd85-pctgt\" (UID: \"7710d3a2-0914-427a-aa34-6941e5886b4e\") " pod="odh-ai-gateway-infra/maas-api-56b468fd85-pctgt"
Jul 21 14:42:01.197622 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:01.197598    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7710d3a2-0914-427a-aa34-6941e5886b4e-openshift-service-ca\") pod \"maas-api-56b468fd85-pctgt\" (UID: \"7710d3a2-0914-427a-aa34-6941e5886b4e\") " pod="odh-ai-gateway-infra/maas-api-56b468fd85-pctgt"
Jul 21 14:42:01.199372 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:01.199357    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7710d3a2-0914-427a-aa34-6941e5886b4e-maas-api-tls\") pod \"maas-api-56b468fd85-pctgt\" (UID: \"7710d3a2-0914-427a-aa34-6941e5886b4e\") " pod="odh-ai-gateway-infra/maas-api-56b468fd85-pctgt"
Jul 21 14:42:01.207604 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:01.207576    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmkhw\" (UniqueName: \"kubernetes.io/projected/7710d3a2-0914-427a-aa34-6941e5886b4e-kube-api-access-wmkhw\") pod \"maas-api-56b468fd85-pctgt\" (UID: \"7710d3a2-0914-427a-aa34-6941e5886b4e\") " pod="odh-ai-gateway-infra/maas-api-56b468fd85-pctgt"
Jul 21 14:42:01.311359 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:01.311236    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-56b468fd85-pctgt"
Jul 21 14:42:01.446610 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:01.446586    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-56b468fd85-pctgt"]
Jul 21 14:42:01.448302 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:42:01.448259    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7710d3a2_0914_427a_aa34_6941e5886b4e.slice/crio-d9e25eb724961294a9c12ed80a4b55c49f30f305e35eb79cfb965343d6c0ab42 WatchSource:0}: Error finding container d9e25eb724961294a9c12ed80a4b55c49f30f305e35eb79cfb965343d6c0ab42: Status 404 returned error can't find the container with id d9e25eb724961294a9c12ed80a4b55c49f30f305e35eb79cfb965343d6c0ab42
Jul 21 14:42:01.458149 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:01.458121    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-56b468fd85-pctgt" event={"ID":"7710d3a2-0914-427a-aa34-6941e5886b4e","Type":"ContainerStarted","Data":"d9e25eb724961294a9c12ed80a4b55c49f30f305e35eb79cfb965343d6c0ab42"}
Jul 21 14:42:01.591245 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:01.591171    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-79nqm"]
Jul 21 14:42:01.596002 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:01.595979    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-79nqm"
Jul 21 14:42:01.598850 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:01.598821    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-gp6pj\""
Jul 21 14:42:01.598967 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:01.598894    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 21 14:42:01.600578 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:01.600558    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6h22\" (UniqueName: \"kubernetes.io/projected/66ba0e99-974c-4ff9-8b52-f2ee4d591271-kube-api-access-p6h22\") pod \"payload-processing-5bf47cd988-79nqm\" (UID: \"66ba0e99-974c-4ff9-8b52-f2ee4d591271\") " pod="openshift-ingress/payload-processing-5bf47cd988-79nqm"
Jul 21 14:42:01.600671 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:01.600596    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66ba0e99-974c-4ff9-8b52-f2ee4d591271-plugins-config-volume\") pod \"payload-processing-5bf47cd988-79nqm\" (UID: \"66ba0e99-974c-4ff9-8b52-f2ee4d591271\") " pod="openshift-ingress/payload-processing-5bf47cd988-79nqm"
Jul 21 14:42:01.607073 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:01.607052    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-79nqm"]
Jul 21 14:42:01.701313 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:01.701284    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6h22\" (UniqueName: \"kubernetes.io/projected/66ba0e99-974c-4ff9-8b52-f2ee4d591271-kube-api-access-p6h22\") pod \"payload-processing-5bf47cd988-79nqm\" (UID: \"66ba0e99-974c-4ff9-8b52-f2ee4d591271\") " pod="openshift-ingress/payload-processing-5bf47cd988-79nqm"
Jul 21 14:42:01.701477 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:01.701332    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66ba0e99-974c-4ff9-8b52-f2ee4d591271-plugins-config-volume\") pod \"payload-processing-5bf47cd988-79nqm\" (UID: \"66ba0e99-974c-4ff9-8b52-f2ee4d591271\") " pod="openshift-ingress/payload-processing-5bf47cd988-79nqm"
Jul 21 14:42:01.702004 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:01.701985    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/66ba0e99-974c-4ff9-8b52-f2ee4d591271-plugins-config-volume\") pod \"payload-processing-5bf47cd988-79nqm\" (UID: \"66ba0e99-974c-4ff9-8b52-f2ee4d591271\") " pod="openshift-ingress/payload-processing-5bf47cd988-79nqm"
Jul 21 14:42:01.710127 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:01.710105    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6h22\" (UniqueName: \"kubernetes.io/projected/66ba0e99-974c-4ff9-8b52-f2ee4d591271-kube-api-access-p6h22\") pod \"payload-processing-5bf47cd988-79nqm\" (UID: \"66ba0e99-974c-4ff9-8b52-f2ee4d591271\") " pod="openshift-ingress/payload-processing-5bf47cd988-79nqm"
Jul 21 14:42:01.909003 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:01.908966    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-79nqm"
Jul 21 14:42:02.039195 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:02.039138    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-79nqm"]
Jul 21 14:42:02.465052 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:02.465007    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-79nqm" event={"ID":"66ba0e99-974c-4ff9-8b52-f2ee4d591271","Type":"ContainerStarted","Data":"8187c2ce5e490f7b01e620edd2afaa612ea2b92d74f30cd89180beaf2cbba3df"}
Jul 21 14:42:03.471326 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:03.471228    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-56b468fd85-pctgt" event={"ID":"7710d3a2-0914-427a-aa34-6941e5886b4e","Type":"ContainerStarted","Data":"7e345877cdaab1bed5502a9527daac9b6bb4ce810c384aa801b54f0b677df347"}
Jul 21 14:42:03.471697 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:03.471426    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-56b468fd85-pctgt"
Jul 21 14:42:03.492291 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:03.490644    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-56b468fd85-pctgt" podStartSLOduration=1.7223056460000001 podStartE2EDuration="3.490622301s" totalImagesPullingTime="1.768316655s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:42:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:42:03.488002567 +0000 UTC m=+579.837745978" watchObservedRunningTime="2026-07-21 14:42:03.490622301 +0000 UTC m=+579.840365714"
Jul 21 14:42:05.482743 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:05.482707    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-79nqm" event={"ID":"66ba0e99-974c-4ff9-8b52-f2ee4d591271","Type":"ContainerStarted","Data":"f758559151b8cb928698e11ad32d4bb12d68d3516279fe2633e4b5015c15d1fe"}
Jul 21 14:42:05.483114 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:05.482830    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-79nqm"
Jul 21 14:42:05.499739 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:05.499696    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-79nqm" podStartSLOduration=1.9388943589999998 podStartE2EDuration="4.499681683s" totalImagesPullingTime="2.560787324s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:42:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:42:05.497937212 +0000 UTC m=+581.847680649" watchObservedRunningTime="2026-07-21 14:42:05.499681683 +0000 UTC m=+581.849425095"
Jul 21 14:42:09.482705 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:09.482674    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-56b468fd85-pctgt"
Jul 21 14:42:16.489314 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:16.489263    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-79nqm"
Jul 21 14:42:17.999306 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:17.999262    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74"]
Jul 21 14:42:18.002373 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:18.002357    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74"
Jul 21 14:42:18.005054 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:18.005036    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-pq8h2\""
Jul 21 14:42:18.005171 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:18.005041    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 21 14:42:18.005171 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:18.005142    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 21 14:42:18.005285 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:18.005167    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 21 14:42:18.010515 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:18.010491    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74"]
Jul 21 14:42:18.043031 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:18.043008    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b075fa64-0f4e-4011-ac25-9d876655aae9-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74\" (UID: \"b075fa64-0f4e-4011-ac25-9d876655aae9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74"
Jul 21 14:42:18.043175 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:18.043036    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b075fa64-0f4e-4011-ac25-9d876655aae9-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74\" (UID: \"b075fa64-0f4e-4011-ac25-9d876655aae9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74"
Jul 21 14:42:18.043175 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:18.043054    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b075fa64-0f4e-4011-ac25-9d876655aae9-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74\" (UID: \"b075fa64-0f4e-4011-ac25-9d876655aae9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74"
Jul 21 14:42:18.043175 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:18.043155    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b075fa64-0f4e-4011-ac25-9d876655aae9-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74\" (UID: \"b075fa64-0f4e-4011-ac25-9d876655aae9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74"
Jul 21 14:42:18.043313 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:18.043190    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4qls\" (UniqueName: \"kubernetes.io/projected/b075fa64-0f4e-4011-ac25-9d876655aae9-kube-api-access-z4qls\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74\" (UID: \"b075fa64-0f4e-4011-ac25-9d876655aae9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74"
Jul 21 14:42:18.043313 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:18.043213    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b075fa64-0f4e-4011-ac25-9d876655aae9-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74\" (UID: \"b075fa64-0f4e-4011-ac25-9d876655aae9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74"
Jul 21 14:42:18.144066 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:18.144028    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b075fa64-0f4e-4011-ac25-9d876655aae9-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74\" (UID: \"b075fa64-0f4e-4011-ac25-9d876655aae9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74"
Jul 21 14:42:18.144066 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:18.144064    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4qls\" (UniqueName: \"kubernetes.io/projected/b075fa64-0f4e-4011-ac25-9d876655aae9-kube-api-access-z4qls\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74\" (UID: \"b075fa64-0f4e-4011-ac25-9d876655aae9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74"
Jul 21 14:42:18.144326 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:18.144083    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b075fa64-0f4e-4011-ac25-9d876655aae9-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74\" (UID: \"b075fa64-0f4e-4011-ac25-9d876655aae9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74"
Jul 21 14:42:18.144326 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:18.144131    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b075fa64-0f4e-4011-ac25-9d876655aae9-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74\" (UID: \"b075fa64-0f4e-4011-ac25-9d876655aae9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74"
Jul 21 14:42:18.144326 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:18.144149    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b075fa64-0f4e-4011-ac25-9d876655aae9-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74\" (UID: \"b075fa64-0f4e-4011-ac25-9d876655aae9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74"
Jul 21 14:42:18.144326 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:18.144166    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b075fa64-0f4e-4011-ac25-9d876655aae9-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74\" (UID: \"b075fa64-0f4e-4011-ac25-9d876655aae9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74"
Jul 21 14:42:18.144556 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:18.144527    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b075fa64-0f4e-4011-ac25-9d876655aae9-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74\" (UID: \"b075fa64-0f4e-4011-ac25-9d876655aae9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74"
Jul 21 14:42:18.144681 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:18.144535    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b075fa64-0f4e-4011-ac25-9d876655aae9-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74\" (UID: \"b075fa64-0f4e-4011-ac25-9d876655aae9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74"
Jul 21 14:42:18.144681 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:18.144624    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b075fa64-0f4e-4011-ac25-9d876655aae9-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74\" (UID: \"b075fa64-0f4e-4011-ac25-9d876655aae9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74"
Jul 21 14:42:18.146355 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:18.146334    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b075fa64-0f4e-4011-ac25-9d876655aae9-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74\" (UID: \"b075fa64-0f4e-4011-ac25-9d876655aae9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74"
Jul 21 14:42:18.146629 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:18.146612    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b075fa64-0f4e-4011-ac25-9d876655aae9-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74\" (UID: \"b075fa64-0f4e-4011-ac25-9d876655aae9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74"
Jul 21 14:42:18.152035 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:18.152009    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4qls\" (UniqueName: \"kubernetes.io/projected/b075fa64-0f4e-4011-ac25-9d876655aae9-kube-api-access-z4qls\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74\" (UID: \"b075fa64-0f4e-4011-ac25-9d876655aae9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74"
Jul 21 14:42:18.313809 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:18.313739    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74"
Jul 21 14:42:18.444407 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:18.444373    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74"]
Jul 21 14:42:18.447241 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:42:18.447216    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb075fa64_0f4e_4011_ac25_9d876655aae9.slice/crio-20a793e79640c4a1d759aabd50bb1fe9590354929897adb8bea393e8e1f4bc98 WatchSource:0}: Error finding container 20a793e79640c4a1d759aabd50bb1fe9590354929897adb8bea393e8e1f4bc98: Status 404 returned error can't find the container with id 20a793e79640c4a1d759aabd50bb1fe9590354929897adb8bea393e8e1f4bc98
Jul 21 14:42:18.540743 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:18.540711    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74" event={"ID":"b075fa64-0f4e-4011-ac25-9d876655aae9","Type":"ContainerStarted","Data":"20a793e79640c4a1d759aabd50bb1fe9590354929897adb8bea393e8e1f4bc98"}
Jul 21 14:42:20.550626 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:20.550589    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74" event={"ID":"b075fa64-0f4e-4011-ac25-9d876655aae9","Type":"ContainerStarted","Data":"15da69d7f8f4673a32169c24e219b2cd371e73bea50adf47ec3d2a16e47f5b4e"}
Jul 21 14:42:20.566703 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:20.566658    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74" podStartSLOduration=2.3324841259999998 podStartE2EDuration="3.566643787s" totalImagesPullingTime="1.234159661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:42:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:42:20.565987162 +0000 UTC m=+596.915730598" watchObservedRunningTime="2026-07-21 14:42:20.566643787 +0000 UTC m=+596.916387199"
Jul 21 14:42:28.314564 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:28.314534    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74"
Jul 21 14:42:28.314972 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:28.314591    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74"
Jul 21 14:42:28.326810 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:28.326788    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74"
Jul 21 14:42:28.597483 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:28.597401    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74"
Jul 21 14:42:35.189081 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:35.189049    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-ff2mt"]
Jul 21 14:42:35.194034 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:35.194011    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ff2mt"
Jul 21 14:42:35.196652 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:35.196627    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 21 14:42:35.203815 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:35.203787    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-ff2mt"]
Jul 21 14:42:35.301509 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:35.301478    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3288f187-7e07-48a9-9376-6c521e448f47-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ff2mt\" (UID: \"3288f187-7e07-48a9-9376-6c521e448f47\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ff2mt"
Jul 21 14:42:35.301650 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:35.301530    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-244n7\" (UniqueName: \"kubernetes.io/projected/3288f187-7e07-48a9-9376-6c521e448f47-kube-api-access-244n7\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ff2mt\" (UID: \"3288f187-7e07-48a9-9376-6c521e448f47\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ff2mt"
Jul 21 14:42:35.301650 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:35.301560    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3288f187-7e07-48a9-9376-6c521e448f47-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ff2mt\" (UID: \"3288f187-7e07-48a9-9376-6c521e448f47\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ff2mt"
Jul 21 14:42:35.301650 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:35.301593    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3288f187-7e07-48a9-9376-6c521e448f47-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ff2mt\" (UID: \"3288f187-7e07-48a9-9376-6c521e448f47\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ff2mt"
Jul 21 14:42:35.301650 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:35.301637    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3288f187-7e07-48a9-9376-6c521e448f47-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ff2mt\" (UID: \"3288f187-7e07-48a9-9376-6c521e448f47\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ff2mt"
Jul 21 14:42:35.301796 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:35.301669    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3288f187-7e07-48a9-9376-6c521e448f47-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ff2mt\" (UID: \"3288f187-7e07-48a9-9376-6c521e448f47\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ff2mt"
Jul 21 14:42:35.402747 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:35.402717    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3288f187-7e07-48a9-9376-6c521e448f47-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ff2mt\" (UID: \"3288f187-7e07-48a9-9376-6c521e448f47\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ff2mt"
Jul 21 14:42:35.402912 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:35.402765    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3288f187-7e07-48a9-9376-6c521e448f47-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ff2mt\" (UID: \"3288f187-7e07-48a9-9376-6c521e448f47\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ff2mt"
Jul 21 14:42:35.402912 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:35.402802    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-244n7\" (UniqueName: \"kubernetes.io/projected/3288f187-7e07-48a9-9376-6c521e448f47-kube-api-access-244n7\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ff2mt\" (UID: \"3288f187-7e07-48a9-9376-6c521e448f47\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ff2mt"
Jul 21 14:42:35.402912 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:35.402833    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3288f187-7e07-48a9-9376-6c521e448f47-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ff2mt\" (UID: \"3288f187-7e07-48a9-9376-6c521e448f47\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ff2mt"
Jul 21 14:42:35.402912 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:35.402850    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3288f187-7e07-48a9-9376-6c521e448f47-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ff2mt\" (UID: \"3288f187-7e07-48a9-9376-6c521e448f47\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ff2mt"
Jul 21 14:42:35.402912 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:35.402871    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3288f187-7e07-48a9-9376-6c521e448f47-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ff2mt\" (UID: \"3288f187-7e07-48a9-9376-6c521e448f47\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ff2mt"
Jul 21 14:42:35.403214 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:35.403189    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3288f187-7e07-48a9-9376-6c521e448f47-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ff2mt\" (UID: \"3288f187-7e07-48a9-9376-6c521e448f47\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ff2mt"
Jul 21 14:42:35.403320 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:35.403209    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3288f187-7e07-48a9-9376-6c521e448f47-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ff2mt\" (UID: \"3288f187-7e07-48a9-9376-6c521e448f47\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ff2mt"
Jul 21 14:42:35.403320 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:35.403288    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3288f187-7e07-48a9-9376-6c521e448f47-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ff2mt\" (UID: \"3288f187-7e07-48a9-9376-6c521e448f47\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ff2mt"
Jul 21 14:42:35.405075 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:35.405054    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3288f187-7e07-48a9-9376-6c521e448f47-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ff2mt\" (UID: \"3288f187-7e07-48a9-9376-6c521e448f47\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ff2mt"
Jul 21 14:42:35.405239 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:35.405224    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3288f187-7e07-48a9-9376-6c521e448f47-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ff2mt\" (UID: \"3288f187-7e07-48a9-9376-6c521e448f47\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ff2mt"
Jul 21 14:42:35.411977 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:35.411955    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-244n7\" (UniqueName: \"kubernetes.io/projected/3288f187-7e07-48a9-9376-6c521e448f47-kube-api-access-244n7\") pod \"e2e-trlp-test-simulated-kserve-598466548d-ff2mt\" (UID: \"3288f187-7e07-48a9-9376-6c521e448f47\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ff2mt"
Jul 21 14:42:35.506076 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:35.506001    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ff2mt"
Jul 21 14:42:35.645143 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:35.645120    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-ff2mt"]
Jul 21 14:42:35.647281 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:42:35.647223    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3288f187_7e07_48a9_9376_6c521e448f47.slice/crio-ff7010b2f535acbb79a62d396b011a24adf6125c4fede64c239c36102f2ee66a WatchSource:0}: Error finding container ff7010b2f535acbb79a62d396b011a24adf6125c4fede64c239c36102f2ee66a: Status 404 returned error can't find the container with id ff7010b2f535acbb79a62d396b011a24adf6125c4fede64c239c36102f2ee66a
Jul 21 14:42:36.619194 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:36.619158    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ff2mt" event={"ID":"3288f187-7e07-48a9-9376-6c521e448f47","Type":"ContainerStarted","Data":"759ede365a895342cb2067415b1eb6acbe0d09e88e55656afdfa48035d80fb37"}
Jul 21 14:42:36.622556 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:36.619218    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ff2mt" event={"ID":"3288f187-7e07-48a9-9376-6c521e448f47","Type":"ContainerStarted","Data":"ff7010b2f535acbb79a62d396b011a24adf6125c4fede64c239c36102f2ee66a"}
Jul 21 14:42:36.636456 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:36.636407    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ff2mt" podStartSLOduration=1.41658763 podStartE2EDuration="1.63639277s" totalImagesPullingTime="219.80514ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:42:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:42:36.635818525 +0000 UTC m=+612.985561938" watchObservedRunningTime="2026-07-21 14:42:36.63639277 +0000 UTC m=+612.986136181"
Jul 21 14:42:45.506424 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:45.506388    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ff2mt"
Jul 21 14:42:45.506886 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:45.506448    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ff2mt"
Jul 21 14:42:45.518716 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:45.518693    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ff2mt"
Jul 21 14:42:45.668228 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:42:45.668192    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-ff2mt"
Jul 21 14:45:00.154665 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:45:00.154632    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744085-k6ptx"]
Jul 21 14:45:00.157893 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:45:00.157877    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744085-k6ptx"
Jul 21 14:45:00.161074 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:45:00.161054    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-494sx\""
Jul 21 14:45:00.167495 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:45:00.167473    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744085-k6ptx"
Jul 21 14:45:00.173474 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:45:00.173456    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744085-k6ptx"]
Jul 21 14:45:00.314824 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:45:00.314800    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744085-k6ptx"]
Jul 21 14:45:00.317445 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:45:00.317398    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d9c49b4_d4f6_4131_9769_2628dd20d27c.slice/crio-8d4fe39e097514215a2a3f5a2e29173d7f74fb2512ad280c8fadc0ecb2bc4f42 WatchSource:0}: Error finding container 8d4fe39e097514215a2a3f5a2e29173d7f74fb2512ad280c8fadc0ecb2bc4f42: Status 404 returned error can't find the container with id 8d4fe39e097514215a2a3f5a2e29173d7f74fb2512ad280c8fadc0ecb2bc4f42
Jul 21 14:45:00.319405 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:45:00.319389    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 14:45:01.248137 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:45:01.248099    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744085-k6ptx" event={"ID":"3d9c49b4-d4f6-4131-9769-2628dd20d27c","Type":"ContainerStarted","Data":"8d4fe39e097514215a2a3f5a2e29173d7f74fb2512ad280c8fadc0ecb2bc4f42"}
Jul 21 14:45:03.259951 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:45:03.259923    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744085-k6ptx" event={"ID":"3d9c49b4-d4f6-4131-9769-2628dd20d27c","Type":"ContainerStarted","Data":"0c152f60496a5028c2048a0f229e40fd5867e4a1459a5267255c96875ad8dc64"}
Jul 21 14:45:03.273209 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:45:03.273159    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744085-k6ptx" podStartSLOduration=0.75101076 podStartE2EDuration="3.27314473s" totalImagesPullingTime="2.52213397s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:45:03.272990137 +0000 UTC m=+759.622733549" watchObservedRunningTime="2026-07-21 14:45:03.27314473 +0000 UTC m=+759.622888142"
Jul 21 14:45:04.265111 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:45:04.265078    2579 generic.go:356] "Generic (PLEG): container finished" podID="3d9c49b4-d4f6-4131-9769-2628dd20d27c" containerID="0c152f60496a5028c2048a0f229e40fd5867e4a1459a5267255c96875ad8dc64" exitCode=0
Jul 21 14:45:04.265549 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:45:04.265110    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744085-k6ptx" event={"ID":"3d9c49b4-d4f6-4131-9769-2628dd20d27c","Type":"ContainerDied","Data":"0c152f60496a5028c2048a0f229e40fd5867e4a1459a5267255c96875ad8dc64"}
Jul 21 14:45:05.400823 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:45:05.400802    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744085-k6ptx"
Jul 21 14:45:06.275534 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:45:06.275499    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744085-k6ptx" event={"ID":"3d9c49b4-d4f6-4131-9769-2628dd20d27c","Type":"ContainerDied","Data":"8d4fe39e097514215a2a3f5a2e29173d7f74fb2512ad280c8fadc0ecb2bc4f42"}
Jul 21 14:45:06.275534 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:45:06.275531    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744085-k6ptx"
Jul 21 14:45:06.275754 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:45:06.275546    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8d4fe39e097514215a2a3f5a2e29173d7f74fb2512ad280c8fadc0ecb2bc4f42"
Jul 21 14:45:57.823089 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:45:57.823053    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7487796bf9-jpsvm"]
Jul 21 14:45:57.823675 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:45:57.823521    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7487796bf9-jpsvm" podUID="a944ad88-63b9-4e0b-ba20-159794e0f55e" containerName="manager" containerID="cri-o://1b464fd3bc013e3b883163cb9c4deb6b445359b24b8c496c9f73968116f49f0e" gracePeriod=30
Jul 21 14:45:58.070213 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:45:58.070190    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7487796bf9-jpsvm"
Jul 21 14:45:58.129159 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:45:58.129081    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a944ad88-63b9-4e0b-ba20-159794e0f55e-webhook-cert\") pod \"a944ad88-63b9-4e0b-ba20-159794e0f55e\" (UID: \"a944ad88-63b9-4e0b-ba20-159794e0f55e\") "
Jul 21 14:45:58.129409 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:45:58.129185    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6897r\" (UniqueName: \"kubernetes.io/projected/a944ad88-63b9-4e0b-ba20-159794e0f55e-kube-api-access-6897r\") pod \"a944ad88-63b9-4e0b-ba20-159794e0f55e\" (UID: \"a944ad88-63b9-4e0b-ba20-159794e0f55e\") "
Jul 21 14:45:58.131005 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:45:58.130978    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a944ad88-63b9-4e0b-ba20-159794e0f55e-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "a944ad88-63b9-4e0b-ba20-159794e0f55e" (UID: "a944ad88-63b9-4e0b-ba20-159794e0f55e"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:45:58.131223 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:45:58.131201    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a944ad88-63b9-4e0b-ba20-159794e0f55e-kube-api-access-6897r" (OuterVolumeSpecName: "kube-api-access-6897r") pod "a944ad88-63b9-4e0b-ba20-159794e0f55e" (UID: "a944ad88-63b9-4e0b-ba20-159794e0f55e"). InnerVolumeSpecName "kube-api-access-6897r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:45:58.230622 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:45:58.230590    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6897r\" (UniqueName: \"kubernetes.io/projected/a944ad88-63b9-4e0b-ba20-159794e0f55e-kube-api-access-6897r\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:45:58.230622 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:45:58.230616    2579 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a944ad88-63b9-4e0b-ba20-159794e0f55e-webhook-cert\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:45:58.498440 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:45:58.498407    2579 generic.go:356] "Generic (PLEG): container finished" podID="a944ad88-63b9-4e0b-ba20-159794e0f55e" containerID="1b464fd3bc013e3b883163cb9c4deb6b445359b24b8c496c9f73968116f49f0e" exitCode=0
Jul 21 14:45:58.498589 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:45:58.498454    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7487796bf9-jpsvm"
Jul 21 14:45:58.498589 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:45:58.498454    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7487796bf9-jpsvm" event={"ID":"a944ad88-63b9-4e0b-ba20-159794e0f55e","Type":"ContainerDied","Data":"1b464fd3bc013e3b883163cb9c4deb6b445359b24b8c496c9f73968116f49f0e"}
Jul 21 14:45:58.498589 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:45:58.498560    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7487796bf9-jpsvm" event={"ID":"a944ad88-63b9-4e0b-ba20-159794e0f55e","Type":"ContainerDied","Data":"873fa09f8b8e5360e6e1ab4cf01076df4298f5fce1f1fc665688bfab02974d86"}
Jul 21 14:45:58.498589 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:45:58.498588    2579 scope.go:117] "RemoveContainer" containerID="1b464fd3bc013e3b883163cb9c4deb6b445359b24b8c496c9f73968116f49f0e"
Jul 21 14:45:58.506685 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:45:58.506668    2579 scope.go:117] "RemoveContainer" containerID="1b464fd3bc013e3b883163cb9c4deb6b445359b24b8c496c9f73968116f49f0e"
Jul 21 14:45:58.506950 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:45:58.506932    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b464fd3bc013e3b883163cb9c4deb6b445359b24b8c496c9f73968116f49f0e\": container with ID starting with 1b464fd3bc013e3b883163cb9c4deb6b445359b24b8c496c9f73968116f49f0e not found: ID does not exist" containerID="1b464fd3bc013e3b883163cb9c4deb6b445359b24b8c496c9f73968116f49f0e"
Jul 21 14:45:58.506993 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:45:58.506958    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b464fd3bc013e3b883163cb9c4deb6b445359b24b8c496c9f73968116f49f0e"} err="failed to get container status \"1b464fd3bc013e3b883163cb9c4deb6b445359b24b8c496c9f73968116f49f0e\": rpc error: code = NotFound desc = could not find container \"1b464fd3bc013e3b883163cb9c4deb6b445359b24b8c496c9f73968116f49f0e\": container with ID starting with 1b464fd3bc013e3b883163cb9c4deb6b445359b24b8c496c9f73968116f49f0e not found: ID does not exist"
Jul 21 14:45:58.516043 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:45:58.516021    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7487796bf9-jpsvm"]
Jul 21 14:45:58.519453 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:45:58.519433    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7487796bf9-jpsvm"]
Jul 21 14:46:00.172232 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:46:00.172196    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a944ad88-63b9-4e0b-ba20-159794e0f55e" path="/var/lib/kubelet/pods/a944ad88-63b9-4e0b-ba20-159794e0f55e/volumes"
Jul 21 14:46:59.369651 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:46:59.369624    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7487796bf9-c22l8"]
Jul 21 14:46:59.370094 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:46:59.369936    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d9c49b4-d4f6-4131-9769-2628dd20d27c" containerName="cleanup"
Jul 21 14:46:59.370094 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:46:59.369946    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d9c49b4-d4f6-4131-9769-2628dd20d27c" containerName="cleanup"
Jul 21 14:46:59.370094 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:46:59.369958    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a944ad88-63b9-4e0b-ba20-159794e0f55e" containerName="manager"
Jul 21 14:46:59.370094 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:46:59.369964    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a944ad88-63b9-4e0b-ba20-159794e0f55e" containerName="manager"
Jul 21 14:46:59.370094 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:46:59.370020    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a944ad88-63b9-4e0b-ba20-159794e0f55e" containerName="manager"
Jul 21 14:46:59.370094 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:46:59.370028    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d9c49b4-d4f6-4131-9769-2628dd20d27c" containerName="cleanup"
Jul 21 14:46:59.373054 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:46:59.373037    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7487796bf9-c22l8"
Jul 21 14:46:59.375572 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:46:59.375552    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 21 14:46:59.376569 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:46:59.376546    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-bbcx6\""
Jul 21 14:46:59.376687 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:46:59.376632    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 21 14:46:59.382001 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:46:59.381979    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7487796bf9-c22l8"]
Jul 21 14:46:59.430068 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:46:59.430043    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a673785c-a75a-4084-ba0a-5bcd97b5900f-webhook-cert\") pod \"maas-controller-7487796bf9-c22l8\" (UID: \"a673785c-a75a-4084-ba0a-5bcd97b5900f\") " pod="opendatahub/maas-controller-7487796bf9-c22l8"
Jul 21 14:46:59.430171 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:46:59.430069    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szlzr\" (UniqueName: \"kubernetes.io/projected/a673785c-a75a-4084-ba0a-5bcd97b5900f-kube-api-access-szlzr\") pod \"maas-controller-7487796bf9-c22l8\" (UID: \"a673785c-a75a-4084-ba0a-5bcd97b5900f\") " pod="opendatahub/maas-controller-7487796bf9-c22l8"
Jul 21 14:46:59.530778 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:46:59.530749    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a673785c-a75a-4084-ba0a-5bcd97b5900f-webhook-cert\") pod \"maas-controller-7487796bf9-c22l8\" (UID: \"a673785c-a75a-4084-ba0a-5bcd97b5900f\") " pod="opendatahub/maas-controller-7487796bf9-c22l8"
Jul 21 14:46:59.530778 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:46:59.530784    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szlzr\" (UniqueName: \"kubernetes.io/projected/a673785c-a75a-4084-ba0a-5bcd97b5900f-kube-api-access-szlzr\") pod \"maas-controller-7487796bf9-c22l8\" (UID: \"a673785c-a75a-4084-ba0a-5bcd97b5900f\") " pod="opendatahub/maas-controller-7487796bf9-c22l8"
Jul 21 14:46:59.533056 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:46:59.533038    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a673785c-a75a-4084-ba0a-5bcd97b5900f-webhook-cert\") pod \"maas-controller-7487796bf9-c22l8\" (UID: \"a673785c-a75a-4084-ba0a-5bcd97b5900f\") " pod="opendatahub/maas-controller-7487796bf9-c22l8"
Jul 21 14:46:59.539364 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:46:59.539346    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szlzr\" (UniqueName: \"kubernetes.io/projected/a673785c-a75a-4084-ba0a-5bcd97b5900f-kube-api-access-szlzr\") pod \"maas-controller-7487796bf9-c22l8\" (UID: \"a673785c-a75a-4084-ba0a-5bcd97b5900f\") " pod="opendatahub/maas-controller-7487796bf9-c22l8"
Jul 21 14:46:59.684541 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:46:59.684513    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7487796bf9-c22l8"
Jul 21 14:46:59.816927 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:46:59.816813    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7487796bf9-c22l8"]
Jul 21 14:46:59.819064 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:46:59.819035    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda673785c_a75a_4084_ba0a_5bcd97b5900f.slice/crio-a98b08530cbf2c5d761d1e284757003bb2078d259e4cd146de8d4c1f39ba7958 WatchSource:0}: Error finding container a98b08530cbf2c5d761d1e284757003bb2078d259e4cd146de8d4c1f39ba7958: Status 404 returned error can't find the container with id a98b08530cbf2c5d761d1e284757003bb2078d259e4cd146de8d4c1f39ba7958
Jul 21 14:47:00.780106 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:00.780074    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7487796bf9-c22l8" event={"ID":"a673785c-a75a-4084-ba0a-5bcd97b5900f","Type":"ContainerStarted","Data":"f0f01901ae31070c44a7b0fd549852e06556ed8a6c61513126bb1f053d46d95f"}
Jul 21 14:47:00.780542 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:00.780124    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7487796bf9-c22l8" event={"ID":"a673785c-a75a-4084-ba0a-5bcd97b5900f","Type":"ContainerStarted","Data":"a98b08530cbf2c5d761d1e284757003bb2078d259e4cd146de8d4c1f39ba7958"}
Jul 21 14:47:00.780542 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:00.780173    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7487796bf9-c22l8"
Jul 21 14:47:00.798330 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:00.798286    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7487796bf9-c22l8" podStartSLOduration=1.263460223 podStartE2EDuration="1.798260314s" totalImagesPullingTime="534.800091ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:46:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:47:00.796306685 +0000 UTC m=+877.146050099" watchObservedRunningTime="2026-07-21 14:47:00.798260314 +0000 UTC m=+877.148003725"
Jul 21 14:47:11.787307 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:11.787256    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7487796bf9-c22l8"
Jul 21 14:47:16.441553 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:16.441480    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7487796bf9-c22l8"]
Jul 21 14:47:16.442050 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:16.441669    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7487796bf9-c22l8" podUID="a673785c-a75a-4084-ba0a-5bcd97b5900f" containerName="manager" containerID="cri-o://f0f01901ae31070c44a7b0fd549852e06556ed8a6c61513126bb1f053d46d95f" gracePeriod=30
Jul 21 14:47:16.695224 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:16.695169    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7487796bf9-c22l8"
Jul 21 14:47:16.853743 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:16.853714    2579 generic.go:356] "Generic (PLEG): container finished" podID="a673785c-a75a-4084-ba0a-5bcd97b5900f" containerID="f0f01901ae31070c44a7b0fd549852e06556ed8a6c61513126bb1f053d46d95f" exitCode=0
Jul 21 14:47:16.853906 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:16.853749    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7487796bf9-c22l8" event={"ID":"a673785c-a75a-4084-ba0a-5bcd97b5900f","Type":"ContainerDied","Data":"f0f01901ae31070c44a7b0fd549852e06556ed8a6c61513126bb1f053d46d95f"}
Jul 21 14:47:16.853906 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:16.853758    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7487796bf9-c22l8"
Jul 21 14:47:16.853906 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:16.853785    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7487796bf9-c22l8" event={"ID":"a673785c-a75a-4084-ba0a-5bcd97b5900f","Type":"ContainerDied","Data":"a98b08530cbf2c5d761d1e284757003bb2078d259e4cd146de8d4c1f39ba7958"}
Jul 21 14:47:16.853906 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:16.853806    2579 scope.go:117] "RemoveContainer" containerID="f0f01901ae31070c44a7b0fd549852e06556ed8a6c61513126bb1f053d46d95f"
Jul 21 14:47:16.861804 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:16.861785    2579 scope.go:117] "RemoveContainer" containerID="f0f01901ae31070c44a7b0fd549852e06556ed8a6c61513126bb1f053d46d95f"
Jul 21 14:47:16.862035 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:47:16.862017    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0f01901ae31070c44a7b0fd549852e06556ed8a6c61513126bb1f053d46d95f\": container with ID starting with f0f01901ae31070c44a7b0fd549852e06556ed8a6c61513126bb1f053d46d95f not found: ID does not exist" containerID="f0f01901ae31070c44a7b0fd549852e06556ed8a6c61513126bb1f053d46d95f"
Jul 21 14:47:16.862087 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:16.862046    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0f01901ae31070c44a7b0fd549852e06556ed8a6c61513126bb1f053d46d95f"} err="failed to get container status \"f0f01901ae31070c44a7b0fd549852e06556ed8a6c61513126bb1f053d46d95f\": rpc error: code = NotFound desc = could not find container \"f0f01901ae31070c44a7b0fd549852e06556ed8a6c61513126bb1f053d46d95f\": container with ID starting with f0f01901ae31070c44a7b0fd549852e06556ed8a6c61513126bb1f053d46d95f not found: ID does not exist"
Jul 21 14:47:16.875356 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:16.875334    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a673785c-a75a-4084-ba0a-5bcd97b5900f-webhook-cert\") pod \"a673785c-a75a-4084-ba0a-5bcd97b5900f\" (UID: \"a673785c-a75a-4084-ba0a-5bcd97b5900f\") "
Jul 21 14:47:16.875449 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:16.875374    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-szlzr\" (UniqueName: \"kubernetes.io/projected/a673785c-a75a-4084-ba0a-5bcd97b5900f-kube-api-access-szlzr\") pod \"a673785c-a75a-4084-ba0a-5bcd97b5900f\" (UID: \"a673785c-a75a-4084-ba0a-5bcd97b5900f\") "
Jul 21 14:47:16.877194 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:16.877174    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a673785c-a75a-4084-ba0a-5bcd97b5900f-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "a673785c-a75a-4084-ba0a-5bcd97b5900f" (UID: "a673785c-a75a-4084-ba0a-5bcd97b5900f"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:47:16.877278 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:16.877207    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a673785c-a75a-4084-ba0a-5bcd97b5900f-kube-api-access-szlzr" (OuterVolumeSpecName: "kube-api-access-szlzr") pod "a673785c-a75a-4084-ba0a-5bcd97b5900f" (UID: "a673785c-a75a-4084-ba0a-5bcd97b5900f"). InnerVolumeSpecName "kube-api-access-szlzr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:47:16.976906 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:16.976859    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szlzr\" (UniqueName: \"kubernetes.io/projected/a673785c-a75a-4084-ba0a-5bcd97b5900f-kube-api-access-szlzr\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:47:16.976906 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:16.976881    2579 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a673785c-a75a-4084-ba0a-5bcd97b5900f-webhook-cert\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:47:17.180008 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:17.179976    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7487796bf9-c22l8"]
Jul 21 14:47:17.183719 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:17.183697    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7487796bf9-c22l8"]
Jul 21 14:47:17.950498 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:17.950469    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7487796bf9-sgk7l"]
Jul 21 14:47:17.950836 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:17.950772    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a673785c-a75a-4084-ba0a-5bcd97b5900f" containerName="manager"
Jul 21 14:47:17.950836 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:17.950781    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a673785c-a75a-4084-ba0a-5bcd97b5900f" containerName="manager"
Jul 21 14:47:17.950905 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:17.950849    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a673785c-a75a-4084-ba0a-5bcd97b5900f" containerName="manager"
Jul 21 14:47:17.955162 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:17.955142    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7487796bf9-sgk7l"
Jul 21 14:47:17.957620 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:17.957600    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 21 14:47:17.958744 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:17.958726    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-bbcx6\""
Jul 21 14:47:17.958824 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:17.958763    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 21 14:47:17.965864 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:17.965842    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7487796bf9-sgk7l"]
Jul 21 14:47:18.085895 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:18.085857    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mww7k\" (UniqueName: \"kubernetes.io/projected/8a1187a0-2f58-4791-ad69-8dd726b9d35e-kube-api-access-mww7k\") pod \"maas-controller-7487796bf9-sgk7l\" (UID: \"8a1187a0-2f58-4791-ad69-8dd726b9d35e\") " pod="opendatahub/maas-controller-7487796bf9-sgk7l"
Jul 21 14:47:18.086062 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:18.085917    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8a1187a0-2f58-4791-ad69-8dd726b9d35e-webhook-cert\") pod \"maas-controller-7487796bf9-sgk7l\" (UID: \"8a1187a0-2f58-4791-ad69-8dd726b9d35e\") " pod="opendatahub/maas-controller-7487796bf9-sgk7l"
Jul 21 14:47:18.172808 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:18.172779    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a673785c-a75a-4084-ba0a-5bcd97b5900f" path="/var/lib/kubelet/pods/a673785c-a75a-4084-ba0a-5bcd97b5900f/volumes"
Jul 21 14:47:18.186660 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:18.186632    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mww7k\" (UniqueName: \"kubernetes.io/projected/8a1187a0-2f58-4791-ad69-8dd726b9d35e-kube-api-access-mww7k\") pod \"maas-controller-7487796bf9-sgk7l\" (UID: \"8a1187a0-2f58-4791-ad69-8dd726b9d35e\") " pod="opendatahub/maas-controller-7487796bf9-sgk7l"
Jul 21 14:47:18.186800 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:18.186677    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8a1187a0-2f58-4791-ad69-8dd726b9d35e-webhook-cert\") pod \"maas-controller-7487796bf9-sgk7l\" (UID: \"8a1187a0-2f58-4791-ad69-8dd726b9d35e\") " pod="opendatahub/maas-controller-7487796bf9-sgk7l"
Jul 21 14:47:18.188991 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:18.188964    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8a1187a0-2f58-4791-ad69-8dd726b9d35e-webhook-cert\") pod \"maas-controller-7487796bf9-sgk7l\" (UID: \"8a1187a0-2f58-4791-ad69-8dd726b9d35e\") " pod="opendatahub/maas-controller-7487796bf9-sgk7l"
Jul 21 14:47:18.195543 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:18.195519    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mww7k\" (UniqueName: \"kubernetes.io/projected/8a1187a0-2f58-4791-ad69-8dd726b9d35e-kube-api-access-mww7k\") pod \"maas-controller-7487796bf9-sgk7l\" (UID: \"8a1187a0-2f58-4791-ad69-8dd726b9d35e\") " pod="opendatahub/maas-controller-7487796bf9-sgk7l"
Jul 21 14:47:18.264814 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:18.264748    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7487796bf9-sgk7l"
Jul 21 14:47:18.397908 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:18.397882    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7487796bf9-sgk7l"]
Jul 21 14:47:18.400407 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:47:18.400377    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a1187a0_2f58_4791_ad69_8dd726b9d35e.slice/crio-bb10db3344fe3b42714fe0c1e4a0df69e8f12d0dcd795c037c76031cd9a251d8 WatchSource:0}: Error finding container bb10db3344fe3b42714fe0c1e4a0df69e8f12d0dcd795c037c76031cd9a251d8: Status 404 returned error can't find the container with id bb10db3344fe3b42714fe0c1e4a0df69e8f12d0dcd795c037c76031cd9a251d8
Jul 21 14:47:18.865080 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:18.865045    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7487796bf9-sgk7l" event={"ID":"8a1187a0-2f58-4791-ad69-8dd726b9d35e","Type":"ContainerStarted","Data":"bb10db3344fe3b42714fe0c1e4a0df69e8f12d0dcd795c037c76031cd9a251d8"}
Jul 21 14:47:19.871811 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:19.871770    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7487796bf9-sgk7l" event={"ID":"8a1187a0-2f58-4791-ad69-8dd726b9d35e","Type":"ContainerStarted","Data":"313d9dc28428de6a68125d7dee89f2d8b9b681081a8ff6c811eb353d22077c3e"}
Jul 21 14:47:19.872242 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:19.871906    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7487796bf9-sgk7l"
Jul 21 14:47:19.890068 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:19.890023    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7487796bf9-sgk7l" podStartSLOduration=2.344274248 podStartE2EDuration="2.890009931s" totalImagesPullingTime="545.735683ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:47:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:47:19.889022626 +0000 UTC m=+896.238766037" watchObservedRunningTime="2026-07-21 14:47:19.890009931 +0000 UTC m=+896.239753342"
Jul 21 14:47:30.878771 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:47:30.878739    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7487796bf9-sgk7l"
Jul 21 14:55:53.583633 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:55:53.583605    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z"]
Jul 21 14:55:53.586672 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:55:53.586651    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z"
Jul 21 14:55:53.589416 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:55:53.589399    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-3d46f128-kserve-self-signed-certs\""
Jul 21 14:55:53.598873 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:55:53.598841    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z"]
Jul 21 14:55:53.642869 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:55:53.642839    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b4f8bd0-8343-44e9-b74c-96c9a24a38f4-dshm\") pod \"e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z\" (UID: \"8b4f8bd0-8343-44e9-b74c-96c9a24a38f4\") " pod="llm/e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z"
Jul 21 14:55:53.643002 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:55:53.642873    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b4f8bd0-8343-44e9-b74c-96c9a24a38f4-tls-certs\") pod \"e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z\" (UID: \"8b4f8bd0-8343-44e9-b74c-96c9a24a38f4\") " pod="llm/e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z"
Jul 21 14:55:53.643002 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:55:53.642899    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fh7b6\" (UniqueName: \"kubernetes.io/projected/8b4f8bd0-8343-44e9-b74c-96c9a24a38f4-kube-api-access-fh7b6\") pod \"e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z\" (UID: \"8b4f8bd0-8343-44e9-b74c-96c9a24a38f4\") " pod="llm/e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z"
Jul 21 14:55:53.643002 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:55:53.642986    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b4f8bd0-8343-44e9-b74c-96c9a24a38f4-model-cache\") pod \"e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z\" (UID: \"8b4f8bd0-8343-44e9-b74c-96c9a24a38f4\") " pod="llm/e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z"
Jul 21 14:55:53.643120 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:55:53.643015    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b4f8bd0-8343-44e9-b74c-96c9a24a38f4-tmp-dir\") pod \"e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z\" (UID: \"8b4f8bd0-8343-44e9-b74c-96c9a24a38f4\") " pod="llm/e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z"
Jul 21 14:55:53.643120 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:55:53.643077    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b4f8bd0-8343-44e9-b74c-96c9a24a38f4-home\") pod \"e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z\" (UID: \"8b4f8bd0-8343-44e9-b74c-96c9a24a38f4\") " pod="llm/e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z"
Jul 21 14:55:53.743647 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:55:53.743617    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b4f8bd0-8343-44e9-b74c-96c9a24a38f4-model-cache\") pod \"e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z\" (UID: \"8b4f8bd0-8343-44e9-b74c-96c9a24a38f4\") " pod="llm/e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z"
Jul 21 14:55:53.743805 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:55:53.743656    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b4f8bd0-8343-44e9-b74c-96c9a24a38f4-tmp-dir\") pod \"e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z\" (UID: \"8b4f8bd0-8343-44e9-b74c-96c9a24a38f4\") " pod="llm/e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z"
Jul 21 14:55:53.743805 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:55:53.743718    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b4f8bd0-8343-44e9-b74c-96c9a24a38f4-home\") pod \"e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z\" (UID: \"8b4f8bd0-8343-44e9-b74c-96c9a24a38f4\") " pod="llm/e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z"
Jul 21 14:55:53.743882 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:55:53.743834    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b4f8bd0-8343-44e9-b74c-96c9a24a38f4-dshm\") pod \"e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z\" (UID: \"8b4f8bd0-8343-44e9-b74c-96c9a24a38f4\") " pod="llm/e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z"
Jul 21 14:55:53.743882 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:55:53.743865    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b4f8bd0-8343-44e9-b74c-96c9a24a38f4-tls-certs\") pod \"e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z\" (UID: \"8b4f8bd0-8343-44e9-b74c-96c9a24a38f4\") " pod="llm/e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z"
Jul 21 14:55:53.743954 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:55:53.743885    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fh7b6\" (UniqueName: \"kubernetes.io/projected/8b4f8bd0-8343-44e9-b74c-96c9a24a38f4-kube-api-access-fh7b6\") pod \"e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z\" (UID: \"8b4f8bd0-8343-44e9-b74c-96c9a24a38f4\") " pod="llm/e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z"
Jul 21 14:55:53.744042 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:55:53.744021    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b4f8bd0-8343-44e9-b74c-96c9a24a38f4-model-cache\") pod \"e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z\" (UID: \"8b4f8bd0-8343-44e9-b74c-96c9a24a38f4\") " pod="llm/e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z"
Jul 21 14:55:53.744159 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:55:53.744044    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b4f8bd0-8343-44e9-b74c-96c9a24a38f4-home\") pod \"e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z\" (UID: \"8b4f8bd0-8343-44e9-b74c-96c9a24a38f4\") " pod="llm/e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z"
Jul 21 14:55:53.744159 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:55:53.744102    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b4f8bd0-8343-44e9-b74c-96c9a24a38f4-tmp-dir\") pod \"e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z\" (UID: \"8b4f8bd0-8343-44e9-b74c-96c9a24a38f4\") " pod="llm/e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z"
Jul 21 14:55:53.746175 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:55:53.746150    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b4f8bd0-8343-44e9-b74c-96c9a24a38f4-dshm\") pod \"e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z\" (UID: \"8b4f8bd0-8343-44e9-b74c-96c9a24a38f4\") " pod="llm/e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z"
Jul 21 14:55:53.746349 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:55:53.746332    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b4f8bd0-8343-44e9-b74c-96c9a24a38f4-tls-certs\") pod \"e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z\" (UID: \"8b4f8bd0-8343-44e9-b74c-96c9a24a38f4\") " pod="llm/e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z"
Jul 21 14:55:53.751700 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:55:53.751681    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fh7b6\" (UniqueName: \"kubernetes.io/projected/8b4f8bd0-8343-44e9-b74c-96c9a24a38f4-kube-api-access-fh7b6\") pod \"e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z\" (UID: \"8b4f8bd0-8343-44e9-b74c-96c9a24a38f4\") " pod="llm/e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z"
Jul 21 14:55:53.895943 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:55:53.895917    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z"
Jul 21 14:55:54.025750 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:55:54.025722    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z"]
Jul 21 14:55:54.028077 ip-10-0-130-83 kubenswrapper[2579]: W0721 14:55:54.028048    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8b4f8bd0_8343_44e9_b74c_96c9a24a38f4.slice/crio-960e7e970460b60a5a6f9b6207baff9df3b6157509ad5817b676f8ad31ca4243 WatchSource:0}: Error finding container 960e7e970460b60a5a6f9b6207baff9df3b6157509ad5817b676f8ad31ca4243: Status 404 returned error can't find the container with id 960e7e970460b60a5a6f9b6207baff9df3b6157509ad5817b676f8ad31ca4243
Jul 21 14:55:54.128581 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:55:54.128545    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z" event={"ID":"8b4f8bd0-8343-44e9-b74c-96c9a24a38f4","Type":"ContainerStarted","Data":"345e4f90b8a819f446052ec60dee2a60eefc7eba525e8ee9280dcac376ed5a77"}
Jul 21 14:55:54.128725 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:55:54.128601    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z" event={"ID":"8b4f8bd0-8343-44e9-b74c-96c9a24a38f4","Type":"ContainerStarted","Data":"960e7e970460b60a5a6f9b6207baff9df3b6157509ad5817b676f8ad31ca4243"}
Jul 21 14:55:54.146775 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:55:54.146694    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z" podStartSLOduration=1.146678779 podStartE2EDuration="1.146678779s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 14:55:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 14:55:54.145005719 +0000 UTC m=+1410.494749131" watchObservedRunningTime="2026-07-21 14:55:54.146678779 +0000 UTC m=+1410.496422189"
Jul 21 14:56:03.896180 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:56:03.896148    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z"
Jul 21 14:56:03.896596 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:56:03.896233    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z"
Jul 21 14:56:03.908804 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:56:03.908784    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z"
Jul 21 14:56:04.185286 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:56:04.185194    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z"
Jul 21 14:56:40.593916 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:56:40.593821    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z"]
Jul 21 14:56:55.614817 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:56:55.614769    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z" podUID="8b4f8bd0-8343-44e9-b74c-96c9a24a38f4" containerName="main" containerID="cri-o://345e4f90b8a819f446052ec60dee2a60eefc7eba525e8ee9280dcac376ed5a77" gracePeriod=45
Jul 21 14:56:55.878834 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:56:55.878783    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z"
Jul 21 14:56:55.981747 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:56:55.981716    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b4f8bd0-8343-44e9-b74c-96c9a24a38f4-tmp-dir\") pod \"8b4f8bd0-8343-44e9-b74c-96c9a24a38f4\" (UID: \"8b4f8bd0-8343-44e9-b74c-96c9a24a38f4\") "
Jul 21 14:56:55.981887 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:56:55.981753    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b4f8bd0-8343-44e9-b74c-96c9a24a38f4-dshm\") pod \"8b4f8bd0-8343-44e9-b74c-96c9a24a38f4\" (UID: \"8b4f8bd0-8343-44e9-b74c-96c9a24a38f4\") "
Jul 21 14:56:55.981887 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:56:55.981778    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fh7b6\" (UniqueName: \"kubernetes.io/projected/8b4f8bd0-8343-44e9-b74c-96c9a24a38f4-kube-api-access-fh7b6\") pod \"8b4f8bd0-8343-44e9-b74c-96c9a24a38f4\" (UID: \"8b4f8bd0-8343-44e9-b74c-96c9a24a38f4\") "
Jul 21 14:56:55.981887 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:56:55.981819    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b4f8bd0-8343-44e9-b74c-96c9a24a38f4-home\") pod \"8b4f8bd0-8343-44e9-b74c-96c9a24a38f4\" (UID: \"8b4f8bd0-8343-44e9-b74c-96c9a24a38f4\") "
Jul 21 14:56:55.981887 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:56:55.981869    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b4f8bd0-8343-44e9-b74c-96c9a24a38f4-model-cache\") pod \"8b4f8bd0-8343-44e9-b74c-96c9a24a38f4\" (UID: \"8b4f8bd0-8343-44e9-b74c-96c9a24a38f4\") "
Jul 21 14:56:55.981887 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:56:55.981886    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b4f8bd0-8343-44e9-b74c-96c9a24a38f4-tls-certs\") pod \"8b4f8bd0-8343-44e9-b74c-96c9a24a38f4\" (UID: \"8b4f8bd0-8343-44e9-b74c-96c9a24a38f4\") "
Jul 21 14:56:55.982153 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:56:55.981989    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b4f8bd0-8343-44e9-b74c-96c9a24a38f4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8b4f8bd0-8343-44e9-b74c-96c9a24a38f4" (UID: "8b4f8bd0-8343-44e9-b74c-96c9a24a38f4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 14:56:55.982153 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:56:55.982123    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b4f8bd0-8343-44e9-b74c-96c9a24a38f4-home" (OuterVolumeSpecName: "home") pod "8b4f8bd0-8343-44e9-b74c-96c9a24a38f4" (UID: "8b4f8bd0-8343-44e9-b74c-96c9a24a38f4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 14:56:55.982244 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:56:55.982139    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b4f8bd0-8343-44e9-b74c-96c9a24a38f4-model-cache" (OuterVolumeSpecName: "model-cache") pod "8b4f8bd0-8343-44e9-b74c-96c9a24a38f4" (UID: "8b4f8bd0-8343-44e9-b74c-96c9a24a38f4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 14:56:55.982244 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:56:55.982205    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8b4f8bd0-8343-44e9-b74c-96c9a24a38f4-tmp-dir\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:56:55.982244 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:56:55.982226    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8b4f8bd0-8343-44e9-b74c-96c9a24a38f4-home\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:56:55.983953 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:56:55.983932    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b4f8bd0-8343-44e9-b74c-96c9a24a38f4-kube-api-access-fh7b6" (OuterVolumeSpecName: "kube-api-access-fh7b6") pod "8b4f8bd0-8343-44e9-b74c-96c9a24a38f4" (UID: "8b4f8bd0-8343-44e9-b74c-96c9a24a38f4"). InnerVolumeSpecName "kube-api-access-fh7b6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 14:56:55.984007 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:56:55.983970    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b4f8bd0-8343-44e9-b74c-96c9a24a38f4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8b4f8bd0-8343-44e9-b74c-96c9a24a38f4" (UID: "8b4f8bd0-8343-44e9-b74c-96c9a24a38f4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 14:56:55.984007 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:56:55.983997    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b4f8bd0-8343-44e9-b74c-96c9a24a38f4-dshm" (OuterVolumeSpecName: "dshm") pod "8b4f8bd0-8343-44e9-b74c-96c9a24a38f4" (UID: "8b4f8bd0-8343-44e9-b74c-96c9a24a38f4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 14:56:56.083406 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:56:56.083368    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8b4f8bd0-8343-44e9-b74c-96c9a24a38f4-dshm\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:56:56.083406 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:56:56.083402    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fh7b6\" (UniqueName: \"kubernetes.io/projected/8b4f8bd0-8343-44e9-b74c-96c9a24a38f4-kube-api-access-fh7b6\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:56:56.083566 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:56:56.083419    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8b4f8bd0-8343-44e9-b74c-96c9a24a38f4-model-cache\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:56:56.083566 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:56:56.083434    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b4f8bd0-8343-44e9-b74c-96c9a24a38f4-tls-certs\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 14:56:56.405426 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:56:56.405381    2579 generic.go:356] "Generic (PLEG): container finished" podID="8b4f8bd0-8343-44e9-b74c-96c9a24a38f4" containerID="345e4f90b8a819f446052ec60dee2a60eefc7eba525e8ee9280dcac376ed5a77" exitCode=0
Jul 21 14:56:56.405426 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:56:56.405424    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z"
Jul 21 14:56:56.405426 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:56:56.405430    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z" event={"ID":"8b4f8bd0-8343-44e9-b74c-96c9a24a38f4","Type":"ContainerDied","Data":"345e4f90b8a819f446052ec60dee2a60eefc7eba525e8ee9280dcac376ed5a77"}
Jul 21 14:56:56.405688 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:56:56.405469    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z" event={"ID":"8b4f8bd0-8343-44e9-b74c-96c9a24a38f4","Type":"ContainerDied","Data":"960e7e970460b60a5a6f9b6207baff9df3b6157509ad5817b676f8ad31ca4243"}
Jul 21 14:56:56.405688 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:56:56.405490    2579 scope.go:117] "RemoveContainer" containerID="345e4f90b8a819f446052ec60dee2a60eefc7eba525e8ee9280dcac376ed5a77"
Jul 21 14:56:56.413575 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:56:56.413557    2579 scope.go:117] "RemoveContainer" containerID="345e4f90b8a819f446052ec60dee2a60eefc7eba525e8ee9280dcac376ed5a77"
Jul 21 14:56:56.413863 ip-10-0-130-83 kubenswrapper[2579]: E0721 14:56:56.413839    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"345e4f90b8a819f446052ec60dee2a60eefc7eba525e8ee9280dcac376ed5a77\": container with ID starting with 345e4f90b8a819f446052ec60dee2a60eefc7eba525e8ee9280dcac376ed5a77 not found: ID does not exist" containerID="345e4f90b8a819f446052ec60dee2a60eefc7eba525e8ee9280dcac376ed5a77"
Jul 21 14:56:56.413924 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:56:56.413872    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"345e4f90b8a819f446052ec60dee2a60eefc7eba525e8ee9280dcac376ed5a77"} err="failed to get container status \"345e4f90b8a819f446052ec60dee2a60eefc7eba525e8ee9280dcac376ed5a77\": rpc error: code = NotFound desc = could not find container \"345e4f90b8a819f446052ec60dee2a60eefc7eba525e8ee9280dcac376ed5a77\": container with ID starting with 345e4f90b8a819f446052ec60dee2a60eefc7eba525e8ee9280dcac376ed5a77 not found: ID does not exist"
Jul 21 14:56:56.421301 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:56:56.421263    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z"]
Jul 21 14:56:56.424427 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:56:56.424407    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-3d46f128-kserve-74d494fb75-qbm4z"]
Jul 21 14:56:58.172181 ip-10-0-130-83 kubenswrapper[2579]: I0721 14:56:58.172152    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b4f8bd0-8343-44e9-b74c-96c9a24a38f4" path="/var/lib/kubelet/pods/8b4f8bd0-8343-44e9-b74c-96c9a24a38f4/volumes"
Jul 21 15:00:00.134629 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:00:00.134596    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744100-d2wpr"]
Jul 21 15:00:00.137009 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:00:00.134967    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b4f8bd0-8343-44e9-b74c-96c9a24a38f4" containerName="main"
Jul 21 15:00:00.137009 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:00:00.134978    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b4f8bd0-8343-44e9-b74c-96c9a24a38f4" containerName="main"
Jul 21 15:00:00.137009 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:00:00.135033    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b4f8bd0-8343-44e9-b74c-96c9a24a38f4" containerName="main"
Jul 21 15:00:00.137871 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:00:00.137854    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744100-d2wpr"
Jul 21 15:00:00.140505 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:00:00.140485    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-494sx\""
Jul 21 15:00:00.147261 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:00:00.147244    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744100-d2wpr"
Jul 21 15:00:00.152098 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:00:00.152071    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744100-d2wpr"]
Jul 21 15:00:00.282923 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:00:00.282899    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744100-d2wpr"]
Jul 21 15:00:00.285819 ip-10-0-130-83 kubenswrapper[2579]: W0721 15:00:00.285789    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6f98b27_7bec_4afc_9487_2a1ed68ae2eb.slice/crio-a939fccff835392594476c8b3133382d94c9c775e99d19318d83e90fbc2d1e3f WatchSource:0}: Error finding container a939fccff835392594476c8b3133382d94c9c775e99d19318d83e90fbc2d1e3f: Status 404 returned error can't find the container with id a939fccff835392594476c8b3133382d94c9c775e99d19318d83e90fbc2d1e3f
Jul 21 15:00:01.225899 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:00:01.225867    2579 generic.go:356] "Generic (PLEG): container finished" podID="d6f98b27-7bec-4afc-9487-2a1ed68ae2eb" containerID="7bc2a902dcd28f20ea0d63f40080eda9a6068d18214355c7f6446072e2d1cc6d" exitCode=0
Jul 21 15:00:01.226256 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:00:01.225950    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744100-d2wpr" event={"ID":"d6f98b27-7bec-4afc-9487-2a1ed68ae2eb","Type":"ContainerDied","Data":"7bc2a902dcd28f20ea0d63f40080eda9a6068d18214355c7f6446072e2d1cc6d"}
Jul 21 15:00:01.226256 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:00:01.226001    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744100-d2wpr" event={"ID":"d6f98b27-7bec-4afc-9487-2a1ed68ae2eb","Type":"ContainerStarted","Data":"a939fccff835392594476c8b3133382d94c9c775e99d19318d83e90fbc2d1e3f"}
Jul 21 15:00:02.358144 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:00:02.358122    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744100-d2wpr"
Jul 21 15:00:03.235736 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:00:03.235709    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744100-d2wpr"
Jul 21 15:00:03.235901 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:00:03.235702    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744100-d2wpr" event={"ID":"d6f98b27-7bec-4afc-9487-2a1ed68ae2eb","Type":"ContainerDied","Data":"a939fccff835392594476c8b3133382d94c9c775e99d19318d83e90fbc2d1e3f"}
Jul 21 15:00:03.235901 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:00:03.235829    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a939fccff835392594476c8b3133382d94c9c775e99d19318d83e90fbc2d1e3f"
Jul 21 15:00:44.754431 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:00:44.754398    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c42bc55b-677cc8f96b-ccvk8"]
Jul 21 15:00:44.755033 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:00:44.754729    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6f98b27-7bec-4afc-9487-2a1ed68ae2eb" containerName="cleanup"
Jul 21 15:00:44.755033 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:00:44.754739    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6f98b27-7bec-4afc-9487-2a1ed68ae2eb" containerName="cleanup"
Jul 21 15:00:44.755033 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:00:44.754807    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6f98b27-7bec-4afc-9487-2a1ed68ae2eb" containerName="cleanup"
Jul 21 15:00:44.757938 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:00:44.757917    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c42bc55b-677cc8f96b-ccvk8"
Jul 21 15:00:44.761957 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:00:44.761937    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-c42bc55b-serving-cert\""
Jul 21 15:00:44.791872 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:00:44.791850    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c42bc55b-677cc8f96b-ccvk8"]
Jul 21 15:00:44.943761 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:00:44.943731    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0047408a-8b23-48d6-8024-de2f2ea937cc-openshift-service-ca\") pod \"maas-api-e2e-ait-c42bc55b-677cc8f96b-ccvk8\" (UID: \"0047408a-8b23-48d6-8024-de2f2ea937cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c42bc55b-677cc8f96b-ccvk8"
Jul 21 15:00:44.943761 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:00:44.943765    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zghbb\" (UniqueName: \"kubernetes.io/projected/0047408a-8b23-48d6-8024-de2f2ea937cc-kube-api-access-zghbb\") pod \"maas-api-e2e-ait-c42bc55b-677cc8f96b-ccvk8\" (UID: \"0047408a-8b23-48d6-8024-de2f2ea937cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c42bc55b-677cc8f96b-ccvk8"
Jul 21 15:00:44.943943 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:00:44.943908    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0047408a-8b23-48d6-8024-de2f2ea937cc-maas-api-tls\") pod \"maas-api-e2e-ait-c42bc55b-677cc8f96b-ccvk8\" (UID: \"0047408a-8b23-48d6-8024-de2f2ea937cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c42bc55b-677cc8f96b-ccvk8"
Jul 21 15:00:45.044596 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:00:45.044512    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0047408a-8b23-48d6-8024-de2f2ea937cc-maas-api-tls\") pod \"maas-api-e2e-ait-c42bc55b-677cc8f96b-ccvk8\" (UID: \"0047408a-8b23-48d6-8024-de2f2ea937cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c42bc55b-677cc8f96b-ccvk8"
Jul 21 15:00:45.044748 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:00:45.044609    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0047408a-8b23-48d6-8024-de2f2ea937cc-openshift-service-ca\") pod \"maas-api-e2e-ait-c42bc55b-677cc8f96b-ccvk8\" (UID: \"0047408a-8b23-48d6-8024-de2f2ea937cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c42bc55b-677cc8f96b-ccvk8"
Jul 21 15:00:45.044748 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:00:45.044636    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zghbb\" (UniqueName: \"kubernetes.io/projected/0047408a-8b23-48d6-8024-de2f2ea937cc-kube-api-access-zghbb\") pod \"maas-api-e2e-ait-c42bc55b-677cc8f96b-ccvk8\" (UID: \"0047408a-8b23-48d6-8024-de2f2ea937cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c42bc55b-677cc8f96b-ccvk8"
Jul 21 15:00:45.044748 ip-10-0-130-83 kubenswrapper[2579]: E0721 15:00:45.044678    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-c42bc55b-serving-cert: secret "maas-api-e2e-ait-c42bc55b-serving-cert" not found
Jul 21 15:00:45.044748 ip-10-0-130-83 kubenswrapper[2579]: E0721 15:00:45.044737    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0047408a-8b23-48d6-8024-de2f2ea937cc-maas-api-tls podName:0047408a-8b23-48d6-8024-de2f2ea937cc nodeName:}" failed. No retries permitted until 2026-07-21 15:00:45.544720241 +0000 UTC m=+1701.894463635 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0047408a-8b23-48d6-8024-de2f2ea937cc-maas-api-tls") pod "maas-api-e2e-ait-c42bc55b-677cc8f96b-ccvk8" (UID: "0047408a-8b23-48d6-8024-de2f2ea937cc") : secret "maas-api-e2e-ait-c42bc55b-serving-cert" not found
Jul 21 15:00:45.045415 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:00:45.045391    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0047408a-8b23-48d6-8024-de2f2ea937cc-openshift-service-ca\") pod \"maas-api-e2e-ait-c42bc55b-677cc8f96b-ccvk8\" (UID: \"0047408a-8b23-48d6-8024-de2f2ea937cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c42bc55b-677cc8f96b-ccvk8"
Jul 21 15:00:45.058602 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:00:45.058577    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zghbb\" (UniqueName: \"kubernetes.io/projected/0047408a-8b23-48d6-8024-de2f2ea937cc-kube-api-access-zghbb\") pod \"maas-api-e2e-ait-c42bc55b-677cc8f96b-ccvk8\" (UID: \"0047408a-8b23-48d6-8024-de2f2ea937cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c42bc55b-677cc8f96b-ccvk8"
Jul 21 15:00:45.548814 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:00:45.548780    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0047408a-8b23-48d6-8024-de2f2ea937cc-maas-api-tls\") pod \"maas-api-e2e-ait-c42bc55b-677cc8f96b-ccvk8\" (UID: \"0047408a-8b23-48d6-8024-de2f2ea937cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c42bc55b-677cc8f96b-ccvk8"
Jul 21 15:00:45.551143 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:00:45.551117    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0047408a-8b23-48d6-8024-de2f2ea937cc-maas-api-tls\") pod \"maas-api-e2e-ait-c42bc55b-677cc8f96b-ccvk8\" (UID: \"0047408a-8b23-48d6-8024-de2f2ea937cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c42bc55b-677cc8f96b-ccvk8"
Jul 21 15:00:45.668993 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:00:45.668957    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c42bc55b-677cc8f96b-ccvk8"
Jul 21 15:00:45.802583 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:00:45.802559    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c42bc55b-677cc8f96b-ccvk8"]
Jul 21 15:00:45.804537 ip-10-0-130-83 kubenswrapper[2579]: W0721 15:00:45.804510    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0047408a_8b23_48d6_8024_de2f2ea937cc.slice/crio-b1d62a0572a2cfa2385b6063658ca96272b5fa259a35b2e1d856febba7503215 WatchSource:0}: Error finding container b1d62a0572a2cfa2385b6063658ca96272b5fa259a35b2e1d856febba7503215: Status 404 returned error can't find the container with id b1d62a0572a2cfa2385b6063658ca96272b5fa259a35b2e1d856febba7503215
Jul 21 15:00:45.805903 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:00:45.805883    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 15:00:46.426896 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:00:46.426850    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c42bc55b-677cc8f96b-ccvk8" event={"ID":"0047408a-8b23-48d6-8024-de2f2ea937cc","Type":"ContainerStarted","Data":"b1d62a0572a2cfa2385b6063658ca96272b5fa259a35b2e1d856febba7503215"}
Jul 21 15:00:48.437841 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:00:48.437811    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c42bc55b-677cc8f96b-ccvk8" event={"ID":"0047408a-8b23-48d6-8024-de2f2ea937cc","Type":"ContainerStarted","Data":"708e642a20270b6f2b870b924b1e91782b66efded3ff9f2c9c91007ba55276ab"}
Jul 21 15:00:48.438205 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:00:48.437942    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c42bc55b-677cc8f96b-ccvk8"
Jul 21 15:00:48.456513 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:00:48.456458    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c42bc55b-677cc8f96b-ccvk8" podStartSLOduration=1.989587123 podStartE2EDuration="4.456440286s" totalImagesPullingTime="2.466853163s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 15:00:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 15:00:48.454374935 +0000 UTC m=+1704.804118352" watchObservedRunningTime="2026-07-21 15:00:48.456440286 +0000 UTC m=+1704.806183698"
Jul 21 15:00:54.447715 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:00:54.447685    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c42bc55b-677cc8f96b-ccvk8"
Jul 21 15:01:10.071178 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:10.071136    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c42bc55b-4z59m"]
Jul 21 15:01:10.078854 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:10.076324    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c42bc55b-4z59m"
Jul 21 15:01:10.079896 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:10.079631    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-494sx\""
Jul 21 15:01:10.082994 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:10.082967    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c42bc55b-4z59m"]
Jul 21 15:01:10.130133 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:10.130107    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2259f5d9-d0b5-4b29-8e6a-cc066bb419ac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c42bc55b-4z59m\" (UID: \"2259f5d9-d0b5-4b29-8e6a-cc066bb419ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c42bc55b-4z59m"
Jul 21 15:01:10.231360 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:10.231337    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2259f5d9-d0b5-4b29-8e6a-cc066bb419ac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c42bc55b-4z59m\" (UID: \"2259f5d9-d0b5-4b29-8e6a-cc066bb419ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c42bc55b-4z59m"
Jul 21 15:01:10.232066 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:10.232042    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2259f5d9-d0b5-4b29-8e6a-cc066bb419ac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c42bc55b-4z59m\" (UID: \"2259f5d9-d0b5-4b29-8e6a-cc066bb419ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c42bc55b-4z59m"
Jul 21 15:01:10.390317 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:10.390295    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c42bc55b-4z59m"
Jul 21 15:01:10.519286 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:10.519240    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c42bc55b-4z59m"]
Jul 21 15:01:10.520750 ip-10-0-130-83 kubenswrapper[2579]: W0721 15:01:10.520720    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2259f5d9_d0b5_4b29_8e6a_cc066bb419ac.slice/crio-3a2d659d487a2ead7bee90936731c664024dd2565646f41f064a529b2d81fc6e WatchSource:0}: Error finding container 3a2d659d487a2ead7bee90936731c664024dd2565646f41f064a529b2d81fc6e: Status 404 returned error can't find the container with id 3a2d659d487a2ead7bee90936731c664024dd2565646f41f064a529b2d81fc6e
Jul 21 15:01:10.539181 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:10.539150    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c42bc55b-4z59m" event={"ID":"2259f5d9-d0b5-4b29-8e6a-cc066bb419ac","Type":"ContainerStarted","Data":"3a2d659d487a2ead7bee90936731c664024dd2565646f41f064a529b2d81fc6e"}
Jul 21 15:01:11.545106 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:11.545026    2579 generic.go:356] "Generic (PLEG): container finished" podID="2259f5d9-d0b5-4b29-8e6a-cc066bb419ac" containerID="49410cfebdee6674a0298f172625ef0732c3e7fd7fcb9c61066134706e4a3129" exitCode=0
Jul 21 15:01:11.545106 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:11.545084    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c42bc55b-4z59m" event={"ID":"2259f5d9-d0b5-4b29-8e6a-cc066bb419ac","Type":"ContainerDied","Data":"49410cfebdee6674a0298f172625ef0732c3e7fd7fcb9c61066134706e4a3129"}
Jul 21 15:01:12.675967 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:12.675945    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c42bc55b-4z59m"
Jul 21 15:01:12.748651 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:12.748630    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2259f5d9-d0b5-4b29-8e6a-cc066bb419ac-maas-api-service-ca\") pod \"2259f5d9-d0b5-4b29-8e6a-cc066bb419ac\" (UID: \"2259f5d9-d0b5-4b29-8e6a-cc066bb419ac\") "
Jul 21 15:01:12.748939 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:12.748920    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2259f5d9-d0b5-4b29-8e6a-cc066bb419ac-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2259f5d9-d0b5-4b29-8e6a-cc066bb419ac" (UID: "2259f5d9-d0b5-4b29-8e6a-cc066bb419ac"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 15:01:12.849551 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:12.849498    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2259f5d9-d0b5-4b29-8e6a-cc066bb419ac-maas-api-service-ca\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:01:13.556101 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:13.556074    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c42bc55b-4z59m"
Jul 21 15:01:13.556297 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:13.556073    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c42bc55b-4z59m" event={"ID":"2259f5d9-d0b5-4b29-8e6a-cc066bb419ac","Type":"ContainerDied","Data":"3a2d659d487a2ead7bee90936731c664024dd2565646f41f064a529b2d81fc6e"}
Jul 21 15:01:13.556297 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:13.556209    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3a2d659d487a2ead7bee90936731c664024dd2565646f41f064a529b2d81fc6e"
Jul 21 15:01:20.533247 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:20.533208    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c42bc55b-677cc8f96b-ccvk8"]
Jul 21 15:01:20.533885 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:20.533474    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c42bc55b-677cc8f96b-ccvk8" podUID="0047408a-8b23-48d6-8024-de2f2ea937cc" containerName="maas-api" containerID="cri-o://708e642a20270b6f2b870b924b1e91782b66efded3ff9f2c9c91007ba55276ab" gracePeriod=30
Jul 21 15:01:20.779467 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:20.779447    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c42bc55b-677cc8f96b-ccvk8"
Jul 21 15:01:20.912816 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:20.912791    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0047408a-8b23-48d6-8024-de2f2ea937cc-openshift-service-ca\") pod \"0047408a-8b23-48d6-8024-de2f2ea937cc\" (UID: \"0047408a-8b23-48d6-8024-de2f2ea937cc\") "
Jul 21 15:01:20.912951 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:20.912846    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0047408a-8b23-48d6-8024-de2f2ea937cc-maas-api-tls\") pod \"0047408a-8b23-48d6-8024-de2f2ea937cc\" (UID: \"0047408a-8b23-48d6-8024-de2f2ea937cc\") "
Jul 21 15:01:20.912951 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:20.912870    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zghbb\" (UniqueName: \"kubernetes.io/projected/0047408a-8b23-48d6-8024-de2f2ea937cc-kube-api-access-zghbb\") pod \"0047408a-8b23-48d6-8024-de2f2ea937cc\" (UID: \"0047408a-8b23-48d6-8024-de2f2ea937cc\") "
Jul 21 15:01:20.913184 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:20.913155    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0047408a-8b23-48d6-8024-de2f2ea937cc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0047408a-8b23-48d6-8024-de2f2ea937cc" (UID: "0047408a-8b23-48d6-8024-de2f2ea937cc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 15:01:20.914975 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:20.914946    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0047408a-8b23-48d6-8024-de2f2ea937cc-kube-api-access-zghbb" (OuterVolumeSpecName: "kube-api-access-zghbb") pod "0047408a-8b23-48d6-8024-de2f2ea937cc" (UID: "0047408a-8b23-48d6-8024-de2f2ea937cc"). InnerVolumeSpecName "kube-api-access-zghbb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 15:01:20.914975 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:20.914948    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0047408a-8b23-48d6-8024-de2f2ea937cc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0047408a-8b23-48d6-8024-de2f2ea937cc" (UID: "0047408a-8b23-48d6-8024-de2f2ea937cc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 15:01:21.013906 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:21.013885    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0047408a-8b23-48d6-8024-de2f2ea937cc-maas-api-tls\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:01:21.013906 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:21.013905    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zghbb\" (UniqueName: \"kubernetes.io/projected/0047408a-8b23-48d6-8024-de2f2ea937cc-kube-api-access-zghbb\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:01:21.014032 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:21.013915    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0047408a-8b23-48d6-8024-de2f2ea937cc-openshift-service-ca\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:01:21.591807 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:21.591773    2579 generic.go:356] "Generic (PLEG): container finished" podID="0047408a-8b23-48d6-8024-de2f2ea937cc" containerID="708e642a20270b6f2b870b924b1e91782b66efded3ff9f2c9c91007ba55276ab" exitCode=0
Jul 21 15:01:21.592206 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:21.591818    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c42bc55b-677cc8f96b-ccvk8"
Jul 21 15:01:21.592206 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:21.591825    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c42bc55b-677cc8f96b-ccvk8" event={"ID":"0047408a-8b23-48d6-8024-de2f2ea937cc","Type":"ContainerDied","Data":"708e642a20270b6f2b870b924b1e91782b66efded3ff9f2c9c91007ba55276ab"}
Jul 21 15:01:21.592206 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:21.591875    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c42bc55b-677cc8f96b-ccvk8" event={"ID":"0047408a-8b23-48d6-8024-de2f2ea937cc","Type":"ContainerDied","Data":"b1d62a0572a2cfa2385b6063658ca96272b5fa259a35b2e1d856febba7503215"}
Jul 21 15:01:21.592206 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:21.591906    2579 scope.go:117] "RemoveContainer" containerID="708e642a20270b6f2b870b924b1e91782b66efded3ff9f2c9c91007ba55276ab"
Jul 21 15:01:21.606222 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:21.606194    2579 scope.go:117] "RemoveContainer" containerID="708e642a20270b6f2b870b924b1e91782b66efded3ff9f2c9c91007ba55276ab"
Jul 21 15:01:21.606504 ip-10-0-130-83 kubenswrapper[2579]: E0721 15:01:21.606482    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"708e642a20270b6f2b870b924b1e91782b66efded3ff9f2c9c91007ba55276ab\": container with ID starting with 708e642a20270b6f2b870b924b1e91782b66efded3ff9f2c9c91007ba55276ab not found: ID does not exist" containerID="708e642a20270b6f2b870b924b1e91782b66efded3ff9f2c9c91007ba55276ab"
Jul 21 15:01:21.606558 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:21.606512    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"708e642a20270b6f2b870b924b1e91782b66efded3ff9f2c9c91007ba55276ab"} err="failed to get container status \"708e642a20270b6f2b870b924b1e91782b66efded3ff9f2c9c91007ba55276ab\": rpc error: code = NotFound desc = could not find container \"708e642a20270b6f2b870b924b1e91782b66efded3ff9f2c9c91007ba55276ab\": container with ID starting with 708e642a20270b6f2b870b924b1e91782b66efded3ff9f2c9c91007ba55276ab not found: ID does not exist"
Jul 21 15:01:21.616074 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:21.616051    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c42bc55b-677cc8f96b-ccvk8"]
Jul 21 15:01:21.619446 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:21.619425    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c42bc55b-677cc8f96b-ccvk8"]
Jul 21 15:01:22.173114 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:22.173083    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0047408a-8b23-48d6-8024-de2f2ea937cc" path="/var/lib/kubelet/pods/0047408a-8b23-48d6-8024-de2f2ea937cc/volumes"
Jul 21 15:01:25.160833 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:25.160806    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c42bc55b-4z59m"]
Jul 21 15:01:25.163752 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:25.163725    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c42bc55b-4z59m"]
Jul 21 15:01:26.172136 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:26.172100    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2259f5d9-d0b5-4b29-8e6a-cc066bb419ac" path="/var/lib/kubelet/pods/2259f5d9-d0b5-4b29-8e6a-cc066bb419ac/volumes"
Jul 21 15:01:28.993317 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:28.993283    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7fd9f867-799ddcd7c8-r4dpd"]
Jul 21 15:01:28.993698 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:28.993612    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2259f5d9-d0b5-4b29-8e6a-cc066bb419ac" containerName="revoke-keys"
Jul 21 15:01:28.993698 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:28.993624    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="2259f5d9-d0b5-4b29-8e6a-cc066bb419ac" containerName="revoke-keys"
Jul 21 15:01:28.993698 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:28.993645    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0047408a-8b23-48d6-8024-de2f2ea937cc" containerName="maas-api"
Jul 21 15:01:28.993698 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:28.993651    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0047408a-8b23-48d6-8024-de2f2ea937cc" containerName="maas-api"
Jul 21 15:01:28.993698 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:28.993698    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="2259f5d9-d0b5-4b29-8e6a-cc066bb419ac" containerName="revoke-keys"
Jul 21 15:01:28.993878 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:28.993707    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0047408a-8b23-48d6-8024-de2f2ea937cc" containerName="maas-api"
Jul 21 15:01:28.996620 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:28.996594    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7fd9f867-799ddcd7c8-r4dpd"
Jul 21 15:01:28.999451 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:28.999424    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-7fd9f867-serving-cert\""
Jul 21 15:01:29.005551 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:29.005529    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7fd9f867-799ddcd7c8-r4dpd"]
Jul 21 15:01:29.080904 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:29.080864    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23c7e6b8-e505-494c-ab2b-35790d5a1ddb-maas-api-tls\") pod \"maas-api-e2e-migrate-7fd9f867-799ddcd7c8-r4dpd\" (UID: \"23c7e6b8-e505-494c-ab2b-35790d5a1ddb\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7fd9f867-799ddcd7c8-r4dpd"
Jul 21 15:01:29.081056 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:29.080944    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23c7e6b8-e505-494c-ab2b-35790d5a1ddb-openshift-service-ca\") pod \"maas-api-e2e-migrate-7fd9f867-799ddcd7c8-r4dpd\" (UID: \"23c7e6b8-e505-494c-ab2b-35790d5a1ddb\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7fd9f867-799ddcd7c8-r4dpd"
Jul 21 15:01:29.081056 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:29.080973    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4l5kq\" (UniqueName: \"kubernetes.io/projected/23c7e6b8-e505-494c-ab2b-35790d5a1ddb-kube-api-access-4l5kq\") pod \"maas-api-e2e-migrate-7fd9f867-799ddcd7c8-r4dpd\" (UID: \"23c7e6b8-e505-494c-ab2b-35790d5a1ddb\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7fd9f867-799ddcd7c8-r4dpd"
Jul 21 15:01:29.181913 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:29.181879    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23c7e6b8-e505-494c-ab2b-35790d5a1ddb-maas-api-tls\") pod \"maas-api-e2e-migrate-7fd9f867-799ddcd7c8-r4dpd\" (UID: \"23c7e6b8-e505-494c-ab2b-35790d5a1ddb\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7fd9f867-799ddcd7c8-r4dpd"
Jul 21 15:01:29.182073 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:29.181952    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23c7e6b8-e505-494c-ab2b-35790d5a1ddb-openshift-service-ca\") pod \"maas-api-e2e-migrate-7fd9f867-799ddcd7c8-r4dpd\" (UID: \"23c7e6b8-e505-494c-ab2b-35790d5a1ddb\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7fd9f867-799ddcd7c8-r4dpd"
Jul 21 15:01:29.182073 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:29.181993    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4l5kq\" (UniqueName: \"kubernetes.io/projected/23c7e6b8-e505-494c-ab2b-35790d5a1ddb-kube-api-access-4l5kq\") pod \"maas-api-e2e-migrate-7fd9f867-799ddcd7c8-r4dpd\" (UID: \"23c7e6b8-e505-494c-ab2b-35790d5a1ddb\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7fd9f867-799ddcd7c8-r4dpd"
Jul 21 15:01:29.182700 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:29.182682    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23c7e6b8-e505-494c-ab2b-35790d5a1ddb-openshift-service-ca\") pod \"maas-api-e2e-migrate-7fd9f867-799ddcd7c8-r4dpd\" (UID: \"23c7e6b8-e505-494c-ab2b-35790d5a1ddb\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7fd9f867-799ddcd7c8-r4dpd"
Jul 21 15:01:29.184257 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:29.184239    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23c7e6b8-e505-494c-ab2b-35790d5a1ddb-maas-api-tls\") pod \"maas-api-e2e-migrate-7fd9f867-799ddcd7c8-r4dpd\" (UID: \"23c7e6b8-e505-494c-ab2b-35790d5a1ddb\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7fd9f867-799ddcd7c8-r4dpd"
Jul 21 15:01:29.189960 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:29.189938    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4l5kq\" (UniqueName: \"kubernetes.io/projected/23c7e6b8-e505-494c-ab2b-35790d5a1ddb-kube-api-access-4l5kq\") pod \"maas-api-e2e-migrate-7fd9f867-799ddcd7c8-r4dpd\" (UID: \"23c7e6b8-e505-494c-ab2b-35790d5a1ddb\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7fd9f867-799ddcd7c8-r4dpd"
Jul 21 15:01:29.308233 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:29.308158    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7fd9f867-799ddcd7c8-r4dpd"
Jul 21 15:01:29.648852 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:29.648819    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7fd9f867-799ddcd7c8-r4dpd"]
Jul 21 15:01:29.651933 ip-10-0-130-83 kubenswrapper[2579]: W0721 15:01:29.651901    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23c7e6b8_e505_494c_ab2b_35790d5a1ddb.slice/crio-4c65406d73bc9b9a4cb1d4c6beb9639d3c97e1ff3e0421c8de67be862c7c1a9e WatchSource:0}: Error finding container 4c65406d73bc9b9a4cb1d4c6beb9639d3c97e1ff3e0421c8de67be862c7c1a9e: Status 404 returned error can't find the container with id 4c65406d73bc9b9a4cb1d4c6beb9639d3c97e1ff3e0421c8de67be862c7c1a9e
Jul 21 15:01:30.635454 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:30.635424    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7fd9f867-799ddcd7c8-r4dpd" event={"ID":"23c7e6b8-e505-494c-ab2b-35790d5a1ddb","Type":"ContainerStarted","Data":"4c65406d73bc9b9a4cb1d4c6beb9639d3c97e1ff3e0421c8de67be862c7c1a9e"}
Jul 21 15:01:31.641509 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:31.641468    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7fd9f867-799ddcd7c8-r4dpd" event={"ID":"23c7e6b8-e505-494c-ab2b-35790d5a1ddb","Type":"ContainerStarted","Data":"d15ae673214b8769f7aded2165f9ed210221c664e2a7d57ae09e4499214db596"}
Jul 21 15:01:31.641894 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:31.641548    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7fd9f867-799ddcd7c8-r4dpd"
Jul 21 15:01:31.660587 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:31.660545    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7fd9f867-799ddcd7c8-r4dpd" podStartSLOduration=2.686156111 podStartE2EDuration="3.660532766s" totalImagesPullingTime="974.376655ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 15:01:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 15:01:31.659472067 +0000 UTC m=+1748.009215476" watchObservedRunningTime="2026-07-21 15:01:31.660532766 +0000 UTC m=+1748.010276177"
Jul 21 15:01:37.652164 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:37.652136    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7fd9f867-799ddcd7c8-r4dpd"
Jul 21 15:01:53.887144 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:53.887112    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7fd9f867-qjzw7"]
Jul 21 15:01:53.890827 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:53.890805    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7fd9f867-qjzw7"
Jul 21 15:01:53.893203 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:53.893184    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-494sx\""
Jul 21 15:01:53.896971 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:53.896943    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7fd9f867-qjzw7"]
Jul 21 15:01:53.985340 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:53.985313    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51b6f972-6e31-4776-86c2-54709f2d0e2d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7fd9f867-qjzw7\" (UID: \"51b6f972-6e31-4776-86c2-54709f2d0e2d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7fd9f867-qjzw7"
Jul 21 15:01:54.086542 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:54.086517    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51b6f972-6e31-4776-86c2-54709f2d0e2d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7fd9f867-qjzw7\" (UID: \"51b6f972-6e31-4776-86c2-54709f2d0e2d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7fd9f867-qjzw7"
Jul 21 15:01:54.087114 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:54.087093    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51b6f972-6e31-4776-86c2-54709f2d0e2d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7fd9f867-qjzw7\" (UID: \"51b6f972-6e31-4776-86c2-54709f2d0e2d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7fd9f867-qjzw7"
Jul 21 15:01:54.204161 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:54.204102    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7fd9f867-qjzw7"
Jul 21 15:01:54.336417 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:54.336392    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7fd9f867-qjzw7"]
Jul 21 15:01:54.338749 ip-10-0-130-83 kubenswrapper[2579]: W0721 15:01:54.338721    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51b6f972_6e31_4776_86c2_54709f2d0e2d.slice/crio-a74fb8a00a9a28d0936c32f0e47da5285385c49c1afcfe178248f8464030a2a9 WatchSource:0}: Error finding container a74fb8a00a9a28d0936c32f0e47da5285385c49c1afcfe178248f8464030a2a9: Status 404 returned error can't find the container with id a74fb8a00a9a28d0936c32f0e47da5285385c49c1afcfe178248f8464030a2a9
Jul 21 15:01:54.747352 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:54.747321    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7fd9f867-qjzw7" event={"ID":"51b6f972-6e31-4776-86c2-54709f2d0e2d","Type":"ContainerStarted","Data":"51a9c14f475a2bc5162668e0f436bf6322f86926b11d76639705961bd43275c1"}
Jul 21 15:01:54.747503 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:54.747369    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7fd9f867-qjzw7" event={"ID":"51b6f972-6e31-4776-86c2-54709f2d0e2d","Type":"ContainerStarted","Data":"a74fb8a00a9a28d0936c32f0e47da5285385c49c1afcfe178248f8464030a2a9"}
Jul 21 15:01:54.764644 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:54.764594    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7fd9f867-qjzw7" podStartSLOduration=1.764579296 podStartE2EDuration="1.764579296s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 15:01:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 15:01:54.763150561 +0000 UTC m=+1771.112893997" watchObservedRunningTime="2026-07-21 15:01:54.764579296 +0000 UTC m=+1771.114322707"
Jul 21 15:01:55.752883 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:55.752849    2579 generic.go:356] "Generic (PLEG): container finished" podID="51b6f972-6e31-4776-86c2-54709f2d0e2d" containerID="51a9c14f475a2bc5162668e0f436bf6322f86926b11d76639705961bd43275c1" exitCode=0
Jul 21 15:01:55.753258 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:55.752931    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7fd9f867-qjzw7" event={"ID":"51b6f972-6e31-4776-86c2-54709f2d0e2d","Type":"ContainerDied","Data":"51a9c14f475a2bc5162668e0f436bf6322f86926b11d76639705961bd43275c1"}
Jul 21 15:01:56.888650 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:56.888625    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7fd9f867-qjzw7"
Jul 21 15:01:57.010033 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:57.009999    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51b6f972-6e31-4776-86c2-54709f2d0e2d-maas-api-service-ca\") pod \"51b6f972-6e31-4776-86c2-54709f2d0e2d\" (UID: \"51b6f972-6e31-4776-86c2-54709f2d0e2d\") "
Jul 21 15:01:57.010392 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:57.010373    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51b6f972-6e31-4776-86c2-54709f2d0e2d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "51b6f972-6e31-4776-86c2-54709f2d0e2d" (UID: "51b6f972-6e31-4776-86c2-54709f2d0e2d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 15:01:57.111318 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:57.111252    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/51b6f972-6e31-4776-86c2-54709f2d0e2d-maas-api-service-ca\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:01:57.768600 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:57.768569    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7fd9f867-qjzw7"
Jul 21 15:01:57.768777 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:57.768606    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7fd9f867-qjzw7" event={"ID":"51b6f972-6e31-4776-86c2-54709f2d0e2d","Type":"ContainerDied","Data":"a74fb8a00a9a28d0936c32f0e47da5285385c49c1afcfe178248f8464030a2a9"}
Jul 21 15:01:57.768777 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:01:57.768661    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a74fb8a00a9a28d0936c32f0e47da5285385c49c1afcfe178248f8464030a2a9"
Jul 21 15:02:04.418850 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:04.418815    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7fd9f867-799ddcd7c8-r4dpd"]
Jul 21 15:02:04.419254 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:04.419058    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7fd9f867-799ddcd7c8-r4dpd" podUID="23c7e6b8-e505-494c-ab2b-35790d5a1ddb" containerName="maas-api" containerID="cri-o://d15ae673214b8769f7aded2165f9ed210221c664e2a7d57ae09e4499214db596" gracePeriod=30
Jul 21 15:02:04.667828 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:04.667807    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7fd9f867-799ddcd7c8-r4dpd"
Jul 21 15:02:04.771788 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:04.771717    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23c7e6b8-e505-494c-ab2b-35790d5a1ddb-openshift-service-ca\") pod \"23c7e6b8-e505-494c-ab2b-35790d5a1ddb\" (UID: \"23c7e6b8-e505-494c-ab2b-35790d5a1ddb\") "
Jul 21 15:02:04.771788 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:04.771783    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23c7e6b8-e505-494c-ab2b-35790d5a1ddb-maas-api-tls\") pod \"23c7e6b8-e505-494c-ab2b-35790d5a1ddb\" (UID: \"23c7e6b8-e505-494c-ab2b-35790d5a1ddb\") "
Jul 21 15:02:04.772006 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:04.771817    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4l5kq\" (UniqueName: \"kubernetes.io/projected/23c7e6b8-e505-494c-ab2b-35790d5a1ddb-kube-api-access-4l5kq\") pod \"23c7e6b8-e505-494c-ab2b-35790d5a1ddb\" (UID: \"23c7e6b8-e505-494c-ab2b-35790d5a1ddb\") "
Jul 21 15:02:04.772165 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:04.772143    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23c7e6b8-e505-494c-ab2b-35790d5a1ddb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "23c7e6b8-e505-494c-ab2b-35790d5a1ddb" (UID: "23c7e6b8-e505-494c-ab2b-35790d5a1ddb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 15:02:04.773797 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:04.773777    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23c7e6b8-e505-494c-ab2b-35790d5a1ddb-kube-api-access-4l5kq" (OuterVolumeSpecName: "kube-api-access-4l5kq") pod "23c7e6b8-e505-494c-ab2b-35790d5a1ddb" (UID: "23c7e6b8-e505-494c-ab2b-35790d5a1ddb"). InnerVolumeSpecName "kube-api-access-4l5kq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 15:02:04.773867 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:04.773826    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23c7e6b8-e505-494c-ab2b-35790d5a1ddb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "23c7e6b8-e505-494c-ab2b-35790d5a1ddb" (UID: "23c7e6b8-e505-494c-ab2b-35790d5a1ddb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 15:02:04.801909 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:04.801879    2579 generic.go:356] "Generic (PLEG): container finished" podID="23c7e6b8-e505-494c-ab2b-35790d5a1ddb" containerID="d15ae673214b8769f7aded2165f9ed210221c664e2a7d57ae09e4499214db596" exitCode=0
Jul 21 15:02:04.802001 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:04.801924    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7fd9f867-799ddcd7c8-r4dpd"
Jul 21 15:02:04.802067 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:04.801922    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7fd9f867-799ddcd7c8-r4dpd" event={"ID":"23c7e6b8-e505-494c-ab2b-35790d5a1ddb","Type":"ContainerDied","Data":"d15ae673214b8769f7aded2165f9ed210221c664e2a7d57ae09e4499214db596"}
Jul 21 15:02:04.802067 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:04.802043    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7fd9f867-799ddcd7c8-r4dpd" event={"ID":"23c7e6b8-e505-494c-ab2b-35790d5a1ddb","Type":"ContainerDied","Data":"4c65406d73bc9b9a4cb1d4c6beb9639d3c97e1ff3e0421c8de67be862c7c1a9e"}
Jul 21 15:02:04.802067 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:04.802064    2579 scope.go:117] "RemoveContainer" containerID="d15ae673214b8769f7aded2165f9ed210221c664e2a7d57ae09e4499214db596"
Jul 21 15:02:04.811110 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:04.811094    2579 scope.go:117] "RemoveContainer" containerID="d15ae673214b8769f7aded2165f9ed210221c664e2a7d57ae09e4499214db596"
Jul 21 15:02:04.811371 ip-10-0-130-83 kubenswrapper[2579]: E0721 15:02:04.811356    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d15ae673214b8769f7aded2165f9ed210221c664e2a7d57ae09e4499214db596\": container with ID starting with d15ae673214b8769f7aded2165f9ed210221c664e2a7d57ae09e4499214db596 not found: ID does not exist" containerID="d15ae673214b8769f7aded2165f9ed210221c664e2a7d57ae09e4499214db596"
Jul 21 15:02:04.811427 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:04.811378    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d15ae673214b8769f7aded2165f9ed210221c664e2a7d57ae09e4499214db596"} err="failed to get container status \"d15ae673214b8769f7aded2165f9ed210221c664e2a7d57ae09e4499214db596\": rpc error: code = NotFound desc = could not find container \"d15ae673214b8769f7aded2165f9ed210221c664e2a7d57ae09e4499214db596\": container with ID starting with d15ae673214b8769f7aded2165f9ed210221c664e2a7d57ae09e4499214db596 not found: ID does not exist"
Jul 21 15:02:04.822505 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:04.822485    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7fd9f867-799ddcd7c8-r4dpd"]
Jul 21 15:02:04.826255 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:04.826237    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7fd9f867-799ddcd7c8-r4dpd"]
Jul 21 15:02:04.873122 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:04.873098    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23c7e6b8-e505-494c-ab2b-35790d5a1ddb-openshift-service-ca\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:02:04.873122 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:04.873119    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23c7e6b8-e505-494c-ab2b-35790d5a1ddb-maas-api-tls\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:02:04.873221 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:04.873128    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4l5kq\" (UniqueName: \"kubernetes.io/projected/23c7e6b8-e505-494c-ab2b-35790d5a1ddb-kube-api-access-4l5kq\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:02:06.172885 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:06.172850    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23c7e6b8-e505-494c-ab2b-35790d5a1ddb" path="/var/lib/kubelet/pods/23c7e6b8-e505-494c-ab2b-35790d5a1ddb/volumes"
Jul 21 15:02:08.967005 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:08.966976    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7fd9f867-qjzw7"]
Jul 21 15:02:08.970263 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:08.970237    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7fd9f867-qjzw7"]
Jul 21 15:02:10.172255 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:10.172222    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51b6f972-6e31-4776-86c2-54709f2d0e2d" path="/var/lib/kubelet/pods/51b6f972-6e31-4776-86c2-54709f2d0e2d/volumes"
Jul 21 15:02:12.157777 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:12.157750    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2cd63d23-bb6876f75-gpdvn"]
Jul 21 15:02:12.158134 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:12.158120    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51b6f972-6e31-4776-86c2-54709f2d0e2d" containerName="revoke-keys"
Jul 21 15:02:12.158185 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:12.158135    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="51b6f972-6e31-4776-86c2-54709f2d0e2d" containerName="revoke-keys"
Jul 21 15:02:12.158185 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:12.158165    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23c7e6b8-e505-494c-ab2b-35790d5a1ddb" containerName="maas-api"
Jul 21 15:02:12.158185 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:12.158171    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="23c7e6b8-e505-494c-ab2b-35790d5a1ddb" containerName="maas-api"
Jul 21 15:02:12.158308 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:12.158225    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="51b6f972-6e31-4776-86c2-54709f2d0e2d" containerName="revoke-keys"
Jul 21 15:02:12.158308 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:12.158237    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="23c7e6b8-e505-494c-ab2b-35790d5a1ddb" containerName="maas-api"
Jul 21 15:02:12.162313 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:12.162291    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2cd63d23-bb6876f75-gpdvn"
Jul 21 15:02:12.164922 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:12.164900    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-2cd63d23-serving-cert\""
Jul 21 15:02:12.174640 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:12.174619    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2cd63d23-bb6876f75-gpdvn"]
Jul 21 15:02:12.232634 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:12.232600    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9d168a17-61b3-4fc0-8517-e0a48be94ffe-maas-api-tls\") pod \"maas-api-e2e-ait-2cd63d23-bb6876f75-gpdvn\" (UID: \"9d168a17-61b3-4fc0-8517-e0a48be94ffe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2cd63d23-bb6876f75-gpdvn"
Jul 21 15:02:12.232806 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:12.232642    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d168a17-61b3-4fc0-8517-e0a48be94ffe-openshift-service-ca\") pod \"maas-api-e2e-ait-2cd63d23-bb6876f75-gpdvn\" (UID: \"9d168a17-61b3-4fc0-8517-e0a48be94ffe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2cd63d23-bb6876f75-gpdvn"
Jul 21 15:02:12.232806 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:12.232709    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p5zz\" (UniqueName: \"kubernetes.io/projected/9d168a17-61b3-4fc0-8517-e0a48be94ffe-kube-api-access-8p5zz\") pod \"maas-api-e2e-ait-2cd63d23-bb6876f75-gpdvn\" (UID: \"9d168a17-61b3-4fc0-8517-e0a48be94ffe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2cd63d23-bb6876f75-gpdvn"
Jul 21 15:02:12.334094 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:12.334048    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d168a17-61b3-4fc0-8517-e0a48be94ffe-openshift-service-ca\") pod \"maas-api-e2e-ait-2cd63d23-bb6876f75-gpdvn\" (UID: \"9d168a17-61b3-4fc0-8517-e0a48be94ffe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2cd63d23-bb6876f75-gpdvn"
Jul 21 15:02:12.334474 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:12.334447    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8p5zz\" (UniqueName: \"kubernetes.io/projected/9d168a17-61b3-4fc0-8517-e0a48be94ffe-kube-api-access-8p5zz\") pod \"maas-api-e2e-ait-2cd63d23-bb6876f75-gpdvn\" (UID: \"9d168a17-61b3-4fc0-8517-e0a48be94ffe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2cd63d23-bb6876f75-gpdvn"
Jul 21 15:02:12.334804 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:12.334728    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9d168a17-61b3-4fc0-8517-e0a48be94ffe-maas-api-tls\") pod \"maas-api-e2e-ait-2cd63d23-bb6876f75-gpdvn\" (UID: \"9d168a17-61b3-4fc0-8517-e0a48be94ffe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2cd63d23-bb6876f75-gpdvn"
Jul 21 15:02:12.336019 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:12.334990    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d168a17-61b3-4fc0-8517-e0a48be94ffe-openshift-service-ca\") pod \"maas-api-e2e-ait-2cd63d23-bb6876f75-gpdvn\" (UID: \"9d168a17-61b3-4fc0-8517-e0a48be94ffe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2cd63d23-bb6876f75-gpdvn"
Jul 21 15:02:12.337954 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:12.337934    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9d168a17-61b3-4fc0-8517-e0a48be94ffe-maas-api-tls\") pod \"maas-api-e2e-ait-2cd63d23-bb6876f75-gpdvn\" (UID: \"9d168a17-61b3-4fc0-8517-e0a48be94ffe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2cd63d23-bb6876f75-gpdvn"
Jul 21 15:02:12.343260 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:12.343240    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p5zz\" (UniqueName: \"kubernetes.io/projected/9d168a17-61b3-4fc0-8517-e0a48be94ffe-kube-api-access-8p5zz\") pod \"maas-api-e2e-ait-2cd63d23-bb6876f75-gpdvn\" (UID: \"9d168a17-61b3-4fc0-8517-e0a48be94ffe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-2cd63d23-bb6876f75-gpdvn"
Jul 21 15:02:12.473827 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:12.473747    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2cd63d23-bb6876f75-gpdvn"
Jul 21 15:02:12.624899 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:12.624873    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2cd63d23-bb6876f75-gpdvn"]
Jul 21 15:02:12.627286 ip-10-0-130-83 kubenswrapper[2579]: W0721 15:02:12.627248    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d168a17_61b3_4fc0_8517_e0a48be94ffe.slice/crio-524d1c3a4d79366422491e79eb10000e7deca834f97f371d7bb67789a57a3e69 WatchSource:0}: Error finding container 524d1c3a4d79366422491e79eb10000e7deca834f97f371d7bb67789a57a3e69: Status 404 returned error can't find the container with id 524d1c3a4d79366422491e79eb10000e7deca834f97f371d7bb67789a57a3e69
Jul 21 15:02:12.842071 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:12.841989    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2cd63d23-bb6876f75-gpdvn" event={"ID":"9d168a17-61b3-4fc0-8517-e0a48be94ffe","Type":"ContainerStarted","Data":"524d1c3a4d79366422491e79eb10000e7deca834f97f371d7bb67789a57a3e69"}
Jul 21 15:02:13.848365 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:13.848329    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2cd63d23-bb6876f75-gpdvn" event={"ID":"9d168a17-61b3-4fc0-8517-e0a48be94ffe","Type":"ContainerStarted","Data":"7d9ab131eb4fcf46ab1f769c90a16f5f664a17889c8df6c9f874f3746f3812b4"}
Jul 21 15:02:13.848747 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:13.848508    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2cd63d23-bb6876f75-gpdvn"
Jul 21 15:02:13.865800 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:13.865750    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2cd63d23-bb6876f75-gpdvn" podStartSLOduration=0.794766532 podStartE2EDuration="1.865735724s" totalImagesPullingTime="1.070969192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 15:02:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 15:02:13.863589294 +0000 UTC m=+1790.213332706" watchObservedRunningTime="2026-07-21 15:02:13.865735724 +0000 UTC m=+1790.215479158"
Jul 21 15:02:19.859094 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:19.859018    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2cd63d23-bb6876f75-gpdvn"
Jul 21 15:02:37.952911 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:37.952881    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2cd63d23-675fn"]
Jul 21 15:02:37.956384 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:37.956363    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2cd63d23-675fn"
Jul 21 15:02:37.958645 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:37.958622    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-494sx\""
Jul 21 15:02:37.963530 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:37.963507    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2cd63d23-675fn"]
Jul 21 15:02:38.041099 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:38.041072    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5ce2cf7-a6b5-4218-917c-d11b876964d2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2cd63d23-675fn\" (UID: \"e5ce2cf7-a6b5-4218-917c-d11b876964d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2cd63d23-675fn"
Jul 21 15:02:38.142170 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:38.142140    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5ce2cf7-a6b5-4218-917c-d11b876964d2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2cd63d23-675fn\" (UID: \"e5ce2cf7-a6b5-4218-917c-d11b876964d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2cd63d23-675fn"
Jul 21 15:02:38.142739 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:38.142722    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5ce2cf7-a6b5-4218-917c-d11b876964d2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-2cd63d23-675fn\" (UID: \"e5ce2cf7-a6b5-4218-917c-d11b876964d2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2cd63d23-675fn"
Jul 21 15:02:38.268118 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:38.268042    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2cd63d23-675fn"
Jul 21 15:02:38.403828 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:38.403799    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2cd63d23-675fn"]
Jul 21 15:02:38.406937 ip-10-0-130-83 kubenswrapper[2579]: W0721 15:02:38.406910    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5ce2cf7_a6b5_4218_917c_d11b876964d2.slice/crio-63a1bb5e637cd23389d1d82df4aab4a634effe8b202fefef1b3a96c3a8acae4b WatchSource:0}: Error finding container 63a1bb5e637cd23389d1d82df4aab4a634effe8b202fefef1b3a96c3a8acae4b: Status 404 returned error can't find the container with id 63a1bb5e637cd23389d1d82df4aab4a634effe8b202fefef1b3a96c3a8acae4b
Jul 21 15:02:38.964845 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:38.964762    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2cd63d23-675fn" event={"ID":"e5ce2cf7-a6b5-4218-917c-d11b876964d2","Type":"ContainerStarted","Data":"228b7a992f833136c1fd53fdbf76c06236a6ae5032dd9d490b09e507cee6c2a5"}
Jul 21 15:02:38.964845 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:38.964821    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2cd63d23-675fn" event={"ID":"e5ce2cf7-a6b5-4218-917c-d11b876964d2","Type":"ContainerStarted","Data":"63a1bb5e637cd23389d1d82df4aab4a634effe8b202fefef1b3a96c3a8acae4b"}
Jul 21 15:02:38.984548 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:38.984507    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2cd63d23-675fn" podStartSLOduration=1.984492425 podStartE2EDuration="1.984492425s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 15:02:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 15:02:38.983214882 +0000 UTC m=+1815.332958303" watchObservedRunningTime="2026-07-21 15:02:38.984492425 +0000 UTC m=+1815.334235836"
Jul 21 15:02:39.971070 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:39.971036    2579 generic.go:356] "Generic (PLEG): container finished" podID="e5ce2cf7-a6b5-4218-917c-d11b876964d2" containerID="228b7a992f833136c1fd53fdbf76c06236a6ae5032dd9d490b09e507cee6c2a5" exitCode=0
Jul 21 15:02:39.971492 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:39.971132    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2cd63d23-675fn" event={"ID":"e5ce2cf7-a6b5-4218-917c-d11b876964d2","Type":"ContainerDied","Data":"228b7a992f833136c1fd53fdbf76c06236a6ae5032dd9d490b09e507cee6c2a5"}
Jul 21 15:02:41.105160 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:41.105137    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2cd63d23-675fn"
Jul 21 15:02:41.269533 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:41.269476    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5ce2cf7-a6b5-4218-917c-d11b876964d2-maas-api-service-ca\") pod \"e5ce2cf7-a6b5-4218-917c-d11b876964d2\" (UID: \"e5ce2cf7-a6b5-4218-917c-d11b876964d2\") "
Jul 21 15:02:41.269803 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:41.269782    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5ce2cf7-a6b5-4218-917c-d11b876964d2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e5ce2cf7-a6b5-4218-917c-d11b876964d2" (UID: "e5ce2cf7-a6b5-4218-917c-d11b876964d2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 15:02:41.370067 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:41.370039    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e5ce2cf7-a6b5-4218-917c-d11b876964d2-maas-api-service-ca\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:02:41.981489 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:41.981461    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2cd63d23-675fn"
Jul 21 15:02:41.981640 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:41.981491    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2cd63d23-675fn" event={"ID":"e5ce2cf7-a6b5-4218-917c-d11b876964d2","Type":"ContainerDied","Data":"63a1bb5e637cd23389d1d82df4aab4a634effe8b202fefef1b3a96c3a8acae4b"}
Jul 21 15:02:41.981640 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:41.981534    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="63a1bb5e637cd23389d1d82df4aab4a634effe8b202fefef1b3a96c3a8acae4b"
Jul 21 15:02:48.490739 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:48.490702    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2cd63d23-bb6876f75-gpdvn"]
Jul 21 15:02:48.491210 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:48.491023    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2cd63d23-bb6876f75-gpdvn" podUID="9d168a17-61b3-4fc0-8517-e0a48be94ffe" containerName="maas-api" containerID="cri-o://7d9ab131eb4fcf46ab1f769c90a16f5f664a17889c8df6c9f874f3746f3812b4" gracePeriod=30
Jul 21 15:02:48.740439 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:48.740407    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2cd63d23-bb6876f75-gpdvn"
Jul 21 15:02:48.933694 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:48.933661    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d168a17-61b3-4fc0-8517-e0a48be94ffe-openshift-service-ca\") pod \"9d168a17-61b3-4fc0-8517-e0a48be94ffe\" (UID: \"9d168a17-61b3-4fc0-8517-e0a48be94ffe\") "
Jul 21 15:02:48.933854 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:48.933711    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9d168a17-61b3-4fc0-8517-e0a48be94ffe-maas-api-tls\") pod \"9d168a17-61b3-4fc0-8517-e0a48be94ffe\" (UID: \"9d168a17-61b3-4fc0-8517-e0a48be94ffe\") "
Jul 21 15:02:48.933854 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:48.933777    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8p5zz\" (UniqueName: \"kubernetes.io/projected/9d168a17-61b3-4fc0-8517-e0a48be94ffe-kube-api-access-8p5zz\") pod \"9d168a17-61b3-4fc0-8517-e0a48be94ffe\" (UID: \"9d168a17-61b3-4fc0-8517-e0a48be94ffe\") "
Jul 21 15:02:48.934051 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:48.934032    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d168a17-61b3-4fc0-8517-e0a48be94ffe-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9d168a17-61b3-4fc0-8517-e0a48be94ffe" (UID: "9d168a17-61b3-4fc0-8517-e0a48be94ffe"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 15:02:48.935704 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:48.935668    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d168a17-61b3-4fc0-8517-e0a48be94ffe-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9d168a17-61b3-4fc0-8517-e0a48be94ffe" (UID: "9d168a17-61b3-4fc0-8517-e0a48be94ffe"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 15:02:48.935811 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:48.935765    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d168a17-61b3-4fc0-8517-e0a48be94ffe-kube-api-access-8p5zz" (OuterVolumeSpecName: "kube-api-access-8p5zz") pod "9d168a17-61b3-4fc0-8517-e0a48be94ffe" (UID: "9d168a17-61b3-4fc0-8517-e0a48be94ffe"). InnerVolumeSpecName "kube-api-access-8p5zz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 15:02:49.012554 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:49.012528    2579 generic.go:356] "Generic (PLEG): container finished" podID="9d168a17-61b3-4fc0-8517-e0a48be94ffe" containerID="7d9ab131eb4fcf46ab1f769c90a16f5f664a17889c8df6c9f874f3746f3812b4" exitCode=0
Jul 21 15:02:49.012681 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:49.012568    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2cd63d23-bb6876f75-gpdvn"
Jul 21 15:02:49.012681 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:49.012578    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2cd63d23-bb6876f75-gpdvn" event={"ID":"9d168a17-61b3-4fc0-8517-e0a48be94ffe","Type":"ContainerDied","Data":"7d9ab131eb4fcf46ab1f769c90a16f5f664a17889c8df6c9f874f3746f3812b4"}
Jul 21 15:02:49.012681 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:49.012620    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-2cd63d23-bb6876f75-gpdvn" event={"ID":"9d168a17-61b3-4fc0-8517-e0a48be94ffe","Type":"ContainerDied","Data":"524d1c3a4d79366422491e79eb10000e7deca834f97f371d7bb67789a57a3e69"}
Jul 21 15:02:49.012681 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:49.012648    2579 scope.go:117] "RemoveContainer" containerID="7d9ab131eb4fcf46ab1f769c90a16f5f664a17889c8df6c9f874f3746f3812b4"
Jul 21 15:02:49.021754 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:49.021739    2579 scope.go:117] "RemoveContainer" containerID="7d9ab131eb4fcf46ab1f769c90a16f5f664a17889c8df6c9f874f3746f3812b4"
Jul 21 15:02:49.021992 ip-10-0-130-83 kubenswrapper[2579]: E0721 15:02:49.021977    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d9ab131eb4fcf46ab1f769c90a16f5f664a17889c8df6c9f874f3746f3812b4\": container with ID starting with 7d9ab131eb4fcf46ab1f769c90a16f5f664a17889c8df6c9f874f3746f3812b4 not found: ID does not exist" containerID="7d9ab131eb4fcf46ab1f769c90a16f5f664a17889c8df6c9f874f3746f3812b4"
Jul 21 15:02:49.022055 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:49.022005    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d9ab131eb4fcf46ab1f769c90a16f5f664a17889c8df6c9f874f3746f3812b4"} err="failed to get container status \"7d9ab131eb4fcf46ab1f769c90a16f5f664a17889c8df6c9f874f3746f3812b4\": rpc error: code = NotFound desc = could not find container \"7d9ab131eb4fcf46ab1f769c90a16f5f664a17889c8df6c9f874f3746f3812b4\": container with ID starting with 7d9ab131eb4fcf46ab1f769c90a16f5f664a17889c8df6c9f874f3746f3812b4 not found: ID does not exist"
Jul 21 15:02:49.034478 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:49.034455    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2cd63d23-bb6876f75-gpdvn"]
Jul 21 15:02:49.034732 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:49.034716    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d168a17-61b3-4fc0-8517-e0a48be94ffe-openshift-service-ca\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:02:49.034777 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:49.034736    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9d168a17-61b3-4fc0-8517-e0a48be94ffe-maas-api-tls\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:02:49.034777 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:49.034746    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8p5zz\" (UniqueName: \"kubernetes.io/projected/9d168a17-61b3-4fc0-8517-e0a48be94ffe-kube-api-access-8p5zz\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:02:49.038296 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:49.038264    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-2cd63d23-bb6876f75-gpdvn"]
Jul 21 15:02:50.171369 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:50.171331    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d168a17-61b3-4fc0-8517-e0a48be94ffe" path="/var/lib/kubelet/pods/9d168a17-61b3-4fc0-8517-e0a48be94ffe/volumes"
Jul 21 15:02:53.032520 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:53.032483    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2cd63d23-675fn"]
Jul 21 15:02:53.036818 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:53.036794    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-2cd63d23-675fn"]
Jul 21 15:02:54.171533 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:54.171499    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5ce2cf7-a6b5-4218-917c-d11b876964d2" path="/var/lib/kubelet/pods/e5ce2cf7-a6b5-4218-917c-d11b876964d2/volumes"
Jul 21 15:02:57.442331 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:57.442300    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-068a27f9-75dd5cd684-sfr55"]
Jul 21 15:02:57.442785 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:57.442769    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d168a17-61b3-4fc0-8517-e0a48be94ffe" containerName="maas-api"
Jul 21 15:02:57.442841 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:57.442788    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d168a17-61b3-4fc0-8517-e0a48be94ffe" containerName="maas-api"
Jul 21 15:02:57.442841 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:57.442811    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5ce2cf7-a6b5-4218-917c-d11b876964d2" containerName="revoke-keys"
Jul 21 15:02:57.442841 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:57.442820    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5ce2cf7-a6b5-4218-917c-d11b876964d2" containerName="revoke-keys"
Jul 21 15:02:57.442931 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:57.442911    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5ce2cf7-a6b5-4218-917c-d11b876964d2" containerName="revoke-keys"
Jul 21 15:02:57.442931 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:57.442923    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d168a17-61b3-4fc0-8517-e0a48be94ffe" containerName="maas-api"
Jul 21 15:02:57.446038 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:57.446021    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-068a27f9-75dd5cd684-sfr55"
Jul 21 15:02:57.448765 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:57.448744    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-068a27f9-serving-cert\""
Jul 21 15:02:57.458132 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:57.458108    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-068a27f9-75dd5cd684-sfr55"]
Jul 21 15:02:57.502991 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:57.502965    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqwbf\" (UniqueName: \"kubernetes.io/projected/bb3422dc-476a-49d8-834c-05dd06ea03f8-kube-api-access-bqwbf\") pod \"maas-api-e2e-derive-068a27f9-75dd5cd684-sfr55\" (UID: \"bb3422dc-476a-49d8-834c-05dd06ea03f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-068a27f9-75dd5cd684-sfr55"
Jul 21 15:02:57.503117 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:57.503044    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb3422dc-476a-49d8-834c-05dd06ea03f8-maas-api-tls\") pod \"maas-api-e2e-derive-068a27f9-75dd5cd684-sfr55\" (UID: \"bb3422dc-476a-49d8-834c-05dd06ea03f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-068a27f9-75dd5cd684-sfr55"
Jul 21 15:02:57.503200 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:57.503168    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb3422dc-476a-49d8-834c-05dd06ea03f8-openshift-service-ca\") pod \"maas-api-e2e-derive-068a27f9-75dd5cd684-sfr55\" (UID: \"bb3422dc-476a-49d8-834c-05dd06ea03f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-068a27f9-75dd5cd684-sfr55"
Jul 21 15:02:57.604387 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:57.604354    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb3422dc-476a-49d8-834c-05dd06ea03f8-openshift-service-ca\") pod \"maas-api-e2e-derive-068a27f9-75dd5cd684-sfr55\" (UID: \"bb3422dc-476a-49d8-834c-05dd06ea03f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-068a27f9-75dd5cd684-sfr55"
Jul 21 15:02:57.604565 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:57.604403    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqwbf\" (UniqueName: \"kubernetes.io/projected/bb3422dc-476a-49d8-834c-05dd06ea03f8-kube-api-access-bqwbf\") pod \"maas-api-e2e-derive-068a27f9-75dd5cd684-sfr55\" (UID: \"bb3422dc-476a-49d8-834c-05dd06ea03f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-068a27f9-75dd5cd684-sfr55"
Jul 21 15:02:57.604565 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:57.604453    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb3422dc-476a-49d8-834c-05dd06ea03f8-maas-api-tls\") pod \"maas-api-e2e-derive-068a27f9-75dd5cd684-sfr55\" (UID: \"bb3422dc-476a-49d8-834c-05dd06ea03f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-068a27f9-75dd5cd684-sfr55"
Jul 21 15:02:57.604689 ip-10-0-130-83 kubenswrapper[2579]: E0721 15:02:57.604568    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-068a27f9-serving-cert: secret "maas-api-e2e-derive-068a27f9-serving-cert" not found
Jul 21 15:02:57.604689 ip-10-0-130-83 kubenswrapper[2579]: E0721 15:02:57.604652    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb3422dc-476a-49d8-834c-05dd06ea03f8-maas-api-tls podName:bb3422dc-476a-49d8-834c-05dd06ea03f8 nodeName:}" failed. No retries permitted until 2026-07-21 15:02:58.104626713 +0000 UTC m=+1834.454370120 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bb3422dc-476a-49d8-834c-05dd06ea03f8-maas-api-tls") pod "maas-api-e2e-derive-068a27f9-75dd5cd684-sfr55" (UID: "bb3422dc-476a-49d8-834c-05dd06ea03f8") : secret "maas-api-e2e-derive-068a27f9-serving-cert" not found
Jul 21 15:02:57.604971 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:57.604952    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb3422dc-476a-49d8-834c-05dd06ea03f8-openshift-service-ca\") pod \"maas-api-e2e-derive-068a27f9-75dd5cd684-sfr55\" (UID: \"bb3422dc-476a-49d8-834c-05dd06ea03f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-068a27f9-75dd5cd684-sfr55"
Jul 21 15:02:57.614165 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:57.614141    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqwbf\" (UniqueName: \"kubernetes.io/projected/bb3422dc-476a-49d8-834c-05dd06ea03f8-kube-api-access-bqwbf\") pod \"maas-api-e2e-derive-068a27f9-75dd5cd684-sfr55\" (UID: \"bb3422dc-476a-49d8-834c-05dd06ea03f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-068a27f9-75dd5cd684-sfr55"
Jul 21 15:02:58.108068 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:58.108038    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb3422dc-476a-49d8-834c-05dd06ea03f8-maas-api-tls\") pod \"maas-api-e2e-derive-068a27f9-75dd5cd684-sfr55\" (UID: \"bb3422dc-476a-49d8-834c-05dd06ea03f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-068a27f9-75dd5cd684-sfr55"
Jul 21 15:02:58.110619 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:58.110592    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb3422dc-476a-49d8-834c-05dd06ea03f8-maas-api-tls\") pod \"maas-api-e2e-derive-068a27f9-75dd5cd684-sfr55\" (UID: \"bb3422dc-476a-49d8-834c-05dd06ea03f8\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-068a27f9-75dd5cd684-sfr55"
Jul 21 15:02:58.359715 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:58.359641    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-068a27f9-75dd5cd684-sfr55"
Jul 21 15:02:58.496494 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:58.496463    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-068a27f9-75dd5cd684-sfr55"]
Jul 21 15:02:58.498749 ip-10-0-130-83 kubenswrapper[2579]: W0721 15:02:58.498720    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb3422dc_476a_49d8_834c_05dd06ea03f8.slice/crio-522deddda36c3d6ee8049f84f971597f6bfd6788154fb15cdfaa315ddd287298 WatchSource:0}: Error finding container 522deddda36c3d6ee8049f84f971597f6bfd6788154fb15cdfaa315ddd287298: Status 404 returned error can't find the container with id 522deddda36c3d6ee8049f84f971597f6bfd6788154fb15cdfaa315ddd287298
Jul 21 15:02:59.059684 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:59.059656    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-068a27f9-75dd5cd684-sfr55" event={"ID":"bb3422dc-476a-49d8-834c-05dd06ea03f8","Type":"ContainerStarted","Data":"6879d014e8a2ca344c6a6f634b875c7d103447b85a8aec06d0e19a455ce4cde6"}
Jul 21 15:02:59.059805 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:59.059706    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-068a27f9-75dd5cd684-sfr55" event={"ID":"bb3422dc-476a-49d8-834c-05dd06ea03f8","Type":"ContainerStarted","Data":"522deddda36c3d6ee8049f84f971597f6bfd6788154fb15cdfaa315ddd287298"}
Jul 21 15:02:59.059805 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:59.059735    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-068a27f9-75dd5cd684-sfr55"
Jul 21 15:02:59.083986 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:02:59.083941    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-068a27f9-75dd5cd684-sfr55" podStartSLOduration=1.605282924 podStartE2EDuration="2.08392709s" totalImagesPullingTime="478.644166ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 15:02:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 15:02:59.081722403 +0000 UTC m=+1835.431465811" watchObservedRunningTime="2026-07-21 15:02:59.08392709 +0000 UTC m=+1835.433670499"
Jul 21 15:03:05.069723 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:05.069697    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-068a27f9-75dd5cd684-sfr55"
Jul 21 15:03:22.226045 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:22.226013    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-068a27f9-97bw9"]
Jul 21 15:03:22.229648 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:22.229631    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-068a27f9-97bw9"
Jul 21 15:03:22.232603 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:22.232584    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-494sx\""
Jul 21 15:03:22.237753 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:22.237732    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-068a27f9-97bw9"]
Jul 21 15:03:22.298929 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:22.298903    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81c158a3-05a4-4ea3-aa50-031778aaa654-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-068a27f9-97bw9\" (UID: \"81c158a3-05a4-4ea3-aa50-031778aaa654\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-068a27f9-97bw9"
Jul 21 15:03:22.399848 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:22.399817    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81c158a3-05a4-4ea3-aa50-031778aaa654-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-068a27f9-97bw9\" (UID: \"81c158a3-05a4-4ea3-aa50-031778aaa654\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-068a27f9-97bw9"
Jul 21 15:03:22.400371 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:22.400355    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81c158a3-05a4-4ea3-aa50-031778aaa654-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-068a27f9-97bw9\" (UID: \"81c158a3-05a4-4ea3-aa50-031778aaa654\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-068a27f9-97bw9"
Jul 21 15:03:22.541154 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:22.541086    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-068a27f9-97bw9"
Jul 21 15:03:22.671017 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:22.670977    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-068a27f9-97bw9"]
Jul 21 15:03:22.677750 ip-10-0-130-83 kubenswrapper[2579]: W0721 15:03:22.677724    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81c158a3_05a4_4ea3_aa50_031778aaa654.slice/crio-b0ae2b20383297b5be623beed96819283eb08d4344419a1f1b5a889e0c19cf86 WatchSource:0}: Error finding container b0ae2b20383297b5be623beed96819283eb08d4344419a1f1b5a889e0c19cf86: Status 404 returned error can't find the container with id b0ae2b20383297b5be623beed96819283eb08d4344419a1f1b5a889e0c19cf86
Jul 21 15:03:23.180655 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:23.180617    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-068a27f9-97bw9" event={"ID":"81c158a3-05a4-4ea3-aa50-031778aaa654","Type":"ContainerStarted","Data":"c06eee07cf7a0faaddaa7b8605c9598eec1aa891e8049a7996686605b4613a62"}
Jul 21 15:03:23.180818 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:23.180669    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-068a27f9-97bw9" event={"ID":"81c158a3-05a4-4ea3-aa50-031778aaa654","Type":"ContainerStarted","Data":"b0ae2b20383297b5be623beed96819283eb08d4344419a1f1b5a889e0c19cf86"}
Jul 21 15:03:23.199085 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:23.199038    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-068a27f9-97bw9" podStartSLOduration=1.199025705 podStartE2EDuration="1.199025705s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 15:03:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 15:03:23.198634401 +0000 UTC m=+1859.548377812" watchObservedRunningTime="2026-07-21 15:03:23.199025705 +0000 UTC m=+1859.548769116"
Jul 21 15:03:24.185891 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:24.185857    2579 generic.go:356] "Generic (PLEG): container finished" podID="81c158a3-05a4-4ea3-aa50-031778aaa654" containerID="c06eee07cf7a0faaddaa7b8605c9598eec1aa891e8049a7996686605b4613a62" exitCode=0
Jul 21 15:03:24.186247 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:24.185917    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-068a27f9-97bw9" event={"ID":"81c158a3-05a4-4ea3-aa50-031778aaa654","Type":"ContainerDied","Data":"c06eee07cf7a0faaddaa7b8605c9598eec1aa891e8049a7996686605b4613a62"}
Jul 21 15:03:25.314057 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:25.314033    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-068a27f9-97bw9"
Jul 21 15:03:25.426006 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:25.425979    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81c158a3-05a4-4ea3-aa50-031778aaa654-maas-api-service-ca\") pod \"81c158a3-05a4-4ea3-aa50-031778aaa654\" (UID: \"81c158a3-05a4-4ea3-aa50-031778aaa654\") "
Jul 21 15:03:25.426286 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:25.426250    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81c158a3-05a4-4ea3-aa50-031778aaa654-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "81c158a3-05a4-4ea3-aa50-031778aaa654" (UID: "81c158a3-05a4-4ea3-aa50-031778aaa654"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 15:03:25.526983 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:25.526926    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/81c158a3-05a4-4ea3-aa50-031778aaa654-maas-api-service-ca\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:03:26.196397 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:26.196370    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-068a27f9-97bw9"
Jul 21 15:03:26.196397 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:26.196378    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-068a27f9-97bw9" event={"ID":"81c158a3-05a4-4ea3-aa50-031778aaa654","Type":"ContainerDied","Data":"b0ae2b20383297b5be623beed96819283eb08d4344419a1f1b5a889e0c19cf86"}
Jul 21 15:03:26.196566 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:26.196424    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b0ae2b20383297b5be623beed96819283eb08d4344419a1f1b5a889e0c19cf86"
Jul 21 15:03:32.633473 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:32.633424    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-068a27f9-75dd5cd684-sfr55"]
Jul 21 15:03:32.633923 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:32.633678    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-068a27f9-75dd5cd684-sfr55" podUID="bb3422dc-476a-49d8-834c-05dd06ea03f8" containerName="maas-api" containerID="cri-o://6879d014e8a2ca344c6a6f634b875c7d103447b85a8aec06d0e19a455ce4cde6" gracePeriod=30
Jul 21 15:03:32.881950 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:32.881928    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-068a27f9-75dd5cd684-sfr55"
Jul 21 15:03:32.987177 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:32.987107    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb3422dc-476a-49d8-834c-05dd06ea03f8-maas-api-tls\") pod \"bb3422dc-476a-49d8-834c-05dd06ea03f8\" (UID: \"bb3422dc-476a-49d8-834c-05dd06ea03f8\") "
Jul 21 15:03:32.987177 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:32.987171    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb3422dc-476a-49d8-834c-05dd06ea03f8-openshift-service-ca\") pod \"bb3422dc-476a-49d8-834c-05dd06ea03f8\" (UID: \"bb3422dc-476a-49d8-834c-05dd06ea03f8\") "
Jul 21 15:03:32.987381 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:32.987213    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bqwbf\" (UniqueName: \"kubernetes.io/projected/bb3422dc-476a-49d8-834c-05dd06ea03f8-kube-api-access-bqwbf\") pod \"bb3422dc-476a-49d8-834c-05dd06ea03f8\" (UID: \"bb3422dc-476a-49d8-834c-05dd06ea03f8\") "
Jul 21 15:03:32.987580 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:32.987554    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb3422dc-476a-49d8-834c-05dd06ea03f8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bb3422dc-476a-49d8-834c-05dd06ea03f8" (UID: "bb3422dc-476a-49d8-834c-05dd06ea03f8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 15:03:32.989295 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:32.989250    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb3422dc-476a-49d8-834c-05dd06ea03f8-kube-api-access-bqwbf" (OuterVolumeSpecName: "kube-api-access-bqwbf") pod "bb3422dc-476a-49d8-834c-05dd06ea03f8" (UID: "bb3422dc-476a-49d8-834c-05dd06ea03f8"). InnerVolumeSpecName "kube-api-access-bqwbf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 15:03:32.989393 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:32.989321    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb3422dc-476a-49d8-834c-05dd06ea03f8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bb3422dc-476a-49d8-834c-05dd06ea03f8" (UID: "bb3422dc-476a-49d8-834c-05dd06ea03f8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 15:03:33.088571 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:33.088540    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb3422dc-476a-49d8-834c-05dd06ea03f8-maas-api-tls\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:03:33.088571 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:33.088567    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb3422dc-476a-49d8-834c-05dd06ea03f8-openshift-service-ca\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:03:33.088571 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:33.088577    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqwbf\" (UniqueName: \"kubernetes.io/projected/bb3422dc-476a-49d8-834c-05dd06ea03f8-kube-api-access-bqwbf\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:03:33.228180 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:33.228138    2579 generic.go:356] "Generic (PLEG): container finished" podID="bb3422dc-476a-49d8-834c-05dd06ea03f8" containerID="6879d014e8a2ca344c6a6f634b875c7d103447b85a8aec06d0e19a455ce4cde6" exitCode=0
Jul 21 15:03:33.228180 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:33.228183    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-068a27f9-75dd5cd684-sfr55"
Jul 21 15:03:33.228454 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:33.228220    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-068a27f9-75dd5cd684-sfr55" event={"ID":"bb3422dc-476a-49d8-834c-05dd06ea03f8","Type":"ContainerDied","Data":"6879d014e8a2ca344c6a6f634b875c7d103447b85a8aec06d0e19a455ce4cde6"}
Jul 21 15:03:33.228454 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:33.228301    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-068a27f9-75dd5cd684-sfr55" event={"ID":"bb3422dc-476a-49d8-834c-05dd06ea03f8","Type":"ContainerDied","Data":"522deddda36c3d6ee8049f84f971597f6bfd6788154fb15cdfaa315ddd287298"}
Jul 21 15:03:33.228454 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:33.228334    2579 scope.go:117] "RemoveContainer" containerID="6879d014e8a2ca344c6a6f634b875c7d103447b85a8aec06d0e19a455ce4cde6"
Jul 21 15:03:33.238182 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:33.238164    2579 scope.go:117] "RemoveContainer" containerID="6879d014e8a2ca344c6a6f634b875c7d103447b85a8aec06d0e19a455ce4cde6"
Jul 21 15:03:33.238505 ip-10-0-130-83 kubenswrapper[2579]: E0721 15:03:33.238487    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6879d014e8a2ca344c6a6f634b875c7d103447b85a8aec06d0e19a455ce4cde6\": container with ID starting with 6879d014e8a2ca344c6a6f634b875c7d103447b85a8aec06d0e19a455ce4cde6 not found: ID does not exist" containerID="6879d014e8a2ca344c6a6f634b875c7d103447b85a8aec06d0e19a455ce4cde6"
Jul 21 15:03:33.238557 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:33.238515    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6879d014e8a2ca344c6a6f634b875c7d103447b85a8aec06d0e19a455ce4cde6"} err="failed to get container status \"6879d014e8a2ca344c6a6f634b875c7d103447b85a8aec06d0e19a455ce4cde6\": rpc error: code = NotFound desc = could not find container \"6879d014e8a2ca344c6a6f634b875c7d103447b85a8aec06d0e19a455ce4cde6\": container with ID starting with 6879d014e8a2ca344c6a6f634b875c7d103447b85a8aec06d0e19a455ce4cde6 not found: ID does not exist"
Jul 21 15:03:33.249526 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:33.249500    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-068a27f9-75dd5cd684-sfr55"]
Jul 21 15:03:33.253214 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:33.253189    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-068a27f9-75dd5cd684-sfr55"]
Jul 21 15:03:34.171863 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:34.171826    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb3422dc-476a-49d8-834c-05dd06ea03f8" path="/var/lib/kubelet/pods/bb3422dc-476a-49d8-834c-05dd06ea03f8/volumes"
Jul 21 15:03:37.302790 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:37.302753    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-068a27f9-97bw9"]
Jul 21 15:03:37.305120 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:37.305097    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-068a27f9-97bw9"]
Jul 21 15:03:38.172593 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:03:38.172559    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81c158a3-05a4-4ea3-aa50-031778aaa654" path="/var/lib/kubelet/pods/81c158a3-05a4-4ea3-aa50-031778aaa654/volumes"
Jul 21 15:05:23.095402 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:23.095326    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8f17ef39-58f8dbffdd-x546v"]
Jul 21 15:05:23.097498 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:23.095675    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb3422dc-476a-49d8-834c-05dd06ea03f8" containerName="maas-api"
Jul 21 15:05:23.097498 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:23.095687    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb3422dc-476a-49d8-834c-05dd06ea03f8" containerName="maas-api"
Jul 21 15:05:23.097498 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:23.095703    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81c158a3-05a4-4ea3-aa50-031778aaa654" containerName="revoke-keys"
Jul 21 15:05:23.097498 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:23.095709    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="81c158a3-05a4-4ea3-aa50-031778aaa654" containerName="revoke-keys"
Jul 21 15:05:23.097498 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:23.095764    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="81c158a3-05a4-4ea3-aa50-031778aaa654" containerName="revoke-keys"
Jul 21 15:05:23.097498 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:23.095773    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb3422dc-476a-49d8-834c-05dd06ea03f8" containerName="maas-api"
Jul 21 15:05:23.098547 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:23.098531    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f17ef39-58f8dbffdd-x546v"
Jul 21 15:05:23.101364 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:23.101336    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-8f17ef39-serving-cert\""
Jul 21 15:05:23.110080 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:23.110056    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8f17ef39-58f8dbffdd-x546v"]
Jul 21 15:05:23.132064 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:23.132036    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4zhw\" (UniqueName: \"kubernetes.io/projected/ce4764fc-e7b2-499d-829f-67636f634305-kube-api-access-v4zhw\") pod \"maas-api-e2e-mt-8f17ef39-58f8dbffdd-x546v\" (UID: \"ce4764fc-e7b2-499d-829f-67636f634305\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f17ef39-58f8dbffdd-x546v"
Jul 21 15:05:23.132201 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:23.132069    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce4764fc-e7b2-499d-829f-67636f634305-maas-api-tls\") pod \"maas-api-e2e-mt-8f17ef39-58f8dbffdd-x546v\" (UID: \"ce4764fc-e7b2-499d-829f-67636f634305\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f17ef39-58f8dbffdd-x546v"
Jul 21 15:05:23.132201 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:23.132177    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce4764fc-e7b2-499d-829f-67636f634305-openshift-service-ca\") pod \"maas-api-e2e-mt-8f17ef39-58f8dbffdd-x546v\" (UID: \"ce4764fc-e7b2-499d-829f-67636f634305\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f17ef39-58f8dbffdd-x546v"
Jul 21 15:05:23.232725 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:23.232686    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce4764fc-e7b2-499d-829f-67636f634305-openshift-service-ca\") pod \"maas-api-e2e-mt-8f17ef39-58f8dbffdd-x546v\" (UID: \"ce4764fc-e7b2-499d-829f-67636f634305\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f17ef39-58f8dbffdd-x546v"
Jul 21 15:05:23.232929 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:23.232761    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4zhw\" (UniqueName: \"kubernetes.io/projected/ce4764fc-e7b2-499d-829f-67636f634305-kube-api-access-v4zhw\") pod \"maas-api-e2e-mt-8f17ef39-58f8dbffdd-x546v\" (UID: \"ce4764fc-e7b2-499d-829f-67636f634305\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f17ef39-58f8dbffdd-x546v"
Jul 21 15:05:23.232929 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:23.232798    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce4764fc-e7b2-499d-829f-67636f634305-maas-api-tls\") pod \"maas-api-e2e-mt-8f17ef39-58f8dbffdd-x546v\" (UID: \"ce4764fc-e7b2-499d-829f-67636f634305\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f17ef39-58f8dbffdd-x546v"
Jul 21 15:05:23.233018 ip-10-0-130-83 kubenswrapper[2579]: E0721 15:05:23.232955    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-8f17ef39-serving-cert: secret "maas-api-e2e-mt-8f17ef39-serving-cert" not found
Jul 21 15:05:23.233053 ip-10-0-130-83 kubenswrapper[2579]: E0721 15:05:23.233020    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce4764fc-e7b2-499d-829f-67636f634305-maas-api-tls podName:ce4764fc-e7b2-499d-829f-67636f634305 nodeName:}" failed. No retries permitted until 2026-07-21 15:05:23.733004334 +0000 UTC m=+1980.082747724 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ce4764fc-e7b2-499d-829f-67636f634305-maas-api-tls") pod "maas-api-e2e-mt-8f17ef39-58f8dbffdd-x546v" (UID: "ce4764fc-e7b2-499d-829f-67636f634305") : secret "maas-api-e2e-mt-8f17ef39-serving-cert" not found
Jul 21 15:05:23.233455 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:23.233435    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce4764fc-e7b2-499d-829f-67636f634305-openshift-service-ca\") pod \"maas-api-e2e-mt-8f17ef39-58f8dbffdd-x546v\" (UID: \"ce4764fc-e7b2-499d-829f-67636f634305\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f17ef39-58f8dbffdd-x546v"
Jul 21 15:05:23.242534 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:23.242507    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4zhw\" (UniqueName: \"kubernetes.io/projected/ce4764fc-e7b2-499d-829f-67636f634305-kube-api-access-v4zhw\") pod \"maas-api-e2e-mt-8f17ef39-58f8dbffdd-x546v\" (UID: \"ce4764fc-e7b2-499d-829f-67636f634305\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f17ef39-58f8dbffdd-x546v"
Jul 21 15:05:23.736440 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:23.736400    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce4764fc-e7b2-499d-829f-67636f634305-maas-api-tls\") pod \"maas-api-e2e-mt-8f17ef39-58f8dbffdd-x546v\" (UID: \"ce4764fc-e7b2-499d-829f-67636f634305\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f17ef39-58f8dbffdd-x546v"
Jul 21 15:05:23.738783 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:23.738762    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce4764fc-e7b2-499d-829f-67636f634305-maas-api-tls\") pod \"maas-api-e2e-mt-8f17ef39-58f8dbffdd-x546v\" (UID: \"ce4764fc-e7b2-499d-829f-67636f634305\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f17ef39-58f8dbffdd-x546v"
Jul 21 15:05:24.009154 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:24.009071    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f17ef39-58f8dbffdd-x546v"
Jul 21 15:05:24.142472 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:24.142439    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8f17ef39-58f8dbffdd-x546v"]
Jul 21 15:05:24.145757 ip-10-0-130-83 kubenswrapper[2579]: W0721 15:05:24.145726    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce4764fc_e7b2_499d_829f_67636f634305.slice/crio-806edd7e19bcda41f49ccf11855c0041edb6fff7b326893d6128bc3573499669 WatchSource:0}: Error finding container 806edd7e19bcda41f49ccf11855c0041edb6fff7b326893d6128bc3573499669: Status 404 returned error can't find the container with id 806edd7e19bcda41f49ccf11855c0041edb6fff7b326893d6128bc3573499669
Jul 21 15:05:24.748830 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:24.748801    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f17ef39-58f8dbffdd-x546v" event={"ID":"ce4764fc-e7b2-499d-829f-67636f634305","Type":"ContainerStarted","Data":"806edd7e19bcda41f49ccf11855c0041edb6fff7b326893d6128bc3573499669"}
Jul 21 15:05:25.754754 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:25.754723    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f17ef39-58f8dbffdd-x546v" event={"ID":"ce4764fc-e7b2-499d-829f-67636f634305","Type":"ContainerStarted","Data":"e0ff13816a755485c4b9c94e71911c53c1d1b0876643db5b84eadb261cc391c1"}
Jul 21 15:05:25.755137 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:25.754834    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f17ef39-58f8dbffdd-x546v"
Jul 21 15:05:25.774820 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:25.774774    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f17ef39-58f8dbffdd-x546v" podStartSLOduration=2.206228543 podStartE2EDuration="2.774760711s" totalImagesPullingTime="568.532168ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 15:05:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 15:05:25.772312403 +0000 UTC m=+1982.122055815" watchObservedRunningTime="2026-07-21 15:05:25.774760711 +0000 UTC m=+1982.124504123"
Jul 21 15:05:31.765673 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:31.765644    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f17ef39-58f8dbffdd-x546v"
Jul 21 15:05:34.464955 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:34.464917    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8f17ef39-df8bk"]
Jul 21 15:05:34.468596 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:34.468580    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8f17ef39-df8bk"
Jul 21 15:05:34.471563 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:34.471546    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-494sx\""
Jul 21 15:05:34.477305 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:34.477283    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8f17ef39-df8bk"]
Jul 21 15:05:34.523972 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:34.523945    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1943539e-7ca4-49ca-a7dc-8ba8e2a5cdf3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-8f17ef39-df8bk\" (UID: \"1943539e-7ca4-49ca-a7dc-8ba8e2a5cdf3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8f17ef39-df8bk"
Jul 21 15:05:34.624499 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:34.624469    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1943539e-7ca4-49ca-a7dc-8ba8e2a5cdf3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-8f17ef39-df8bk\" (UID: \"1943539e-7ca4-49ca-a7dc-8ba8e2a5cdf3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8f17ef39-df8bk"
Jul 21 15:05:34.625036 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:34.625018    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1943539e-7ca4-49ca-a7dc-8ba8e2a5cdf3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-8f17ef39-df8bk\" (UID: \"1943539e-7ca4-49ca-a7dc-8ba8e2a5cdf3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8f17ef39-df8bk"
Jul 21 15:05:34.779412 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:34.779340    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8f17ef39-df8bk"
Jul 21 15:05:34.910161 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:34.910139    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8f17ef39-df8bk"]
Jul 21 15:05:34.911957 ip-10-0-130-83 kubenswrapper[2579]: W0721 15:05:34.911926    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1943539e_7ca4_49ca_a7dc_8ba8e2a5cdf3.slice/crio-580e60c2d83cc07bae0a252f2701858ab2a36c1adea2e882a19cb1b3ff3968e1 WatchSource:0}: Error finding container 580e60c2d83cc07bae0a252f2701858ab2a36c1adea2e882a19cb1b3ff3968e1: Status 404 returned error can't find the container with id 580e60c2d83cc07bae0a252f2701858ab2a36c1adea2e882a19cb1b3ff3968e1
Jul 21 15:05:35.806649 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:35.806622    2579 generic.go:356] "Generic (PLEG): container finished" podID="1943539e-7ca4-49ca-a7dc-8ba8e2a5cdf3" containerID="75270c0dad8bbc921854cc48b4c9e58dedb8c7d5445d78f01d4edf4de53397c6" exitCode=0
Jul 21 15:05:35.806997 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:35.806683    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8f17ef39-df8bk" event={"ID":"1943539e-7ca4-49ca-a7dc-8ba8e2a5cdf3","Type":"ContainerDied","Data":"75270c0dad8bbc921854cc48b4c9e58dedb8c7d5445d78f01d4edf4de53397c6"}
Jul 21 15:05:35.806997 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:35.806721    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8f17ef39-df8bk" event={"ID":"1943539e-7ca4-49ca-a7dc-8ba8e2a5cdf3","Type":"ContainerStarted","Data":"580e60c2d83cc07bae0a252f2701858ab2a36c1adea2e882a19cb1b3ff3968e1"}
Jul 21 15:05:36.943425 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:36.943403    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8f17ef39-df8bk"
Jul 21 15:05:37.046079 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:37.046052    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1943539e-7ca4-49ca-a7dc-8ba8e2a5cdf3-maas-api-service-ca\") pod \"1943539e-7ca4-49ca-a7dc-8ba8e2a5cdf3\" (UID: \"1943539e-7ca4-49ca-a7dc-8ba8e2a5cdf3\") "
Jul 21 15:05:37.046399 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:37.046377    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1943539e-7ca4-49ca-a7dc-8ba8e2a5cdf3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1943539e-7ca4-49ca-a7dc-8ba8e2a5cdf3" (UID: "1943539e-7ca4-49ca-a7dc-8ba8e2a5cdf3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 15:05:37.147229 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:37.147204    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1943539e-7ca4-49ca-a7dc-8ba8e2a5cdf3-maas-api-service-ca\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:05:37.819482 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:37.819444    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8f17ef39-df8bk" event={"ID":"1943539e-7ca4-49ca-a7dc-8ba8e2a5cdf3","Type":"ContainerDied","Data":"580e60c2d83cc07bae0a252f2701858ab2a36c1adea2e882a19cb1b3ff3968e1"}
Jul 21 15:05:37.819482 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:37.819471    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8f17ef39-df8bk"
Jul 21 15:05:37.819694 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:37.819494    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="580e60c2d83cc07bae0a252f2701858ab2a36c1adea2e882a19cb1b3ff3968e1"
Jul 21 15:05:45.121512 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:45.121477    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8f17ef39-58f8dbffdd-x546v"]
Jul 21 15:05:45.121985 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:45.121719    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f17ef39-58f8dbffdd-x546v" podUID="ce4764fc-e7b2-499d-829f-67636f634305" containerName="maas-api" containerID="cri-o://e0ff13816a755485c4b9c94e71911c53c1d1b0876643db5b84eadb261cc391c1" gracePeriod=30
Jul 21 15:05:45.373314 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:45.373255    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f17ef39-58f8dbffdd-x546v"
Jul 21 15:05:45.419649 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:45.419619    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4zhw\" (UniqueName: \"kubernetes.io/projected/ce4764fc-e7b2-499d-829f-67636f634305-kube-api-access-v4zhw\") pod \"ce4764fc-e7b2-499d-829f-67636f634305\" (UID: \"ce4764fc-e7b2-499d-829f-67636f634305\") "
Jul 21 15:05:45.419826 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:45.419667    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce4764fc-e7b2-499d-829f-67636f634305-maas-api-tls\") pod \"ce4764fc-e7b2-499d-829f-67636f634305\" (UID: \"ce4764fc-e7b2-499d-829f-67636f634305\") "
Jul 21 15:05:45.419826 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:45.419748    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce4764fc-e7b2-499d-829f-67636f634305-openshift-service-ca\") pod \"ce4764fc-e7b2-499d-829f-67636f634305\" (UID: \"ce4764fc-e7b2-499d-829f-67636f634305\") "
Jul 21 15:05:45.420134 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:45.420111    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce4764fc-e7b2-499d-829f-67636f634305-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ce4764fc-e7b2-499d-829f-67636f634305" (UID: "ce4764fc-e7b2-499d-829f-67636f634305"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 15:05:45.421652 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:45.421626    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce4764fc-e7b2-499d-829f-67636f634305-kube-api-access-v4zhw" (OuterVolumeSpecName: "kube-api-access-v4zhw") pod "ce4764fc-e7b2-499d-829f-67636f634305" (UID: "ce4764fc-e7b2-499d-829f-67636f634305"). InnerVolumeSpecName "kube-api-access-v4zhw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 15:05:45.421728 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:45.421648    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce4764fc-e7b2-499d-829f-67636f634305-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ce4764fc-e7b2-499d-829f-67636f634305" (UID: "ce4764fc-e7b2-499d-829f-67636f634305"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 15:05:45.521303 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:45.521251    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce4764fc-e7b2-499d-829f-67636f634305-openshift-service-ca\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:05:45.521303 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:45.521299    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4zhw\" (UniqueName: \"kubernetes.io/projected/ce4764fc-e7b2-499d-829f-67636f634305-kube-api-access-v4zhw\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:05:45.521478 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:45.521311    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce4764fc-e7b2-499d-829f-67636f634305-maas-api-tls\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:05:45.858658 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:45.858627    2579 generic.go:356] "Generic (PLEG): container finished" podID="ce4764fc-e7b2-499d-829f-67636f634305" containerID="e0ff13816a755485c4b9c94e71911c53c1d1b0876643db5b84eadb261cc391c1" exitCode=0
Jul 21 15:05:45.858816 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:45.858665    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f17ef39-58f8dbffdd-x546v"
Jul 21 15:05:45.858816 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:45.858706    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f17ef39-58f8dbffdd-x546v" event={"ID":"ce4764fc-e7b2-499d-829f-67636f634305","Type":"ContainerDied","Data":"e0ff13816a755485c4b9c94e71911c53c1d1b0876643db5b84eadb261cc391c1"}
Jul 21 15:05:45.858816 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:45.858762    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8f17ef39-58f8dbffdd-x546v" event={"ID":"ce4764fc-e7b2-499d-829f-67636f634305","Type":"ContainerDied","Data":"806edd7e19bcda41f49ccf11855c0041edb6fff7b326893d6128bc3573499669"}
Jul 21 15:05:45.858816 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:45.858787    2579 scope.go:117] "RemoveContainer" containerID="e0ff13816a755485c4b9c94e71911c53c1d1b0876643db5b84eadb261cc391c1"
Jul 21 15:05:45.867623 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:45.867606    2579 scope.go:117] "RemoveContainer" containerID="e0ff13816a755485c4b9c94e71911c53c1d1b0876643db5b84eadb261cc391c1"
Jul 21 15:05:45.867873 ip-10-0-130-83 kubenswrapper[2579]: E0721 15:05:45.867854    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0ff13816a755485c4b9c94e71911c53c1d1b0876643db5b84eadb261cc391c1\": container with ID starting with e0ff13816a755485c4b9c94e71911c53c1d1b0876643db5b84eadb261cc391c1 not found: ID does not exist" containerID="e0ff13816a755485c4b9c94e71911c53c1d1b0876643db5b84eadb261cc391c1"
Jul 21 15:05:45.867956 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:45.867884    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0ff13816a755485c4b9c94e71911c53c1d1b0876643db5b84eadb261cc391c1"} err="failed to get container status \"e0ff13816a755485c4b9c94e71911c53c1d1b0876643db5b84eadb261cc391c1\": rpc error: code = NotFound desc = could not find container \"e0ff13816a755485c4b9c94e71911c53c1d1b0876643db5b84eadb261cc391c1\": container with ID starting with e0ff13816a755485c4b9c94e71911c53c1d1b0876643db5b84eadb261cc391c1 not found: ID does not exist"
Jul 21 15:05:45.879547 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:45.879527    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8f17ef39-58f8dbffdd-x546v"]
Jul 21 15:05:45.883610 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:45.883585    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8f17ef39-58f8dbffdd-x546v"]
Jul 21 15:05:46.171444 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:46.171415    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce4764fc-e7b2-499d-829f-67636f634305" path="/var/lib/kubelet/pods/ce4764fc-e7b2-499d-829f-67636f634305/volumes"
Jul 21 15:05:49.626017 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:49.625982    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8f17ef39-df8bk"]
Jul 21 15:05:49.629031 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:49.629008    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8f17ef39-df8bk"]
Jul 21 15:05:50.171894 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:05:50.171859    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1943539e-7ca4-49ca-a7dc-8ba8e2a5cdf3" path="/var/lib/kubelet/pods/1943539e-7ca4-49ca-a7dc-8ba8e2a5cdf3/volumes"
Jul 21 15:06:19.090189 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:19.090154    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5e94d052-d7f7499b9-2r9n2"]
Jul 21 15:06:19.090790 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:19.090756    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce4764fc-e7b2-499d-829f-67636f634305" containerName="maas-api"
Jul 21 15:06:19.090790 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:19.090775    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce4764fc-e7b2-499d-829f-67636f634305" containerName="maas-api"
Jul 21 15:06:19.090913 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:19.090790    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1943539e-7ca4-49ca-a7dc-8ba8e2a5cdf3" containerName="revoke-keys"
Jul 21 15:06:19.090913 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:19.090800    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1943539e-7ca4-49ca-a7dc-8ba8e2a5cdf3" containerName="revoke-keys"
Jul 21 15:06:19.090913 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:19.090896    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1943539e-7ca4-49ca-a7dc-8ba8e2a5cdf3" containerName="revoke-keys"
Jul 21 15:06:19.090913 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:19.090908    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce4764fc-e7b2-499d-829f-67636f634305" containerName="maas-api"
Jul 21 15:06:19.094492 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:19.094473    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e94d052-d7f7499b9-2r9n2"
Jul 21 15:06:19.098536 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:19.098503    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-5e94d052-serving-cert\""
Jul 21 15:06:19.107751 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:19.107729    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5e94d052-d7f7499b9-2r9n2"]
Jul 21 15:06:19.111606 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:19.111584    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p48zk\" (UniqueName: \"kubernetes.io/projected/0b5adc09-b1de-4d0a-b78f-45a354bbe23b-kube-api-access-p48zk\") pod \"maas-api-e2e-mt-5e94d052-d7f7499b9-2r9n2\" (UID: \"0b5adc09-b1de-4d0a-b78f-45a354bbe23b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e94d052-d7f7499b9-2r9n2"
Jul 21 15:06:19.111703 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:19.111632    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b5adc09-b1de-4d0a-b78f-45a354bbe23b-openshift-service-ca\") pod \"maas-api-e2e-mt-5e94d052-d7f7499b9-2r9n2\" (UID: \"0b5adc09-b1de-4d0a-b78f-45a354bbe23b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e94d052-d7f7499b9-2r9n2"
Jul 21 15:06:19.111775 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:19.111742    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b5adc09-b1de-4d0a-b78f-45a354bbe23b-maas-api-tls\") pod \"maas-api-e2e-mt-5e94d052-d7f7499b9-2r9n2\" (UID: \"0b5adc09-b1de-4d0a-b78f-45a354bbe23b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e94d052-d7f7499b9-2r9n2"
Jul 21 15:06:19.212514 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:19.212474    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p48zk\" (UniqueName: \"kubernetes.io/projected/0b5adc09-b1de-4d0a-b78f-45a354bbe23b-kube-api-access-p48zk\") pod \"maas-api-e2e-mt-5e94d052-d7f7499b9-2r9n2\" (UID: \"0b5adc09-b1de-4d0a-b78f-45a354bbe23b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e94d052-d7f7499b9-2r9n2"
Jul 21 15:06:19.212706 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:19.212555    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b5adc09-b1de-4d0a-b78f-45a354bbe23b-openshift-service-ca\") pod \"maas-api-e2e-mt-5e94d052-d7f7499b9-2r9n2\" (UID: \"0b5adc09-b1de-4d0a-b78f-45a354bbe23b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e94d052-d7f7499b9-2r9n2"
Jul 21 15:06:19.212706 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:19.212691    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b5adc09-b1de-4d0a-b78f-45a354bbe23b-maas-api-tls\") pod \"maas-api-e2e-mt-5e94d052-d7f7499b9-2r9n2\" (UID: \"0b5adc09-b1de-4d0a-b78f-45a354bbe23b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e94d052-d7f7499b9-2r9n2"
Jul 21 15:06:19.212835 ip-10-0-130-83 kubenswrapper[2579]: E0721 15:06:19.212813    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-5e94d052-serving-cert: secret "maas-api-e2e-mt-5e94d052-serving-cert" not found
Jul 21 15:06:19.212901 ip-10-0-130-83 kubenswrapper[2579]: E0721 15:06:19.212888    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b5adc09-b1de-4d0a-b78f-45a354bbe23b-maas-api-tls podName:0b5adc09-b1de-4d0a-b78f-45a354bbe23b nodeName:}" failed. No retries permitted until 2026-07-21 15:06:19.712869181 +0000 UTC m=+2036.062612571 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0b5adc09-b1de-4d0a-b78f-45a354bbe23b-maas-api-tls") pod "maas-api-e2e-mt-5e94d052-d7f7499b9-2r9n2" (UID: "0b5adc09-b1de-4d0a-b78f-45a354bbe23b") : secret "maas-api-e2e-mt-5e94d052-serving-cert" not found
Jul 21 15:06:19.213321 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:19.213264    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b5adc09-b1de-4d0a-b78f-45a354bbe23b-openshift-service-ca\") pod \"maas-api-e2e-mt-5e94d052-d7f7499b9-2r9n2\" (UID: \"0b5adc09-b1de-4d0a-b78f-45a354bbe23b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e94d052-d7f7499b9-2r9n2"
Jul 21 15:06:19.223260 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:19.223232    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p48zk\" (UniqueName: \"kubernetes.io/projected/0b5adc09-b1de-4d0a-b78f-45a354bbe23b-kube-api-access-p48zk\") pod \"maas-api-e2e-mt-5e94d052-d7f7499b9-2r9n2\" (UID: \"0b5adc09-b1de-4d0a-b78f-45a354bbe23b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e94d052-d7f7499b9-2r9n2"
Jul 21 15:06:19.716565 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:19.716529    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b5adc09-b1de-4d0a-b78f-45a354bbe23b-maas-api-tls\") pod \"maas-api-e2e-mt-5e94d052-d7f7499b9-2r9n2\" (UID: \"0b5adc09-b1de-4d0a-b78f-45a354bbe23b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e94d052-d7f7499b9-2r9n2"
Jul 21 15:06:19.719144 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:19.719110    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b5adc09-b1de-4d0a-b78f-45a354bbe23b-maas-api-tls\") pod \"maas-api-e2e-mt-5e94d052-d7f7499b9-2r9n2\" (UID: \"0b5adc09-b1de-4d0a-b78f-45a354bbe23b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e94d052-d7f7499b9-2r9n2"
Jul 21 15:06:20.004706 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:20.004611    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e94d052-d7f7499b9-2r9n2"
Jul 21 15:06:20.140172 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:20.140148    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5e94d052-d7f7499b9-2r9n2"]
Jul 21 15:06:20.142947 ip-10-0-130-83 kubenswrapper[2579]: W0721 15:06:20.142917    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b5adc09_b1de_4d0a_b78f_45a354bbe23b.slice/crio-f1f8d13774b25232727fdc4664d54687e890f6348a47b69e84c6b9c765e8f336 WatchSource:0}: Error finding container f1f8d13774b25232727fdc4664d54687e890f6348a47b69e84c6b9c765e8f336: Status 404 returned error can't find the container with id f1f8d13774b25232727fdc4664d54687e890f6348a47b69e84c6b9c765e8f336
Jul 21 15:06:20.144228 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:20.144211    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 15:06:21.032304 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:21.032260    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e94d052-d7f7499b9-2r9n2" event={"ID":"0b5adc09-b1de-4d0a-b78f-45a354bbe23b","Type":"ContainerStarted","Data":"3dc04e8e86bd60e872659d3164c6229e8d7797f977122943e8de987c5ac25897"}
Jul 21 15:06:21.032464 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:21.032317    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e94d052-d7f7499b9-2r9n2" event={"ID":"0b5adc09-b1de-4d0a-b78f-45a354bbe23b","Type":"ContainerStarted","Data":"f1f8d13774b25232727fdc4664d54687e890f6348a47b69e84c6b9c765e8f336"}
Jul 21 15:06:21.032464 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:21.032399    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e94d052-d7f7499b9-2r9n2"
Jul 21 15:06:21.050072 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:21.050003    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e94d052-d7f7499b9-2r9n2" podStartSLOduration=1.61061137 podStartE2EDuration="2.049986866s" totalImagesPullingTime="439.375496ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 15:06:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 15:06:21.049769843 +0000 UTC m=+2037.399513276" watchObservedRunningTime="2026-07-21 15:06:21.049986866 +0000 UTC m=+2037.399730281"
Jul 21 15:06:27.043833 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:27.043802    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e94d052-d7f7499b9-2r9n2"
Jul 21 15:06:51.703740 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:51.703646    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5e94d052-mxhr5"]
Jul 21 15:06:51.707242 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:51.707222    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5e94d052-mxhr5"
Jul 21 15:06:51.709746 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:51.709731    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-494sx\""
Jul 21 15:06:51.714776 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:51.714753    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5e94d052-mxhr5"]
Jul 21 15:06:51.781664 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:51.781617    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5311002-7052-4f15-9018-84502b301443-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-5e94d052-mxhr5\" (UID: \"c5311002-7052-4f15-9018-84502b301443\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5e94d052-mxhr5"
Jul 21 15:06:51.882676 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:51.882638    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5311002-7052-4f15-9018-84502b301443-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-5e94d052-mxhr5\" (UID: \"c5311002-7052-4f15-9018-84502b301443\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5e94d052-mxhr5"
Jul 21 15:06:51.883240 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:51.883219    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5311002-7052-4f15-9018-84502b301443-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-5e94d052-mxhr5\" (UID: \"c5311002-7052-4f15-9018-84502b301443\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5e94d052-mxhr5"
Jul 21 15:06:52.019148 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:52.019059    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5e94d052-mxhr5"
Jul 21 15:06:52.145918 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:52.145885    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5e94d052-mxhr5"]
Jul 21 15:06:52.149005 ip-10-0-130-83 kubenswrapper[2579]: W0721 15:06:52.148978    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5311002_7052_4f15_9018_84502b301443.slice/crio-cd613e8746570d9ba367de4738d13e16a816c1fa7c0c63f5fa1f82dbf7189492 WatchSource:0}: Error finding container cd613e8746570d9ba367de4738d13e16a816c1fa7c0c63f5fa1f82dbf7189492: Status 404 returned error can't find the container with id cd613e8746570d9ba367de4738d13e16a816c1fa7c0c63f5fa1f82dbf7189492
Jul 21 15:06:52.181914 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:52.181885    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5e94d052-mxhr5" event={"ID":"c5311002-7052-4f15-9018-84502b301443","Type":"ContainerStarted","Data":"cd613e8746570d9ba367de4738d13e16a816c1fa7c0c63f5fa1f82dbf7189492"}
Jul 21 15:06:53.187914 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:53.187882    2579 generic.go:356] "Generic (PLEG): container finished" podID="c5311002-7052-4f15-9018-84502b301443" containerID="d26ec526352184ab46e9bcdc5295f19e85ca36ccbaa45184ce6f05c804dd0267" exitCode=0
Jul 21 15:06:53.188297 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:53.187965    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5e94d052-mxhr5" event={"ID":"c5311002-7052-4f15-9018-84502b301443","Type":"ContainerDied","Data":"d26ec526352184ab46e9bcdc5295f19e85ca36ccbaa45184ce6f05c804dd0267"}
Jul 21 15:06:54.319612 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:54.319588    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5e94d052-mxhr5"
Jul 21 15:06:54.406028 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:54.406003    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5311002-7052-4f15-9018-84502b301443-maas-api-service-ca\") pod \"c5311002-7052-4f15-9018-84502b301443\" (UID: \"c5311002-7052-4f15-9018-84502b301443\") "
Jul 21 15:06:54.406370 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:54.406341    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5311002-7052-4f15-9018-84502b301443-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c5311002-7052-4f15-9018-84502b301443" (UID: "c5311002-7052-4f15-9018-84502b301443"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 15:06:54.507185 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:54.507119    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5311002-7052-4f15-9018-84502b301443-maas-api-service-ca\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:06:55.197866 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:55.197837    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5e94d052-mxhr5"
Jul 21 15:06:55.197866 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:55.197860    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5e94d052-mxhr5" event={"ID":"c5311002-7052-4f15-9018-84502b301443","Type":"ContainerDied","Data":"cd613e8746570d9ba367de4738d13e16a816c1fa7c0c63f5fa1f82dbf7189492"}
Jul 21 15:06:55.198122 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:06:55.197899    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cd613e8746570d9ba367de4738d13e16a816c1fa7c0c63f5fa1f82dbf7189492"
Jul 21 15:07:07.149466 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:07:07.149433    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5e94d052-d7f7499b9-2r9n2"]
Jul 21 15:07:07.149991 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:07:07.149636    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e94d052-d7f7499b9-2r9n2" podUID="0b5adc09-b1de-4d0a-b78f-45a354bbe23b" containerName="maas-api" containerID="cri-o://3dc04e8e86bd60e872659d3164c6229e8d7797f977122943e8de987c5ac25897" gracePeriod=30
Jul 21 15:07:07.399140 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:07:07.399117    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e94d052-d7f7499b9-2r9n2"
Jul 21 15:07:07.416906 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:07:07.416844    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p48zk\" (UniqueName: \"kubernetes.io/projected/0b5adc09-b1de-4d0a-b78f-45a354bbe23b-kube-api-access-p48zk\") pod \"0b5adc09-b1de-4d0a-b78f-45a354bbe23b\" (UID: \"0b5adc09-b1de-4d0a-b78f-45a354bbe23b\") "
Jul 21 15:07:07.417020 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:07:07.416936    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b5adc09-b1de-4d0a-b78f-45a354bbe23b-maas-api-tls\") pod \"0b5adc09-b1de-4d0a-b78f-45a354bbe23b\" (UID: \"0b5adc09-b1de-4d0a-b78f-45a354bbe23b\") "
Jul 21 15:07:07.417020 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:07:07.416986    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b5adc09-b1de-4d0a-b78f-45a354bbe23b-openshift-service-ca\") pod \"0b5adc09-b1de-4d0a-b78f-45a354bbe23b\" (UID: \"0b5adc09-b1de-4d0a-b78f-45a354bbe23b\") "
Jul 21 15:07:07.417612 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:07:07.417536    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b5adc09-b1de-4d0a-b78f-45a354bbe23b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0b5adc09-b1de-4d0a-b78f-45a354bbe23b" (UID: "0b5adc09-b1de-4d0a-b78f-45a354bbe23b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 15:07:07.419559 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:07:07.419503    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b5adc09-b1de-4d0a-b78f-45a354bbe23b-kube-api-access-p48zk" (OuterVolumeSpecName: "kube-api-access-p48zk") pod "0b5adc09-b1de-4d0a-b78f-45a354bbe23b" (UID: "0b5adc09-b1de-4d0a-b78f-45a354bbe23b"). InnerVolumeSpecName "kube-api-access-p48zk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 15:07:07.419559 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:07:07.419717    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b5adc09-b1de-4d0a-b78f-45a354bbe23b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0b5adc09-b1de-4d0a-b78f-45a354bbe23b" (UID: "0b5adc09-b1de-4d0a-b78f-45a354bbe23b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 15:07:07.517678 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:07:07.517651    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p48zk\" (UniqueName: \"kubernetes.io/projected/0b5adc09-b1de-4d0a-b78f-45a354bbe23b-kube-api-access-p48zk\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:07:07.517678 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:07:07.517675    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0b5adc09-b1de-4d0a-b78f-45a354bbe23b-maas-api-tls\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:07:07.517833 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:07:07.517685    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b5adc09-b1de-4d0a-b78f-45a354bbe23b-openshift-service-ca\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:07:08.259474 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:07:08.259440    2579 generic.go:356] "Generic (PLEG): container finished" podID="0b5adc09-b1de-4d0a-b78f-45a354bbe23b" containerID="3dc04e8e86bd60e872659d3164c6229e8d7797f977122943e8de987c5ac25897" exitCode=0
Jul 21 15:07:08.259474 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:07:08.259476    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e94d052-d7f7499b9-2r9n2"
Jul 21 15:07:08.259913 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:07:08.259518    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e94d052-d7f7499b9-2r9n2" event={"ID":"0b5adc09-b1de-4d0a-b78f-45a354bbe23b","Type":"ContainerDied","Data":"3dc04e8e86bd60e872659d3164c6229e8d7797f977122943e8de987c5ac25897"}
Jul 21 15:07:08.259913 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:07:08.259571    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5e94d052-d7f7499b9-2r9n2" event={"ID":"0b5adc09-b1de-4d0a-b78f-45a354bbe23b","Type":"ContainerDied","Data":"f1f8d13774b25232727fdc4664d54687e890f6348a47b69e84c6b9c765e8f336"}
Jul 21 15:07:08.259913 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:07:08.259605    2579 scope.go:117] "RemoveContainer" containerID="3dc04e8e86bd60e872659d3164c6229e8d7797f977122943e8de987c5ac25897"
Jul 21 15:07:08.268026 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:07:08.268003    2579 scope.go:117] "RemoveContainer" containerID="3dc04e8e86bd60e872659d3164c6229e8d7797f977122943e8de987c5ac25897"
Jul 21 15:07:08.268298 ip-10-0-130-83 kubenswrapper[2579]: E0721 15:07:08.268277    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3dc04e8e86bd60e872659d3164c6229e8d7797f977122943e8de987c5ac25897\": container with ID starting with 3dc04e8e86bd60e872659d3164c6229e8d7797f977122943e8de987c5ac25897 not found: ID does not exist" containerID="3dc04e8e86bd60e872659d3164c6229e8d7797f977122943e8de987c5ac25897"
Jul 21 15:07:08.268373 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:07:08.268307    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3dc04e8e86bd60e872659d3164c6229e8d7797f977122943e8de987c5ac25897"} err="failed to get container status \"3dc04e8e86bd60e872659d3164c6229e8d7797f977122943e8de987c5ac25897\": rpc error: code = NotFound desc = could not find container \"3dc04e8e86bd60e872659d3164c6229e8d7797f977122943e8de987c5ac25897\": container with ID starting with 3dc04e8e86bd60e872659d3164c6229e8d7797f977122943e8de987c5ac25897 not found: ID does not exist"
Jul 21 15:07:08.276407 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:07:08.276389    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5e94d052-d7f7499b9-2r9n2"]
Jul 21 15:07:08.279637 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:07:08.279612    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5e94d052-d7f7499b9-2r9n2"]
Jul 21 15:07:10.171557 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:07:10.171524    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b5adc09-b1de-4d0a-b78f-45a354bbe23b" path="/var/lib/kubelet/pods/0b5adc09-b1de-4d0a-b78f-45a354bbe23b/volumes"
Jul 21 15:07:11.775167 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:07:11.775135    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5e94d052-mxhr5"]
Jul 21 15:07:11.777537 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:07:11.777511    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5e94d052-mxhr5"]
Jul 21 15:07:12.171421 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:07:12.171389    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5311002-7052-4f15-9018-84502b301443" path="/var/lib/kubelet/pods/c5311002-7052-4f15-9018-84502b301443/volumes"
Jul 21 15:07:24.324482 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:07:24.324456    2579 scope.go:117] "RemoveContainer" containerID="49410cfebdee6674a0298f172625ef0732c3e7fd7fcb9c61066134706e4a3129"
Jul 21 15:08:24.347295 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:08:24.347199    2579 scope.go:117] "RemoveContainer" containerID="51a9c14f475a2bc5162668e0f436bf6322f86926b11d76639705961bd43275c1"
Jul 21 15:09:01.106368 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:01.106333    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d6d63a-6794cbf6cf-8qmmf"]
Jul 21 15:09:01.107412 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:01.107387    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5311002-7052-4f15-9018-84502b301443" containerName="revoke-keys"
Jul 21 15:09:01.107503 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:01.107414    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5311002-7052-4f15-9018-84502b301443" containerName="revoke-keys"
Jul 21 15:09:01.107503 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:01.107428    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b5adc09-b1de-4d0a-b78f-45a354bbe23b" containerName="maas-api"
Jul 21 15:09:01.107503 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:01.107436    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b5adc09-b1de-4d0a-b78f-45a354bbe23b" containerName="maas-api"
Jul 21 15:09:01.107670 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:01.107533    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b5adc09-b1de-4d0a-b78f-45a354bbe23b" containerName="maas-api"
Jul 21 15:09:01.107670 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:01.107546    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5311002-7052-4f15-9018-84502b301443" containerName="revoke-keys"
Jul 21 15:09:01.110805 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:01.110785    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d6d63a-6794cbf6cf-8qmmf"
Jul 21 15:09:01.113415 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:01.113395    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-d6d63a-serving-cert\""
Jul 21 15:09:01.120654 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:01.120625    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d6d63a-6794cbf6cf-8qmmf"]
Jul 21 15:09:01.233991 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:01.233949    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4488c28f-36c4-469e-8fe2-de9ae527651d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-d6d63a-6794cbf6cf-8qmmf\" (UID: \"4488c28f-36c4-469e-8fe2-de9ae527651d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d6d63a-6794cbf6cf-8qmmf"
Jul 21 15:09:01.234335 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:01.233995    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phqmb\" (UniqueName: \"kubernetes.io/projected/4488c28f-36c4-469e-8fe2-de9ae527651d-kube-api-access-phqmb\") pod \"maas-api-e2e-inf-a-d6d63a-6794cbf6cf-8qmmf\" (UID: \"4488c28f-36c4-469e-8fe2-de9ae527651d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d6d63a-6794cbf6cf-8qmmf"
Jul 21 15:09:01.234335 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:01.234066    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4488c28f-36c4-469e-8fe2-de9ae527651d-maas-api-tls\") pod \"maas-api-e2e-inf-a-d6d63a-6794cbf6cf-8qmmf\" (UID: \"4488c28f-36c4-469e-8fe2-de9ae527651d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d6d63a-6794cbf6cf-8qmmf"
Jul 21 15:09:01.335121 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:01.335088    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4488c28f-36c4-469e-8fe2-de9ae527651d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-d6d63a-6794cbf6cf-8qmmf\" (UID: \"4488c28f-36c4-469e-8fe2-de9ae527651d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d6d63a-6794cbf6cf-8qmmf"
Jul 21 15:09:01.335308 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:01.335135    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phqmb\" (UniqueName: \"kubernetes.io/projected/4488c28f-36c4-469e-8fe2-de9ae527651d-kube-api-access-phqmb\") pod \"maas-api-e2e-inf-a-d6d63a-6794cbf6cf-8qmmf\" (UID: \"4488c28f-36c4-469e-8fe2-de9ae527651d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d6d63a-6794cbf6cf-8qmmf"
Jul 21 15:09:01.335308 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:01.335190    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4488c28f-36c4-469e-8fe2-de9ae527651d-maas-api-tls\") pod \"maas-api-e2e-inf-a-d6d63a-6794cbf6cf-8qmmf\" (UID: \"4488c28f-36c4-469e-8fe2-de9ae527651d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d6d63a-6794cbf6cf-8qmmf"
Jul 21 15:09:01.335787 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:01.335768    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4488c28f-36c4-469e-8fe2-de9ae527651d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-d6d63a-6794cbf6cf-8qmmf\" (UID: \"4488c28f-36c4-469e-8fe2-de9ae527651d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d6d63a-6794cbf6cf-8qmmf"
Jul 21 15:09:01.337918 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:01.337891    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4488c28f-36c4-469e-8fe2-de9ae527651d-maas-api-tls\") pod \"maas-api-e2e-inf-a-d6d63a-6794cbf6cf-8qmmf\" (UID: \"4488c28f-36c4-469e-8fe2-de9ae527651d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d6d63a-6794cbf6cf-8qmmf"
Jul 21 15:09:01.343235 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:01.343212    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phqmb\" (UniqueName: \"kubernetes.io/projected/4488c28f-36c4-469e-8fe2-de9ae527651d-kube-api-access-phqmb\") pod \"maas-api-e2e-inf-a-d6d63a-6794cbf6cf-8qmmf\" (UID: \"4488c28f-36c4-469e-8fe2-de9ae527651d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d6d63a-6794cbf6cf-8qmmf"
Jul 21 15:09:01.424088 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:01.424067    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d6d63a-6794cbf6cf-8qmmf"
Jul 21 15:09:01.560819 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:01.560719    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d6d63a-6794cbf6cf-8qmmf"]
Jul 21 15:09:01.563737 ip-10-0-130-83 kubenswrapper[2579]: W0721 15:09:01.563710    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4488c28f_36c4_469e_8fe2_de9ae527651d.slice/crio-ef98ef3d3676dbd6ed935c95fe9ae6e9d2fed1a5944eeb092c170b49eff2e487 WatchSource:0}: Error finding container ef98ef3d3676dbd6ed935c95fe9ae6e9d2fed1a5944eeb092c170b49eff2e487: Status 404 returned error can't find the container with id ef98ef3d3676dbd6ed935c95fe9ae6e9d2fed1a5944eeb092c170b49eff2e487
Jul 21 15:09:01.801089 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:01.801008    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d6d63a-6794cbf6cf-8qmmf" event={"ID":"4488c28f-36c4-469e-8fe2-de9ae527651d","Type":"ContainerStarted","Data":"ef98ef3d3676dbd6ed935c95fe9ae6e9d2fed1a5944eeb092c170b49eff2e487"}
Jul 21 15:09:02.807069 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:02.806992    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d6d63a-6794cbf6cf-8qmmf" event={"ID":"4488c28f-36c4-469e-8fe2-de9ae527651d","Type":"ContainerStarted","Data":"8bcf549241a1c8db66b8acb075d91ed09a6d76fe73fa694fee3b51953c15301e"}
Jul 21 15:09:02.807446 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:02.807125    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d6d63a-6794cbf6cf-8qmmf"
Jul 21 15:09:02.825804 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:02.825758    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d6d63a-6794cbf6cf-8qmmf" podStartSLOduration=0.868076032 podStartE2EDuration="1.825741801s" totalImagesPullingTime="957.665769ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 15:09:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 15:09:02.824857638 +0000 UTC m=+2199.174601107" watchObservedRunningTime="2026-07-21 15:09:02.825741801 +0000 UTC m=+2199.175485213"
Jul 21 15:09:08.819398 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:08.819370    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d6d63a-6794cbf6cf-8qmmf"
Jul 21 15:09:09.205346 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:09.205311    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5ab9b1-f4db4f96b-zvvjx"]
Jul 21 15:09:09.209489 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:09.209466    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ab9b1-f4db4f96b-zvvjx"
Jul 21 15:09:09.212112 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:09.212091    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-5ab9b1-serving-cert\""
Jul 21 15:09:09.218837 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:09.218812    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5ab9b1-f4db4f96b-zvvjx"]
Jul 21 15:09:09.301774 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:09.301739    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab0b7792-3a3d-48f4-aef7-cd50a60275e0-maas-api-tls\") pod \"maas-api-e2e-inf-b-5ab9b1-f4db4f96b-zvvjx\" (UID: \"ab0b7792-3a3d-48f4-aef7-cd50a60275e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ab9b1-f4db4f96b-zvvjx"
Jul 21 15:09:09.301931 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:09.301896    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab0b7792-3a3d-48f4-aef7-cd50a60275e0-openshift-service-ca\") pod \"maas-api-e2e-inf-b-5ab9b1-f4db4f96b-zvvjx\" (UID: \"ab0b7792-3a3d-48f4-aef7-cd50a60275e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ab9b1-f4db4f96b-zvvjx"
Jul 21 15:09:09.301985 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:09.301934    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzq6w\" (UniqueName: \"kubernetes.io/projected/ab0b7792-3a3d-48f4-aef7-cd50a60275e0-kube-api-access-bzq6w\") pod \"maas-api-e2e-inf-b-5ab9b1-f4db4f96b-zvvjx\" (UID: \"ab0b7792-3a3d-48f4-aef7-cd50a60275e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ab9b1-f4db4f96b-zvvjx"
Jul 21 15:09:09.402828 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:09.402791    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab0b7792-3a3d-48f4-aef7-cd50a60275e0-openshift-service-ca\") pod \"maas-api-e2e-inf-b-5ab9b1-f4db4f96b-zvvjx\" (UID: \"ab0b7792-3a3d-48f4-aef7-cd50a60275e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ab9b1-f4db4f96b-zvvjx"
Jul 21 15:09:09.402828 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:09.402830    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzq6w\" (UniqueName: \"kubernetes.io/projected/ab0b7792-3a3d-48f4-aef7-cd50a60275e0-kube-api-access-bzq6w\") pod \"maas-api-e2e-inf-b-5ab9b1-f4db4f96b-zvvjx\" (UID: \"ab0b7792-3a3d-48f4-aef7-cd50a60275e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ab9b1-f4db4f96b-zvvjx"
Jul 21 15:09:09.403044 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:09.402861    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab0b7792-3a3d-48f4-aef7-cd50a60275e0-maas-api-tls\") pod \"maas-api-e2e-inf-b-5ab9b1-f4db4f96b-zvvjx\" (UID: \"ab0b7792-3a3d-48f4-aef7-cd50a60275e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ab9b1-f4db4f96b-zvvjx"
Jul 21 15:09:09.403044 ip-10-0-130-83 kubenswrapper[2579]: E0721 15:09:09.403029    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-5ab9b1-serving-cert: secret "maas-api-e2e-inf-b-5ab9b1-serving-cert" not found
Jul 21 15:09:09.403126 ip-10-0-130-83 kubenswrapper[2579]: E0721 15:09:09.403102    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab0b7792-3a3d-48f4-aef7-cd50a60275e0-maas-api-tls podName:ab0b7792-3a3d-48f4-aef7-cd50a60275e0 nodeName:}" failed. No retries permitted until 2026-07-21 15:09:09.90308459 +0000 UTC m=+2206.252827985 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ab0b7792-3a3d-48f4-aef7-cd50a60275e0-maas-api-tls") pod "maas-api-e2e-inf-b-5ab9b1-f4db4f96b-zvvjx" (UID: "ab0b7792-3a3d-48f4-aef7-cd50a60275e0") : secret "maas-api-e2e-inf-b-5ab9b1-serving-cert" not found
Jul 21 15:09:09.403500 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:09.403480    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab0b7792-3a3d-48f4-aef7-cd50a60275e0-openshift-service-ca\") pod \"maas-api-e2e-inf-b-5ab9b1-f4db4f96b-zvvjx\" (UID: \"ab0b7792-3a3d-48f4-aef7-cd50a60275e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ab9b1-f4db4f96b-zvvjx"
Jul 21 15:09:09.411222 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:09.411190    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzq6w\" (UniqueName: \"kubernetes.io/projected/ab0b7792-3a3d-48f4-aef7-cd50a60275e0-kube-api-access-bzq6w\") pod \"maas-api-e2e-inf-b-5ab9b1-f4db4f96b-zvvjx\" (UID: \"ab0b7792-3a3d-48f4-aef7-cd50a60275e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ab9b1-f4db4f96b-zvvjx"
Jul 21 15:09:09.907148 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:09.907110    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab0b7792-3a3d-48f4-aef7-cd50a60275e0-maas-api-tls\") pod \"maas-api-e2e-inf-b-5ab9b1-f4db4f96b-zvvjx\" (UID: \"ab0b7792-3a3d-48f4-aef7-cd50a60275e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ab9b1-f4db4f96b-zvvjx"
Jul 21 15:09:09.909611 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:09.909587    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab0b7792-3a3d-48f4-aef7-cd50a60275e0-maas-api-tls\") pod \"maas-api-e2e-inf-b-5ab9b1-f4db4f96b-zvvjx\" (UID: \"ab0b7792-3a3d-48f4-aef7-cd50a60275e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ab9b1-f4db4f96b-zvvjx"
Jul 21 15:09:10.121426 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:10.121386    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ab9b1-f4db4f96b-zvvjx"
Jul 21 15:09:10.254499 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:10.254468    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5ab9b1-f4db4f96b-zvvjx"]
Jul 21 15:09:10.257217 ip-10-0-130-83 kubenswrapper[2579]: W0721 15:09:10.257189    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab0b7792_3a3d_48f4_aef7_cd50a60275e0.slice/crio-838dab6cf6e4bb6489188f3f3f7e6c38ee4c7099aebaddd30b0f0b637e3ccfb4 WatchSource:0}: Error finding container 838dab6cf6e4bb6489188f3f3f7e6c38ee4c7099aebaddd30b0f0b637e3ccfb4: Status 404 returned error can't find the container with id 838dab6cf6e4bb6489188f3f3f7e6c38ee4c7099aebaddd30b0f0b637e3ccfb4
Jul 21 15:09:10.848903 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:10.848870    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ab9b1-f4db4f96b-zvvjx" event={"ID":"ab0b7792-3a3d-48f4-aef7-cd50a60275e0","Type":"ContainerStarted","Data":"961961c184977834b1747b34d428367dacf172c28db5e6f07070b52e2b8bb542"}
Jul 21 15:09:10.849059 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:10.848929    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ab9b1-f4db4f96b-zvvjx" event={"ID":"ab0b7792-3a3d-48f4-aef7-cd50a60275e0","Type":"ContainerStarted","Data":"838dab6cf6e4bb6489188f3f3f7e6c38ee4c7099aebaddd30b0f0b637e3ccfb4"}
Jul 21 15:09:10.849059 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:10.849013    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ab9b1-f4db4f96b-zvvjx"
Jul 21 15:09:10.868497 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:10.868455    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ab9b1-f4db4f96b-zvvjx" podStartSLOduration=1.385487906 podStartE2EDuration="1.86844151s" totalImagesPullingTime="482.953604ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 15:09:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 15:09:10.866561067 +0000 UTC m=+2207.216304505" watchObservedRunningTime="2026-07-21 15:09:10.86844151 +0000 UTC m=+2207.218184921"
Jul 21 15:09:16.859534 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:16.859500    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ab9b1-f4db4f96b-zvvjx"
Jul 21 15:09:24.366293 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:24.366243    2579 scope.go:117] "RemoveContainer" containerID="c06eee07cf7a0faaddaa7b8605c9598eec1aa891e8049a7996686605b4613a62"
Jul 21 15:09:24.374304 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:24.374265    2579 scope.go:117] "RemoveContainer" containerID="228b7a992f833136c1fd53fdbf76c06236a6ae5032dd9d490b09e507cee6c2a5"
Jul 21 15:09:55.490880 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:55.490805    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-5ab9b1/test-model-5ab9b1-kserve-f4bbc5c57-ghhsq"]
Jul 21 15:09:55.494251 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:55.494227    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-5ab9b1/test-model-5ab9b1-kserve-f4bbc5c57-ghhsq"
Jul 21 15:09:55.496745 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:55.496725    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-5ab9b1\"/\"default-dockercfg-q52tm\""
Jul 21 15:09:55.497713 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:55.497694    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-5ab9b1\"/\"test-model-5ab9b1-kserve-self-signed-certs\""
Jul 21 15:09:55.497827 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:55.497722    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-5ab9b1\"/\"openshift-service-ca.crt\""
Jul 21 15:09:55.497827 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:55.497727    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-5ab9b1\"/\"kube-root-ca.crt\""
Jul 21 15:09:55.502979 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:55.502959    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-5ab9b1/test-model-5ab9b1-kserve-f4bbc5c57-ghhsq"]
Jul 21 15:09:55.579234 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:55.579205    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9d99940-06d6-4d58-907c-ba5113fb98e0-model-cache\") pod \"test-model-5ab9b1-kserve-f4bbc5c57-ghhsq\" (UID: \"f9d99940-06d6-4d58-907c-ba5113fb98e0\") " pod="ai-tenant-e2e-inf-b-5ab9b1/test-model-5ab9b1-kserve-f4bbc5c57-ghhsq"
Jul 21 15:09:55.579416 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:55.579245    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9d99940-06d6-4d58-907c-ba5113fb98e0-home\") pod \"test-model-5ab9b1-kserve-f4bbc5c57-ghhsq\" (UID: \"f9d99940-06d6-4d58-907c-ba5113fb98e0\") " pod="ai-tenant-e2e-inf-b-5ab9b1/test-model-5ab9b1-kserve-f4bbc5c57-ghhsq"
Jul 21 15:09:55.579416 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:55.579286    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jp9vj\" (UniqueName: \"kubernetes.io/projected/f9d99940-06d6-4d58-907c-ba5113fb98e0-kube-api-access-jp9vj\") pod \"test-model-5ab9b1-kserve-f4bbc5c57-ghhsq\" (UID: \"f9d99940-06d6-4d58-907c-ba5113fb98e0\") " pod="ai-tenant-e2e-inf-b-5ab9b1/test-model-5ab9b1-kserve-f4bbc5c57-ghhsq"
Jul 21 15:09:55.579416 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:55.579343    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9d99940-06d6-4d58-907c-ba5113fb98e0-dshm\") pod \"test-model-5ab9b1-kserve-f4bbc5c57-ghhsq\" (UID: \"f9d99940-06d6-4d58-907c-ba5113fb98e0\") " pod="ai-tenant-e2e-inf-b-5ab9b1/test-model-5ab9b1-kserve-f4bbc5c57-ghhsq"
Jul 21 15:09:55.579581 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:55.579440    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9d99940-06d6-4d58-907c-ba5113fb98e0-tls-certs\") pod \"test-model-5ab9b1-kserve-f4bbc5c57-ghhsq\" (UID: \"f9d99940-06d6-4d58-907c-ba5113fb98e0\") " pod="ai-tenant-e2e-inf-b-5ab9b1/test-model-5ab9b1-kserve-f4bbc5c57-ghhsq"
Jul 21 15:09:55.579581 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:55.579485    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9d99940-06d6-4d58-907c-ba5113fb98e0-tmp-dir\") pod \"test-model-5ab9b1-kserve-f4bbc5c57-ghhsq\" (UID: \"f9d99940-06d6-4d58-907c-ba5113fb98e0\") " pod="ai-tenant-e2e-inf-b-5ab9b1/test-model-5ab9b1-kserve-f4bbc5c57-ghhsq"
Jul 21 15:09:55.680701 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:55.680664    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9d99940-06d6-4d58-907c-ba5113fb98e0-tls-certs\") pod \"test-model-5ab9b1-kserve-f4bbc5c57-ghhsq\" (UID: \"f9d99940-06d6-4d58-907c-ba5113fb98e0\") " pod="ai-tenant-e2e-inf-b-5ab9b1/test-model-5ab9b1-kserve-f4bbc5c57-ghhsq"
Jul 21 15:09:55.680701 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:55.680711    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9d99940-06d6-4d58-907c-ba5113fb98e0-tmp-dir\") pod \"test-model-5ab9b1-kserve-f4bbc5c57-ghhsq\" (UID: \"f9d99940-06d6-4d58-907c-ba5113fb98e0\") " pod="ai-tenant-e2e-inf-b-5ab9b1/test-model-5ab9b1-kserve-f4bbc5c57-ghhsq"
Jul 21 15:09:55.680942 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:55.680743    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9d99940-06d6-4d58-907c-ba5113fb98e0-model-cache\") pod \"test-model-5ab9b1-kserve-f4bbc5c57-ghhsq\" (UID: \"f9d99940-06d6-4d58-907c-ba5113fb98e0\") " pod="ai-tenant-e2e-inf-b-5ab9b1/test-model-5ab9b1-kserve-f4bbc5c57-ghhsq"
Jul 21 15:09:55.680942 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:55.680769    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9d99940-06d6-4d58-907c-ba5113fb98e0-home\") pod \"test-model-5ab9b1-kserve-f4bbc5c57-ghhsq\" (UID: \"f9d99940-06d6-4d58-907c-ba5113fb98e0\") " pod="ai-tenant-e2e-inf-b-5ab9b1/test-model-5ab9b1-kserve-f4bbc5c57-ghhsq"
Jul 21 15:09:55.680942 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:55.680805    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jp9vj\" (UniqueName: \"kubernetes.io/projected/f9d99940-06d6-4d58-907c-ba5113fb98e0-kube-api-access-jp9vj\") pod \"test-model-5ab9b1-kserve-f4bbc5c57-ghhsq\" (UID: \"f9d99940-06d6-4d58-907c-ba5113fb98e0\") " pod="ai-tenant-e2e-inf-b-5ab9b1/test-model-5ab9b1-kserve-f4bbc5c57-ghhsq"
Jul 21 15:09:55.680942 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:55.680858    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9d99940-06d6-4d58-907c-ba5113fb98e0-dshm\") pod \"test-model-5ab9b1-kserve-f4bbc5c57-ghhsq\" (UID: \"f9d99940-06d6-4d58-907c-ba5113fb98e0\") " pod="ai-tenant-e2e-inf-b-5ab9b1/test-model-5ab9b1-kserve-f4bbc5c57-ghhsq"
Jul 21 15:09:55.681297 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:55.681242    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9d99940-06d6-4d58-907c-ba5113fb98e0-tmp-dir\") pod \"test-model-5ab9b1-kserve-f4bbc5c57-ghhsq\" (UID: \"f9d99940-06d6-4d58-907c-ba5113fb98e0\") " pod="ai-tenant-e2e-inf-b-5ab9b1/test-model-5ab9b1-kserve-f4bbc5c57-ghhsq"
Jul 21 15:09:55.681390 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:55.681254    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9d99940-06d6-4d58-907c-ba5113fb98e0-model-cache\") pod \"test-model-5ab9b1-kserve-f4bbc5c57-ghhsq\" (UID: \"f9d99940-06d6-4d58-907c-ba5113fb98e0\") " pod="ai-tenant-e2e-inf-b-5ab9b1/test-model-5ab9b1-kserve-f4bbc5c57-ghhsq"
Jul 21 15:09:55.681390 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:55.681320    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9d99940-06d6-4d58-907c-ba5113fb98e0-home\") pod \"test-model-5ab9b1-kserve-f4bbc5c57-ghhsq\" (UID: \"f9d99940-06d6-4d58-907c-ba5113fb98e0\") " pod="ai-tenant-e2e-inf-b-5ab9b1/test-model-5ab9b1-kserve-f4bbc5c57-ghhsq"
Jul 21 15:09:55.683168 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:55.683140    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9d99940-06d6-4d58-907c-ba5113fb98e0-dshm\") pod \"test-model-5ab9b1-kserve-f4bbc5c57-ghhsq\" (UID: \"f9d99940-06d6-4d58-907c-ba5113fb98e0\") " pod="ai-tenant-e2e-inf-b-5ab9b1/test-model-5ab9b1-kserve-f4bbc5c57-ghhsq"
Jul 21 15:09:55.683367 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:55.683348    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9d99940-06d6-4d58-907c-ba5113fb98e0-tls-certs\") pod \"test-model-5ab9b1-kserve-f4bbc5c57-ghhsq\" (UID: \"f9d99940-06d6-4d58-907c-ba5113fb98e0\") " pod="ai-tenant-e2e-inf-b-5ab9b1/test-model-5ab9b1-kserve-f4bbc5c57-ghhsq"
Jul 21 15:09:55.689191 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:55.689171    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jp9vj\" (UniqueName: \"kubernetes.io/projected/f9d99940-06d6-4d58-907c-ba5113fb98e0-kube-api-access-jp9vj\") pod \"test-model-5ab9b1-kserve-f4bbc5c57-ghhsq\" (UID: \"f9d99940-06d6-4d58-907c-ba5113fb98e0\") " pod="ai-tenant-e2e-inf-b-5ab9b1/test-model-5ab9b1-kserve-f4bbc5c57-ghhsq"
Jul 21 15:09:55.806662 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:55.806568    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-5ab9b1/test-model-5ab9b1-kserve-f4bbc5c57-ghhsq"
Jul 21 15:09:55.950878 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:55.950857    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-5ab9b1/test-model-5ab9b1-kserve-f4bbc5c57-ghhsq"]
Jul 21 15:09:55.953015 ip-10-0-130-83 kubenswrapper[2579]: W0721 15:09:55.952981    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf9d99940_06d6_4d58_907c_ba5113fb98e0.slice/crio-d075b803ac2ebd2630f64470a4b3f69ab74bd8028d7a39eb1646924630783830 WatchSource:0}: Error finding container d075b803ac2ebd2630f64470a4b3f69ab74bd8028d7a39eb1646924630783830: Status 404 returned error can't find the container with id d075b803ac2ebd2630f64470a4b3f69ab74bd8028d7a39eb1646924630783830
Jul 21 15:09:56.065307 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:56.065215    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-5ab9b1/test-model-5ab9b1-kserve-f4bbc5c57-ghhsq" event={"ID":"f9d99940-06d6-4d58-907c-ba5113fb98e0","Type":"ContainerStarted","Data":"751c0610fd4c4fe888db210a43b30431563d01e0f83158438a3d88fde738a6b8"}
Jul 21 15:09:56.065307 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:56.065256    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-5ab9b1/test-model-5ab9b1-kserve-f4bbc5c57-ghhsq" event={"ID":"f9d99940-06d6-4d58-907c-ba5113fb98e0","Type":"ContainerStarted","Data":"d075b803ac2ebd2630f64470a4b3f69ab74bd8028d7a39eb1646924630783830"}
Jul 21 15:09:56.081396 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:09:56.081356    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-5ab9b1/test-model-5ab9b1-kserve-f4bbc5c57-ghhsq" podStartSLOduration=1.081342687 podStartE2EDuration="1.081342687s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 15:09:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 15:09:56.081023537 +0000 UTC m=+2252.430766950" watchObservedRunningTime="2026-07-21 15:09:56.081342687 +0000 UTC m=+2252.431086152"
Jul 21 15:10:05.807020 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:05.806979    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-5ab9b1/test-model-5ab9b1-kserve-f4bbc5c57-ghhsq"
Jul 21 15:10:05.807428 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:05.807042    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-5ab9b1/test-model-5ab9b1-kserve-f4bbc5c57-ghhsq"
Jul 21 15:10:05.819724 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:05.819696    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-5ab9b1/test-model-5ab9b1-kserve-f4bbc5c57-ghhsq"
Jul 21 15:10:06.123584 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:06.123513    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-5ab9b1/test-model-5ab9b1-kserve-f4bbc5c57-ghhsq"
Jul 21 15:10:39.267408 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:39.267376    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-5ab9b1/test-model-5ab9b1-kserve-f4bbc5c57-ghhsq"]
Jul 21 15:10:39.419544 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:39.419512    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d6d63a-6n9zg"]
Jul 21 15:10:39.422944 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:39.422927    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d6d63a-6n9zg"
Jul 21 15:10:39.425340 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:39.425323    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-494sx\""
Jul 21 15:10:39.431803 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:39.431777    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d6d63a-6n9zg"]
Jul 21 15:10:39.536972 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:39.536892    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/339d3833-82f7-4592-8c44-c38bc6df1088-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-d6d63a-6n9zg\" (UID: \"339d3833-82f7-4592-8c44-c38bc6df1088\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d6d63a-6n9zg"
Jul 21 15:10:39.637872 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:39.637842    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/339d3833-82f7-4592-8c44-c38bc6df1088-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-d6d63a-6n9zg\" (UID: \"339d3833-82f7-4592-8c44-c38bc6df1088\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d6d63a-6n9zg"
Jul 21 15:10:39.638455 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:39.638438    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/339d3833-82f7-4592-8c44-c38bc6df1088-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-d6d63a-6n9zg\" (UID: \"339d3833-82f7-4592-8c44-c38bc6df1088\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d6d63a-6n9zg"
Jul 21 15:10:39.736376 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:39.736347    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d6d63a-6n9zg"
Jul 21 15:10:40.081712 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:40.081683    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d6d63a-6n9zg"]
Jul 21 15:10:40.083460 ip-10-0-130-83 kubenswrapper[2579]: W0721 15:10:40.083425    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod339d3833_82f7_4592_8c44_c38bc6df1088.slice/crio-966a5def3d3bb05bb0cba75b287f300a696b26a4e239ed400949365866d7ab6d WatchSource:0}: Error finding container 966a5def3d3bb05bb0cba75b287f300a696b26a4e239ed400949365866d7ab6d: Status 404 returned error can't find the container with id 966a5def3d3bb05bb0cba75b287f300a696b26a4e239ed400949365866d7ab6d
Jul 21 15:10:40.269361 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:40.269327    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d6d63a-6n9zg" event={"ID":"339d3833-82f7-4592-8c44-c38bc6df1088","Type":"ContainerStarted","Data":"966a5def3d3bb05bb0cba75b287f300a696b26a4e239ed400949365866d7ab6d"}
Jul 21 15:10:41.276172 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:41.276138    2579 generic.go:356] "Generic (PLEG): container finished" podID="339d3833-82f7-4592-8c44-c38bc6df1088" containerID="5a7d4e7ddc4fbc7566614f75dec9dd68533919541c6133470a4f127e8e2eadbd" exitCode=0
Jul 21 15:10:41.276571 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:41.276222    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d6d63a-6n9zg" event={"ID":"339d3833-82f7-4592-8c44-c38bc6df1088","Type":"ContainerDied","Data":"5a7d4e7ddc4fbc7566614f75dec9dd68533919541c6133470a4f127e8e2eadbd"}
Jul 21 15:10:42.409470 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:42.409448    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d6d63a-6n9zg"
Jul 21 15:10:42.461395 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:42.461367    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/339d3833-82f7-4592-8c44-c38bc6df1088-maas-api-service-ca\") pod \"339d3833-82f7-4592-8c44-c38bc6df1088\" (UID: \"339d3833-82f7-4592-8c44-c38bc6df1088\") "
Jul 21 15:10:42.461673 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:42.461653    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/339d3833-82f7-4592-8c44-c38bc6df1088-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "339d3833-82f7-4592-8c44-c38bc6df1088" (UID: "339d3833-82f7-4592-8c44-c38bc6df1088"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 15:10:42.561965 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:42.561900    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/339d3833-82f7-4592-8c44-c38bc6df1088-maas-api-service-ca\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:10:43.287466 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:43.287443    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d6d63a-6n9zg"
Jul 21 15:10:43.287466 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:43.287448    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d6d63a-6n9zg" event={"ID":"339d3833-82f7-4592-8c44-c38bc6df1088","Type":"ContainerDied","Data":"966a5def3d3bb05bb0cba75b287f300a696b26a4e239ed400949365866d7ab6d"}
Jul 21 15:10:43.287666 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:43.287493    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="966a5def3d3bb05bb0cba75b287f300a696b26a4e239ed400949365866d7ab6d"
Jul 21 15:10:49.901826 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:49.901792    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d6d63a-6794cbf6cf-8qmmf"]
Jul 21 15:10:49.902328 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:49.902023    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d6d63a-6794cbf6cf-8qmmf" podUID="4488c28f-36c4-469e-8fe2-de9ae527651d" containerName="maas-api" containerID="cri-o://8bcf549241a1c8db66b8acb075d91ed09a6d76fe73fa694fee3b51953c15301e" gracePeriod=30
Jul 21 15:10:50.321179 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:50.321147    2579 generic.go:356] "Generic (PLEG): container finished" podID="4488c28f-36c4-469e-8fe2-de9ae527651d" containerID="8bcf549241a1c8db66b8acb075d91ed09a6d76fe73fa694fee3b51953c15301e" exitCode=0
Jul 21 15:10:50.321358 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:50.321220    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d6d63a-6794cbf6cf-8qmmf" event={"ID":"4488c28f-36c4-469e-8fe2-de9ae527651d","Type":"ContainerDied","Data":"8bcf549241a1c8db66b8acb075d91ed09a6d76fe73fa694fee3b51953c15301e"}
Jul 21 15:10:50.353333 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:50.353306    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d6d63a-6794cbf6cf-8qmmf"
Jul 21 15:10:50.432573 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:50.432513    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4488c28f-36c4-469e-8fe2-de9ae527651d-maas-api-tls\") pod \"4488c28f-36c4-469e-8fe2-de9ae527651d\" (UID: \"4488c28f-36c4-469e-8fe2-de9ae527651d\") "
Jul 21 15:10:50.432573 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:50.432552    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4488c28f-36c4-469e-8fe2-de9ae527651d-openshift-service-ca\") pod \"4488c28f-36c4-469e-8fe2-de9ae527651d\" (UID: \"4488c28f-36c4-469e-8fe2-de9ae527651d\") "
Jul 21 15:10:50.432751 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:50.432589    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-phqmb\" (UniqueName: \"kubernetes.io/projected/4488c28f-36c4-469e-8fe2-de9ae527651d-kube-api-access-phqmb\") pod \"4488c28f-36c4-469e-8fe2-de9ae527651d\" (UID: \"4488c28f-36c4-469e-8fe2-de9ae527651d\") "
Jul 21 15:10:50.432948 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:50.432927    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4488c28f-36c4-469e-8fe2-de9ae527651d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4488c28f-36c4-469e-8fe2-de9ae527651d" (UID: "4488c28f-36c4-469e-8fe2-de9ae527651d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 15:10:50.434715 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:50.434690    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4488c28f-36c4-469e-8fe2-de9ae527651d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4488c28f-36c4-469e-8fe2-de9ae527651d" (UID: "4488c28f-36c4-469e-8fe2-de9ae527651d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 15:10:50.434818 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:50.434716    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4488c28f-36c4-469e-8fe2-de9ae527651d-kube-api-access-phqmb" (OuterVolumeSpecName: "kube-api-access-phqmb") pod "4488c28f-36c4-469e-8fe2-de9ae527651d" (UID: "4488c28f-36c4-469e-8fe2-de9ae527651d"). InnerVolumeSpecName "kube-api-access-phqmb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 15:10:50.533495 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:50.533470    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phqmb\" (UniqueName: \"kubernetes.io/projected/4488c28f-36c4-469e-8fe2-de9ae527651d-kube-api-access-phqmb\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:10:50.533619 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:50.533498    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4488c28f-36c4-469e-8fe2-de9ae527651d-maas-api-tls\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:10:50.533619 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:50.533509    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4488c28f-36c4-469e-8fe2-de9ae527651d-openshift-service-ca\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:10:51.327036 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:51.327006    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d6d63a-6794cbf6cf-8qmmf" event={"ID":"4488c28f-36c4-469e-8fe2-de9ae527651d","Type":"ContainerDied","Data":"ef98ef3d3676dbd6ed935c95fe9ae6e9d2fed1a5944eeb092c170b49eff2e487"}
Jul 21 15:10:51.327036 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:51.327022    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d6d63a-6794cbf6cf-8qmmf"
Jul 21 15:10:51.327523 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:51.327056    2579 scope.go:117] "RemoveContainer" containerID="8bcf549241a1c8db66b8acb075d91ed09a6d76fe73fa694fee3b51953c15301e"
Jul 21 15:10:51.347440 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:51.347413    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d6d63a-6794cbf6cf-8qmmf"]
Jul 21 15:10:51.349625 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:51.349605    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d6d63a-6794cbf6cf-8qmmf"]
Jul 21 15:10:52.172167 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:52.172134    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4488c28f-36c4-469e-8fe2-de9ae527651d" path="/var/lib/kubelet/pods/4488c28f-36c4-469e-8fe2-de9ae527651d/volumes"
Jul 21 15:10:54.287810 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:54.287778    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-5ab9b1/test-model-5ab9b1-kserve-f4bbc5c57-ghhsq" podUID="f9d99940-06d6-4d58-907c-ba5113fb98e0" containerName="main" containerID="cri-o://751c0610fd4c4fe888db210a43b30431563d01e0f83158438a3d88fde738a6b8" gracePeriod=45
Jul 21 15:10:54.521110 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:54.521076    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d6d63a-6n9zg"]
Jul 21 15:10:54.525124 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:54.525098    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-d6d63a-6n9zg"]
Jul 21 15:10:54.546923 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:54.546871    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-5ab9b1/test-model-5ab9b1-kserve-f4bbc5c57-ghhsq"
Jul 21 15:10:54.668416 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:54.668392    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9d99940-06d6-4d58-907c-ba5113fb98e0-home\") pod \"f9d99940-06d6-4d58-907c-ba5113fb98e0\" (UID: \"f9d99940-06d6-4d58-907c-ba5113fb98e0\") "
Jul 21 15:10:54.668607 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:54.668461    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jp9vj\" (UniqueName: \"kubernetes.io/projected/f9d99940-06d6-4d58-907c-ba5113fb98e0-kube-api-access-jp9vj\") pod \"f9d99940-06d6-4d58-907c-ba5113fb98e0\" (UID: \"f9d99940-06d6-4d58-907c-ba5113fb98e0\") "
Jul 21 15:10:54.668607 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:54.668478    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9d99940-06d6-4d58-907c-ba5113fb98e0-dshm\") pod \"f9d99940-06d6-4d58-907c-ba5113fb98e0\" (UID: \"f9d99940-06d6-4d58-907c-ba5113fb98e0\") "
Jul 21 15:10:54.668607 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:54.668494    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9d99940-06d6-4d58-907c-ba5113fb98e0-model-cache\") pod \"f9d99940-06d6-4d58-907c-ba5113fb98e0\" (UID: \"f9d99940-06d6-4d58-907c-ba5113fb98e0\") "
Jul 21 15:10:54.668607 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:54.668543    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9d99940-06d6-4d58-907c-ba5113fb98e0-tmp-dir\") pod \"f9d99940-06d6-4d58-907c-ba5113fb98e0\" (UID: \"f9d99940-06d6-4d58-907c-ba5113fb98e0\") "
Jul 21 15:10:54.668857 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:54.668618    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9d99940-06d6-4d58-907c-ba5113fb98e0-tls-certs\") pod \"f9d99940-06d6-4d58-907c-ba5113fb98e0\" (UID: \"f9d99940-06d6-4d58-907c-ba5113fb98e0\") "
Jul 21 15:10:54.668857 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:54.668693    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9d99940-06d6-4d58-907c-ba5113fb98e0-home" (OuterVolumeSpecName: "home") pod "f9d99940-06d6-4d58-907c-ba5113fb98e0" (UID: "f9d99940-06d6-4d58-907c-ba5113fb98e0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 15:10:54.668857 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:54.668760    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9d99940-06d6-4d58-907c-ba5113fb98e0-model-cache" (OuterVolumeSpecName: "model-cache") pod "f9d99940-06d6-4d58-907c-ba5113fb98e0" (UID: "f9d99940-06d6-4d58-907c-ba5113fb98e0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 15:10:54.669015 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:54.668899    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9d99940-06d6-4d58-907c-ba5113fb98e0-home\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:10:54.669015 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:54.668905    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9d99940-06d6-4d58-907c-ba5113fb98e0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f9d99940-06d6-4d58-907c-ba5113fb98e0" (UID: "f9d99940-06d6-4d58-907c-ba5113fb98e0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 15:10:54.669015 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:54.668916    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9d99940-06d6-4d58-907c-ba5113fb98e0-model-cache\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:10:54.670968 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:54.670945    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9d99940-06d6-4d58-907c-ba5113fb98e0-kube-api-access-jp9vj" (OuterVolumeSpecName: "kube-api-access-jp9vj") pod "f9d99940-06d6-4d58-907c-ba5113fb98e0" (UID: "f9d99940-06d6-4d58-907c-ba5113fb98e0"). InnerVolumeSpecName "kube-api-access-jp9vj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 15:10:54.670968 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:54.670946    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9d99940-06d6-4d58-907c-ba5113fb98e0-dshm" (OuterVolumeSpecName: "dshm") pod "f9d99940-06d6-4d58-907c-ba5113fb98e0" (UID: "f9d99940-06d6-4d58-907c-ba5113fb98e0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 15:10:54.671122 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:54.671038    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9d99940-06d6-4d58-907c-ba5113fb98e0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f9d99940-06d6-4d58-907c-ba5113fb98e0" (UID: "f9d99940-06d6-4d58-907c-ba5113fb98e0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 15:10:54.769953 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:54.769928    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9d99940-06d6-4d58-907c-ba5113fb98e0-tls-certs\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:10:54.769953 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:54.769951    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jp9vj\" (UniqueName: \"kubernetes.io/projected/f9d99940-06d6-4d58-907c-ba5113fb98e0-kube-api-access-jp9vj\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:10:54.770085 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:54.769961    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9d99940-06d6-4d58-907c-ba5113fb98e0-dshm\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:10:54.770085 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:54.769969    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9d99940-06d6-4d58-907c-ba5113fb98e0-tmp-dir\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:10:55.346781 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:55.346752    2579 generic.go:356] "Generic (PLEG): container finished" podID="f9d99940-06d6-4d58-907c-ba5113fb98e0" containerID="751c0610fd4c4fe888db210a43b30431563d01e0f83158438a3d88fde738a6b8" exitCode=0
Jul 21 15:10:55.347196 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:55.346790    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-5ab9b1/test-model-5ab9b1-kserve-f4bbc5c57-ghhsq"
Jul 21 15:10:55.347196 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:55.346805    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-5ab9b1/test-model-5ab9b1-kserve-f4bbc5c57-ghhsq" event={"ID":"f9d99940-06d6-4d58-907c-ba5113fb98e0","Type":"ContainerDied","Data":"751c0610fd4c4fe888db210a43b30431563d01e0f83158438a3d88fde738a6b8"}
Jul 21 15:10:55.347196 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:55.346833    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-5ab9b1/test-model-5ab9b1-kserve-f4bbc5c57-ghhsq" event={"ID":"f9d99940-06d6-4d58-907c-ba5113fb98e0","Type":"ContainerDied","Data":"d075b803ac2ebd2630f64470a4b3f69ab74bd8028d7a39eb1646924630783830"}
Jul 21 15:10:55.347196 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:55.346849    2579 scope.go:117] "RemoveContainer" containerID="751c0610fd4c4fe888db210a43b30431563d01e0f83158438a3d88fde738a6b8"
Jul 21 15:10:55.355675 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:55.355659    2579 scope.go:117] "RemoveContainer" containerID="751c0610fd4c4fe888db210a43b30431563d01e0f83158438a3d88fde738a6b8"
Jul 21 15:10:55.355920 ip-10-0-130-83 kubenswrapper[2579]: E0721 15:10:55.355900    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"751c0610fd4c4fe888db210a43b30431563d01e0f83158438a3d88fde738a6b8\": container with ID starting with 751c0610fd4c4fe888db210a43b30431563d01e0f83158438a3d88fde738a6b8 not found: ID does not exist" containerID="751c0610fd4c4fe888db210a43b30431563d01e0f83158438a3d88fde738a6b8"
Jul 21 15:10:55.355973 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:55.355926    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"751c0610fd4c4fe888db210a43b30431563d01e0f83158438a3d88fde738a6b8"} err="failed to get container status \"751c0610fd4c4fe888db210a43b30431563d01e0f83158438a3d88fde738a6b8\": rpc error: code = NotFound desc = could not find container \"751c0610fd4c4fe888db210a43b30431563d01e0f83158438a3d88fde738a6b8\": container with ID starting with 751c0610fd4c4fe888db210a43b30431563d01e0f83158438a3d88fde738a6b8 not found: ID does not exist"
Jul 21 15:10:55.373852 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:55.373826    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-5ab9b1/test-model-5ab9b1-kserve-f4bbc5c57-ghhsq"]
Jul 21 15:10:55.377110 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:55.377085    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-5ab9b1/test-model-5ab9b1-kserve-f4bbc5c57-ghhsq"]
Jul 21 15:10:56.172161 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:56.172125    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="339d3833-82f7-4592-8c44-c38bc6df1088" path="/var/lib/kubelet/pods/339d3833-82f7-4592-8c44-c38bc6df1088/volumes"
Jul 21 15:10:56.173093 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:10:56.173063    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9d99940-06d6-4d58-907c-ba5113fb98e0" path="/var/lib/kubelet/pods/f9d99940-06d6-4d58-907c-ba5113fb98e0/volumes"
Jul 21 15:11:01.287646 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:01.287612    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ab9b1-7cz4k"]
Jul 21 15:11:01.288101 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:01.287943    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4488c28f-36c4-469e-8fe2-de9ae527651d" containerName="maas-api"
Jul 21 15:11:01.288101 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:01.287954    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="4488c28f-36c4-469e-8fe2-de9ae527651d" containerName="maas-api"
Jul 21 15:11:01.288101 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:01.287962    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9d99940-06d6-4d58-907c-ba5113fb98e0" containerName="main"
Jul 21 15:11:01.288101 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:01.287967    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9d99940-06d6-4d58-907c-ba5113fb98e0" containerName="main"
Jul 21 15:11:01.288101 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:01.287987    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="339d3833-82f7-4592-8c44-c38bc6df1088" containerName="revoke-keys"
Jul 21 15:11:01.288101 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:01.287993    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="339d3833-82f7-4592-8c44-c38bc6df1088" containerName="revoke-keys"
Jul 21 15:11:01.288101 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:01.288049    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="339d3833-82f7-4592-8c44-c38bc6df1088" containerName="revoke-keys"
Jul 21 15:11:01.288101 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:01.288061    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9d99940-06d6-4d58-907c-ba5113fb98e0" containerName="main"
Jul 21 15:11:01.288101 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:01.288067    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="4488c28f-36c4-469e-8fe2-de9ae527651d" containerName="maas-api"
Jul 21 15:11:01.290809 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:01.290793    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ab9b1-7cz4k"
Jul 21 15:11:01.293612 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:01.293595    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-494sx\""
Jul 21 15:11:01.298598 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:01.298573    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ab9b1-7cz4k"]
Jul 21 15:11:01.427180 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:01.427148    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/969db7a9-0cb8-46fd-a99f-e3d0be4234c1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-5ab9b1-7cz4k\" (UID: \"969db7a9-0cb8-46fd-a99f-e3d0be4234c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ab9b1-7cz4k"
Jul 21 15:11:01.528403 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:01.528374    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/969db7a9-0cb8-46fd-a99f-e3d0be4234c1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-5ab9b1-7cz4k\" (UID: \"969db7a9-0cb8-46fd-a99f-e3d0be4234c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ab9b1-7cz4k"
Jul 21 15:11:01.529022 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:01.529004    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/969db7a9-0cb8-46fd-a99f-e3d0be4234c1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-5ab9b1-7cz4k\" (UID: \"969db7a9-0cb8-46fd-a99f-e3d0be4234c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ab9b1-7cz4k"
Jul 21 15:11:01.603120 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:01.603044    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ab9b1-7cz4k"
Jul 21 15:11:01.735989 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:01.735965    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ab9b1-7cz4k"]
Jul 21 15:11:01.738968 ip-10-0-130-83 kubenswrapper[2579]: W0721 15:11:01.738889    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod969db7a9_0cb8_46fd_a99f_e3d0be4234c1.slice/crio-685d5d3f09a16a8d91e631b7fc71fbdfafcf1a26684d7db1388f6d9422fc8c56 WatchSource:0}: Error finding container 685d5d3f09a16a8d91e631b7fc71fbdfafcf1a26684d7db1388f6d9422fc8c56: Status 404 returned error can't find the container with id 685d5d3f09a16a8d91e631b7fc71fbdfafcf1a26684d7db1388f6d9422fc8c56
Jul 21 15:11:02.383632 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:02.383599    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ab9b1-7cz4k" event={"ID":"969db7a9-0cb8-46fd-a99f-e3d0be4234c1","Type":"ContainerStarted","Data":"0626f993c568ef538ccf3b17c20ba4c151955323094b6c8342d5c8243a769040"}
Jul 21 15:11:02.383982 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:02.383653    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ab9b1-7cz4k" event={"ID":"969db7a9-0cb8-46fd-a99f-e3d0be4234c1","Type":"ContainerStarted","Data":"685d5d3f09a16a8d91e631b7fc71fbdfafcf1a26684d7db1388f6d9422fc8c56"}
Jul 21 15:11:02.399347 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:02.399293    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ab9b1-7cz4k" podStartSLOduration=1.399263441 podStartE2EDuration="1.399263441s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 15:11:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 15:11:02.397074078 +0000 UTC m=+2318.746817490" watchObservedRunningTime="2026-07-21 15:11:02.399263441 +0000 UTC m=+2318.749006853"
Jul 21 15:11:03.389394 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:03.389361    2579 generic.go:356] "Generic (PLEG): container finished" podID="969db7a9-0cb8-46fd-a99f-e3d0be4234c1" containerID="0626f993c568ef538ccf3b17c20ba4c151955323094b6c8342d5c8243a769040" exitCode=0
Jul 21 15:11:03.389755 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:03.389437    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ab9b1-7cz4k" event={"ID":"969db7a9-0cb8-46fd-a99f-e3d0be4234c1","Type":"ContainerDied","Data":"0626f993c568ef538ccf3b17c20ba4c151955323094b6c8342d5c8243a769040"}
Jul 21 15:11:04.526208 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:04.526178    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ab9b1-7cz4k"
Jul 21 15:11:04.655855 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:04.655767    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/969db7a9-0cb8-46fd-a99f-e3d0be4234c1-maas-api-service-ca\") pod \"969db7a9-0cb8-46fd-a99f-e3d0be4234c1\" (UID: \"969db7a9-0cb8-46fd-a99f-e3d0be4234c1\") "
Jul 21 15:11:04.656155 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:04.656131    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/969db7a9-0cb8-46fd-a99f-e3d0be4234c1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "969db7a9-0cb8-46fd-a99f-e3d0be4234c1" (UID: "969db7a9-0cb8-46fd-a99f-e3d0be4234c1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 15:11:04.756820 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:04.756788    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/969db7a9-0cb8-46fd-a99f-e3d0be4234c1-maas-api-service-ca\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:11:05.399730 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:05.399701    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ab9b1-7cz4k"
Jul 21 15:11:05.399898 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:05.399696    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ab9b1-7cz4k" event={"ID":"969db7a9-0cb8-46fd-a99f-e3d0be4234c1","Type":"ContainerDied","Data":"685d5d3f09a16a8d91e631b7fc71fbdfafcf1a26684d7db1388f6d9422fc8c56"}
Jul 21 15:11:05.399898 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:05.399832    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="685d5d3f09a16a8d91e631b7fc71fbdfafcf1a26684d7db1388f6d9422fc8c56"
Jul 21 15:11:11.703609 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:11.703571    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5ab9b1-f4db4f96b-zvvjx"]
Jul 21 15:11:11.704039 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:11.703812    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ab9b1-f4db4f96b-zvvjx" podUID="ab0b7792-3a3d-48f4-aef7-cd50a60275e0" containerName="maas-api" containerID="cri-o://961961c184977834b1747b34d428367dacf172c28db5e6f07070b52e2b8bb542" gracePeriod=30
Jul 21 15:11:11.855514 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:11.855476    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ab9b1-f4db4f96b-zvvjx" podUID="ab0b7792-3a3d-48f4-aef7-cd50a60275e0" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.57:8443/health\": dial tcp 10.133.0.57:8443: connect: connection refused"
Jul 21 15:11:12.169640 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:12.169615    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ab9b1-f4db4f96b-zvvjx"
Jul 21 15:11:12.207880 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:12.207854    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bzq6w\" (UniqueName: \"kubernetes.io/projected/ab0b7792-3a3d-48f4-aef7-cd50a60275e0-kube-api-access-bzq6w\") pod \"ab0b7792-3a3d-48f4-aef7-cd50a60275e0\" (UID: \"ab0b7792-3a3d-48f4-aef7-cd50a60275e0\") "
Jul 21 15:11:12.207986 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:12.207906    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab0b7792-3a3d-48f4-aef7-cd50a60275e0-maas-api-tls\") pod \"ab0b7792-3a3d-48f4-aef7-cd50a60275e0\" (UID: \"ab0b7792-3a3d-48f4-aef7-cd50a60275e0\") "
Jul 21 15:11:12.208037 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:12.207993    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab0b7792-3a3d-48f4-aef7-cd50a60275e0-openshift-service-ca\") pod \"ab0b7792-3a3d-48f4-aef7-cd50a60275e0\" (UID: \"ab0b7792-3a3d-48f4-aef7-cd50a60275e0\") "
Jul 21 15:11:12.208372 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:12.208345    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab0b7792-3a3d-48f4-aef7-cd50a60275e0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ab0b7792-3a3d-48f4-aef7-cd50a60275e0" (UID: "ab0b7792-3a3d-48f4-aef7-cd50a60275e0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 15:11:12.209931 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:12.209910    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab0b7792-3a3d-48f4-aef7-cd50a60275e0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ab0b7792-3a3d-48f4-aef7-cd50a60275e0" (UID: "ab0b7792-3a3d-48f4-aef7-cd50a60275e0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 15:11:12.210019 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:12.209945    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab0b7792-3a3d-48f4-aef7-cd50a60275e0-kube-api-access-bzq6w" (OuterVolumeSpecName: "kube-api-access-bzq6w") pod "ab0b7792-3a3d-48f4-aef7-cd50a60275e0" (UID: "ab0b7792-3a3d-48f4-aef7-cd50a60275e0"). InnerVolumeSpecName "kube-api-access-bzq6w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 15:11:12.309133 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:12.309064    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab0b7792-3a3d-48f4-aef7-cd50a60275e0-openshift-service-ca\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:11:12.309133 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:12.309089    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bzq6w\" (UniqueName: \"kubernetes.io/projected/ab0b7792-3a3d-48f4-aef7-cd50a60275e0-kube-api-access-bzq6w\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:11:12.309133 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:12.309099    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab0b7792-3a3d-48f4-aef7-cd50a60275e0-maas-api-tls\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:11:12.433203 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:12.433174    2579 generic.go:356] "Generic (PLEG): container finished" podID="ab0b7792-3a3d-48f4-aef7-cd50a60275e0" containerID="961961c184977834b1747b34d428367dacf172c28db5e6f07070b52e2b8bb542" exitCode=0
Jul 21 15:11:12.433376 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:12.433229    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ab9b1-f4db4f96b-zvvjx"
Jul 21 15:11:12.433376 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:12.433264    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ab9b1-f4db4f96b-zvvjx" event={"ID":"ab0b7792-3a3d-48f4-aef7-cd50a60275e0","Type":"ContainerDied","Data":"961961c184977834b1747b34d428367dacf172c28db5e6f07070b52e2b8bb542"}
Jul 21 15:11:12.433376 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:12.433345    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ab9b1-f4db4f96b-zvvjx" event={"ID":"ab0b7792-3a3d-48f4-aef7-cd50a60275e0","Type":"ContainerDied","Data":"838dab6cf6e4bb6489188f3f3f7e6c38ee4c7099aebaddd30b0f0b637e3ccfb4"}
Jul 21 15:11:12.433488 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:12.433378    2579 scope.go:117] "RemoveContainer" containerID="961961c184977834b1747b34d428367dacf172c28db5e6f07070b52e2b8bb542"
Jul 21 15:11:12.441875 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:12.441856    2579 scope.go:117] "RemoveContainer" containerID="961961c184977834b1747b34d428367dacf172c28db5e6f07070b52e2b8bb542"
Jul 21 15:11:12.442150 ip-10-0-130-83 kubenswrapper[2579]: E0721 15:11:12.442115    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"961961c184977834b1747b34d428367dacf172c28db5e6f07070b52e2b8bb542\": container with ID starting with 961961c184977834b1747b34d428367dacf172c28db5e6f07070b52e2b8bb542 not found: ID does not exist" containerID="961961c184977834b1747b34d428367dacf172c28db5e6f07070b52e2b8bb542"
Jul 21 15:11:12.442261 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:12.442157    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"961961c184977834b1747b34d428367dacf172c28db5e6f07070b52e2b8bb542"} err="failed to get container status \"961961c184977834b1747b34d428367dacf172c28db5e6f07070b52e2b8bb542\": rpc error: code = NotFound desc = could not find container \"961961c184977834b1747b34d428367dacf172c28db5e6f07070b52e2b8bb542\": container with ID starting with 961961c184977834b1747b34d428367dacf172c28db5e6f07070b52e2b8bb542 not found: ID does not exist"
Jul 21 15:11:12.455398 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:12.455373    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5ab9b1-f4db4f96b-zvvjx"]
Jul 21 15:11:12.460723 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:12.460702    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5ab9b1-f4db4f96b-zvvjx"]
Jul 21 15:11:14.171925 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:14.171889    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab0b7792-3a3d-48f4-aef7-cd50a60275e0" path="/var/lib/kubelet/pods/ab0b7792-3a3d-48f4-aef7-cd50a60275e0/volumes"
Jul 21 15:11:16.369184 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:16.369112    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ab9b1-7cz4k"]
Jul 21 15:11:16.371260 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:16.371233    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-5ab9b1-7cz4k"]
Jul 21 15:11:18.171969 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:18.171938    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="969db7a9-0cb8-46fd-a99f-e3d0be4234c1" path="/var/lib/kubelet/pods/969db7a9-0cb8-46fd-a99f-e3d0be4234c1/volumes"
Jul 21 15:11:26.899525 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:26.899487    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-712877-7977458698-d6kps"]
Jul 21 15:11:26.899989 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:26.899812    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab0b7792-3a3d-48f4-aef7-cd50a60275e0" containerName="maas-api"
Jul 21 15:11:26.899989 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:26.899823    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab0b7792-3a3d-48f4-aef7-cd50a60275e0" containerName="maas-api"
Jul 21 15:11:26.899989 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:26.899845    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="969db7a9-0cb8-46fd-a99f-e3d0be4234c1" containerName="revoke-keys"
Jul 21 15:11:26.899989 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:26.899851    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="969db7a9-0cb8-46fd-a99f-e3d0be4234c1" containerName="revoke-keys"
Jul 21 15:11:26.899989 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:26.899900    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab0b7792-3a3d-48f4-aef7-cd50a60275e0" containerName="maas-api"
Jul 21 15:11:26.899989 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:26.899910    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="969db7a9-0cb8-46fd-a99f-e3d0be4234c1" containerName="revoke-keys"
Jul 21 15:11:26.904413 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:26.904393    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-712877-7977458698-d6kps"
Jul 21 15:11:26.908575 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:26.908555    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-712877-serving-cert\""
Jul 21 15:11:26.915575 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:26.915553    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-712877-7977458698-d6kps"]
Jul 21 15:11:27.023223 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:27.023192    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16590d60-acfc-46a5-bcca-96b2ef80938b-maas-api-tls\") pod \"maas-api-e2e-api-a-712877-7977458698-d6kps\" (UID: \"16590d60-acfc-46a5-bcca-96b2ef80938b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-712877-7977458698-d6kps"
Jul 21 15:11:27.023389 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:27.023230    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrw8h\" (UniqueName: \"kubernetes.io/projected/16590d60-acfc-46a5-bcca-96b2ef80938b-kube-api-access-mrw8h\") pod \"maas-api-e2e-api-a-712877-7977458698-d6kps\" (UID: \"16590d60-acfc-46a5-bcca-96b2ef80938b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-712877-7977458698-d6kps"
Jul 21 15:11:27.023389 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:27.023293    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16590d60-acfc-46a5-bcca-96b2ef80938b-openshift-service-ca\") pod \"maas-api-e2e-api-a-712877-7977458698-d6kps\" (UID: \"16590d60-acfc-46a5-bcca-96b2ef80938b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-712877-7977458698-d6kps"
Jul 21 15:11:27.124583 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:27.124552    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16590d60-acfc-46a5-bcca-96b2ef80938b-maas-api-tls\") pod \"maas-api-e2e-api-a-712877-7977458698-d6kps\" (UID: \"16590d60-acfc-46a5-bcca-96b2ef80938b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-712877-7977458698-d6kps"
Jul 21 15:11:27.124583 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:27.124586    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrw8h\" (UniqueName: \"kubernetes.io/projected/16590d60-acfc-46a5-bcca-96b2ef80938b-kube-api-access-mrw8h\") pod \"maas-api-e2e-api-a-712877-7977458698-d6kps\" (UID: \"16590d60-acfc-46a5-bcca-96b2ef80938b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-712877-7977458698-d6kps"
Jul 21 15:11:27.124777 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:27.124614    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16590d60-acfc-46a5-bcca-96b2ef80938b-openshift-service-ca\") pod \"maas-api-e2e-api-a-712877-7977458698-d6kps\" (UID: \"16590d60-acfc-46a5-bcca-96b2ef80938b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-712877-7977458698-d6kps"
Jul 21 15:11:27.125177 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:27.125159    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16590d60-acfc-46a5-bcca-96b2ef80938b-openshift-service-ca\") pod \"maas-api-e2e-api-a-712877-7977458698-d6kps\" (UID: \"16590d60-acfc-46a5-bcca-96b2ef80938b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-712877-7977458698-d6kps"
Jul 21 15:11:27.126975 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:27.126959    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16590d60-acfc-46a5-bcca-96b2ef80938b-maas-api-tls\") pod \"maas-api-e2e-api-a-712877-7977458698-d6kps\" (UID: \"16590d60-acfc-46a5-bcca-96b2ef80938b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-712877-7977458698-d6kps"
Jul 21 15:11:27.137334 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:27.137305    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrw8h\" (UniqueName: \"kubernetes.io/projected/16590d60-acfc-46a5-bcca-96b2ef80938b-kube-api-access-mrw8h\") pod \"maas-api-e2e-api-a-712877-7977458698-d6kps\" (UID: \"16590d60-acfc-46a5-bcca-96b2ef80938b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-712877-7977458698-d6kps"
Jul 21 15:11:27.215603 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:27.215526    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-712877-7977458698-d6kps"
Jul 21 15:11:27.347743 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:27.347719    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-712877-7977458698-d6kps"]
Jul 21 15:11:27.349948 ip-10-0-130-83 kubenswrapper[2579]: W0721 15:11:27.349923    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16590d60_acfc_46a5_bcca_96b2ef80938b.slice/crio-d2e26d5cea8b6df62ccd8c7697f1eeaa029165032af36705c5b9e520c80e0cc0 WatchSource:0}: Error finding container d2e26d5cea8b6df62ccd8c7697f1eeaa029165032af36705c5b9e520c80e0cc0: Status 404 returned error can't find the container with id d2e26d5cea8b6df62ccd8c7697f1eeaa029165032af36705c5b9e520c80e0cc0
Jul 21 15:11:27.351213 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:27.351189    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 15:11:27.504227 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:27.504156    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-712877-7977458698-d6kps" event={"ID":"16590d60-acfc-46a5-bcca-96b2ef80938b","Type":"ContainerStarted","Data":"d2e26d5cea8b6df62ccd8c7697f1eeaa029165032af36705c5b9e520c80e0cc0"}
Jul 21 15:11:28.511660 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:28.511625    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-712877-7977458698-d6kps" event={"ID":"16590d60-acfc-46a5-bcca-96b2ef80938b","Type":"ContainerStarted","Data":"4b50a5577cb1e899e822c7f693a935cfe9c39bb2711015a94ac635bde6b57f6d"}
Jul 21 15:11:28.512045 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:28.511759    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-712877-7977458698-d6kps"
Jul 21 15:11:28.545285 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:28.545231    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-712877-7977458698-d6kps" podStartSLOduration=2.046571702 podStartE2EDuration="2.545215634s" totalImagesPullingTime="498.643932ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 15:11:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 15:11:28.542301026 +0000 UTC m=+2344.892044438" watchObservedRunningTime="2026-07-21 15:11:28.545215634 +0000 UTC m=+2344.894959046"
Jul 21 15:11:34.521401 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:34.521374    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-712877-7977458698-d6kps"
Jul 21 15:11:35.106039 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:35.106008    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-99a04e-65476f6cfc-64zq2"]
Jul 21 15:11:35.109491 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:35.109470    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-99a04e-65476f6cfc-64zq2"
Jul 21 15:11:35.111892 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:35.111869    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-99a04e-serving-cert\""
Jul 21 15:11:35.121312 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:35.121263    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-99a04e-65476f6cfc-64zq2"]
Jul 21 15:11:35.196375 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:35.196342    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efa53695-ce63-48b8-b98a-1600423b528a-maas-api-tls\") pod \"maas-api-e2e-api-b-99a04e-65476f6cfc-64zq2\" (UID: \"efa53695-ce63-48b8-b98a-1600423b528a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-99a04e-65476f6cfc-64zq2"
Jul 21 15:11:35.196587 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:35.196410    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efa53695-ce63-48b8-b98a-1600423b528a-openshift-service-ca\") pod \"maas-api-e2e-api-b-99a04e-65476f6cfc-64zq2\" (UID: \"efa53695-ce63-48b8-b98a-1600423b528a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-99a04e-65476f6cfc-64zq2"
Jul 21 15:11:35.196587 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:35.196484    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-654kl\" (UniqueName: \"kubernetes.io/projected/efa53695-ce63-48b8-b98a-1600423b528a-kube-api-access-654kl\") pod \"maas-api-e2e-api-b-99a04e-65476f6cfc-64zq2\" (UID: \"efa53695-ce63-48b8-b98a-1600423b528a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-99a04e-65476f6cfc-64zq2"
Jul 21 15:11:35.297329 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:35.297290    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-654kl\" (UniqueName: \"kubernetes.io/projected/efa53695-ce63-48b8-b98a-1600423b528a-kube-api-access-654kl\") pod \"maas-api-e2e-api-b-99a04e-65476f6cfc-64zq2\" (UID: \"efa53695-ce63-48b8-b98a-1600423b528a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-99a04e-65476f6cfc-64zq2"
Jul 21 15:11:35.297522 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:35.297409    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efa53695-ce63-48b8-b98a-1600423b528a-maas-api-tls\") pod \"maas-api-e2e-api-b-99a04e-65476f6cfc-64zq2\" (UID: \"efa53695-ce63-48b8-b98a-1600423b528a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-99a04e-65476f6cfc-64zq2"
Jul 21 15:11:35.297522 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:35.297499    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efa53695-ce63-48b8-b98a-1600423b528a-openshift-service-ca\") pod \"maas-api-e2e-api-b-99a04e-65476f6cfc-64zq2\" (UID: \"efa53695-ce63-48b8-b98a-1600423b528a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-99a04e-65476f6cfc-64zq2"
Jul 21 15:11:35.298088 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:35.298072    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efa53695-ce63-48b8-b98a-1600423b528a-openshift-service-ca\") pod \"maas-api-e2e-api-b-99a04e-65476f6cfc-64zq2\" (UID: \"efa53695-ce63-48b8-b98a-1600423b528a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-99a04e-65476f6cfc-64zq2"
Jul 21 15:11:35.299851 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:35.299831    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efa53695-ce63-48b8-b98a-1600423b528a-maas-api-tls\") pod \"maas-api-e2e-api-b-99a04e-65476f6cfc-64zq2\" (UID: \"efa53695-ce63-48b8-b98a-1600423b528a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-99a04e-65476f6cfc-64zq2"
Jul 21 15:11:35.304892 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:35.304875    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-654kl\" (UniqueName: \"kubernetes.io/projected/efa53695-ce63-48b8-b98a-1600423b528a-kube-api-access-654kl\") pod \"maas-api-e2e-api-b-99a04e-65476f6cfc-64zq2\" (UID: \"efa53695-ce63-48b8-b98a-1600423b528a\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-99a04e-65476f6cfc-64zq2"
Jul 21 15:11:35.421864 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:35.421838    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-99a04e-65476f6cfc-64zq2"
Jul 21 15:11:35.582032 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:35.582009    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-99a04e-65476f6cfc-64zq2"]
Jul 21 15:11:35.584825 ip-10-0-130-83 kubenswrapper[2579]: W0721 15:11:35.584794    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefa53695_ce63_48b8_b98a_1600423b528a.slice/crio-e94604d3a5bf39a700db6f06b46236140f0c498f253a34e66da33ed5a67308de WatchSource:0}: Error finding container e94604d3a5bf39a700db6f06b46236140f0c498f253a34e66da33ed5a67308de: Status 404 returned error can't find the container with id e94604d3a5bf39a700db6f06b46236140f0c498f253a34e66da33ed5a67308de
Jul 21 15:11:36.559353 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:36.559314    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-99a04e-65476f6cfc-64zq2" event={"ID":"efa53695-ce63-48b8-b98a-1600423b528a","Type":"ContainerStarted","Data":"e94604d3a5bf39a700db6f06b46236140f0c498f253a34e66da33ed5a67308de"}
Jul 21 15:11:38.570622 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:38.570533    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-99a04e-65476f6cfc-64zq2" event={"ID":"efa53695-ce63-48b8-b98a-1600423b528a","Type":"ContainerStarted","Data":"599a1d4c923bd87d57f54925e4f367441a321822a61f7642168377855321553d"}
Jul 21 15:11:38.571050 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:38.570668    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-99a04e-65476f6cfc-64zq2"
Jul 21 15:11:38.589796 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:38.589748    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-99a04e-65476f6cfc-64zq2" podStartSLOduration=0.982236395 podStartE2EDuration="3.589734371s" totalImagesPullingTime="2.607497976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 15:11:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 15:11:38.587150008 +0000 UTC m=+2354.936893432" watchObservedRunningTime="2026-07-21 15:11:38.589734371 +0000 UTC m=+2354.939477788"
Jul 21 15:11:44.580791 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:44.580762    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-99a04e-65476f6cfc-64zq2"
Jul 21 15:11:47.375162 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:47.375116    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-712877-54v4r"]
Jul 21 15:11:47.378810 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:47.378786    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-712877-54v4r"
Jul 21 15:11:47.381439 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:47.381418    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-494sx\""
Jul 21 15:11:47.385819 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:47.385312    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-712877-54v4r"]
Jul 21 15:11:47.496882 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:47.496856    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2e8af8c-b4a9-45ce-8092-8ef14ec3fc6d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-712877-54v4r\" (UID: \"c2e8af8c-b4a9-45ce-8092-8ef14ec3fc6d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-712877-54v4r"
Jul 21 15:11:47.597544 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:47.597509    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2e8af8c-b4a9-45ce-8092-8ef14ec3fc6d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-712877-54v4r\" (UID: \"c2e8af8c-b4a9-45ce-8092-8ef14ec3fc6d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-712877-54v4r"
Jul 21 15:11:47.598103 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:47.598083    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2e8af8c-b4a9-45ce-8092-8ef14ec3fc6d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-712877-54v4r\" (UID: \"c2e8af8c-b4a9-45ce-8092-8ef14ec3fc6d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-712877-54v4r"
Jul 21 15:11:47.691730 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:47.691695    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-712877-54v4r"
Jul 21 15:11:47.825691 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:47.825666    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-712877-54v4r"]
Jul 21 15:11:47.827398 ip-10-0-130-83 kubenswrapper[2579]: W0721 15:11:47.827372    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2e8af8c_b4a9_45ce_8092_8ef14ec3fc6d.slice/crio-91251864ea80e9352a62952765fb5485ef297f163f35e0a0560e29a33717bc70 WatchSource:0}: Error finding container 91251864ea80e9352a62952765fb5485ef297f163f35e0a0560e29a33717bc70: Status 404 returned error can't find the container with id 91251864ea80e9352a62952765fb5485ef297f163f35e0a0560e29a33717bc70
Jul 21 15:11:48.628025 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:48.627985    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-712877-54v4r" event={"ID":"c2e8af8c-b4a9-45ce-8092-8ef14ec3fc6d","Type":"ContainerStarted","Data":"ad70423f35ee24e8d923724241a710371093d06ab54202da92e8efb8d843ff6e"}
Jul 21 15:11:48.628423 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:48.628048    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-712877-54v4r" event={"ID":"c2e8af8c-b4a9-45ce-8092-8ef14ec3fc6d","Type":"ContainerStarted","Data":"91251864ea80e9352a62952765fb5485ef297f163f35e0a0560e29a33717bc70"}
Jul 21 15:11:48.644100 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:48.644056    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-712877-54v4r" podStartSLOduration=1.644041113 podStartE2EDuration="1.644041113s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 15:11:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 15:11:48.642109377 +0000 UTC m=+2364.991852788" watchObservedRunningTime="2026-07-21 15:11:48.644041113 +0000 UTC m=+2364.993784524"
Jul 21 15:11:49.634351 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:49.634318    2579 generic.go:356] "Generic (PLEG): container finished" podID="c2e8af8c-b4a9-45ce-8092-8ef14ec3fc6d" containerID="ad70423f35ee24e8d923724241a710371093d06ab54202da92e8efb8d843ff6e" exitCode=0
Jul 21 15:11:49.634732 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:49.634392    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-712877-54v4r" event={"ID":"c2e8af8c-b4a9-45ce-8092-8ef14ec3fc6d","Type":"ContainerDied","Data":"ad70423f35ee24e8d923724241a710371093d06ab54202da92e8efb8d843ff6e"}
Jul 21 15:11:50.773414 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:50.773393    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-712877-54v4r"
Jul 21 15:11:50.926319 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:50.926242    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2e8af8c-b4a9-45ce-8092-8ef14ec3fc6d-maas-api-service-ca\") pod \"c2e8af8c-b4a9-45ce-8092-8ef14ec3fc6d\" (UID: \"c2e8af8c-b4a9-45ce-8092-8ef14ec3fc6d\") "
Jul 21 15:11:50.926577 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:50.926555    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2e8af8c-b4a9-45ce-8092-8ef14ec3fc6d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c2e8af8c-b4a9-45ce-8092-8ef14ec3fc6d" (UID: "c2e8af8c-b4a9-45ce-8092-8ef14ec3fc6d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 15:11:51.026805 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:51.026778    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2e8af8c-b4a9-45ce-8092-8ef14ec3fc6d-maas-api-service-ca\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:11:51.645208 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:51.645184    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-712877-54v4r"
Jul 21 15:11:51.645208 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:51.645191    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-712877-54v4r" event={"ID":"c2e8af8c-b4a9-45ce-8092-8ef14ec3fc6d","Type":"ContainerDied","Data":"91251864ea80e9352a62952765fb5485ef297f163f35e0a0560e29a33717bc70"}
Jul 21 15:11:51.645430 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:11:51.645232    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="91251864ea80e9352a62952765fb5485ef297f163f35e0a0560e29a33717bc70"
Jul 21 15:12:03.299178 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:03.299130    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-712877-7977458698-d6kps"]
Jul 21 15:12:03.299657 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:03.299462    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-712877-7977458698-d6kps" podUID="16590d60-acfc-46a5-bcca-96b2ef80938b" containerName="maas-api" containerID="cri-o://4b50a5577cb1e899e822c7f693a935cfe9c39bb2711015a94ac635bde6b57f6d" gracePeriod=30
Jul 21 15:12:03.552517 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:03.552463    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-712877-7977458698-d6kps"
Jul 21 15:12:03.708697 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:03.708658    2579 generic.go:356] "Generic (PLEG): container finished" podID="16590d60-acfc-46a5-bcca-96b2ef80938b" containerID="4b50a5577cb1e899e822c7f693a935cfe9c39bb2711015a94ac635bde6b57f6d" exitCode=0
Jul 21 15:12:03.708890 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:03.708715    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-712877-7977458698-d6kps"
Jul 21 15:12:03.708890 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:03.708732    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-712877-7977458698-d6kps" event={"ID":"16590d60-acfc-46a5-bcca-96b2ef80938b","Type":"ContainerDied","Data":"4b50a5577cb1e899e822c7f693a935cfe9c39bb2711015a94ac635bde6b57f6d"}
Jul 21 15:12:03.708890 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:03.708801    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-712877-7977458698-d6kps" event={"ID":"16590d60-acfc-46a5-bcca-96b2ef80938b","Type":"ContainerDied","Data":"d2e26d5cea8b6df62ccd8c7697f1eeaa029165032af36705c5b9e520c80e0cc0"}
Jul 21 15:12:03.708890 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:03.708822    2579 scope.go:117] "RemoveContainer" containerID="4b50a5577cb1e899e822c7f693a935cfe9c39bb2711015a94ac635bde6b57f6d"
Jul 21 15:12:03.717028 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:03.717012    2579 scope.go:117] "RemoveContainer" containerID="4b50a5577cb1e899e822c7f693a935cfe9c39bb2711015a94ac635bde6b57f6d"
Jul 21 15:12:03.717298 ip-10-0-130-83 kubenswrapper[2579]: E0721 15:12:03.717257    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b50a5577cb1e899e822c7f693a935cfe9c39bb2711015a94ac635bde6b57f6d\": container with ID starting with 4b50a5577cb1e899e822c7f693a935cfe9c39bb2711015a94ac635bde6b57f6d not found: ID does not exist" containerID="4b50a5577cb1e899e822c7f693a935cfe9c39bb2711015a94ac635bde6b57f6d"
Jul 21 15:12:03.717362 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:03.717307    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b50a5577cb1e899e822c7f693a935cfe9c39bb2711015a94ac635bde6b57f6d"} err="failed to get container status \"4b50a5577cb1e899e822c7f693a935cfe9c39bb2711015a94ac635bde6b57f6d\": rpc error: code = NotFound desc = could not find container \"4b50a5577cb1e899e822c7f693a935cfe9c39bb2711015a94ac635bde6b57f6d\": container with ID starting with 4b50a5577cb1e899e822c7f693a935cfe9c39bb2711015a94ac635bde6b57f6d not found: ID does not exist"
Jul 21 15:12:03.735776 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:03.735751    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16590d60-acfc-46a5-bcca-96b2ef80938b-maas-api-tls\") pod \"16590d60-acfc-46a5-bcca-96b2ef80938b\" (UID: \"16590d60-acfc-46a5-bcca-96b2ef80938b\") "
Jul 21 15:12:03.735865 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:03.735847    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16590d60-acfc-46a5-bcca-96b2ef80938b-openshift-service-ca\") pod \"16590d60-acfc-46a5-bcca-96b2ef80938b\" (UID: \"16590d60-acfc-46a5-bcca-96b2ef80938b\") "
Jul 21 15:12:03.735903 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:03.735879    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mrw8h\" (UniqueName: \"kubernetes.io/projected/16590d60-acfc-46a5-bcca-96b2ef80938b-kube-api-access-mrw8h\") pod \"16590d60-acfc-46a5-bcca-96b2ef80938b\" (UID: \"16590d60-acfc-46a5-bcca-96b2ef80938b\") "
Jul 21 15:12:03.736256 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:03.736230    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16590d60-acfc-46a5-bcca-96b2ef80938b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "16590d60-acfc-46a5-bcca-96b2ef80938b" (UID: "16590d60-acfc-46a5-bcca-96b2ef80938b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 15:12:03.737894 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:03.737870    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16590d60-acfc-46a5-bcca-96b2ef80938b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "16590d60-acfc-46a5-bcca-96b2ef80938b" (UID: "16590d60-acfc-46a5-bcca-96b2ef80938b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 15:12:03.737894 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:03.737882    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16590d60-acfc-46a5-bcca-96b2ef80938b-kube-api-access-mrw8h" (OuterVolumeSpecName: "kube-api-access-mrw8h") pod "16590d60-acfc-46a5-bcca-96b2ef80938b" (UID: "16590d60-acfc-46a5-bcca-96b2ef80938b"). InnerVolumeSpecName "kube-api-access-mrw8h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 15:12:03.836495 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:03.836409    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16590d60-acfc-46a5-bcca-96b2ef80938b-openshift-service-ca\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:12:03.836495 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:03.836437    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrw8h\" (UniqueName: \"kubernetes.io/projected/16590d60-acfc-46a5-bcca-96b2ef80938b-kube-api-access-mrw8h\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:12:03.836495 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:03.836449    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16590d60-acfc-46a5-bcca-96b2ef80938b-maas-api-tls\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:12:04.030291 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:04.030236    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-712877-7977458698-d6kps"]
Jul 21 15:12:04.034721 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:04.034697    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-712877-7977458698-d6kps"]
Jul 21 15:12:04.172366 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:04.172334    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16590d60-acfc-46a5-bcca-96b2ef80938b" path="/var/lib/kubelet/pods/16590d60-acfc-46a5-bcca-96b2ef80938b/volumes"
Jul 21 15:12:07.462175 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:07.462140    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-712877-54v4r"]
Jul 21 15:12:07.464605 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:07.464580    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-712877-54v4r"]
Jul 21 15:12:08.172939 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:08.172911    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2e8af8c-b4a9-45ce-8092-8ef14ec3fc6d" path="/var/lib/kubelet/pods/c2e8af8c-b4a9-45ce-8092-8ef14ec3fc6d/volumes"
Jul 21 15:12:14.392943 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:14.392911    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-99a04e-ckhrq"]
Jul 21 15:12:14.393333 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:14.393254    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16590d60-acfc-46a5-bcca-96b2ef80938b" containerName="maas-api"
Jul 21 15:12:14.393333 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:14.393282    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="16590d60-acfc-46a5-bcca-96b2ef80938b" containerName="maas-api"
Jul 21 15:12:14.393333 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:14.393296    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2e8af8c-b4a9-45ce-8092-8ef14ec3fc6d" containerName="revoke-keys"
Jul 21 15:12:14.393333 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:14.393303    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2e8af8c-b4a9-45ce-8092-8ef14ec3fc6d" containerName="revoke-keys"
Jul 21 15:12:14.393468 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:14.393371    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="16590d60-acfc-46a5-bcca-96b2ef80938b" containerName="maas-api"
Jul 21 15:12:14.393468 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:14.393381    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2e8af8c-b4a9-45ce-8092-8ef14ec3fc6d" containerName="revoke-keys"
Jul 21 15:12:14.397558 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:14.397536    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-99a04e-ckhrq"
Jul 21 15:12:14.400195 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:14.400176    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-494sx\""
Jul 21 15:12:14.406382 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:14.406361    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-99a04e-ckhrq"]
Jul 21 15:12:14.420856 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:14.420829    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eff635d0-19b7-408c-9a29-087e1b315773-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-99a04e-ckhrq\" (UID: \"eff635d0-19b7-408c-9a29-087e1b315773\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-99a04e-ckhrq"
Jul 21 15:12:14.521332 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:14.521304    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eff635d0-19b7-408c-9a29-087e1b315773-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-99a04e-ckhrq\" (UID: \"eff635d0-19b7-408c-9a29-087e1b315773\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-99a04e-ckhrq"
Jul 21 15:12:14.521887 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:14.521869    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eff635d0-19b7-408c-9a29-087e1b315773-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-99a04e-ckhrq\" (UID: \"eff635d0-19b7-408c-9a29-087e1b315773\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-99a04e-ckhrq"
Jul 21 15:12:14.707663 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:14.707587    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-99a04e-ckhrq"
Jul 21 15:12:14.842577 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:14.842391    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-99a04e-ckhrq"]
Jul 21 15:12:14.845322 ip-10-0-130-83 kubenswrapper[2579]: W0721 15:12:14.845295    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeff635d0_19b7_408c_9a29_087e1b315773.slice/crio-d849f7ae3ceb3d2c8bde59f1a0ae3c5e6e334f45413d26920fc00c9e28b1d235 WatchSource:0}: Error finding container d849f7ae3ceb3d2c8bde59f1a0ae3c5e6e334f45413d26920fc00c9e28b1d235: Status 404 returned error can't find the container with id d849f7ae3ceb3d2c8bde59f1a0ae3c5e6e334f45413d26920fc00c9e28b1d235
Jul 21 15:12:15.773941 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:15.773903    2579 generic.go:356] "Generic (PLEG): container finished" podID="eff635d0-19b7-408c-9a29-087e1b315773" containerID="90c62fccc01225e2a010d7df2fb41ed544c543edec5de56f8e5d2ddf8649ce37" exitCode=0
Jul 21 15:12:15.774369 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:15.773995    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-99a04e-ckhrq" event={"ID":"eff635d0-19b7-408c-9a29-087e1b315773","Type":"ContainerDied","Data":"90c62fccc01225e2a010d7df2fb41ed544c543edec5de56f8e5d2ddf8649ce37"}
Jul 21 15:12:15.774369 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:15.774058    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-99a04e-ckhrq" event={"ID":"eff635d0-19b7-408c-9a29-087e1b315773","Type":"ContainerStarted","Data":"d849f7ae3ceb3d2c8bde59f1a0ae3c5e6e334f45413d26920fc00c9e28b1d235"}
Jul 21 15:12:16.910352 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:16.910329    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-99a04e-ckhrq"
Jul 21 15:12:16.944340 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:16.944319    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eff635d0-19b7-408c-9a29-087e1b315773-maas-api-service-ca\") pod \"eff635d0-19b7-408c-9a29-087e1b315773\" (UID: \"eff635d0-19b7-408c-9a29-087e1b315773\") "
Jul 21 15:12:16.944669 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:16.944637    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eff635d0-19b7-408c-9a29-087e1b315773-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "eff635d0-19b7-408c-9a29-087e1b315773" (UID: "eff635d0-19b7-408c-9a29-087e1b315773"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 15:12:17.045514 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:17.045461    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eff635d0-19b7-408c-9a29-087e1b315773-maas-api-service-ca\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:12:17.785549 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:17.785520    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-99a04e-ckhrq"
Jul 21 15:12:17.785549 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:17.785541    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-99a04e-ckhrq" event={"ID":"eff635d0-19b7-408c-9a29-087e1b315773","Type":"ContainerDied","Data":"d849f7ae3ceb3d2c8bde59f1a0ae3c5e6e334f45413d26920fc00c9e28b1d235"}
Jul 21 15:12:17.785761 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:17.785583    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d849f7ae3ceb3d2c8bde59f1a0ae3c5e6e334f45413d26920fc00c9e28b1d235"
Jul 21 15:12:24.420926 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:24.420895    2579 scope.go:117] "RemoveContainer" containerID="75270c0dad8bbc921854cc48b4c9e58dedb8c7d5445d78f01d4edf4de53397c6"
Jul 21 15:12:29.942194 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:29.942163    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-99a04e-65476f6cfc-64zq2"]
Jul 21 15:12:29.942609 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:29.942373    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-99a04e-65476f6cfc-64zq2" podUID="efa53695-ce63-48b8-b98a-1600423b528a" containerName="maas-api" containerID="cri-o://599a1d4c923bd87d57f54925e4f367441a321822a61f7642168377855321553d" gracePeriod=30
Jul 21 15:12:30.189419 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:30.189396    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-99a04e-65476f6cfc-64zq2"
Jul 21 15:12:30.255394 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:30.255365    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efa53695-ce63-48b8-b98a-1600423b528a-maas-api-tls\") pod \"efa53695-ce63-48b8-b98a-1600423b528a\" (UID: \"efa53695-ce63-48b8-b98a-1600423b528a\") "
Jul 21 15:12:30.255527 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:30.255417    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efa53695-ce63-48b8-b98a-1600423b528a-openshift-service-ca\") pod \"efa53695-ce63-48b8-b98a-1600423b528a\" (UID: \"efa53695-ce63-48b8-b98a-1600423b528a\") "
Jul 21 15:12:30.255527 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:30.255442    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-654kl\" (UniqueName: \"kubernetes.io/projected/efa53695-ce63-48b8-b98a-1600423b528a-kube-api-access-654kl\") pod \"efa53695-ce63-48b8-b98a-1600423b528a\" (UID: \"efa53695-ce63-48b8-b98a-1600423b528a\") "
Jul 21 15:12:30.255771 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:30.255748    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efa53695-ce63-48b8-b98a-1600423b528a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "efa53695-ce63-48b8-b98a-1600423b528a" (UID: "efa53695-ce63-48b8-b98a-1600423b528a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 15:12:30.257308 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:30.257281    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efa53695-ce63-48b8-b98a-1600423b528a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "efa53695-ce63-48b8-b98a-1600423b528a" (UID: "efa53695-ce63-48b8-b98a-1600423b528a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 15:12:30.257422 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:30.257379    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efa53695-ce63-48b8-b98a-1600423b528a-kube-api-access-654kl" (OuterVolumeSpecName: "kube-api-access-654kl") pod "efa53695-ce63-48b8-b98a-1600423b528a" (UID: "efa53695-ce63-48b8-b98a-1600423b528a"). InnerVolumeSpecName "kube-api-access-654kl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 15:12:30.356355 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:30.356327    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efa53695-ce63-48b8-b98a-1600423b528a-maas-api-tls\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:12:30.356355 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:30.356351    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efa53695-ce63-48b8-b98a-1600423b528a-openshift-service-ca\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:12:30.356355 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:30.356360    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-654kl\" (UniqueName: \"kubernetes.io/projected/efa53695-ce63-48b8-b98a-1600423b528a-kube-api-access-654kl\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:12:30.849955 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:30.849927    2579 generic.go:356] "Generic (PLEG): container finished" podID="efa53695-ce63-48b8-b98a-1600423b528a" containerID="599a1d4c923bd87d57f54925e4f367441a321822a61f7642168377855321553d" exitCode=0
Jul 21 15:12:30.850120 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:30.849974    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-99a04e-65476f6cfc-64zq2"
Jul 21 15:12:30.850120 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:30.849990    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-99a04e-65476f6cfc-64zq2" event={"ID":"efa53695-ce63-48b8-b98a-1600423b528a","Type":"ContainerDied","Data":"599a1d4c923bd87d57f54925e4f367441a321822a61f7642168377855321553d"}
Jul 21 15:12:30.850120 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:30.850033    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-99a04e-65476f6cfc-64zq2" event={"ID":"efa53695-ce63-48b8-b98a-1600423b528a","Type":"ContainerDied","Data":"e94604d3a5bf39a700db6f06b46236140f0c498f253a34e66da33ed5a67308de"}
Jul 21 15:12:30.850120 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:30.850054    2579 scope.go:117] "RemoveContainer" containerID="599a1d4c923bd87d57f54925e4f367441a321822a61f7642168377855321553d"
Jul 21 15:12:30.859232 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:30.859215    2579 scope.go:117] "RemoveContainer" containerID="599a1d4c923bd87d57f54925e4f367441a321822a61f7642168377855321553d"
Jul 21 15:12:30.859514 ip-10-0-130-83 kubenswrapper[2579]: E0721 15:12:30.859495    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"599a1d4c923bd87d57f54925e4f367441a321822a61f7642168377855321553d\": container with ID starting with 599a1d4c923bd87d57f54925e4f367441a321822a61f7642168377855321553d not found: ID does not exist" containerID="599a1d4c923bd87d57f54925e4f367441a321822a61f7642168377855321553d"
Jul 21 15:12:30.859568 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:30.859521    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"599a1d4c923bd87d57f54925e4f367441a321822a61f7642168377855321553d"} err="failed to get container status \"599a1d4c923bd87d57f54925e4f367441a321822a61f7642168377855321553d\": rpc error: code = NotFound desc = could not find container \"599a1d4c923bd87d57f54925e4f367441a321822a61f7642168377855321553d\": container with ID starting with 599a1d4c923bd87d57f54925e4f367441a321822a61f7642168377855321553d not found: ID does not exist"
Jul 21 15:12:30.871323 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:30.871301    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-99a04e-65476f6cfc-64zq2"]
Jul 21 15:12:30.874562 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:30.874543    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-99a04e-65476f6cfc-64zq2"]
Jul 21 15:12:32.171254 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:32.171223    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efa53695-ce63-48b8-b98a-1600423b528a" path="/var/lib/kubelet/pods/efa53695-ce63-48b8-b98a-1600423b528a/volumes"
Jul 21 15:12:34.469870 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:34.469837    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-99a04e-ckhrq"]
Jul 21 15:12:34.473383 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:34.473359    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-99a04e-ckhrq"]
Jul 21 15:12:36.171801 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:36.171769    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eff635d0-19b7-408c-9a29-087e1b315773" path="/var/lib/kubelet/pods/eff635d0-19b7-408c-9a29-087e1b315773/volumes"
Jul 21 15:12:45.402464 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:45.402422    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-9b06ae-68d45c75cb-499d2"]
Jul 21 15:12:45.403017 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:45.402845    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efa53695-ce63-48b8-b98a-1600423b528a" containerName="maas-api"
Jul 21 15:12:45.403017 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:45.402860    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="efa53695-ce63-48b8-b98a-1600423b528a" containerName="maas-api"
Jul 21 15:12:45.403017 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:45.402873    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eff635d0-19b7-408c-9a29-087e1b315773" containerName="revoke-keys"
Jul 21 15:12:45.403017 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:45.402879    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="eff635d0-19b7-408c-9a29-087e1b315773" containerName="revoke-keys"
Jul 21 15:12:45.403017 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:45.402960    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="efa53695-ce63-48b8-b98a-1600423b528a" containerName="maas-api"
Jul 21 15:12:45.403017 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:45.402974    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="eff635d0-19b7-408c-9a29-087e1b315773" containerName="revoke-keys"
Jul 21 15:12:45.412970 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:45.412948    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b06ae-68d45c75cb-499d2"
Jul 21 15:12:45.418165 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:45.416387    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-9b06ae-serving-cert\""
Jul 21 15:12:45.418165 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:45.418077    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-9b06ae-68d45c75cb-499d2"]
Jul 21 15:12:45.480531 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:45.480504    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa42b39a-4822-47b5-94d1-7a067f53c113-maas-api-tls\") pod \"maas-api-e2e-shared-a-9b06ae-68d45c75cb-499d2\" (UID: \"aa42b39a-4822-47b5-94d1-7a067f53c113\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b06ae-68d45c75cb-499d2"
Jul 21 15:12:45.480631 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:45.480564    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa42b39a-4822-47b5-94d1-7a067f53c113-openshift-service-ca\") pod \"maas-api-e2e-shared-a-9b06ae-68d45c75cb-499d2\" (UID: \"aa42b39a-4822-47b5-94d1-7a067f53c113\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b06ae-68d45c75cb-499d2"
Jul 21 15:12:45.480711 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:45.480688    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5949t\" (UniqueName: \"kubernetes.io/projected/aa42b39a-4822-47b5-94d1-7a067f53c113-kube-api-access-5949t\") pod \"maas-api-e2e-shared-a-9b06ae-68d45c75cb-499d2\" (UID: \"aa42b39a-4822-47b5-94d1-7a067f53c113\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b06ae-68d45c75cb-499d2"
Jul 21 15:12:45.581148 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:45.581112    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa42b39a-4822-47b5-94d1-7a067f53c113-maas-api-tls\") pod \"maas-api-e2e-shared-a-9b06ae-68d45c75cb-499d2\" (UID: \"aa42b39a-4822-47b5-94d1-7a067f53c113\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b06ae-68d45c75cb-499d2"
Jul 21 15:12:45.581322 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:45.581162    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa42b39a-4822-47b5-94d1-7a067f53c113-openshift-service-ca\") pod \"maas-api-e2e-shared-a-9b06ae-68d45c75cb-499d2\" (UID: \"aa42b39a-4822-47b5-94d1-7a067f53c113\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b06ae-68d45c75cb-499d2"
Jul 21 15:12:45.581322 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:45.581206    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5949t\" (UniqueName: \"kubernetes.io/projected/aa42b39a-4822-47b5-94d1-7a067f53c113-kube-api-access-5949t\") pod \"maas-api-e2e-shared-a-9b06ae-68d45c75cb-499d2\" (UID: \"aa42b39a-4822-47b5-94d1-7a067f53c113\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b06ae-68d45c75cb-499d2"
Jul 21 15:12:45.581322 ip-10-0-130-83 kubenswrapper[2579]: E0721 15:12:45.581301    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-9b06ae-serving-cert: secret "maas-api-e2e-shared-a-9b06ae-serving-cert" not found
Jul 21 15:12:45.581506 ip-10-0-130-83 kubenswrapper[2579]: E0721 15:12:45.581394    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa42b39a-4822-47b5-94d1-7a067f53c113-maas-api-tls podName:aa42b39a-4822-47b5-94d1-7a067f53c113 nodeName:}" failed. No retries permitted until 2026-07-21 15:12:46.081371052 +0000 UTC m=+2422.431114445 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/aa42b39a-4822-47b5-94d1-7a067f53c113-maas-api-tls") pod "maas-api-e2e-shared-a-9b06ae-68d45c75cb-499d2" (UID: "aa42b39a-4822-47b5-94d1-7a067f53c113") : secret "maas-api-e2e-shared-a-9b06ae-serving-cert" not found
Jul 21 15:12:45.581878 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:45.581858    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa42b39a-4822-47b5-94d1-7a067f53c113-openshift-service-ca\") pod \"maas-api-e2e-shared-a-9b06ae-68d45c75cb-499d2\" (UID: \"aa42b39a-4822-47b5-94d1-7a067f53c113\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b06ae-68d45c75cb-499d2"
Jul 21 15:12:45.590365 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:45.590334    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5949t\" (UniqueName: \"kubernetes.io/projected/aa42b39a-4822-47b5-94d1-7a067f53c113-kube-api-access-5949t\") pod \"maas-api-e2e-shared-a-9b06ae-68d45c75cb-499d2\" (UID: \"aa42b39a-4822-47b5-94d1-7a067f53c113\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b06ae-68d45c75cb-499d2"
Jul 21 15:12:46.084996 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:46.084919    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa42b39a-4822-47b5-94d1-7a067f53c113-maas-api-tls\") pod \"maas-api-e2e-shared-a-9b06ae-68d45c75cb-499d2\" (UID: \"aa42b39a-4822-47b5-94d1-7a067f53c113\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b06ae-68d45c75cb-499d2"
Jul 21 15:12:46.087321 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:46.087301    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa42b39a-4822-47b5-94d1-7a067f53c113-maas-api-tls\") pod \"maas-api-e2e-shared-a-9b06ae-68d45c75cb-499d2\" (UID: \"aa42b39a-4822-47b5-94d1-7a067f53c113\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b06ae-68d45c75cb-499d2"
Jul 21 15:12:46.326810 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:46.326779    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b06ae-68d45c75cb-499d2"
Jul 21 15:12:46.464311 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:46.464264    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-9b06ae-68d45c75cb-499d2"]
Jul 21 15:12:46.467298 ip-10-0-130-83 kubenswrapper[2579]: W0721 15:12:46.467258    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa42b39a_4822_47b5_94d1_7a067f53c113.slice/crio-330dc57861c0e21f4dfc75dcce05d45d68d6af7b0412a47c8bce81a66c4145d4 WatchSource:0}: Error finding container 330dc57861c0e21f4dfc75dcce05d45d68d6af7b0412a47c8bce81a66c4145d4: Status 404 returned error can't find the container with id 330dc57861c0e21f4dfc75dcce05d45d68d6af7b0412a47c8bce81a66c4145d4
Jul 21 15:12:46.929063 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:46.929033    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b06ae-68d45c75cb-499d2" event={"ID":"aa42b39a-4822-47b5-94d1-7a067f53c113","Type":"ContainerStarted","Data":"e47f34f9a0631010ba56ce7e6f066a1a82ed1961cb738757cf7a0ee685b7988d"}
Jul 21 15:12:46.929171 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:46.929089    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b06ae-68d45c75cb-499d2" event={"ID":"aa42b39a-4822-47b5-94d1-7a067f53c113","Type":"ContainerStarted","Data":"330dc57861c0e21f4dfc75dcce05d45d68d6af7b0412a47c8bce81a66c4145d4"}
Jul 21 15:12:46.929216 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:46.929167    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b06ae-68d45c75cb-499d2"
Jul 21 15:12:46.947144 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:46.947093    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b06ae-68d45c75cb-499d2" podStartSLOduration=1.578585013 podStartE2EDuration="1.947079517s" totalImagesPullingTime="368.494504ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 15:12:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 15:12:46.945216603 +0000 UTC m=+2423.294960017" watchObservedRunningTime="2026-07-21 15:12:46.947079517 +0000 UTC m=+2423.296822929"
Jul 21 15:12:52.941556 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:52.941525    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b06ae-68d45c75cb-499d2"
Jul 21 15:12:53.551588 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:53.551558    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-26fa51-bd6667659-frvqs"]
Jul 21 15:12:53.555283 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:53.555251    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-26fa51-bd6667659-frvqs"
Jul 21 15:12:53.558134 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:53.558111    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-26fa51-serving-cert\""
Jul 21 15:12:53.566849 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:53.566827    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-26fa51-bd6667659-frvqs"]
Jul 21 15:12:53.651433 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:53.651403    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jd77n\" (UniqueName: \"kubernetes.io/projected/01c22400-ab70-43a9-8a4f-4598db1137f9-kube-api-access-jd77n\") pod \"maas-api-e2e-shared-b-26fa51-bd6667659-frvqs\" (UID: \"01c22400-ab70-43a9-8a4f-4598db1137f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-26fa51-bd6667659-frvqs"
Jul 21 15:12:53.651598 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:53.651447    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01c22400-ab70-43a9-8a4f-4598db1137f9-maas-api-tls\") pod \"maas-api-e2e-shared-b-26fa51-bd6667659-frvqs\" (UID: \"01c22400-ab70-43a9-8a4f-4598db1137f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-26fa51-bd6667659-frvqs"
Jul 21 15:12:53.651598 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:53.651468    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01c22400-ab70-43a9-8a4f-4598db1137f9-openshift-service-ca\") pod \"maas-api-e2e-shared-b-26fa51-bd6667659-frvqs\" (UID: \"01c22400-ab70-43a9-8a4f-4598db1137f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-26fa51-bd6667659-frvqs"
Jul 21 15:12:53.752870 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:53.752839    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jd77n\" (UniqueName: \"kubernetes.io/projected/01c22400-ab70-43a9-8a4f-4598db1137f9-kube-api-access-jd77n\") pod \"maas-api-e2e-shared-b-26fa51-bd6667659-frvqs\" (UID: \"01c22400-ab70-43a9-8a4f-4598db1137f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-26fa51-bd6667659-frvqs"
Jul 21 15:12:53.753045 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:53.752883    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01c22400-ab70-43a9-8a4f-4598db1137f9-maas-api-tls\") pod \"maas-api-e2e-shared-b-26fa51-bd6667659-frvqs\" (UID: \"01c22400-ab70-43a9-8a4f-4598db1137f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-26fa51-bd6667659-frvqs"
Jul 21 15:12:53.753045 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:53.752907    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01c22400-ab70-43a9-8a4f-4598db1137f9-openshift-service-ca\") pod \"maas-api-e2e-shared-b-26fa51-bd6667659-frvqs\" (UID: \"01c22400-ab70-43a9-8a4f-4598db1137f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-26fa51-bd6667659-frvqs"
Jul 21 15:12:53.753140 ip-10-0-130-83 kubenswrapper[2579]: E0721 15:12:53.753086    2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-26fa51-serving-cert: secret "maas-api-e2e-shared-b-26fa51-serving-cert" not found
Jul 21 15:12:53.753193 ip-10-0-130-83 kubenswrapper[2579]: E0721 15:12:53.753180    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01c22400-ab70-43a9-8a4f-4598db1137f9-maas-api-tls podName:01c22400-ab70-43a9-8a4f-4598db1137f9 nodeName:}" failed. No retries permitted until 2026-07-21 15:12:54.253150999 +0000 UTC m=+2430.602894391 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/01c22400-ab70-43a9-8a4f-4598db1137f9-maas-api-tls") pod "maas-api-e2e-shared-b-26fa51-bd6667659-frvqs" (UID: "01c22400-ab70-43a9-8a4f-4598db1137f9") : secret "maas-api-e2e-shared-b-26fa51-serving-cert" not found
Jul 21 15:12:53.753544 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:53.753527    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01c22400-ab70-43a9-8a4f-4598db1137f9-openshift-service-ca\") pod \"maas-api-e2e-shared-b-26fa51-bd6667659-frvqs\" (UID: \"01c22400-ab70-43a9-8a4f-4598db1137f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-26fa51-bd6667659-frvqs"
Jul 21 15:12:53.764914 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:53.764881    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jd77n\" (UniqueName: \"kubernetes.io/projected/01c22400-ab70-43a9-8a4f-4598db1137f9-kube-api-access-jd77n\") pod \"maas-api-e2e-shared-b-26fa51-bd6667659-frvqs\" (UID: \"01c22400-ab70-43a9-8a4f-4598db1137f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-26fa51-bd6667659-frvqs"
Jul 21 15:12:54.257654 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:54.257624    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01c22400-ab70-43a9-8a4f-4598db1137f9-maas-api-tls\") pod \"maas-api-e2e-shared-b-26fa51-bd6667659-frvqs\" (UID: \"01c22400-ab70-43a9-8a4f-4598db1137f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-26fa51-bd6667659-frvqs"
Jul 21 15:12:54.260029 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:54.260005    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01c22400-ab70-43a9-8a4f-4598db1137f9-maas-api-tls\") pod \"maas-api-e2e-shared-b-26fa51-bd6667659-frvqs\" (UID: \"01c22400-ab70-43a9-8a4f-4598db1137f9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-26fa51-bd6667659-frvqs"
Jul 21 15:12:54.466071 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:54.466029    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-26fa51-bd6667659-frvqs"
Jul 21 15:12:54.607682 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:54.607654    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-26fa51-bd6667659-frvqs"]
Jul 21 15:12:54.611128 ip-10-0-130-83 kubenswrapper[2579]: W0721 15:12:54.611103    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01c22400_ab70_43a9_8a4f_4598db1137f9.slice/crio-877cf07e960db3b8ddec60ff03fcdbd42a4fae8e6c8df8d21428add334d14756 WatchSource:0}: Error finding container 877cf07e960db3b8ddec60ff03fcdbd42a4fae8e6c8df8d21428add334d14756: Status 404 returned error can't find the container with id 877cf07e960db3b8ddec60ff03fcdbd42a4fae8e6c8df8d21428add334d14756
Jul 21 15:12:54.969677 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:54.969645    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-26fa51-bd6667659-frvqs" event={"ID":"01c22400-ab70-43a9-8a4f-4598db1137f9","Type":"ContainerStarted","Data":"877cf07e960db3b8ddec60ff03fcdbd42a4fae8e6c8df8d21428add334d14756"}
Jul 21 15:12:55.977180 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:55.977146    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-26fa51-bd6667659-frvqs" event={"ID":"01c22400-ab70-43a9-8a4f-4598db1137f9","Type":"ContainerStarted","Data":"94e574c6bda5437b4d0e1e3f622130930fdf377c9bfc65cf3d79e6679d8be612"}
Jul 21 15:12:55.977691 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:55.977306    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-26fa51-bd6667659-frvqs"
Jul 21 15:12:55.993099 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:12:55.993056    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-26fa51-bd6667659-frvqs" podStartSLOduration=2.441837824 podStartE2EDuration="2.993043163s" totalImagesPullingTime="551.205339ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 15:12:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 15:12:55.992082308 +0000 UTC m=+2432.341825730" watchObservedRunningTime="2026-07-21 15:12:55.993043163 +0000 UTC m=+2432.342786574"
Jul 21 15:13:01.987916 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:13:01.987891    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-26fa51-bd6667659-frvqs"
Jul 21 15:13:24.446027 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:13:24.445997    2579 scope.go:117] "RemoveContainer" containerID="d26ec526352184ab46e9bcdc5295f19e85ca36ccbaa45184ce6f05c804dd0267"
Jul 21 15:13:31.715465 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:13:31.715437    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-26fa51/auth-test-model-26fa51-kserve-7f485b6c5-tccn7"]
Jul 21 15:13:31.718665 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:13:31.718645    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-26fa51/auth-test-model-26fa51-kserve-7f485b6c5-tccn7"
Jul 21 15:13:31.722958 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:13:31.722739    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-26fa51\"/\"auth-test-model-26fa51-kserve-self-signed-certs\""
Jul 21 15:13:31.722958 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:13:31.722752    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-26fa51\"/\"kube-root-ca.crt\""
Jul 21 15:13:31.722958 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:13:31.722830    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-26fa51\"/\"openshift-service-ca.crt\""
Jul 21 15:13:31.722958 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:13:31.722875    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-26fa51\"/\"default-dockercfg-l2tcs\""
Jul 21 15:13:31.742085 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:13:31.742064    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-26fa51/auth-test-model-26fa51-kserve-7f485b6c5-tccn7"]
Jul 21 15:13:31.759564 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:13:31.759539    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af244b07-69cf-4acc-a725-72acbaf85d09-tmp-dir\") pod \"auth-test-model-26fa51-kserve-7f485b6c5-tccn7\" (UID: \"af244b07-69cf-4acc-a725-72acbaf85d09\") " pod="ai-tenant-e2e-shared-b-26fa51/auth-test-model-26fa51-kserve-7f485b6c5-tccn7"
Jul 21 15:13:31.759682 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:13:31.759612    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af244b07-69cf-4acc-a725-72acbaf85d09-tls-certs\") pod \"auth-test-model-26fa51-kserve-7f485b6c5-tccn7\" (UID: \"af244b07-69cf-4acc-a725-72acbaf85d09\") " pod="ai-tenant-e2e-shared-b-26fa51/auth-test-model-26fa51-kserve-7f485b6c5-tccn7"
Jul 21 15:13:31.759737 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:13:31.759709    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af244b07-69cf-4acc-a725-72acbaf85d09-home\") pod \"auth-test-model-26fa51-kserve-7f485b6c5-tccn7\" (UID: \"af244b07-69cf-4acc-a725-72acbaf85d09\") " pod="ai-tenant-e2e-shared-b-26fa51/auth-test-model-26fa51-kserve-7f485b6c5-tccn7"
Jul 21 15:13:31.759780 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:13:31.759757    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af244b07-69cf-4acc-a725-72acbaf85d09-dshm\") pod \"auth-test-model-26fa51-kserve-7f485b6c5-tccn7\" (UID: \"af244b07-69cf-4acc-a725-72acbaf85d09\") " pod="ai-tenant-e2e-shared-b-26fa51/auth-test-model-26fa51-kserve-7f485b6c5-tccn7"
Jul 21 15:13:31.759826 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:13:31.759788    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af244b07-69cf-4acc-a725-72acbaf85d09-model-cache\") pod \"auth-test-model-26fa51-kserve-7f485b6c5-tccn7\" (UID: \"af244b07-69cf-4acc-a725-72acbaf85d09\") " pod="ai-tenant-e2e-shared-b-26fa51/auth-test-model-26fa51-kserve-7f485b6c5-tccn7"
Jul 21 15:13:31.759863 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:13:31.759819    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xhgk\" (UniqueName: \"kubernetes.io/projected/af244b07-69cf-4acc-a725-72acbaf85d09-kube-api-access-8xhgk\") pod \"auth-test-model-26fa51-kserve-7f485b6c5-tccn7\" (UID: \"af244b07-69cf-4acc-a725-72acbaf85d09\") " pod="ai-tenant-e2e-shared-b-26fa51/auth-test-model-26fa51-kserve-7f485b6c5-tccn7"
Jul 21 15:13:31.860423 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:13:31.860394    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af244b07-69cf-4acc-a725-72acbaf85d09-home\") pod \"auth-test-model-26fa51-kserve-7f485b6c5-tccn7\" (UID: \"af244b07-69cf-4acc-a725-72acbaf85d09\") " pod="ai-tenant-e2e-shared-b-26fa51/auth-test-model-26fa51-kserve-7f485b6c5-tccn7"
Jul 21 15:13:31.860539 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:13:31.860432    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af244b07-69cf-4acc-a725-72acbaf85d09-dshm\") pod \"auth-test-model-26fa51-kserve-7f485b6c5-tccn7\" (UID: \"af244b07-69cf-4acc-a725-72acbaf85d09\") " pod="ai-tenant-e2e-shared-b-26fa51/auth-test-model-26fa51-kserve-7f485b6c5-tccn7"
Jul 21 15:13:31.860539 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:13:31.860456    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af244b07-69cf-4acc-a725-72acbaf85d09-model-cache\") pod \"auth-test-model-26fa51-kserve-7f485b6c5-tccn7\" (UID: \"af244b07-69cf-4acc-a725-72acbaf85d09\") " pod="ai-tenant-e2e-shared-b-26fa51/auth-test-model-26fa51-kserve-7f485b6c5-tccn7"
Jul 21 15:13:31.860539 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:13:31.860481    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xhgk\" (UniqueName: \"kubernetes.io/projected/af244b07-69cf-4acc-a725-72acbaf85d09-kube-api-access-8xhgk\") pod \"auth-test-model-26fa51-kserve-7f485b6c5-tccn7\" (UID: \"af244b07-69cf-4acc-a725-72acbaf85d09\") " pod="ai-tenant-e2e-shared-b-26fa51/auth-test-model-26fa51-kserve-7f485b6c5-tccn7"
Jul 21 15:13:31.860539 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:13:31.860523    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af244b07-69cf-4acc-a725-72acbaf85d09-tmp-dir\") pod \"auth-test-model-26fa51-kserve-7f485b6c5-tccn7\" (UID: \"af244b07-69cf-4acc-a725-72acbaf85d09\") " pod="ai-tenant-e2e-shared-b-26fa51/auth-test-model-26fa51-kserve-7f485b6c5-tccn7"
Jul 21 15:13:31.860756 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:13:31.860572    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af244b07-69cf-4acc-a725-72acbaf85d09-tls-certs\") pod \"auth-test-model-26fa51-kserve-7f485b6c5-tccn7\" (UID: \"af244b07-69cf-4acc-a725-72acbaf85d09\") " pod="ai-tenant-e2e-shared-b-26fa51/auth-test-model-26fa51-kserve-7f485b6c5-tccn7"
Jul 21 15:13:31.860829 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:13:31.860808    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af244b07-69cf-4acc-a725-72acbaf85d09-home\") pod \"auth-test-model-26fa51-kserve-7f485b6c5-tccn7\" (UID: \"af244b07-69cf-4acc-a725-72acbaf85d09\") " pod="ai-tenant-e2e-shared-b-26fa51/auth-test-model-26fa51-kserve-7f485b6c5-tccn7"
Jul 21 15:13:31.860876 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:13:31.860855    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af244b07-69cf-4acc-a725-72acbaf85d09-model-cache\") pod \"auth-test-model-26fa51-kserve-7f485b6c5-tccn7\" (UID: \"af244b07-69cf-4acc-a725-72acbaf85d09\") " pod="ai-tenant-e2e-shared-b-26fa51/auth-test-model-26fa51-kserve-7f485b6c5-tccn7"
Jul 21 15:13:31.860968 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:13:31.860953    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af244b07-69cf-4acc-a725-72acbaf85d09-tmp-dir\") pod \"auth-test-model-26fa51-kserve-7f485b6c5-tccn7\" (UID: \"af244b07-69cf-4acc-a725-72acbaf85d09\") " pod="ai-tenant-e2e-shared-b-26fa51/auth-test-model-26fa51-kserve-7f485b6c5-tccn7"
Jul 21 15:13:31.862649 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:13:31.862633    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af244b07-69cf-4acc-a725-72acbaf85d09-dshm\") pod \"auth-test-model-26fa51-kserve-7f485b6c5-tccn7\" (UID: \"af244b07-69cf-4acc-a725-72acbaf85d09\") " pod="ai-tenant-e2e-shared-b-26fa51/auth-test-model-26fa51-kserve-7f485b6c5-tccn7"
Jul 21 15:13:31.863056 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:13:31.863037    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af244b07-69cf-4acc-a725-72acbaf85d09-tls-certs\") pod \"auth-test-model-26fa51-kserve-7f485b6c5-tccn7\" (UID: \"af244b07-69cf-4acc-a725-72acbaf85d09\") " pod="ai-tenant-e2e-shared-b-26fa51/auth-test-model-26fa51-kserve-7f485b6c5-tccn7"
Jul 21 15:13:31.873281 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:13:31.873256    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xhgk\" (UniqueName: \"kubernetes.io/projected/af244b07-69cf-4acc-a725-72acbaf85d09-kube-api-access-8xhgk\") pod \"auth-test-model-26fa51-kserve-7f485b6c5-tccn7\" (UID: \"af244b07-69cf-4acc-a725-72acbaf85d09\") " pod="ai-tenant-e2e-shared-b-26fa51/auth-test-model-26fa51-kserve-7f485b6c5-tccn7"
Jul 21 15:13:32.027953 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:13:32.027876    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-26fa51/auth-test-model-26fa51-kserve-7f485b6c5-tccn7"
Jul 21 15:13:32.168342 ip-10-0-130-83 kubenswrapper[2579]: W0721 15:13:32.168314    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaf244b07_69cf_4acc_a725_72acbaf85d09.slice/crio-fef0999ff53428f5f62e92278eb31213e883632ca77f482eca81c469ff80b6ac WatchSource:0}: Error finding container fef0999ff53428f5f62e92278eb31213e883632ca77f482eca81c469ff80b6ac: Status 404 returned error can't find the container with id fef0999ff53428f5f62e92278eb31213e883632ca77f482eca81c469ff80b6ac
Jul 21 15:13:32.176567 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:13:32.172818    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-26fa51/auth-test-model-26fa51-kserve-7f485b6c5-tccn7"]
Jul 21 15:13:33.166390 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:13:33.166358    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-26fa51/auth-test-model-26fa51-kserve-7f485b6c5-tccn7" event={"ID":"af244b07-69cf-4acc-a725-72acbaf85d09","Type":"ContainerStarted","Data":"4d3584f7710aef45a34b527777955cce1705586d35f2520058a23e5dd366b659"}
Jul 21 15:13:33.166390 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:13:33.166396    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-26fa51/auth-test-model-26fa51-kserve-7f485b6c5-tccn7" event={"ID":"af244b07-69cf-4acc-a725-72acbaf85d09","Type":"ContainerStarted","Data":"fef0999ff53428f5f62e92278eb31213e883632ca77f482eca81c469ff80b6ac"}
Jul 21 15:13:33.183319 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:13:33.183255    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-26fa51/auth-test-model-26fa51-kserve-7f485b6c5-tccn7" podStartSLOduration=2.183239579 podStartE2EDuration="2.183239579s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 15:13:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 15:13:33.182365134 +0000 UTC m=+2469.532108569" watchObservedRunningTime="2026-07-21 15:13:33.183239579 +0000 UTC m=+2469.532982991"
Jul 21 15:13:42.028961 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:13:42.028917    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-26fa51/auth-test-model-26fa51-kserve-7f485b6c5-tccn7"
Jul 21 15:13:42.029417 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:13:42.029142    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-26fa51/auth-test-model-26fa51-kserve-7f485b6c5-tccn7"
Jul 21 15:13:42.041556 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:13:42.041534    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-26fa51/auth-test-model-26fa51-kserve-7f485b6c5-tccn7"
Jul 21 15:13:42.217190 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:13:42.217163    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-26fa51/auth-test-model-26fa51-kserve-7f485b6c5-tccn7"
Jul 21 15:15:00.133900 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:00.133851    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-9b06ae-29744115-77fmw"]
Jul 21 15:15:00.137780 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:00.137759    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-26fa51-29744115-ll25f"]
Jul 21 15:15:00.137929 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:00.137910    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-9b06ae-29744115-77fmw"
Jul 21 15:15:00.140681 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:00.140655    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-494sx\""
Jul 21 15:15:00.141873 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:00.141850    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744115-wphnq"]
Jul 21 15:15:00.142022 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:00.142002    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-26fa51-29744115-ll25f"
Jul 21 15:15:00.146264 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:00.146244    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-26fa51-29744115-ll25f"]
Jul 21 15:15:00.146449 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:00.146390    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744115-wphnq"
Jul 21 15:15:00.149497 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:00.147336    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-9b06ae-29744115-77fmw"]
Jul 21 15:15:00.149953 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:00.149930    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744115-wphnq"]
Jul 21 15:15:00.152760 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:00.152737    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-9b06ae-29744115-77fmw"
Jul 21 15:15:00.161853 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:00.161834    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-26fa51-29744115-ll25f"
Jul 21 15:15:00.198062 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:00.198039    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744115-wphnq"
Jul 21 15:15:00.312330 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:00.311540    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-9b06ae-29744115-77fmw"]
Jul 21 15:15:00.373950 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:00.373786    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744115-wphnq"]
Jul 21 15:15:00.376080 ip-10-0-130-83 kubenswrapper[2579]: W0721 15:15:00.376056    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f888722_c08b_4c62_8150_f93fee5f267c.slice/crio-f8f1edd97b8cee016007aae5c328c3fde91e7fd2e3dabd3260ef5ff36906376c WatchSource:0}: Error finding container f8f1edd97b8cee016007aae5c328c3fde91e7fd2e3dabd3260ef5ff36906376c: Status 404 returned error can't find the container with id f8f1edd97b8cee016007aae5c328c3fde91e7fd2e3dabd3260ef5ff36906376c
Jul 21 15:15:00.549393 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:00.549375    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-26fa51-29744115-ll25f"]
Jul 21 15:15:00.551305 ip-10-0-130-83 kubenswrapper[2579]: W0721 15:15:00.551278    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod173e05e7_f6bf_4f1f_988e_d0191a104dd0.slice/crio-fef3704440c8124879f1e0db07457ed467f15f0bc9aa77522343721de1c1f58c WatchSource:0}: Error finding container fef3704440c8124879f1e0db07457ed467f15f0bc9aa77522343721de1c1f58c: Status 404 returned error can't find the container with id fef3704440c8124879f1e0db07457ed467f15f0bc9aa77522343721de1c1f58c
Jul 21 15:15:00.574449 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:00.574416    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744115-wphnq" event={"ID":"5f888722-c08b-4c62-8150-f93fee5f267c","Type":"ContainerStarted","Data":"f8f1edd97b8cee016007aae5c328c3fde91e7fd2e3dabd3260ef5ff36906376c"}
Jul 21 15:15:00.575479 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:00.575456    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-9b06ae-29744115-77fmw" event={"ID":"5b367ab1-e5c6-4500-a01f-eaf951f63632","Type":"ContainerStarted","Data":"25d6a72f94bbfa31eda2a253feb1312c90e63d03ce836f3dcb9e02f5e004bfab"}
Jul 21 15:15:00.576388 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:00.576367    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-26fa51-29744115-ll25f" event={"ID":"173e05e7-f6bf-4f1f-988e-d0191a104dd0","Type":"ContainerStarted","Data":"fef3704440c8124879f1e0db07457ed467f15f0bc9aa77522343721de1c1f58c"}
Jul 21 15:15:01.582777 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:01.582745    2579 generic.go:356] "Generic (PLEG): container finished" podID="5f888722-c08b-4c62-8150-f93fee5f267c" containerID="6ce48dca735e9f37153b326214156fafaa110b84067433aaf0af3d3fe5927889" exitCode=0
Jul 21 15:15:01.583143 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:01.582808    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744115-wphnq" event={"ID":"5f888722-c08b-4c62-8150-f93fee5f267c","Type":"ContainerDied","Data":"6ce48dca735e9f37153b326214156fafaa110b84067433aaf0af3d3fe5927889"}
Jul 21 15:15:01.584262 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:01.584239    2579 generic.go:356] "Generic (PLEG): container finished" podID="5b367ab1-e5c6-4500-a01f-eaf951f63632" containerID="194fb7b8deef256b52baa5d22e530eb73e901e3424b771c8c20bb91af56b7e6e" exitCode=0
Jul 21 15:15:01.584384 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:01.584301    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-9b06ae-29744115-77fmw" event={"ID":"5b367ab1-e5c6-4500-a01f-eaf951f63632","Type":"ContainerDied","Data":"194fb7b8deef256b52baa5d22e530eb73e901e3424b771c8c20bb91af56b7e6e"}
Jul 21 15:15:01.585781 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:01.585763    2579 generic.go:356] "Generic (PLEG): container finished" podID="173e05e7-f6bf-4f1f-988e-d0191a104dd0" containerID="aa64493cc3b8f18e3288197dd15ff7cbe407a5024f48b35eb258c0596027e0cf" exitCode=0
Jul 21 15:15:01.585877 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:01.585822    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-26fa51-29744115-ll25f" event={"ID":"173e05e7-f6bf-4f1f-988e-d0191a104dd0","Type":"ContainerDied","Data":"aa64493cc3b8f18e3288197dd15ff7cbe407a5024f48b35eb258c0596027e0cf"}
Jul 21 15:15:02.756995 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:02.756972    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-26fa51-29744115-ll25f"
Jul 21 15:15:02.799789 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:02.799758    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744115-wphnq"
Jul 21 15:15:02.803444 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:02.803414    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-9b06ae-29744115-77fmw"
Jul 21 15:15:03.600014 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:03.599979    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744115-wphnq"
Jul 21 15:15:03.600206 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:03.599978    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744115-wphnq" event={"ID":"5f888722-c08b-4c62-8150-f93fee5f267c","Type":"ContainerDied","Data":"f8f1edd97b8cee016007aae5c328c3fde91e7fd2e3dabd3260ef5ff36906376c"}
Jul 21 15:15:03.600206 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:03.600099    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f8f1edd97b8cee016007aae5c328c3fde91e7fd2e3dabd3260ef5ff36906376c"
Jul 21 15:15:03.601307 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:03.601265    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-9b06ae-29744115-77fmw"
Jul 21 15:15:03.601451 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:03.601307    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-9b06ae-29744115-77fmw" event={"ID":"5b367ab1-e5c6-4500-a01f-eaf951f63632","Type":"ContainerDied","Data":"25d6a72f94bbfa31eda2a253feb1312c90e63d03ce836f3dcb9e02f5e004bfab"}
Jul 21 15:15:03.601451 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:03.601345    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="25d6a72f94bbfa31eda2a253feb1312c90e63d03ce836f3dcb9e02f5e004bfab"
Jul 21 15:15:03.602625 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:03.602606    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-26fa51-29744115-ll25f" event={"ID":"173e05e7-f6bf-4f1f-988e-d0191a104dd0","Type":"ContainerDied","Data":"fef3704440c8124879f1e0db07457ed467f15f0bc9aa77522343721de1c1f58c"}
Jul 21 15:15:03.602748 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:03.602625    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-26fa51-29744115-ll25f"
Jul 21 15:15:03.602748 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:03.602635    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fef3704440c8124879f1e0db07457ed467f15f0bc9aa77522343721de1c1f58c"
Jul 21 15:15:43.579362 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:43.579326    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-26fa51/auth-test-model-26fa51-kserve-7f485b6c5-tccn7"]
Jul 21 15:15:58.601499 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:58.601442    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-26fa51/auth-test-model-26fa51-kserve-7f485b6c5-tccn7" podUID="af244b07-69cf-4acc-a725-72acbaf85d09" containerName="main" containerID="cri-o://4d3584f7710aef45a34b527777955cce1705586d35f2520058a23e5dd366b659" gracePeriod=45
Jul 21 15:15:58.857813 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:58.857765    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-26fa51/auth-test-model-26fa51-kserve-7f485b6c5-tccn7"
Jul 21 15:15:58.882822 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:58.882787    2579 generic.go:356] "Generic (PLEG): container finished" podID="af244b07-69cf-4acc-a725-72acbaf85d09" containerID="4d3584f7710aef45a34b527777955cce1705586d35f2520058a23e5dd366b659" exitCode=0
Jul 21 15:15:58.882972 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:58.882833    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-26fa51/auth-test-model-26fa51-kserve-7f485b6c5-tccn7"
Jul 21 15:15:58.882972 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:58.882860    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-26fa51/auth-test-model-26fa51-kserve-7f485b6c5-tccn7" event={"ID":"af244b07-69cf-4acc-a725-72acbaf85d09","Type":"ContainerDied","Data":"4d3584f7710aef45a34b527777955cce1705586d35f2520058a23e5dd366b659"}
Jul 21 15:15:58.882972 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:58.882897    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-26fa51/auth-test-model-26fa51-kserve-7f485b6c5-tccn7" event={"ID":"af244b07-69cf-4acc-a725-72acbaf85d09","Type":"ContainerDied","Data":"fef0999ff53428f5f62e92278eb31213e883632ca77f482eca81c469ff80b6ac"}
Jul 21 15:15:58.882972 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:58.882913    2579 scope.go:117] "RemoveContainer" containerID="4d3584f7710aef45a34b527777955cce1705586d35f2520058a23e5dd366b659"
Jul 21 15:15:58.891943 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:58.891907    2579 scope.go:117] "RemoveContainer" containerID="4d3584f7710aef45a34b527777955cce1705586d35f2520058a23e5dd366b659"
Jul 21 15:15:58.892296 ip-10-0-130-83 kubenswrapper[2579]: E0721 15:15:58.892247    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d3584f7710aef45a34b527777955cce1705586d35f2520058a23e5dd366b659\": container with ID starting with 4d3584f7710aef45a34b527777955cce1705586d35f2520058a23e5dd366b659 not found: ID does not exist" containerID="4d3584f7710aef45a34b527777955cce1705586d35f2520058a23e5dd366b659"
Jul 21 15:15:58.892384 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:58.892316    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d3584f7710aef45a34b527777955cce1705586d35f2520058a23e5dd366b659"} err="failed to get container status \"4d3584f7710aef45a34b527777955cce1705586d35f2520058a23e5dd366b659\": rpc error: code = NotFound desc = could not find container \"4d3584f7710aef45a34b527777955cce1705586d35f2520058a23e5dd366b659\": container with ID starting with 4d3584f7710aef45a34b527777955cce1705586d35f2520058a23e5dd366b659 not found: ID does not exist"
Jul 21 15:15:58.902770 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:58.902749    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af244b07-69cf-4acc-a725-72acbaf85d09-home\") pod \"af244b07-69cf-4acc-a725-72acbaf85d09\" (UID: \"af244b07-69cf-4acc-a725-72acbaf85d09\") "
Jul 21 15:15:58.902863 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:58.902785    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af244b07-69cf-4acc-a725-72acbaf85d09-model-cache\") pod \"af244b07-69cf-4acc-a725-72acbaf85d09\" (UID: \"af244b07-69cf-4acc-a725-72acbaf85d09\") "
Jul 21 15:15:58.902863 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:58.902804    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8xhgk\" (UniqueName: \"kubernetes.io/projected/af244b07-69cf-4acc-a725-72acbaf85d09-kube-api-access-8xhgk\") pod \"af244b07-69cf-4acc-a725-72acbaf85d09\" (UID: \"af244b07-69cf-4acc-a725-72acbaf85d09\") "
Jul 21 15:15:58.902863 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:58.902836    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af244b07-69cf-4acc-a725-72acbaf85d09-tmp-dir\") pod \"af244b07-69cf-4acc-a725-72acbaf85d09\" (UID: \"af244b07-69cf-4acc-a725-72acbaf85d09\") "
Jul 21 15:15:58.903044 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:58.902870    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af244b07-69cf-4acc-a725-72acbaf85d09-dshm\") pod \"af244b07-69cf-4acc-a725-72acbaf85d09\" (UID: \"af244b07-69cf-4acc-a725-72acbaf85d09\") "
Jul 21 15:15:58.903044 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:58.902919    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af244b07-69cf-4acc-a725-72acbaf85d09-tls-certs\") pod \"af244b07-69cf-4acc-a725-72acbaf85d09\" (UID: \"af244b07-69cf-4acc-a725-72acbaf85d09\") "
Jul 21 15:15:58.903044 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:58.903010    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af244b07-69cf-4acc-a725-72acbaf85d09-home" (OuterVolumeSpecName: "home") pod "af244b07-69cf-4acc-a725-72acbaf85d09" (UID: "af244b07-69cf-4acc-a725-72acbaf85d09"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 15:15:58.903044 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:58.903024    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af244b07-69cf-4acc-a725-72acbaf85d09-model-cache" (OuterVolumeSpecName: "model-cache") pod "af244b07-69cf-4acc-a725-72acbaf85d09" (UID: "af244b07-69cf-4acc-a725-72acbaf85d09"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 15:15:58.903285 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:58.903142    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af244b07-69cf-4acc-a725-72acbaf85d09-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "af244b07-69cf-4acc-a725-72acbaf85d09" (UID: "af244b07-69cf-4acc-a725-72acbaf85d09"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 15:15:58.903285 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:58.903214    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/af244b07-69cf-4acc-a725-72acbaf85d09-home\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:15:58.903285 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:58.903232    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/af244b07-69cf-4acc-a725-72acbaf85d09-model-cache\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:15:58.903285 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:58.903247    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/af244b07-69cf-4acc-a725-72acbaf85d09-tmp-dir\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:15:58.904967 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:58.904947    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af244b07-69cf-4acc-a725-72acbaf85d09-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "af244b07-69cf-4acc-a725-72acbaf85d09" (UID: "af244b07-69cf-4acc-a725-72acbaf85d09"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 15:15:58.905066 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:58.904981    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af244b07-69cf-4acc-a725-72acbaf85d09-dshm" (OuterVolumeSpecName: "dshm") pod "af244b07-69cf-4acc-a725-72acbaf85d09" (UID: "af244b07-69cf-4acc-a725-72acbaf85d09"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 15:15:58.905066 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:58.905058    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af244b07-69cf-4acc-a725-72acbaf85d09-kube-api-access-8xhgk" (OuterVolumeSpecName: "kube-api-access-8xhgk") pod "af244b07-69cf-4acc-a725-72acbaf85d09" (UID: "af244b07-69cf-4acc-a725-72acbaf85d09"). InnerVolumeSpecName "kube-api-access-8xhgk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 15:15:59.003874 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:59.003847    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/af244b07-69cf-4acc-a725-72acbaf85d09-tls-certs\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:15:59.003874 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:59.003873    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xhgk\" (UniqueName: \"kubernetes.io/projected/af244b07-69cf-4acc-a725-72acbaf85d09-kube-api-access-8xhgk\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:15:59.004014 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:59.003883    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/af244b07-69cf-4acc-a725-72acbaf85d09-dshm\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:15:59.207167 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:59.207138    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-26fa51/auth-test-model-26fa51-kserve-7f485b6c5-tccn7"]
Jul 21 15:15:59.211217 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:15:59.211196    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-26fa51/auth-test-model-26fa51-kserve-7f485b6c5-tccn7"]
Jul 21 15:16:00.171977 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:00.171942    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af244b07-69cf-4acc-a725-72acbaf85d09" path="/var/lib/kubelet/pods/af244b07-69cf-4acc-a725-72acbaf85d09/volumes"
Jul 21 15:16:30.897995 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:30.897964    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-26fa51/sub-test-model-26fa51-kserve-78cb9949d5-mwtlx"]
Jul 21 15:16:30.898390 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:30.898320    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b367ab1-e5c6-4500-a01f-eaf951f63632" containerName="cleanup"
Jul 21 15:16:30.898390 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:30.898332    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b367ab1-e5c6-4500-a01f-eaf951f63632" containerName="cleanup"
Jul 21 15:16:30.898390 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:30.898341    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="173e05e7-f6bf-4f1f-988e-d0191a104dd0" containerName="cleanup"
Jul 21 15:16:30.898390 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:30.898347    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="173e05e7-f6bf-4f1f-988e-d0191a104dd0" containerName="cleanup"
Jul 21 15:16:30.898390 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:30.898355    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f888722-c08b-4c62-8150-f93fee5f267c" containerName="cleanup"
Jul 21 15:16:30.898390 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:30.898360    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f888722-c08b-4c62-8150-f93fee5f267c" containerName="cleanup"
Jul 21 15:16:30.898390 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:30.898368    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af244b07-69cf-4acc-a725-72acbaf85d09" containerName="main"
Jul 21 15:16:30.898390 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:30.898374    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="af244b07-69cf-4acc-a725-72acbaf85d09" containerName="main"
Jul 21 15:16:30.898655 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:30.898422    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="af244b07-69cf-4acc-a725-72acbaf85d09" containerName="main"
Jul 21 15:16:30.898655 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:30.898432    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b367ab1-e5c6-4500-a01f-eaf951f63632" containerName="cleanup"
Jul 21 15:16:30.898655 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:30.898438    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="173e05e7-f6bf-4f1f-988e-d0191a104dd0" containerName="cleanup"
Jul 21 15:16:30.898655 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:30.898446    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f888722-c08b-4c62-8150-f93fee5f267c" containerName="cleanup"
Jul 21 15:16:30.902701 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:30.902679    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-26fa51/sub-test-model-26fa51-kserve-78cb9949d5-mwtlx"
Jul 21 15:16:30.905576 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:30.905543    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-26fa51\"/\"sub-test-model-26fa51-kserve-self-signed-certs\""
Jul 21 15:16:30.911972 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:30.911943    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-26fa51\"/\"default-dockercfg-l2tcs\""
Jul 21 15:16:30.912077 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:30.911993    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-26fa51\"/\"kube-root-ca.crt\""
Jul 21 15:16:30.912826 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:30.912803    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-26fa51\"/\"openshift-service-ca.crt\""
Jul 21 15:16:30.913866 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:30.913840    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-26fa51/sub-test-model-26fa51-kserve-78cb9949d5-mwtlx"]
Jul 21 15:16:31.066205 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:31.066172    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1febbe5b-4832-46a2-9e9f-5187e9379b37-dshm\") pod \"sub-test-model-26fa51-kserve-78cb9949d5-mwtlx\" (UID: \"1febbe5b-4832-46a2-9e9f-5187e9379b37\") " pod="ai-tenant-e2e-shared-b-26fa51/sub-test-model-26fa51-kserve-78cb9949d5-mwtlx"
Jul 21 15:16:31.066397 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:31.066211    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1febbe5b-4832-46a2-9e9f-5187e9379b37-tls-certs\") pod \"sub-test-model-26fa51-kserve-78cb9949d5-mwtlx\" (UID: \"1febbe5b-4832-46a2-9e9f-5187e9379b37\") " pod="ai-tenant-e2e-shared-b-26fa51/sub-test-model-26fa51-kserve-78cb9949d5-mwtlx"
Jul 21 15:16:31.066397 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:31.066327    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1febbe5b-4832-46a2-9e9f-5187e9379b37-model-cache\") pod \"sub-test-model-26fa51-kserve-78cb9949d5-mwtlx\" (UID: \"1febbe5b-4832-46a2-9e9f-5187e9379b37\") " pod="ai-tenant-e2e-shared-b-26fa51/sub-test-model-26fa51-kserve-78cb9949d5-mwtlx"
Jul 21 15:16:31.066397 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:31.066359    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdrx2\" (UniqueName: \"kubernetes.io/projected/1febbe5b-4832-46a2-9e9f-5187e9379b37-kube-api-access-tdrx2\") pod \"sub-test-model-26fa51-kserve-78cb9949d5-mwtlx\" (UID: \"1febbe5b-4832-46a2-9e9f-5187e9379b37\") " pod="ai-tenant-e2e-shared-b-26fa51/sub-test-model-26fa51-kserve-78cb9949d5-mwtlx"
Jul 21 15:16:31.066397 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:31.066386    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1febbe5b-4832-46a2-9e9f-5187e9379b37-home\") pod \"sub-test-model-26fa51-kserve-78cb9949d5-mwtlx\" (UID: \"1febbe5b-4832-46a2-9e9f-5187e9379b37\") " pod="ai-tenant-e2e-shared-b-26fa51/sub-test-model-26fa51-kserve-78cb9949d5-mwtlx"
Jul 21 15:16:31.066546 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:31.066433    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1febbe5b-4832-46a2-9e9f-5187e9379b37-tmp-dir\") pod \"sub-test-model-26fa51-kserve-78cb9949d5-mwtlx\" (UID: \"1febbe5b-4832-46a2-9e9f-5187e9379b37\") " pod="ai-tenant-e2e-shared-b-26fa51/sub-test-model-26fa51-kserve-78cb9949d5-mwtlx"
Jul 21 15:16:31.167383 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:31.167306    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1febbe5b-4832-46a2-9e9f-5187e9379b37-tmp-dir\") pod \"sub-test-model-26fa51-kserve-78cb9949d5-mwtlx\" (UID: \"1febbe5b-4832-46a2-9e9f-5187e9379b37\") " pod="ai-tenant-e2e-shared-b-26fa51/sub-test-model-26fa51-kserve-78cb9949d5-mwtlx"
Jul 21 15:16:31.167383 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:31.167351    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1febbe5b-4832-46a2-9e9f-5187e9379b37-dshm\") pod \"sub-test-model-26fa51-kserve-78cb9949d5-mwtlx\" (UID: \"1febbe5b-4832-46a2-9e9f-5187e9379b37\") " pod="ai-tenant-e2e-shared-b-26fa51/sub-test-model-26fa51-kserve-78cb9949d5-mwtlx"
Jul 21 15:16:31.167582 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:31.167382    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1febbe5b-4832-46a2-9e9f-5187e9379b37-tls-certs\") pod \"sub-test-model-26fa51-kserve-78cb9949d5-mwtlx\" (UID: \"1febbe5b-4832-46a2-9e9f-5187e9379b37\") " pod="ai-tenant-e2e-shared-b-26fa51/sub-test-model-26fa51-kserve-78cb9949d5-mwtlx"
Jul 21 15:16:31.167582 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:31.167440    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1febbe5b-4832-46a2-9e9f-5187e9379b37-model-cache\") pod \"sub-test-model-26fa51-kserve-78cb9949d5-mwtlx\" (UID: \"1febbe5b-4832-46a2-9e9f-5187e9379b37\") " pod="ai-tenant-e2e-shared-b-26fa51/sub-test-model-26fa51-kserve-78cb9949d5-mwtlx"
Jul 21 15:16:31.167582 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:31.167468    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdrx2\" (UniqueName: \"kubernetes.io/projected/1febbe5b-4832-46a2-9e9f-5187e9379b37-kube-api-access-tdrx2\") pod \"sub-test-model-26fa51-kserve-78cb9949d5-mwtlx\" (UID: \"1febbe5b-4832-46a2-9e9f-5187e9379b37\") " pod="ai-tenant-e2e-shared-b-26fa51/sub-test-model-26fa51-kserve-78cb9949d5-mwtlx"
Jul 21 15:16:31.167582 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:31.167498    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1febbe5b-4832-46a2-9e9f-5187e9379b37-home\") pod \"sub-test-model-26fa51-kserve-78cb9949d5-mwtlx\" (UID: \"1febbe5b-4832-46a2-9e9f-5187e9379b37\") " pod="ai-tenant-e2e-shared-b-26fa51/sub-test-model-26fa51-kserve-78cb9949d5-mwtlx"
Jul 21 15:16:31.167809 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:31.167786    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1febbe5b-4832-46a2-9e9f-5187e9379b37-tmp-dir\") pod \"sub-test-model-26fa51-kserve-78cb9949d5-mwtlx\" (UID: \"1febbe5b-4832-46a2-9e9f-5187e9379b37\") " pod="ai-tenant-e2e-shared-b-26fa51/sub-test-model-26fa51-kserve-78cb9949d5-mwtlx"
Jul 21 15:16:31.167871 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:31.167811    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1febbe5b-4832-46a2-9e9f-5187e9379b37-home\") pod \"sub-test-model-26fa51-kserve-78cb9949d5-mwtlx\" (UID: \"1febbe5b-4832-46a2-9e9f-5187e9379b37\") " pod="ai-tenant-e2e-shared-b-26fa51/sub-test-model-26fa51-kserve-78cb9949d5-mwtlx"
Jul 21 15:16:31.167923 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:31.167870    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1febbe5b-4832-46a2-9e9f-5187e9379b37-model-cache\") pod \"sub-test-model-26fa51-kserve-78cb9949d5-mwtlx\" (UID: \"1febbe5b-4832-46a2-9e9f-5187e9379b37\") " pod="ai-tenant-e2e-shared-b-26fa51/sub-test-model-26fa51-kserve-78cb9949d5-mwtlx"
Jul 21 15:16:31.169493 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:31.169475    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1febbe5b-4832-46a2-9e9f-5187e9379b37-dshm\") pod \"sub-test-model-26fa51-kserve-78cb9949d5-mwtlx\" (UID: \"1febbe5b-4832-46a2-9e9f-5187e9379b37\") " pod="ai-tenant-e2e-shared-b-26fa51/sub-test-model-26fa51-kserve-78cb9949d5-mwtlx"
Jul 21 15:16:31.169878 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:31.169857    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1febbe5b-4832-46a2-9e9f-5187e9379b37-tls-certs\") pod \"sub-test-model-26fa51-kserve-78cb9949d5-mwtlx\" (UID: \"1febbe5b-4832-46a2-9e9f-5187e9379b37\") " pod="ai-tenant-e2e-shared-b-26fa51/sub-test-model-26fa51-kserve-78cb9949d5-mwtlx"
Jul 21 15:16:31.175735 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:31.175714    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdrx2\" (UniqueName: \"kubernetes.io/projected/1febbe5b-4832-46a2-9e9f-5187e9379b37-kube-api-access-tdrx2\") pod \"sub-test-model-26fa51-kserve-78cb9949d5-mwtlx\" (UID: \"1febbe5b-4832-46a2-9e9f-5187e9379b37\") " pod="ai-tenant-e2e-shared-b-26fa51/sub-test-model-26fa51-kserve-78cb9949d5-mwtlx"
Jul 21 15:16:31.220428 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:31.220405    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-26fa51/sub-test-model-26fa51-kserve-78cb9949d5-mwtlx"
Jul 21 15:16:31.350902 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:31.350874    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-26fa51/sub-test-model-26fa51-kserve-78cb9949d5-mwtlx"]
Jul 21 15:16:31.352722 ip-10-0-130-83 kubenswrapper[2579]: W0721 15:16:31.352688    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1febbe5b_4832_46a2_9e9f_5187e9379b37.slice/crio-cdd47ca2bebb3a10d3596b9eb9c79406898f23f9a3acbda49b9bc829eb730640 WatchSource:0}: Error finding container cdd47ca2bebb3a10d3596b9eb9c79406898f23f9a3acbda49b9bc829eb730640: Status 404 returned error can't find the container with id cdd47ca2bebb3a10d3596b9eb9c79406898f23f9a3acbda49b9bc829eb730640
Jul 21 15:16:32.047005 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:32.046970    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-26fa51/sub-test-model-26fa51-kserve-78cb9949d5-mwtlx" event={"ID":"1febbe5b-4832-46a2-9e9f-5187e9379b37","Type":"ContainerStarted","Data":"4b486a1b2062a98006dd4cbb5554edf6787152febd9f457bec1418ffe0d1760a"}
Jul 21 15:16:32.047005 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:32.047009    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-26fa51/sub-test-model-26fa51-kserve-78cb9949d5-mwtlx" event={"ID":"1febbe5b-4832-46a2-9e9f-5187e9379b37","Type":"ContainerStarted","Data":"cdd47ca2bebb3a10d3596b9eb9c79406898f23f9a3acbda49b9bc829eb730640"}
Jul 21 15:16:32.064913 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:32.064838    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-26fa51/sub-test-model-26fa51-kserve-78cb9949d5-mwtlx" podStartSLOduration=2.064820354 podStartE2EDuration="2.064820354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 15:16:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 15:16:32.064178738 +0000 UTC m=+2648.413922169" watchObservedRunningTime="2026-07-21 15:16:32.064820354 +0000 UTC m=+2648.414563766"
Jul 21 15:16:41.221140 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:41.221108    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-26fa51/sub-test-model-26fa51-kserve-78cb9949d5-mwtlx"
Jul 21 15:16:41.221602 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:41.221150    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-26fa51/sub-test-model-26fa51-kserve-78cb9949d5-mwtlx"
Jul 21 15:16:41.233000 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:41.232974    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-26fa51/sub-test-model-26fa51-kserve-78cb9949d5-mwtlx"
Jul 21 15:16:42.112617 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:16:42.112592    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-26fa51/sub-test-model-26fa51-kserve-78cb9949d5-mwtlx"
Jul 21 15:17:18.793454 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:17:18.793377    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-26fa51/sub-test-model-26fa51-kserve-78cb9949d5-mwtlx"]
Jul 21 15:17:24.495077 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:17:24.495041    2579 scope.go:117] "RemoveContainer" containerID="0626f993c568ef538ccf3b17c20ba4c151955323094b6c8342d5c8243a769040"
Jul 21 15:17:24.503033 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:17:24.503013    2579 scope.go:117] "RemoveContainer" containerID="5a7d4e7ddc4fbc7566614f75dec9dd68533919541c6133470a4f127e8e2eadbd"
Jul 21 15:17:33.813315 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:17:33.813229    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-26fa51/sub-test-model-26fa51-kserve-78cb9949d5-mwtlx" podUID="1febbe5b-4832-46a2-9e9f-5187e9379b37" containerName="main" containerID="cri-o://4b486a1b2062a98006dd4cbb5554edf6787152febd9f457bec1418ffe0d1760a" gracePeriod=45
Jul 21 15:17:34.058060 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:17:34.058037    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-26fa51/sub-test-model-26fa51-kserve-78cb9949d5-mwtlx"
Jul 21 15:17:34.171017 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:17:34.170986    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1febbe5b-4832-46a2-9e9f-5187e9379b37-dshm\") pod \"1febbe5b-4832-46a2-9e9f-5187e9379b37\" (UID: \"1febbe5b-4832-46a2-9e9f-5187e9379b37\") "
Jul 21 15:17:34.171017 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:17:34.171014    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1febbe5b-4832-46a2-9e9f-5187e9379b37-model-cache\") pod \"1febbe5b-4832-46a2-9e9f-5187e9379b37\" (UID: \"1febbe5b-4832-46a2-9e9f-5187e9379b37\") "
Jul 21 15:17:34.171261 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:17:34.171075    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1febbe5b-4832-46a2-9e9f-5187e9379b37-home\") pod \"1febbe5b-4832-46a2-9e9f-5187e9379b37\" (UID: \"1febbe5b-4832-46a2-9e9f-5187e9379b37\") "
Jul 21 15:17:34.171261 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:17:34.171115    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tdrx2\" (UniqueName: \"kubernetes.io/projected/1febbe5b-4832-46a2-9e9f-5187e9379b37-kube-api-access-tdrx2\") pod \"1febbe5b-4832-46a2-9e9f-5187e9379b37\" (UID: \"1febbe5b-4832-46a2-9e9f-5187e9379b37\") "
Jul 21 15:17:34.171261 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:17:34.171161    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1febbe5b-4832-46a2-9e9f-5187e9379b37-tls-certs\") pod \"1febbe5b-4832-46a2-9e9f-5187e9379b37\" (UID: \"1febbe5b-4832-46a2-9e9f-5187e9379b37\") "
Jul 21 15:17:34.171261 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:17:34.171213    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1febbe5b-4832-46a2-9e9f-5187e9379b37-tmp-dir\") pod \"1febbe5b-4832-46a2-9e9f-5187e9379b37\" (UID: \"1febbe5b-4832-46a2-9e9f-5187e9379b37\") "
Jul 21 15:17:34.171521 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:17:34.171314    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1febbe5b-4832-46a2-9e9f-5187e9379b37-home" (OuterVolumeSpecName: "home") pod "1febbe5b-4832-46a2-9e9f-5187e9379b37" (UID: "1febbe5b-4832-46a2-9e9f-5187e9379b37"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 15:17:34.171521 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:17:34.171352    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1febbe5b-4832-46a2-9e9f-5187e9379b37-model-cache" (OuterVolumeSpecName: "model-cache") pod "1febbe5b-4832-46a2-9e9f-5187e9379b37" (UID: "1febbe5b-4832-46a2-9e9f-5187e9379b37"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 15:17:34.171631 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:17:34.171550    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1febbe5b-4832-46a2-9e9f-5187e9379b37-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1febbe5b-4832-46a2-9e9f-5187e9379b37" (UID: "1febbe5b-4832-46a2-9e9f-5187e9379b37"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 15:17:34.171631 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:17:34.171609    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1febbe5b-4832-46a2-9e9f-5187e9379b37-home\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:17:34.171731 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:17:34.171630    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1febbe5b-4832-46a2-9e9f-5187e9379b37-model-cache\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:17:34.173159 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:17:34.173135    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1febbe5b-4832-46a2-9e9f-5187e9379b37-dshm" (OuterVolumeSpecName: "dshm") pod "1febbe5b-4832-46a2-9e9f-5187e9379b37" (UID: "1febbe5b-4832-46a2-9e9f-5187e9379b37"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 15:17:34.173322 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:17:34.173188    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1febbe5b-4832-46a2-9e9f-5187e9379b37-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1febbe5b-4832-46a2-9e9f-5187e9379b37" (UID: "1febbe5b-4832-46a2-9e9f-5187e9379b37"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 15:17:34.173322 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:17:34.173193    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1febbe5b-4832-46a2-9e9f-5187e9379b37-kube-api-access-tdrx2" (OuterVolumeSpecName: "kube-api-access-tdrx2") pod "1febbe5b-4832-46a2-9e9f-5187e9379b37" (UID: "1febbe5b-4832-46a2-9e9f-5187e9379b37"). InnerVolumeSpecName "kube-api-access-tdrx2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 15:17:34.272828 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:17:34.272803    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1febbe5b-4832-46a2-9e9f-5187e9379b37-dshm\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:17:34.272828 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:17:34.272826    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tdrx2\" (UniqueName: \"kubernetes.io/projected/1febbe5b-4832-46a2-9e9f-5187e9379b37-kube-api-access-tdrx2\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:17:34.272963 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:17:34.272837    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1febbe5b-4832-46a2-9e9f-5187e9379b37-tls-certs\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:17:34.272963 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:17:34.272846    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1febbe5b-4832-46a2-9e9f-5187e9379b37-tmp-dir\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:17:34.361258 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:17:34.361231    2579 generic.go:356] "Generic (PLEG): container finished" podID="1febbe5b-4832-46a2-9e9f-5187e9379b37" containerID="4b486a1b2062a98006dd4cbb5554edf6787152febd9f457bec1418ffe0d1760a" exitCode=0
Jul 21 15:17:34.361389 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:17:34.361282    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-26fa51/sub-test-model-26fa51-kserve-78cb9949d5-mwtlx"
Jul 21 15:17:34.361389 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:17:34.361324    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-26fa51/sub-test-model-26fa51-kserve-78cb9949d5-mwtlx" event={"ID":"1febbe5b-4832-46a2-9e9f-5187e9379b37","Type":"ContainerDied","Data":"4b486a1b2062a98006dd4cbb5554edf6787152febd9f457bec1418ffe0d1760a"}
Jul 21 15:17:34.361389 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:17:34.361364    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-26fa51/sub-test-model-26fa51-kserve-78cb9949d5-mwtlx" event={"ID":"1febbe5b-4832-46a2-9e9f-5187e9379b37","Type":"ContainerDied","Data":"cdd47ca2bebb3a10d3596b9eb9c79406898f23f9a3acbda49b9bc829eb730640"}
Jul 21 15:17:34.361389 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:17:34.361380    2579 scope.go:117] "RemoveContainer" containerID="4b486a1b2062a98006dd4cbb5554edf6787152febd9f457bec1418ffe0d1760a"
Jul 21 15:17:34.370456 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:17:34.370441    2579 scope.go:117] "RemoveContainer" containerID="4b486a1b2062a98006dd4cbb5554edf6787152febd9f457bec1418ffe0d1760a"
Jul 21 15:17:34.370695 ip-10-0-130-83 kubenswrapper[2579]: E0721 15:17:34.370677    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b486a1b2062a98006dd4cbb5554edf6787152febd9f457bec1418ffe0d1760a\": container with ID starting with 4b486a1b2062a98006dd4cbb5554edf6787152febd9f457bec1418ffe0d1760a not found: ID does not exist" containerID="4b486a1b2062a98006dd4cbb5554edf6787152febd9f457bec1418ffe0d1760a"
Jul 21 15:17:34.370746 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:17:34.370703    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b486a1b2062a98006dd4cbb5554edf6787152febd9f457bec1418ffe0d1760a"} err="failed to get container status \"4b486a1b2062a98006dd4cbb5554edf6787152febd9f457bec1418ffe0d1760a\": rpc error: code = NotFound desc = could not find container \"4b486a1b2062a98006dd4cbb5554edf6787152febd9f457bec1418ffe0d1760a\": container with ID starting with 4b486a1b2062a98006dd4cbb5554edf6787152febd9f457bec1418ffe0d1760a not found: ID does not exist"
Jul 21 15:17:34.384324 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:17:34.384303    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-26fa51/sub-test-model-26fa51-kserve-78cb9949d5-mwtlx"]
Jul 21 15:17:34.388534 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:17:34.388515    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-26fa51/sub-test-model-26fa51-kserve-78cb9949d5-mwtlx"]
Jul 21 15:17:36.172296 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:17:36.172252    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1febbe5b-4832-46a2-9e9f-5187e9379b37" path="/var/lib/kubelet/pods/1febbe5b-4832-46a2-9e9f-5187e9379b37/volumes"
Jul 21 15:18:04.597713 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:18:04.597656    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-26fa51/rate-test-model-26fa51-kserve-cd557d575-d8v9b"]
Jul 21 15:18:04.598367 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:18:04.598211    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1febbe5b-4832-46a2-9e9f-5187e9379b37" containerName="main"
Jul 21 15:18:04.598367 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:18:04.598229    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1febbe5b-4832-46a2-9e9f-5187e9379b37" containerName="main"
Jul 21 15:18:04.598367 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:18:04.598325    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1febbe5b-4832-46a2-9e9f-5187e9379b37" containerName="main"
Jul 21 15:18:04.601244 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:18:04.601224    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-26fa51/rate-test-model-26fa51-kserve-cd557d575-d8v9b"
Jul 21 15:18:04.604865 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:18:04.604838    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-26fa51\"/\"openshift-service-ca.crt\""
Jul 21 15:18:04.605120 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:18:04.605100    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-26fa51\"/\"kube-root-ca.crt\""
Jul 21 15:18:04.605216 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:18:04.605118    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-26fa51\"/\"default-dockercfg-l2tcs\""
Jul 21 15:18:04.605216 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:18:04.605111    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-26fa51\"/\"rate-test-model-26fa51-kserve-self-signed-certs\""
Jul 21 15:18:04.613593 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:18:04.613567    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-26fa51/rate-test-model-26fa51-kserve-cd557d575-d8v9b"]
Jul 21 15:18:04.718368 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:18:04.718334    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7c208ee-234f-4a52-b343-b4c14de499e8-tmp-dir\") pod \"rate-test-model-26fa51-kserve-cd557d575-d8v9b\" (UID: \"e7c208ee-234f-4a52-b343-b4c14de499e8\") " pod="ai-tenant-e2e-shared-b-26fa51/rate-test-model-26fa51-kserve-cd557d575-d8v9b"
Jul 21 15:18:04.718368 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:18:04.718374    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7c208ee-234f-4a52-b343-b4c14de499e8-tls-certs\") pod \"rate-test-model-26fa51-kserve-cd557d575-d8v9b\" (UID: \"e7c208ee-234f-4a52-b343-b4c14de499e8\") " pod="ai-tenant-e2e-shared-b-26fa51/rate-test-model-26fa51-kserve-cd557d575-d8v9b"
Jul 21 15:18:04.718568 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:18:04.718404    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7c208ee-234f-4a52-b343-b4c14de499e8-dshm\") pod \"rate-test-model-26fa51-kserve-cd557d575-d8v9b\" (UID: \"e7c208ee-234f-4a52-b343-b4c14de499e8\") " pod="ai-tenant-e2e-shared-b-26fa51/rate-test-model-26fa51-kserve-cd557d575-d8v9b"
Jul 21 15:18:04.718568 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:18:04.718486    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmmmd\" (UniqueName: \"kubernetes.io/projected/e7c208ee-234f-4a52-b343-b4c14de499e8-kube-api-access-rmmmd\") pod \"rate-test-model-26fa51-kserve-cd557d575-d8v9b\" (UID: \"e7c208ee-234f-4a52-b343-b4c14de499e8\") " pod="ai-tenant-e2e-shared-b-26fa51/rate-test-model-26fa51-kserve-cd557d575-d8v9b"
Jul 21 15:18:04.718568 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:18:04.718538    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7c208ee-234f-4a52-b343-b4c14de499e8-home\") pod \"rate-test-model-26fa51-kserve-cd557d575-d8v9b\" (UID: \"e7c208ee-234f-4a52-b343-b4c14de499e8\") " pod="ai-tenant-e2e-shared-b-26fa51/rate-test-model-26fa51-kserve-cd557d575-d8v9b"
Jul 21 15:18:04.718568 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:18:04.718555    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7c208ee-234f-4a52-b343-b4c14de499e8-model-cache\") pod \"rate-test-model-26fa51-kserve-cd557d575-d8v9b\" (UID: \"e7c208ee-234f-4a52-b343-b4c14de499e8\") " pod="ai-tenant-e2e-shared-b-26fa51/rate-test-model-26fa51-kserve-cd557d575-d8v9b"
Jul 21 15:18:04.819393 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:18:04.819363    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7c208ee-234f-4a52-b343-b4c14de499e8-dshm\") pod \"rate-test-model-26fa51-kserve-cd557d575-d8v9b\" (UID: \"e7c208ee-234f-4a52-b343-b4c14de499e8\") " pod="ai-tenant-e2e-shared-b-26fa51/rate-test-model-26fa51-kserve-cd557d575-d8v9b"
Jul 21 15:18:04.819567 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:18:04.819410    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmmmd\" (UniqueName: \"kubernetes.io/projected/e7c208ee-234f-4a52-b343-b4c14de499e8-kube-api-access-rmmmd\") pod \"rate-test-model-26fa51-kserve-cd557d575-d8v9b\" (UID: \"e7c208ee-234f-4a52-b343-b4c14de499e8\") " pod="ai-tenant-e2e-shared-b-26fa51/rate-test-model-26fa51-kserve-cd557d575-d8v9b"
Jul 21 15:18:04.819567 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:18:04.819441    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7c208ee-234f-4a52-b343-b4c14de499e8-home\") pod \"rate-test-model-26fa51-kserve-cd557d575-d8v9b\" (UID: \"e7c208ee-234f-4a52-b343-b4c14de499e8\") " pod="ai-tenant-e2e-shared-b-26fa51/rate-test-model-26fa51-kserve-cd557d575-d8v9b"
Jul 21 15:18:04.819567 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:18:04.819454    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7c208ee-234f-4a52-b343-b4c14de499e8-model-cache\") pod \"rate-test-model-26fa51-kserve-cd557d575-d8v9b\" (UID: \"e7c208ee-234f-4a52-b343-b4c14de499e8\") " pod="ai-tenant-e2e-shared-b-26fa51/rate-test-model-26fa51-kserve-cd557d575-d8v9b"
Jul 21 15:18:04.819567 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:18:04.819490    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7c208ee-234f-4a52-b343-b4c14de499e8-tmp-dir\") pod \"rate-test-model-26fa51-kserve-cd557d575-d8v9b\" (UID: \"e7c208ee-234f-4a52-b343-b4c14de499e8\") " pod="ai-tenant-e2e-shared-b-26fa51/rate-test-model-26fa51-kserve-cd557d575-d8v9b"
Jul 21 15:18:04.819567 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:18:04.819507    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7c208ee-234f-4a52-b343-b4c14de499e8-tls-certs\") pod \"rate-test-model-26fa51-kserve-cd557d575-d8v9b\" (UID: \"e7c208ee-234f-4a52-b343-b4c14de499e8\") " pod="ai-tenant-e2e-shared-b-26fa51/rate-test-model-26fa51-kserve-cd557d575-d8v9b"
Jul 21 15:18:04.819880 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:18:04.819856    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7c208ee-234f-4a52-b343-b4c14de499e8-home\") pod \"rate-test-model-26fa51-kserve-cd557d575-d8v9b\" (UID: \"e7c208ee-234f-4a52-b343-b4c14de499e8\") " pod="ai-tenant-e2e-shared-b-26fa51/rate-test-model-26fa51-kserve-cd557d575-d8v9b"
Jul 21 15:18:04.819953 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:18:04.819908    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7c208ee-234f-4a52-b343-b4c14de499e8-model-cache\") pod \"rate-test-model-26fa51-kserve-cd557d575-d8v9b\" (UID: \"e7c208ee-234f-4a52-b343-b4c14de499e8\") " pod="ai-tenant-e2e-shared-b-26fa51/rate-test-model-26fa51-kserve-cd557d575-d8v9b"
Jul 21 15:18:04.819953 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:18:04.819928    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7c208ee-234f-4a52-b343-b4c14de499e8-tmp-dir\") pod \"rate-test-model-26fa51-kserve-cd557d575-d8v9b\" (UID: \"e7c208ee-234f-4a52-b343-b4c14de499e8\") " pod="ai-tenant-e2e-shared-b-26fa51/rate-test-model-26fa51-kserve-cd557d575-d8v9b"
Jul 21 15:18:04.821632 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:18:04.821612    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7c208ee-234f-4a52-b343-b4c14de499e8-dshm\") pod \"rate-test-model-26fa51-kserve-cd557d575-d8v9b\" (UID: \"e7c208ee-234f-4a52-b343-b4c14de499e8\") " pod="ai-tenant-e2e-shared-b-26fa51/rate-test-model-26fa51-kserve-cd557d575-d8v9b"
Jul 21 15:18:04.821874 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:18:04.821857    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7c208ee-234f-4a52-b343-b4c14de499e8-tls-certs\") pod \"rate-test-model-26fa51-kserve-cd557d575-d8v9b\" (UID: \"e7c208ee-234f-4a52-b343-b4c14de499e8\") " pod="ai-tenant-e2e-shared-b-26fa51/rate-test-model-26fa51-kserve-cd557d575-d8v9b"
Jul 21 15:18:04.827692 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:18:04.827669    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmmmd\" (UniqueName: \"kubernetes.io/projected/e7c208ee-234f-4a52-b343-b4c14de499e8-kube-api-access-rmmmd\") pod \"rate-test-model-26fa51-kserve-cd557d575-d8v9b\" (UID: \"e7c208ee-234f-4a52-b343-b4c14de499e8\") " pod="ai-tenant-e2e-shared-b-26fa51/rate-test-model-26fa51-kserve-cd557d575-d8v9b"
Jul 21 15:18:04.914624 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:18:04.914601    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-26fa51/rate-test-model-26fa51-kserve-cd557d575-d8v9b"
Jul 21 15:18:05.049184 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:18:05.049158    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-26fa51/rate-test-model-26fa51-kserve-cd557d575-d8v9b"]
Jul 21 15:18:05.051287 ip-10-0-130-83 kubenswrapper[2579]: W0721 15:18:05.051238    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode7c208ee_234f_4a52_b343_b4c14de499e8.slice/crio-8d5c61ef277edc48c74f32d21d9745d46683ed985cfb39af441741aaec469288 WatchSource:0}: Error finding container 8d5c61ef277edc48c74f32d21d9745d46683ed985cfb39af441741aaec469288: Status 404 returned error can't find the container with id 8d5c61ef277edc48c74f32d21d9745d46683ed985cfb39af441741aaec469288
Jul 21 15:18:05.511908 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:18:05.511874    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-26fa51/rate-test-model-26fa51-kserve-cd557d575-d8v9b" event={"ID":"e7c208ee-234f-4a52-b343-b4c14de499e8","Type":"ContainerStarted","Data":"25b7044c4176cbb75243b89289c7f4caee445ca05bac5a5d0a54af7662964c6a"}
Jul 21 15:18:05.511908 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:18:05.511914    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-26fa51/rate-test-model-26fa51-kserve-cd557d575-d8v9b" event={"ID":"e7c208ee-234f-4a52-b343-b4c14de499e8","Type":"ContainerStarted","Data":"8d5c61ef277edc48c74f32d21d9745d46683ed985cfb39af441741aaec469288"}
Jul 21 15:18:05.530738 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:18:05.530684    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-26fa51/rate-test-model-26fa51-kserve-cd557d575-d8v9b" podStartSLOduration=1.530664789 podStartE2EDuration="1.530664789s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 15:18:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 15:18:05.530046844 +0000 UTC m=+2741.879790258" watchObservedRunningTime="2026-07-21 15:18:05.530664789 +0000 UTC m=+2741.880408202"
Jul 21 15:18:14.915055 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:18:14.915017    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-26fa51/rate-test-model-26fa51-kserve-cd557d575-d8v9b"
Jul 21 15:18:14.915568 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:18:14.915084    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-26fa51/rate-test-model-26fa51-kserve-cd557d575-d8v9b"
Jul 21 15:18:14.927369 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:18:14.927347    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-26fa51/rate-test-model-26fa51-kserve-cd557d575-d8v9b"
Jul 21 15:18:15.571687 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:18:15.571655    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-26fa51/rate-test-model-26fa51-kserve-cd557d575-d8v9b"
Jul 21 15:18:24.523858 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:18:24.523827    2579 scope.go:117] "RemoveContainer" containerID="90c62fccc01225e2a010d7df2fb41ed544c543edec5de56f8e5d2ddf8649ce37"
Jul 21 15:18:24.532764 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:18:24.532744    2579 scope.go:117] "RemoveContainer" containerID="ad70423f35ee24e8d923724241a710371093d06ab54202da92e8efb8d843ff6e"
Jul 21 15:19:12.910652 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:12.910618    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-26fa51/rate-test-model-26fa51-kserve-cd557d575-d8v9b"]
Jul 21 15:19:27.930497 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:27.930427    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-26fa51/rate-test-model-26fa51-kserve-cd557d575-d8v9b" podUID="e7c208ee-234f-4a52-b343-b4c14de499e8" containerName="main" containerID="cri-o://25b7044c4176cbb75243b89289c7f4caee445ca05bac5a5d0a54af7662964c6a" gracePeriod=45
Jul 21 15:19:28.181393 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:28.181333    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-26fa51/rate-test-model-26fa51-kserve-cd557d575-d8v9b"
Jul 21 15:19:28.310208 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:28.310181    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7c208ee-234f-4a52-b343-b4c14de499e8-model-cache\") pod \"e7c208ee-234f-4a52-b343-b4c14de499e8\" (UID: \"e7c208ee-234f-4a52-b343-b4c14de499e8\") "
Jul 21 15:19:28.310359 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:28.310243    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7c208ee-234f-4a52-b343-b4c14de499e8-tls-certs\") pod \"e7c208ee-234f-4a52-b343-b4c14de499e8\" (UID: \"e7c208ee-234f-4a52-b343-b4c14de499e8\") "
Jul 21 15:19:28.310359 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:28.310263    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7c208ee-234f-4a52-b343-b4c14de499e8-tmp-dir\") pod \"e7c208ee-234f-4a52-b343-b4c14de499e8\" (UID: \"e7c208ee-234f-4a52-b343-b4c14de499e8\") "
Jul 21 15:19:28.310442 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:28.310363    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7c208ee-234f-4a52-b343-b4c14de499e8-dshm\") pod \"e7c208ee-234f-4a52-b343-b4c14de499e8\" (UID: \"e7c208ee-234f-4a52-b343-b4c14de499e8\") "
Jul 21 15:19:28.310442 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:28.310387    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7c208ee-234f-4a52-b343-b4c14de499e8-home\") pod \"e7c208ee-234f-4a52-b343-b4c14de499e8\" (UID: \"e7c208ee-234f-4a52-b343-b4c14de499e8\") "
Jul 21 15:19:28.310550 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:28.310439    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rmmmd\" (UniqueName: \"kubernetes.io/projected/e7c208ee-234f-4a52-b343-b4c14de499e8-kube-api-access-rmmmd\") pod \"e7c208ee-234f-4a52-b343-b4c14de499e8\" (UID: \"e7c208ee-234f-4a52-b343-b4c14de499e8\") "
Jul 21 15:19:28.310550 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:28.310496    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7c208ee-234f-4a52-b343-b4c14de499e8-model-cache" (OuterVolumeSpecName: "model-cache") pod "e7c208ee-234f-4a52-b343-b4c14de499e8" (UID: "e7c208ee-234f-4a52-b343-b4c14de499e8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 15:19:28.310660 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:28.310579    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7c208ee-234f-4a52-b343-b4c14de499e8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e7c208ee-234f-4a52-b343-b4c14de499e8" (UID: "e7c208ee-234f-4a52-b343-b4c14de499e8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 15:19:28.310769 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:28.310748    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7c208ee-234f-4a52-b343-b4c14de499e8-model-cache\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:19:28.310833 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:28.310778    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7c208ee-234f-4a52-b343-b4c14de499e8-tmp-dir\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:19:28.311002 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:28.310976    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7c208ee-234f-4a52-b343-b4c14de499e8-home" (OuterVolumeSpecName: "home") pod "e7c208ee-234f-4a52-b343-b4c14de499e8" (UID: "e7c208ee-234f-4a52-b343-b4c14de499e8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 15:19:28.312211 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:28.312188    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7c208ee-234f-4a52-b343-b4c14de499e8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e7c208ee-234f-4a52-b343-b4c14de499e8" (UID: "e7c208ee-234f-4a52-b343-b4c14de499e8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 15:19:28.312671 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:28.312643    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7c208ee-234f-4a52-b343-b4c14de499e8-dshm" (OuterVolumeSpecName: "dshm") pod "e7c208ee-234f-4a52-b343-b4c14de499e8" (UID: "e7c208ee-234f-4a52-b343-b4c14de499e8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 15:19:28.312747 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:28.312701    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7c208ee-234f-4a52-b343-b4c14de499e8-kube-api-access-rmmmd" (OuterVolumeSpecName: "kube-api-access-rmmmd") pod "e7c208ee-234f-4a52-b343-b4c14de499e8" (UID: "e7c208ee-234f-4a52-b343-b4c14de499e8"). InnerVolumeSpecName "kube-api-access-rmmmd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 15:19:28.411409 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:28.411383    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7c208ee-234f-4a52-b343-b4c14de499e8-dshm\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:19:28.411409 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:28.411404    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7c208ee-234f-4a52-b343-b4c14de499e8-home\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:19:28.411545 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:28.411413    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmmmd\" (UniqueName: \"kubernetes.io/projected/e7c208ee-234f-4a52-b343-b4c14de499e8-kube-api-access-rmmmd\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:19:28.411545 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:28.411424    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7c208ee-234f-4a52-b343-b4c14de499e8-tls-certs\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:19:28.914118 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:28.914084    2579 generic.go:356] "Generic (PLEG): container finished" podID="e7c208ee-234f-4a52-b343-b4c14de499e8" containerID="25b7044c4176cbb75243b89289c7f4caee445ca05bac5a5d0a54af7662964c6a" exitCode=0
Jul 21 15:19:28.914321 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:28.914131    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-26fa51/rate-test-model-26fa51-kserve-cd557d575-d8v9b"
Jul 21 15:19:28.914321 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:28.914160    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-26fa51/rate-test-model-26fa51-kserve-cd557d575-d8v9b" event={"ID":"e7c208ee-234f-4a52-b343-b4c14de499e8","Type":"ContainerDied","Data":"25b7044c4176cbb75243b89289c7f4caee445ca05bac5a5d0a54af7662964c6a"}
Jul 21 15:19:28.914321 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:28.914202    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-26fa51/rate-test-model-26fa51-kserve-cd557d575-d8v9b" event={"ID":"e7c208ee-234f-4a52-b343-b4c14de499e8","Type":"ContainerDied","Data":"8d5c61ef277edc48c74f32d21d9745d46683ed985cfb39af441741aaec469288"}
Jul 21 15:19:28.914321 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:28.914219    2579 scope.go:117] "RemoveContainer" containerID="25b7044c4176cbb75243b89289c7f4caee445ca05bac5a5d0a54af7662964c6a"
Jul 21 15:19:28.923181 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:28.923162    2579 scope.go:117] "RemoveContainer" containerID="25b7044c4176cbb75243b89289c7f4caee445ca05bac5a5d0a54af7662964c6a"
Jul 21 15:19:28.923484 ip-10-0-130-83 kubenswrapper[2579]: E0721 15:19:28.923466    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25b7044c4176cbb75243b89289c7f4caee445ca05bac5a5d0a54af7662964c6a\": container with ID starting with 25b7044c4176cbb75243b89289c7f4caee445ca05bac5a5d0a54af7662964c6a not found: ID does not exist" containerID="25b7044c4176cbb75243b89289c7f4caee445ca05bac5a5d0a54af7662964c6a"
Jul 21 15:19:28.923533 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:28.923493    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25b7044c4176cbb75243b89289c7f4caee445ca05bac5a5d0a54af7662964c6a"} err="failed to get container status \"25b7044c4176cbb75243b89289c7f4caee445ca05bac5a5d0a54af7662964c6a\": rpc error: code = NotFound desc = could not find container \"25b7044c4176cbb75243b89289c7f4caee445ca05bac5a5d0a54af7662964c6a\": container with ID starting with 25b7044c4176cbb75243b89289c7f4caee445ca05bac5a5d0a54af7662964c6a not found: ID does not exist"
Jul 21 15:19:28.937330 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:28.937304    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-26fa51/rate-test-model-26fa51-kserve-cd557d575-d8v9b"]
Jul 21 15:19:28.940981 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:28.940955    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-26fa51/rate-test-model-26fa51-kserve-cd557d575-d8v9b"]
Jul 21 15:19:30.172620 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:30.172591    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7c208ee-234f-4a52-b343-b4c14de499e8" path="/var/lib/kubelet/pods/e7c208ee-234f-4a52-b343-b4c14de499e8/volumes"
Jul 21 15:19:32.011395 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:32.011362    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9b06ae-lzhgb"]
Jul 21 15:19:32.011760 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:32.011744    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7c208ee-234f-4a52-b343-b4c14de499e8" containerName="main"
Jul 21 15:19:32.011760 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:32.011756    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7c208ee-234f-4a52-b343-b4c14de499e8" containerName="main"
Jul 21 15:19:32.011835 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:32.011829    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7c208ee-234f-4a52-b343-b4c14de499e8" containerName="main"
Jul 21 15:19:32.016356 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:32.016338    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9b06ae-lzhgb"
Jul 21 15:19:32.019561 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:32.019214    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-494sx\""
Jul 21 15:19:32.021915 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:32.021895    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9b06ae-lzhgb"]
Jul 21 15:19:32.143514 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:32.143483    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71b696c2-eb64-4bc2-a564-5c53e5034456-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-9b06ae-lzhgb\" (UID: \"71b696c2-eb64-4bc2-a564-5c53e5034456\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9b06ae-lzhgb"
Jul 21 15:19:32.244375 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:32.244297    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71b696c2-eb64-4bc2-a564-5c53e5034456-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-9b06ae-lzhgb\" (UID: \"71b696c2-eb64-4bc2-a564-5c53e5034456\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9b06ae-lzhgb"
Jul 21 15:19:32.244916 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:32.244894    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71b696c2-eb64-4bc2-a564-5c53e5034456-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-9b06ae-lzhgb\" (UID: \"71b696c2-eb64-4bc2-a564-5c53e5034456\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9b06ae-lzhgb"
Jul 21 15:19:32.327741 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:32.327679    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9b06ae-lzhgb"
Jul 21 15:19:32.458081 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:32.458056    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9b06ae-lzhgb"]
Jul 21 15:19:32.460639 ip-10-0-130-83 kubenswrapper[2579]: W0721 15:19:32.460610    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71b696c2_eb64_4bc2_a564_5c53e5034456.slice/crio-56cb4b318b83c91cecf2757918b025408b2d65d8965c41b7e92e5a03147eedbc WatchSource:0}: Error finding container 56cb4b318b83c91cecf2757918b025408b2d65d8965c41b7e92e5a03147eedbc: Status 404 returned error can't find the container with id 56cb4b318b83c91cecf2757918b025408b2d65d8965c41b7e92e5a03147eedbc
Jul 21 15:19:32.939025 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:32.938936    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9b06ae-lzhgb" event={"ID":"71b696c2-eb64-4bc2-a564-5c53e5034456","Type":"ContainerStarted","Data":"4754c3ecf860535766d03f0e679595123bda1c9f82b143444a22401fe8aa5612"}
Jul 21 15:19:32.939201 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:32.939039    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9b06ae-lzhgb" event={"ID":"71b696c2-eb64-4bc2-a564-5c53e5034456","Type":"ContainerStarted","Data":"56cb4b318b83c91cecf2757918b025408b2d65d8965c41b7e92e5a03147eedbc"}
Jul 21 15:19:32.954788 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:32.954737    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9b06ae-lzhgb" podStartSLOduration=1.9547234439999999 podStartE2EDuration="1.954723444s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 15:19:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 15:19:32.953474962 +0000 UTC m=+2829.303218376" watchObservedRunningTime="2026-07-21 15:19:32.954723444 +0000 UTC m=+2829.304466885"
Jul 21 15:19:33.945468 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:33.945437    2579 generic.go:356] "Generic (PLEG): container finished" podID="71b696c2-eb64-4bc2-a564-5c53e5034456" containerID="4754c3ecf860535766d03f0e679595123bda1c9f82b143444a22401fe8aa5612" exitCode=0
Jul 21 15:19:33.945922 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:33.945517    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9b06ae-lzhgb" event={"ID":"71b696c2-eb64-4bc2-a564-5c53e5034456","Type":"ContainerDied","Data":"4754c3ecf860535766d03f0e679595123bda1c9f82b143444a22401fe8aa5612"}
Jul 21 15:19:35.085570 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:35.085548    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9b06ae-lzhgb"
Jul 21 15:19:35.167909 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:35.167880    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71b696c2-eb64-4bc2-a564-5c53e5034456-maas-api-service-ca\") pod \"71b696c2-eb64-4bc2-a564-5c53e5034456\" (UID: \"71b696c2-eb64-4bc2-a564-5c53e5034456\") "
Jul 21 15:19:35.168220 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:35.168197    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71b696c2-eb64-4bc2-a564-5c53e5034456-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "71b696c2-eb64-4bc2-a564-5c53e5034456" (UID: "71b696c2-eb64-4bc2-a564-5c53e5034456"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 15:19:35.269232 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:35.269164    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/71b696c2-eb64-4bc2-a564-5c53e5034456-maas-api-service-ca\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:19:35.957030 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:35.957000    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9b06ae-lzhgb"
Jul 21 15:19:35.957209 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:35.957041    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9b06ae-lzhgb" event={"ID":"71b696c2-eb64-4bc2-a564-5c53e5034456","Type":"ContainerDied","Data":"56cb4b318b83c91cecf2757918b025408b2d65d8965c41b7e92e5a03147eedbc"}
Jul 21 15:19:35.957209 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:35.957088    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="56cb4b318b83c91cecf2757918b025408b2d65d8965c41b7e92e5a03147eedbc"
Jul 21 15:19:42.410325 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:42.410294    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-9b06ae-68d45c75cb-499d2"]
Jul 21 15:19:42.411074 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:42.411036    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b06ae-68d45c75cb-499d2" podUID="aa42b39a-4822-47b5-94d1-7a067f53c113" containerName="maas-api" containerID="cri-o://e47f34f9a0631010ba56ce7e6f066a1a82ed1961cb738757cf7a0ee685b7988d" gracePeriod=30
Jul 21 15:19:42.440604 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:42.440568    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-9b06ae-29744115-77fmw"]
Jul 21 15:19:42.443218 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:42.443193    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-9b06ae-29744115-77fmw"]
Jul 21 15:19:42.935737 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:42.935702    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b06ae-68d45c75cb-499d2" podUID="aa42b39a-4822-47b5-94d1-7a067f53c113" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.65:8443/health\": dial tcp 10.133.0.65:8443: connect: connection refused"
Jul 21 15:19:43.655867 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:43.655840    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b06ae-68d45c75cb-499d2"
Jul 21 15:19:43.746128 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:43.746049    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa42b39a-4822-47b5-94d1-7a067f53c113-openshift-service-ca\") pod \"aa42b39a-4822-47b5-94d1-7a067f53c113\" (UID: \"aa42b39a-4822-47b5-94d1-7a067f53c113\") "
Jul 21 15:19:43.746128 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:43.746113    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5949t\" (UniqueName: \"kubernetes.io/projected/aa42b39a-4822-47b5-94d1-7a067f53c113-kube-api-access-5949t\") pod \"aa42b39a-4822-47b5-94d1-7a067f53c113\" (UID: \"aa42b39a-4822-47b5-94d1-7a067f53c113\") "
Jul 21 15:19:43.746359 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:43.746138    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa42b39a-4822-47b5-94d1-7a067f53c113-maas-api-tls\") pod \"aa42b39a-4822-47b5-94d1-7a067f53c113\" (UID: \"aa42b39a-4822-47b5-94d1-7a067f53c113\") "
Jul 21 15:19:43.746449 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:43.746424    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa42b39a-4822-47b5-94d1-7a067f53c113-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "aa42b39a-4822-47b5-94d1-7a067f53c113" (UID: "aa42b39a-4822-47b5-94d1-7a067f53c113"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 15:19:43.748200 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:43.748174    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa42b39a-4822-47b5-94d1-7a067f53c113-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "aa42b39a-4822-47b5-94d1-7a067f53c113" (UID: "aa42b39a-4822-47b5-94d1-7a067f53c113"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 15:19:43.748310 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:43.748242    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa42b39a-4822-47b5-94d1-7a067f53c113-kube-api-access-5949t" (OuterVolumeSpecName: "kube-api-access-5949t") pod "aa42b39a-4822-47b5-94d1-7a067f53c113" (UID: "aa42b39a-4822-47b5-94d1-7a067f53c113"). InnerVolumeSpecName "kube-api-access-5949t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 15:19:43.846897 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:43.846871    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5949t\" (UniqueName: \"kubernetes.io/projected/aa42b39a-4822-47b5-94d1-7a067f53c113-kube-api-access-5949t\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:19:43.846897 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:43.846896    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa42b39a-4822-47b5-94d1-7a067f53c113-maas-api-tls\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:19:43.847045 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:43.846905    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa42b39a-4822-47b5-94d1-7a067f53c113-openshift-service-ca\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:19:43.994869 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:43.994829    2579 generic.go:356] "Generic (PLEG): container finished" podID="aa42b39a-4822-47b5-94d1-7a067f53c113" containerID="e47f34f9a0631010ba56ce7e6f066a1a82ed1961cb738757cf7a0ee685b7988d" exitCode=0
Jul 21 15:19:43.995024 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:43.994876    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b06ae-68d45c75cb-499d2"
Jul 21 15:19:43.995024 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:43.994912    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b06ae-68d45c75cb-499d2" event={"ID":"aa42b39a-4822-47b5-94d1-7a067f53c113","Type":"ContainerDied","Data":"e47f34f9a0631010ba56ce7e6f066a1a82ed1961cb738757cf7a0ee685b7988d"}
Jul 21 15:19:43.995024 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:43.994968    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-9b06ae-68d45c75cb-499d2" event={"ID":"aa42b39a-4822-47b5-94d1-7a067f53c113","Type":"ContainerDied","Data":"330dc57861c0e21f4dfc75dcce05d45d68d6af7b0412a47c8bce81a66c4145d4"}
Jul 21 15:19:43.995024 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:43.994991    2579 scope.go:117] "RemoveContainer" containerID="e47f34f9a0631010ba56ce7e6f066a1a82ed1961cb738757cf7a0ee685b7988d"
Jul 21 15:19:44.003861 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:44.003844    2579 scope.go:117] "RemoveContainer" containerID="e47f34f9a0631010ba56ce7e6f066a1a82ed1961cb738757cf7a0ee685b7988d"
Jul 21 15:19:44.004100 ip-10-0-130-83 kubenswrapper[2579]: E0721 15:19:44.004078    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e47f34f9a0631010ba56ce7e6f066a1a82ed1961cb738757cf7a0ee685b7988d\": container with ID starting with e47f34f9a0631010ba56ce7e6f066a1a82ed1961cb738757cf7a0ee685b7988d not found: ID does not exist" containerID="e47f34f9a0631010ba56ce7e6f066a1a82ed1961cb738757cf7a0ee685b7988d"
Jul 21 15:19:44.004177 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:44.004108    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e47f34f9a0631010ba56ce7e6f066a1a82ed1961cb738757cf7a0ee685b7988d"} err="failed to get container status \"e47f34f9a0631010ba56ce7e6f066a1a82ed1961cb738757cf7a0ee685b7988d\": rpc error: code = NotFound desc = could not find container \"e47f34f9a0631010ba56ce7e6f066a1a82ed1961cb738757cf7a0ee685b7988d\": container with ID starting with e47f34f9a0631010ba56ce7e6f066a1a82ed1961cb738757cf7a0ee685b7988d not found: ID does not exist"
Jul 21 15:19:44.017145 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:44.017123    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-9b06ae-68d45c75cb-499d2"]
Jul 21 15:19:44.021057 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:44.021039    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-9b06ae-68d45c75cb-499d2"]
Jul 21 15:19:44.171885 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:44.171853    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b367ab1-e5c6-4500-a01f-eaf951f63632" path="/var/lib/kubelet/pods/5b367ab1-e5c6-4500-a01f-eaf951f63632/volumes"
Jul 21 15:19:44.172197 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:44.172184    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa42b39a-4822-47b5-94d1-7a067f53c113" path="/var/lib/kubelet/pods/aa42b39a-4822-47b5-94d1-7a067f53c113/volumes"
Jul 21 15:19:47.103098 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:47.103067    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9b06ae-lzhgb"]
Jul 21 15:19:47.105577 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:47.105554    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-9b06ae-lzhgb"]
Jul 21 15:19:48.172496 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:48.172465    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71b696c2-eb64-4bc2-a564-5c53e5034456" path="/var/lib/kubelet/pods/71b696c2-eb64-4bc2-a564-5c53e5034456/volumes"
Jul 21 15:19:54.036305 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:54.036254    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-26fa51-htdcp"]
Jul 21 15:19:54.036689 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:54.036642    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa42b39a-4822-47b5-94d1-7a067f53c113" containerName="maas-api"
Jul 21 15:19:54.036689 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:54.036662    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa42b39a-4822-47b5-94d1-7a067f53c113" containerName="maas-api"
Jul 21 15:19:54.036689 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:54.036674    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71b696c2-eb64-4bc2-a564-5c53e5034456" containerName="revoke-keys"
Jul 21 15:19:54.036689 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:54.036679    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="71b696c2-eb64-4bc2-a564-5c53e5034456" containerName="revoke-keys"
Jul 21 15:19:54.036829 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:54.036730    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa42b39a-4822-47b5-94d1-7a067f53c113" containerName="maas-api"
Jul 21 15:19:54.036829 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:54.036742    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="71b696c2-eb64-4bc2-a564-5c53e5034456" containerName="revoke-keys"
Jul 21 15:19:54.040928 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:54.040906    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-26fa51-htdcp"
Jul 21 15:19:54.043337 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:54.043318    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-494sx\""
Jul 21 15:19:54.047351 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:54.047324    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-26fa51-htdcp"]
Jul 21 15:19:54.135499 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:54.135468    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9709b4ca-1d7a-4a1f-9650-1f0d6b6c0a6b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-26fa51-htdcp\" (UID: \"9709b4ca-1d7a-4a1f-9650-1f0d6b6c0a6b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-26fa51-htdcp"
Jul 21 15:19:54.236555 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:54.236530    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9709b4ca-1d7a-4a1f-9650-1f0d6b6c0a6b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-26fa51-htdcp\" (UID: \"9709b4ca-1d7a-4a1f-9650-1f0d6b6c0a6b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-26fa51-htdcp"
Jul 21 15:19:54.237092 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:54.237076    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9709b4ca-1d7a-4a1f-9650-1f0d6b6c0a6b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-26fa51-htdcp\" (UID: \"9709b4ca-1d7a-4a1f-9650-1f0d6b6c0a6b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-26fa51-htdcp"
Jul 21 15:19:54.353762 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:54.353688    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-26fa51-htdcp"
Jul 21 15:19:54.483809 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:54.483739    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-26fa51-htdcp"]
Jul 21 15:19:54.486533 ip-10-0-130-83 kubenswrapper[2579]: W0721 15:19:54.486508    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9709b4ca_1d7a_4a1f_9650_1f0d6b6c0a6b.slice/crio-a9259b7c6f7926d0fb5312ab64b96cf6763f4d8e21a1a47a77a82a3a1d45168c WatchSource:0}: Error finding container a9259b7c6f7926d0fb5312ab64b96cf6763f4d8e21a1a47a77a82a3a1d45168c: Status 404 returned error can't find the container with id a9259b7c6f7926d0fb5312ab64b96cf6763f4d8e21a1a47a77a82a3a1d45168c
Jul 21 15:19:55.048762 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:55.048683    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-26fa51-htdcp" event={"ID":"9709b4ca-1d7a-4a1f-9650-1f0d6b6c0a6b","Type":"ContainerStarted","Data":"16ea72146fd0792fb72bea893b052007aa9b1befec4482d3b01d0bd6b082365e"}
Jul 21 15:19:55.048762 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:55.048747    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-26fa51-htdcp" event={"ID":"9709b4ca-1d7a-4a1f-9650-1f0d6b6c0a6b","Type":"ContainerStarted","Data":"a9259b7c6f7926d0fb5312ab64b96cf6763f4d8e21a1a47a77a82a3a1d45168c"}
Jul 21 15:19:55.062704 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:55.062660    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-26fa51-htdcp" podStartSLOduration=1.062640754 podStartE2EDuration="1.062640754s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 15:19:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 15:19:55.062050678 +0000 UTC m=+2851.411794091" watchObservedRunningTime="2026-07-21 15:19:55.062640754 +0000 UTC m=+2851.412384168"
Jul 21 15:19:56.055527 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:56.055494    2579 generic.go:356] "Generic (PLEG): container finished" podID="9709b4ca-1d7a-4a1f-9650-1f0d6b6c0a6b" containerID="16ea72146fd0792fb72bea893b052007aa9b1befec4482d3b01d0bd6b082365e" exitCode=0
Jul 21 15:19:56.055965 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:56.055573    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-26fa51-htdcp" event={"ID":"9709b4ca-1d7a-4a1f-9650-1f0d6b6c0a6b","Type":"ContainerDied","Data":"16ea72146fd0792fb72bea893b052007aa9b1befec4482d3b01d0bd6b082365e"}
Jul 21 15:19:57.186140 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:57.186116    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-26fa51-htdcp"
Jul 21 15:19:57.362538 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:57.362475    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9709b4ca-1d7a-4a1f-9650-1f0d6b6c0a6b-maas-api-service-ca\") pod \"9709b4ca-1d7a-4a1f-9650-1f0d6b6c0a6b\" (UID: \"9709b4ca-1d7a-4a1f-9650-1f0d6b6c0a6b\") "
Jul 21 15:19:57.362841 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:57.362822    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9709b4ca-1d7a-4a1f-9650-1f0d6b6c0a6b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9709b4ca-1d7a-4a1f-9650-1f0d6b6c0a6b" (UID: "9709b4ca-1d7a-4a1f-9650-1f0d6b6c0a6b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 15:19:57.463179 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:57.463154    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9709b4ca-1d7a-4a1f-9650-1f0d6b6c0a6b-maas-api-service-ca\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:19:58.065528 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:58.065500    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-26fa51-htdcp"
Jul 21 15:19:58.065706 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:58.065543    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-26fa51-htdcp" event={"ID":"9709b4ca-1d7a-4a1f-9650-1f0d6b6c0a6b","Type":"ContainerDied","Data":"a9259b7c6f7926d0fb5312ab64b96cf6763f4d8e21a1a47a77a82a3a1d45168c"}
Jul 21 15:19:58.065706 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:19:58.065593    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a9259b7c6f7926d0fb5312ab64b96cf6763f4d8e21a1a47a77a82a3a1d45168c"
Jul 21 15:20:04.452892 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:04.452857    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-26fa51-bd6667659-frvqs"]
Jul 21 15:20:04.453430 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:04.453126    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-26fa51-bd6667659-frvqs" podUID="01c22400-ab70-43a9-8a4f-4598db1137f9" containerName="maas-api" containerID="cri-o://94e574c6bda5437b4d0e1e3f622130930fdf377c9bfc65cf3d79e6679d8be612" gracePeriod=30
Jul 21 15:20:04.473067 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:04.473039    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-26fa51-29744115-ll25f"]
Jul 21 15:20:04.478624 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:04.478600    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-26fa51-29744115-ll25f"]
Jul 21 15:20:04.700622 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:04.700598    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-26fa51-bd6667659-frvqs"
Jul 21 15:20:04.715390 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:04.715328    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01c22400-ab70-43a9-8a4f-4598db1137f9-maas-api-tls\") pod \"01c22400-ab70-43a9-8a4f-4598db1137f9\" (UID: \"01c22400-ab70-43a9-8a4f-4598db1137f9\") "
Jul 21 15:20:04.715601 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:04.715386    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01c22400-ab70-43a9-8a4f-4598db1137f9-openshift-service-ca\") pod \"01c22400-ab70-43a9-8a4f-4598db1137f9\" (UID: \"01c22400-ab70-43a9-8a4f-4598db1137f9\") "
Jul 21 15:20:04.715601 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:04.715416    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jd77n\" (UniqueName: \"kubernetes.io/projected/01c22400-ab70-43a9-8a4f-4598db1137f9-kube-api-access-jd77n\") pod \"01c22400-ab70-43a9-8a4f-4598db1137f9\" (UID: \"01c22400-ab70-43a9-8a4f-4598db1137f9\") "
Jul 21 15:20:04.715724 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:04.715701    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01c22400-ab70-43a9-8a4f-4598db1137f9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "01c22400-ab70-43a9-8a4f-4598db1137f9" (UID: "01c22400-ab70-43a9-8a4f-4598db1137f9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 15:20:04.717801 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:04.717778    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01c22400-ab70-43a9-8a4f-4598db1137f9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "01c22400-ab70-43a9-8a4f-4598db1137f9" (UID: "01c22400-ab70-43a9-8a4f-4598db1137f9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 15:20:04.717928 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:04.717905    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01c22400-ab70-43a9-8a4f-4598db1137f9-kube-api-access-jd77n" (OuterVolumeSpecName: "kube-api-access-jd77n") pod "01c22400-ab70-43a9-8a4f-4598db1137f9" (UID: "01c22400-ab70-43a9-8a4f-4598db1137f9"). InnerVolumeSpecName "kube-api-access-jd77n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 15:20:04.816082 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:04.816056    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jd77n\" (UniqueName: \"kubernetes.io/projected/01c22400-ab70-43a9-8a4f-4598db1137f9-kube-api-access-jd77n\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:20:04.816082 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:04.816081    2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01c22400-ab70-43a9-8a4f-4598db1137f9-maas-api-tls\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:20:04.816335 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:04.816096    2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01c22400-ab70-43a9-8a4f-4598db1137f9-openshift-service-ca\") on node \"ip-10-0-130-83.ec2.internal\" DevicePath \"\""
Jul 21 15:20:05.098845 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:05.098765    2579 generic.go:356] "Generic (PLEG): container finished" podID="01c22400-ab70-43a9-8a4f-4598db1137f9" containerID="94e574c6bda5437b4d0e1e3f622130930fdf377c9bfc65cf3d79e6679d8be612" exitCode=0
Jul 21 15:20:05.098845 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:05.098807    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-26fa51-bd6667659-frvqs"
Jul 21 15:20:05.099061 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:05.098851    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-26fa51-bd6667659-frvqs" event={"ID":"01c22400-ab70-43a9-8a4f-4598db1137f9","Type":"ContainerDied","Data":"94e574c6bda5437b4d0e1e3f622130930fdf377c9bfc65cf3d79e6679d8be612"}
Jul 21 15:20:05.099061 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:05.098897    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-26fa51-bd6667659-frvqs" event={"ID":"01c22400-ab70-43a9-8a4f-4598db1137f9","Type":"ContainerDied","Data":"877cf07e960db3b8ddec60ff03fcdbd42a4fae8e6c8df8d21428add334d14756"}
Jul 21 15:20:05.099061 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:05.098917    2579 scope.go:117] "RemoveContainer" containerID="94e574c6bda5437b4d0e1e3f622130930fdf377c9bfc65cf3d79e6679d8be612"
Jul 21 15:20:05.108502 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:05.108481    2579 scope.go:117] "RemoveContainer" containerID="94e574c6bda5437b4d0e1e3f622130930fdf377c9bfc65cf3d79e6679d8be612"
Jul 21 15:20:05.108749 ip-10-0-130-83 kubenswrapper[2579]: E0721 15:20:05.108728    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94e574c6bda5437b4d0e1e3f622130930fdf377c9bfc65cf3d79e6679d8be612\": container with ID starting with 94e574c6bda5437b4d0e1e3f622130930fdf377c9bfc65cf3d79e6679d8be612 not found: ID does not exist" containerID="94e574c6bda5437b4d0e1e3f622130930fdf377c9bfc65cf3d79e6679d8be612"
Jul 21 15:20:05.108815 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:05.108756    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94e574c6bda5437b4d0e1e3f622130930fdf377c9bfc65cf3d79e6679d8be612"} err="failed to get container status \"94e574c6bda5437b4d0e1e3f622130930fdf377c9bfc65cf3d79e6679d8be612\": rpc error: code = NotFound desc = could not find container \"94e574c6bda5437b4d0e1e3f622130930fdf377c9bfc65cf3d79e6679d8be612\": container with ID starting with 94e574c6bda5437b4d0e1e3f622130930fdf377c9bfc65cf3d79e6679d8be612 not found: ID does not exist"
Jul 21 15:20:05.120121 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:05.120098    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-26fa51-bd6667659-frvqs"]
Jul 21 15:20:05.126557 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:05.126536    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-26fa51-bd6667659-frvqs"]
Jul 21 15:20:06.171964 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:06.171931    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01c22400-ab70-43a9-8a4f-4598db1137f9" path="/var/lib/kubelet/pods/01c22400-ab70-43a9-8a4f-4598db1137f9/volumes"
Jul 21 15:20:06.172349 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:06.172205    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="173e05e7-f6bf-4f1f-988e-d0191a104dd0" path="/var/lib/kubelet/pods/173e05e7-f6bf-4f1f-988e-d0191a104dd0/volumes"
Jul 21 15:20:09.113779 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:09.113749    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-26fa51-htdcp"]
Jul 21 15:20:09.116536 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:09.116510    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-26fa51-htdcp"]
Jul 21 15:20:10.172045 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:10.172014    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9709b4ca-1d7a-4a1f-9650-1f0d6b6c0a6b" path="/var/lib/kubelet/pods/9709b4ca-1d7a-4a1f-9650-1f0d6b6c0a6b/volumes"
Jul 21 15:20:21.829748 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:21.829667    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-24blz_86ee645c-7f41-4aac-afaf-285233965458/manager/0.log"
Jul 21 15:20:21.956715 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:21.956680    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-7psmb_2cac96bd-ceca-4fc0-b229-601105cb5647/manager/0.log"
Jul 21 15:20:22.091825 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:22.091740    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7487796bf9-sgk7l_8a1187a0-2f58-4791-ad69-8dd726b9d35e/manager/0.log"
Jul 21 15:20:22.339085 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:22.339033    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-76j8p_fae5f2ef-b1c4-44a8-9016-b74864a3dc50/manager/0.log"
Jul 21 15:20:22.456785 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:22.456759    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b978d8c-gpnqz_0e657220-fcc4-4498-b47b-c757abba31c7/copy-manifests/0.log"
Jul 21 15:20:22.475004 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:22.474969    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b978d8c-gpnqz_0e657220-fcc4-4498-b47b-c757abba31c7/manager/0.log"
Jul 21 15:20:23.103510 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:23.103480    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-56b468fd85-pctgt_7710d3a2-0914-427a-aa34-6941e5886b4e/maas-api/0.log"
Jul 21 15:20:23.216558 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:23.216531    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29744085-k6ptx_3d9c49b4-d4f6-4131-9769-2628dd20d27c/cleanup/0.log"
Jul 21 15:20:23.333177 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:23.333153    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29744100-d2wpr_d6f98b27-7bec-4afc-9487-2a1ed68ae2eb/cleanup/0.log"
Jul 21 15:20:23.456461 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:23.456439    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29744115-wphnq_5f888722-c08b-4c62-8150-f93fee5f267c/cleanup/0.log"
Jul 21 15:20:23.577951 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:23.577922    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-wg2sl_bff4c9fc-5579-48fa-bdbf-5efcb4f3c738/postgres/0.log"
Jul 21 15:20:25.202476 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:25.202441    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-xvpks_1bd08cca-a503-4f55-9557-40d4eab4116d/manager/0.log"
Jul 21 15:20:25.467470 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:25.467366    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-ksq67_6e2fa36d-c716-4d5f-b9d6-222eed421ac9/registry-server/0.log"
Jul 21 15:20:25.880560 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:25.880477    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-bvp9p_80b639cf-e471-4034-bbb7-674deed96de1/manager/0.log"
Jul 21 15:20:26.384381 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:26.384350    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-b22v6_88af516f-4d3f-43f0-8701-4dd909af8e39/discovery/0.log"
Jul 21 15:20:26.494939 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:26.494915    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74d9d9b8c-dnpv2_d1c73e43-f9e9-49f3-af4b-e411a0a21f1b/kube-auth-proxy/0.log"
Jul 21 15:20:27.003248 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:27.003223    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-79nqm_66ba0e99-974c-4ff9-8b52-f2ee4d591271/payload-processing/0.log"
Jul 21 15:20:27.464001 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:27.463970    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-wdt74_b075fa64-0f4e-4011-ac25-9d876655aae9/main/0.log"
Jul 21 15:20:27.701455 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:27.701425    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-ff2mt_3288f187-7e07-48a9-9376-6c521e448f47/main/0.log"
Jul 21 15:20:38.050476 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:38.050447    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-78qqf_80d084f2-ce50-4ce6-9a6f-76e108e0215e/global-pull-secret-syncer/0.log"
Jul 21 15:20:38.243695 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:38.243665    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vhndm_442fea02-6da4-40ad-9f33-d7824c635d4f/konnectivity-agent/0.log"
Jul 21 15:20:38.340187 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:38.340108    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-83.ec2.internal_ada9959260df8e5e8a349b59b83b3eed/haproxy/0.log"
Jul 21 15:20:43.726895 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:43.726862    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-xvpks_1bd08cca-a503-4f55-9557-40d4eab4116d/manager/0.log"
Jul 21 15:20:43.829022 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:43.828996    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-ksq67_6e2fa36d-c716-4d5f-b9d6-222eed421ac9/registry-server/0.log"
Jul 21 15:20:44.298254 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:44.298224    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-bvp9p_80b639cf-e471-4034-bbb7-674deed96de1/manager/0.log"
Jul 21 15:20:46.418415 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:46.418387    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sct6c_fbd4dc34-49eb-4fab-95c0-8867f8fec989/node-exporter/0.log"
Jul 21 15:20:46.436918 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:46.436896    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sct6c_fbd4dc34-49eb-4fab-95c0-8867f8fec989/kube-rbac-proxy/0.log"
Jul 21 15:20:46.458446 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:46.458426    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sct6c_fbd4dc34-49eb-4fab-95c0-8867f8fec989/init-textfile/0.log"
Jul 21 15:20:48.001393 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:48.001359    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-scg49/perf-node-gather-daemonset-6xqk2"]
Jul 21 15:20:48.007851 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:48.007816    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01c22400-ab70-43a9-8a4f-4598db1137f9" containerName="maas-api"
Jul 21 15:20:48.007851 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:48.007843    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="01c22400-ab70-43a9-8a4f-4598db1137f9" containerName="maas-api"
Jul 21 15:20:48.007851 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:48.007857    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9709b4ca-1d7a-4a1f-9650-1f0d6b6c0a6b" containerName="revoke-keys"
Jul 21 15:20:48.008132 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:48.007862    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="9709b4ca-1d7a-4a1f-9650-1f0d6b6c0a6b" containerName="revoke-keys"
Jul 21 15:20:48.008132 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:48.007943    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="01c22400-ab70-43a9-8a4f-4598db1137f9" containerName="maas-api"
Jul 21 15:20:48.008132 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:48.007952    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="9709b4ca-1d7a-4a1f-9650-1f0d6b6c0a6b" containerName="revoke-keys"
Jul 21 15:20:48.011141 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:48.011116    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-scg49/perf-node-gather-daemonset-6xqk2"
Jul 21 15:20:48.011939 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:48.011916    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-scg49/perf-node-gather-daemonset-6xqk2"]
Jul 21 15:20:48.013790 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:48.013770    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-scg49\"/\"kube-root-ca.crt\""
Jul 21 15:20:48.014763 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:48.014745    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-scg49\"/\"default-dockercfg-bd55p\""
Jul 21 15:20:48.014877 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:48.014763    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-scg49\"/\"openshift-service-ca.crt\""
Jul 21 15:20:48.163528 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:48.163500    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f3d3ffa6-e1bc-48a3-bfcf-cc695f4e55d1-podres\") pod \"perf-node-gather-daemonset-6xqk2\" (UID: \"f3d3ffa6-e1bc-48a3-bfcf-cc695f4e55d1\") " pod="openshift-must-gather-scg49/perf-node-gather-daemonset-6xqk2"
Jul 21 15:20:48.163706 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:48.163568    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vw2tt\" (UniqueName: \"kubernetes.io/projected/f3d3ffa6-e1bc-48a3-bfcf-cc695f4e55d1-kube-api-access-vw2tt\") pod \"perf-node-gather-daemonset-6xqk2\" (UID: \"f3d3ffa6-e1bc-48a3-bfcf-cc695f4e55d1\") " pod="openshift-must-gather-scg49/perf-node-gather-daemonset-6xqk2"
Jul 21 15:20:48.163706 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:48.163602    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f3d3ffa6-e1bc-48a3-bfcf-cc695f4e55d1-proc\") pod \"perf-node-gather-daemonset-6xqk2\" (UID: \"f3d3ffa6-e1bc-48a3-bfcf-cc695f4e55d1\") " pod="openshift-must-gather-scg49/perf-node-gather-daemonset-6xqk2"
Jul 21 15:20:48.163706 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:48.163696    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f3d3ffa6-e1bc-48a3-bfcf-cc695f4e55d1-sys\") pod \"perf-node-gather-daemonset-6xqk2\" (UID: \"f3d3ffa6-e1bc-48a3-bfcf-cc695f4e55d1\") " pod="openshift-must-gather-scg49/perf-node-gather-daemonset-6xqk2"
Jul 21 15:20:48.163856 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:48.163741    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f3d3ffa6-e1bc-48a3-bfcf-cc695f4e55d1-lib-modules\") pod \"perf-node-gather-daemonset-6xqk2\" (UID: \"f3d3ffa6-e1bc-48a3-bfcf-cc695f4e55d1\") " pod="openshift-must-gather-scg49/perf-node-gather-daemonset-6xqk2"
Jul 21 15:20:48.265012 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:48.264936    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f3d3ffa6-e1bc-48a3-bfcf-cc695f4e55d1-podres\") pod \"perf-node-gather-daemonset-6xqk2\" (UID: \"f3d3ffa6-e1bc-48a3-bfcf-cc695f4e55d1\") " pod="openshift-must-gather-scg49/perf-node-gather-daemonset-6xqk2"
Jul 21 15:20:48.265012 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:48.264986    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vw2tt\" (UniqueName: \"kubernetes.io/projected/f3d3ffa6-e1bc-48a3-bfcf-cc695f4e55d1-kube-api-access-vw2tt\") pod \"perf-node-gather-daemonset-6xqk2\" (UID: \"f3d3ffa6-e1bc-48a3-bfcf-cc695f4e55d1\") " pod="openshift-must-gather-scg49/perf-node-gather-daemonset-6xqk2"
Jul 21 15:20:48.265012 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:48.265005    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f3d3ffa6-e1bc-48a3-bfcf-cc695f4e55d1-proc\") pod \"perf-node-gather-daemonset-6xqk2\" (UID: \"f3d3ffa6-e1bc-48a3-bfcf-cc695f4e55d1\") " pod="openshift-must-gather-scg49/perf-node-gather-daemonset-6xqk2"
Jul 21 15:20:48.265321 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:48.265046    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f3d3ffa6-e1bc-48a3-bfcf-cc695f4e55d1-sys\") pod \"perf-node-gather-daemonset-6xqk2\" (UID: \"f3d3ffa6-e1bc-48a3-bfcf-cc695f4e55d1\") " pod="openshift-must-gather-scg49/perf-node-gather-daemonset-6xqk2"
Jul 21 15:20:48.265321 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:48.265062    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f3d3ffa6-e1bc-48a3-bfcf-cc695f4e55d1-lib-modules\") pod \"perf-node-gather-daemonset-6xqk2\" (UID: \"f3d3ffa6-e1bc-48a3-bfcf-cc695f4e55d1\") " pod="openshift-must-gather-scg49/perf-node-gather-daemonset-6xqk2"
Jul 21 15:20:48.265321 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:48.265084    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f3d3ffa6-e1bc-48a3-bfcf-cc695f4e55d1-podres\") pod \"perf-node-gather-daemonset-6xqk2\" (UID: \"f3d3ffa6-e1bc-48a3-bfcf-cc695f4e55d1\") " pod="openshift-must-gather-scg49/perf-node-gather-daemonset-6xqk2"
Jul 21 15:20:48.265321 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:48.265128    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f3d3ffa6-e1bc-48a3-bfcf-cc695f4e55d1-sys\") pod \"perf-node-gather-daemonset-6xqk2\" (UID: \"f3d3ffa6-e1bc-48a3-bfcf-cc695f4e55d1\") " pod="openshift-must-gather-scg49/perf-node-gather-daemonset-6xqk2"
Jul 21 15:20:48.265321 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:48.265133    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f3d3ffa6-e1bc-48a3-bfcf-cc695f4e55d1-proc\") pod \"perf-node-gather-daemonset-6xqk2\" (UID: \"f3d3ffa6-e1bc-48a3-bfcf-cc695f4e55d1\") " pod="openshift-must-gather-scg49/perf-node-gather-daemonset-6xqk2"
Jul 21 15:20:48.265321 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:48.265218    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f3d3ffa6-e1bc-48a3-bfcf-cc695f4e55d1-lib-modules\") pod \"perf-node-gather-daemonset-6xqk2\" (UID: \"f3d3ffa6-e1bc-48a3-bfcf-cc695f4e55d1\") " pod="openshift-must-gather-scg49/perf-node-gather-daemonset-6xqk2"
Jul 21 15:20:48.272564 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:48.272535    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vw2tt\" (UniqueName: \"kubernetes.io/projected/f3d3ffa6-e1bc-48a3-bfcf-cc695f4e55d1-kube-api-access-vw2tt\") pod \"perf-node-gather-daemonset-6xqk2\" (UID: \"f3d3ffa6-e1bc-48a3-bfcf-cc695f4e55d1\") " pod="openshift-must-gather-scg49/perf-node-gather-daemonset-6xqk2"
Jul 21 15:20:48.320990 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:48.320968    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-scg49/perf-node-gather-daemonset-6xqk2"
Jul 21 15:20:48.448955 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:48.448931    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-scg49/perf-node-gather-daemonset-6xqk2"]
Jul 21 15:20:48.451177 ip-10-0-130-83 kubenswrapper[2579]: W0721 15:20:48.451149    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf3d3ffa6_e1bc_48a3_bfcf_cc695f4e55d1.slice/crio-78ac953f3603cf8960a5ad9300c940b1f4c3eb208e0d6136610e0537dd286293 WatchSource:0}: Error finding container 78ac953f3603cf8960a5ad9300c940b1f4c3eb208e0d6136610e0537dd286293: Status 404 returned error can't find the container with id 78ac953f3603cf8960a5ad9300c940b1f4c3eb208e0d6136610e0537dd286293
Jul 21 15:20:49.302645 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:49.302604    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-scg49/perf-node-gather-daemonset-6xqk2" event={"ID":"f3d3ffa6-e1bc-48a3-bfcf-cc695f4e55d1","Type":"ContainerStarted","Data":"9f56f534c9e7efc86ec8279b663559fab6747a226e83b098557b6c3be604c634"}
Jul 21 15:20:49.303071 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:49.302660    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-scg49/perf-node-gather-daemonset-6xqk2" event={"ID":"f3d3ffa6-e1bc-48a3-bfcf-cc695f4e55d1","Type":"ContainerStarted","Data":"78ac953f3603cf8960a5ad9300c940b1f4c3eb208e0d6136610e0537dd286293"}
Jul 21 15:20:49.303071 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:49.302789    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-scg49/perf-node-gather-daemonset-6xqk2"
Jul 21 15:20:49.320760 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:49.320714    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-scg49/perf-node-gather-daemonset-6xqk2" podStartSLOduration=2.320700905 podStartE2EDuration="2.320700905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 15:20:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 15:20:49.318974951 +0000 UTC m=+2905.668718386" watchObservedRunningTime="2026-07-21 15:20:49.320700905 +0000 UTC m=+2905.670444517"
Jul 21 15:20:49.454454 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:49.454428    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-98b5cf9c5-5c64f_7132eb81-7f77-487d-8b0c-c964da260e10/console/0.log"
Jul 21 15:20:51.125735 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:51.125710    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-s5kpj_eee39cdb-3b6f-423b-b442-ef9153eabb26/dns/0.log"
Jul 21 15:20:51.146737 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:51.146714    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-s5kpj_eee39cdb-3b6f-423b-b442-ef9153eabb26/kube-rbac-proxy/0.log"
Jul 21 15:20:51.269432 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:51.269404    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-gbn95_b81ae5a9-ca37-4ddd-be8a-3a732b73eb43/dns-node-resolver/0.log"
Jul 21 15:20:51.813694 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:51.813662    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6fd455ff8b-m2b7t_a9bde86f-cc13-46cf-a337-9fc75c370062/registry/0.log"
Jul 21 15:20:51.861683 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:51.861662    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-j25df_7845eb46-a515-4a7b-9c2b-88649a1c8ed2/node-ca/0.log"
Jul 21 15:20:53.050949 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:53.050917    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-b22v6_88af516f-4d3f-43f0-8701-4dd909af8e39/discovery/0.log"
Jul 21 15:20:53.071348 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:53.071319    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74d9d9b8c-dnpv2_d1c73e43-f9e9-49f3-af4b-e411a0a21f1b/kube-auth-proxy/0.log"
Jul 21 15:20:53.264069 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:53.264040    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-79nqm_66ba0e99-974c-4ff9-8b52-f2ee4d591271/payload-processing/0.log"
Jul 21 15:20:54.007388 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:54.007359    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-sxjrb_aca305cc-e88b-477b-828f-371769c6bb26/serve-healthcheck-canary/0.log"
Jul 21 15:20:54.553169 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:54.553131    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qpw79_e5ed87de-30a9-4804-a29c-f60695debb7a/kube-rbac-proxy/0.log"
Jul 21 15:20:54.574597 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:54.574565    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qpw79_e5ed87de-30a9-4804-a29c-f60695debb7a/exporter/0.log"
Jul 21 15:20:54.596525 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:54.596503    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qpw79_e5ed87de-30a9-4804-a29c-f60695debb7a/extractor/0.log"
Jul 21 15:20:55.316992 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:55.316961    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-scg49/perf-node-gather-daemonset-6xqk2"
Jul 21 15:20:56.894263 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:56.894234    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-24blz_86ee645c-7f41-4aac-afaf-285233965458/manager/0.log"
Jul 21 15:20:57.113569 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:57.113540    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-7psmb_2cac96bd-ceca-4fc0-b229-601105cb5647/manager/0.log"
Jul 21 15:20:57.329933 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:57.329906    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7487796bf9-sgk7l_8a1187a0-2f58-4791-ad69-8dd726b9d35e/manager/0.log"
Jul 21 15:20:57.464356 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:57.464327    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-76j8p_fae5f2ef-b1c4-44a8-9016-b74864a3dc50/manager/0.log"
Jul 21 15:20:57.585184 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:57.585113    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b978d8c-gpnqz_0e657220-fcc4-4498-b47b-c757abba31c7/manager/0.log"
Jul 21 15:20:57.606572 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:57.606549    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b978d8c-gpnqz_0e657220-fcc4-4498-b47b-c757abba31c7/copy-manifests/0.log"
Jul 21 15:20:58.712437 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:58.712408    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-b22v6_88af516f-4d3f-43f0-8701-4dd909af8e39/discovery/0.log"
Jul 21 15:20:58.726704 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:58.726682    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74d9d9b8c-dnpv2_d1c73e43-f9e9-49f3-af4b-e411a0a21f1b/kube-auth-proxy/0.log"
Jul 21 15:20:58.888221 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:20:58.888196    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-79nqm_66ba0e99-974c-4ff9-8b52-f2ee4d591271/payload-processing/0.log"
Jul 21 15:21:00.372436 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:21:00.372402    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-b22v6_88af516f-4d3f-43f0-8701-4dd909af8e39/discovery/0.log"
Jul 21 15:21:00.384086 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:21:00.384035    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74d9d9b8c-dnpv2_d1c73e43-f9e9-49f3-af4b-e411a0a21f1b/kube-auth-proxy/0.log"
Jul 21 15:21:00.541547 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:21:00.541520    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-79nqm_66ba0e99-974c-4ff9-8b52-f2ee4d591271/payload-processing/0.log"
Jul 21 15:21:05.393167 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:21:05.393139    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gxsg8_37083d35-d10f-4542-871a-a16c12c6bc7f/kube-multus-additional-cni-plugins/0.log"
Jul 21 15:21:05.415027 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:21:05.415006    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gxsg8_37083d35-d10f-4542-871a-a16c12c6bc7f/egress-router-binary-copy/0.log"
Jul 21 15:21:05.436632 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:21:05.436608    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gxsg8_37083d35-d10f-4542-871a-a16c12c6bc7f/cni-plugins/0.log"
Jul 21 15:21:05.459387 ip-10-0-130-83 kubenswrapper[2579]: I0721 15:21:05.459368    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gxsg8_37083d35-d10f-4542-871a-a16c12c6bc7f/bond-cni-plugin/0.log"