Jul 20 09:38:52.190800 ip-10-0-140-134 systemd[1]: Starting Kubernetes Kubelet...
Jul 20 09:38:52.555517 ip-10-0-140-134 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 20 09:38:52.555517 ip-10-0-140-134 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 20 09:38:52.555517 ip-10-0-140-134 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 20 09:38:52.555517 ip-10-0-140-134 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 20 09:38:52.555517 ip-10-0-140-134 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 20 09:38:52.556439 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.556312    2566 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 20 09:38:52.563999 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.563967    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 09:38:52.563999 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.563992    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 09:38:52.563999 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.563995    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 09:38:52.563999 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.563999    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 09:38:52.563999 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564002    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 09:38:52.563999 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564004    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 09:38:52.563999 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564007    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 09:38:52.563999 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564012    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 09:38:52.564304 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564017    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 09:38:52.564304 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564020    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 09:38:52.564304 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564023    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 09:38:52.564304 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564026    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 09:38:52.564304 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564028    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 09:38:52.564304 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564031    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 09:38:52.564304 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564034    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 09:38:52.564304 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564036    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 09:38:52.564304 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564039    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 09:38:52.564304 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564042    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 09:38:52.564304 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564044    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 09:38:52.564304 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564047    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 09:38:52.564304 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564049    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 09:38:52.564304 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564052    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 09:38:52.564304 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564055    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 09:38:52.564304 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564057    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 09:38:52.564304 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564059    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 09:38:52.564304 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564062    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 09:38:52.564304 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564065    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 09:38:52.564304 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564068    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 09:38:52.564791 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564072    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 09:38:52.564791 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564076    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 09:38:52.564791 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564078    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 09:38:52.564791 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564081    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 09:38:52.564791 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564083    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 09:38:52.564791 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564085    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 09:38:52.564791 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564088    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 09:38:52.564791 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564090    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 09:38:52.564791 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564093    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 09:38:52.564791 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564095    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 09:38:52.564791 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564099    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 09:38:52.564791 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564101    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 09:38:52.564791 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564103    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 09:38:52.564791 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564108    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 09:38:52.564791 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564110    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 09:38:52.564791 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564113    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 09:38:52.564791 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564115    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 09:38:52.564791 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564118    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 09:38:52.564791 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564120    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 09:38:52.564791 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564123    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 09:38:52.565322 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564125    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 09:38:52.565322 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564128    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 09:38:52.565322 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564130    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 09:38:52.565322 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564133    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 09:38:52.565322 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564136    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 09:38:52.565322 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564138    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 09:38:52.565322 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564141    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 09:38:52.565322 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564143    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 09:38:52.565322 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564145    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 09:38:52.565322 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564148    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 09:38:52.565322 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564150    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 09:38:52.565322 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564153    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 09:38:52.565322 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564155    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 09:38:52.565322 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564158    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 09:38:52.565322 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564160    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 09:38:52.565322 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564163    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 09:38:52.565322 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564166    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 09:38:52.565322 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564168    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 09:38:52.565322 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564170    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 09:38:52.565322 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564173    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 09:38:52.565817 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564178    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 09:38:52.565817 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564183    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 09:38:52.565817 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564186    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 09:38:52.565817 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564189    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 09:38:52.565817 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564192    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 09:38:52.565817 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564196    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 09:38:52.565817 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564198    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 09:38:52.565817 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564201    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 09:38:52.565817 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564204    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 09:38:52.565817 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564206    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 09:38:52.565817 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564208    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 09:38:52.565817 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564211    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 09:38:52.565817 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564213    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 09:38:52.565817 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564216    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 09:38:52.565817 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564218    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 09:38:52.565817 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564222    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 09:38:52.565817 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564224    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 09:38:52.565817 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564227    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 09:38:52.565817 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564230    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 09:38:52.566276 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564232    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 09:38:52.566276 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564235    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 09:38:52.566276 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564237    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 09:38:52.566276 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564239    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 09:38:52.566276 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564242    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 09:38:52.566276 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564245    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 09:38:52.566276 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564247    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 09:38:52.566276 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564249    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 09:38:52.566276 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564252    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 09:38:52.566276 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564254    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 09:38:52.566276 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564256    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 09:38:52.566276 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.564259    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 09:38:52.566276 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565059    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 09:38:52.566276 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565066    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 09:38:52.566276 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565069    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 09:38:52.566276 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565072    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 09:38:52.566276 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565074    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 09:38:52.566276 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565077    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 09:38:52.566276 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565080    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 09:38:52.566276 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565083    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 09:38:52.566809 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565086    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 09:38:52.566809 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565090    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 09:38:52.566809 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565094    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 09:38:52.566809 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565096    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 09:38:52.566809 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565099    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 09:38:52.566809 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565101    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 09:38:52.566809 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565104    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 09:38:52.566809 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565107    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 09:38:52.566809 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565109    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 09:38:52.566809 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565112    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 09:38:52.566809 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565115    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 09:38:52.566809 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565118    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 09:38:52.566809 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565121    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 09:38:52.566809 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565123    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 09:38:52.566809 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565126    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 09:38:52.566809 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565128    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 09:38:52.566809 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565131    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 09:38:52.566809 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565133    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 09:38:52.566809 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565136    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 09:38:52.567275 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565138    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 09:38:52.567275 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565141    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 09:38:52.567275 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565144    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 09:38:52.567275 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565146    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 09:38:52.567275 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565149    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 09:38:52.567275 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565151    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 09:38:52.567275 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565153    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 09:38:52.567275 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565156    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 09:38:52.567275 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565159    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 09:38:52.567275 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565161    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 09:38:52.567275 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565164    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 09:38:52.567275 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565167    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 09:38:52.567275 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565169    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 09:38:52.567275 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565172    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 09:38:52.567275 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565175    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 09:38:52.567275 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565179    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 09:38:52.567275 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565183    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 09:38:52.567275 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565186    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 09:38:52.567275 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565189    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 09:38:52.567275 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565192    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 09:38:52.567811 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565195    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 09:38:52.567811 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565197    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 09:38:52.567811 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565200    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 09:38:52.567811 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565203    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 09:38:52.567811 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565206    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 09:38:52.567811 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565208    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 09:38:52.567811 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565212    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 09:38:52.567811 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565214    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 09:38:52.567811 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565216    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 09:38:52.567811 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565219    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 09:38:52.567811 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565221    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 09:38:52.567811 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565224    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 09:38:52.567811 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565226    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 09:38:52.567811 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565229    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 09:38:52.567811 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565231    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 09:38:52.567811 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565234    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 09:38:52.567811 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565236    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 09:38:52.567811 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565239    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 09:38:52.567811 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565241    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 09:38:52.567811 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565244    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 09:38:52.568307 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565247    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 09:38:52.568307 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565250    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 09:38:52.568307 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565253    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 09:38:52.568307 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565256    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 09:38:52.568307 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565259    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 09:38:52.568307 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565261    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 09:38:52.568307 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565264    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 09:38:52.568307 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565267    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 09:38:52.568307 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565269    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 09:38:52.568307 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565272    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 09:38:52.568307 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565275    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 09:38:52.568307 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565277    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 09:38:52.568307 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565279    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 09:38:52.568307 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565282    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 09:38:52.568307 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565284    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 09:38:52.568307 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565287    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 09:38:52.568307 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565293    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 09:38:52.568307 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565295    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 09:38:52.568307 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565298    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 09:38:52.568307 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565301    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 09:38:52.568834 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565303    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 09:38:52.568834 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565307    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 09:38:52.568834 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565309    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 09:38:52.568834 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565312    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 09:38:52.568834 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565314    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 09:38:52.568834 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565317    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 09:38:52.568834 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565320    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 09:38:52.568834 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565324    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 09:38:52.568834 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565330    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 09:38:52.568834 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565333    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 09:38:52.568834 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565337    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 09:38:52.568834 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565339    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 09:38:52.568834 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565810    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 09:38:52.568834 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565817    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 09:38:52.568834 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565821    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 09:38:52.568834 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565824    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 09:38:52.568834 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565827    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 09:38:52.568834 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565830    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 09:38:52.568834 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565833    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 09:38:52.569391 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565836    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 09:38:52.569391 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565838    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 09:38:52.569391 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565841    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 09:38:52.569391 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565844    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 09:38:52.569391 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565846    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 09:38:52.569391 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565849    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 09:38:52.569391 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565851    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 09:38:52.569391 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565854    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 09:38:52.569391 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565856    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 09:38:52.569391 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565860    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 09:38:52.569391 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565862    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 09:38:52.569391 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565865    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 09:38:52.569391 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565867    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 09:38:52.569391 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565869    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 09:38:52.569391 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565872    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 09:38:52.569391 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565874    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 09:38:52.569391 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565877    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 09:38:52.569391 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565879    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 09:38:52.569391 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565882    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 09:38:52.569391 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565885    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 09:38:52.569907 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565889    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 09:38:52.569907 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565894    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 09:38:52.569907 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565897    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 09:38:52.569907 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565899    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 09:38:52.569907 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565902    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 09:38:52.569907 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565904    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 09:38:52.569907 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565907    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 09:38:52.569907 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565910    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 09:38:52.569907 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565913    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 09:38:52.569907 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565916    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 09:38:52.569907 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565918    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 09:38:52.569907 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565920    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 09:38:52.569907 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565923    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 09:38:52.569907 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565926    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 09:38:52.569907 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565929    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 09:38:52.569907 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565931    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 09:38:52.569907 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565934    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 09:38:52.569907 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565936    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 09:38:52.569907 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565939    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 09:38:52.570366 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565941    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 09:38:52.570366 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565944    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 09:38:52.570366 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565946    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 09:38:52.570366 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565949    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 09:38:52.570366 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565952    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 09:38:52.570366 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565954    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 09:38:52.570366 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565956    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 09:38:52.570366 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565959    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 09:38:52.570366 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565961    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 09:38:52.570366 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565964    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 09:38:52.570366 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565967    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 09:38:52.570366 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565969    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 09:38:52.570366 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565972    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 09:38:52.570366 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565974    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 09:38:52.570366 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565980    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 09:38:52.570366 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565983    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 09:38:52.570366 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565986    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 09:38:52.570366 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565989    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 09:38:52.570366 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565992    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 09:38:52.570847 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565994    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 09:38:52.570847 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.565997    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 09:38:52.570847 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566000    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 09:38:52.570847 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566003    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 09:38:52.570847 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566006    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 09:38:52.570847 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566008    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 09:38:52.570847 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566011    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 09:38:52.570847 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566014    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 09:38:52.570847 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566016    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 09:38:52.570847 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566019    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 09:38:52.570847 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566021    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 09:38:52.570847 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566023    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 09:38:52.570847 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566026    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 09:38:52.570847 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566028    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 09:38:52.570847 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566031    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 09:38:52.570847 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566034    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 09:38:52.570847 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566036    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 09:38:52.570847 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566039    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 09:38:52.570847 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566042    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 09:38:52.570847 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566044    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 09:38:52.571346 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566047    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 09:38:52.571346 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566049    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 09:38:52.571346 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566051    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 09:38:52.571346 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566054    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 09:38:52.571346 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566056    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 09:38:52.571346 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566059    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 09:38:52.571346 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566064    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 09:38:52.571346 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566068    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 09:38:52.571346 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566071    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 09:38:52.571346 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566075    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 09:38:52.571346 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566078    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 09:38:52.571346 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566081    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 09:38:52.571346 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566084    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 09:38:52.571346 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566087    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 09:38:52.571346 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566167    2566 flags.go:64] FLAG: --address="0.0.0.0"
Jul 20 09:38:52.571346 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566175    2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 20 09:38:52.571346 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566182    2566 flags.go:64] FLAG: --anonymous-auth="true"
Jul 20 09:38:52.571346 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566188    2566 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 20 09:38:52.571346 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566194    2566 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 20 09:38:52.571346 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566199    2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 20 09:38:52.571346 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566205    2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 20 09:38:52.571890 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566212    2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 20 09:38:52.571890 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566216    2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 20 09:38:52.571890 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566220    2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 20 09:38:52.571890 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566224    2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 20 09:38:52.571890 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566227    2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 20 09:38:52.571890 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566230    2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 20 09:38:52.571890 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566233    2566 flags.go:64] FLAG: --cgroup-root=""
Jul 20 09:38:52.571890 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566236    2566 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 20 09:38:52.571890 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566238    2566 flags.go:64] FLAG: --client-ca-file=""
Jul 20 09:38:52.571890 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566242    2566 flags.go:64] FLAG: --cloud-provider="external"
Jul 20 09:38:52.571890 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566245    2566 flags.go:64] FLAG: --cluster-dns="[]"
Jul 20 09:38:52.571890 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566250    2566 flags.go:64] FLAG: --cluster-domain=""
Jul 20 09:38:52.571890 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566253    2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 20 09:38:52.571890 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566256    2566 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 20 09:38:52.571890 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566259    2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 20 09:38:52.571890 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566262    2566 flags.go:64] FLAG: --container-log-max-files="5"
Jul 20 09:38:52.571890 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566266    2566 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 20 09:38:52.571890 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566269    2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 20 09:38:52.571890 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566274    2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 20 09:38:52.571890 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566278    2566 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 20 09:38:52.571890 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566281    2566 flags.go:64] FLAG: --contention-profiling="false"
Jul 20 09:38:52.571890 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566284    2566 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 20 09:38:52.571890 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566287    2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 20 09:38:52.571890 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566290    2566 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 20 09:38:52.572497 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566293    2566 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 20 09:38:52.572497 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566299    2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 20 09:38:52.572497 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566302    2566 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 20 09:38:52.572497 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566305    2566 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 20 09:38:52.572497 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566308    2566 flags.go:64] FLAG: --enable-load-reader="false"
Jul 20 09:38:52.572497 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566311    2566 flags.go:64] FLAG: --enable-server="true"
Jul 20 09:38:52.572497 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566320    2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 20 09:38:52.572497 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566326    2566 flags.go:64] FLAG: --event-burst="100"
Jul 20 09:38:52.572497 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566329    2566 flags.go:64] FLAG: --event-qps="50"
Jul 20 09:38:52.572497 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566332    2566 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 20 09:38:52.572497 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566335    2566 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 20 09:38:52.572497 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566338    2566 flags.go:64] FLAG: --eviction-hard=""
Jul 20 09:38:52.572497 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566341    2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 20 09:38:52.572497 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566344    2566 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 20 09:38:52.572497 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566347    2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 20 09:38:52.572497 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566350    2566 flags.go:64] FLAG: --eviction-soft=""
Jul 20 09:38:52.572497 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566353    2566 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 20 09:38:52.572497 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566355    2566 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 20 09:38:52.572497 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566359    2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 20 09:38:52.572497 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566362    2566 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 20 09:38:52.572497 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566365    2566 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 20 09:38:52.572497 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566368    2566 flags.go:64] FLAG: --fail-swap-on="true"
Jul 20 09:38:52.572497 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566370    2566 flags.go:64] FLAG: --feature-gates=""
Jul 20 09:38:52.572497 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566374    2566 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 20 09:38:52.572497 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566377    2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 20 09:38:52.573194 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566380    2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 20 09:38:52.573194 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566384    2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 20 09:38:52.573194 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566387    2566 flags.go:64] FLAG: --healthz-port="10248"
Jul 20 09:38:52.573194 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566389    2566 flags.go:64] FLAG: --help="false"
Jul 20 09:38:52.573194 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566392    2566 flags.go:64] FLAG: --hostname-override="ip-10-0-140-134.ec2.internal"
Jul 20 09:38:52.573194 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566396    2566 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 20 09:38:52.573194 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566399    2566 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 20 09:38:52.573194 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566401    2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 20 09:38:52.573194 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566406    2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 20 09:38:52.573194 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566410    2566 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 20 09:38:52.573194 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566413    2566 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 20 09:38:52.573194 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566416    2566 flags.go:64] FLAG: --image-service-endpoint=""
Jul 20 09:38:52.573194 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566418    2566 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 20 09:38:52.573194 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566421    2566 flags.go:64] FLAG: --kube-api-burst="100"
Jul 20 09:38:52.573194 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566423    2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 20 09:38:52.573194 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566426    2566 flags.go:64] FLAG: --kube-api-qps="50"
Jul 20 09:38:52.573194 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566429    2566 flags.go:64] FLAG: --kube-reserved=""
Jul 20 09:38:52.573194 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566432    2566 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 20 09:38:52.573194 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566435    2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 20 09:38:52.573194 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566438    2566 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 20 09:38:52.573194 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566440    2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 20 09:38:52.573194 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566443    2566 flags.go:64] FLAG: --lock-file=""
Jul 20 09:38:52.573194 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566446    2566 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 20 09:38:52.573194 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566448    2566 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 20 09:38:52.573852 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566452    2566 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 20 09:38:52.573852 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566470    2566 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 20 09:38:52.573852 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566474    2566 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 20 09:38:52.573852 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566478    2566 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 20 09:38:52.573852 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566482    2566 flags.go:64] FLAG: --logging-format="text"
Jul 20 09:38:52.573852 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566487    2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 20 09:38:52.573852 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566492    2566 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 20 09:38:52.573852 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566496    2566 flags.go:64] FLAG: --manifest-url=""
Jul 20 09:38:52.573852 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566500    2566 flags.go:64] FLAG: --manifest-url-header=""
Jul 20 09:38:52.573852 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566508    2566 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 20 09:38:52.573852 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566513    2566 flags.go:64] FLAG: --max-open-files="1000000"
Jul 20 09:38:52.573852 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566519    2566 flags.go:64] FLAG: --max-pods="110"
Jul 20 09:38:52.573852 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566522    2566 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 20 09:38:52.573852 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566525    2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 20 09:38:52.573852 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566527    2566 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 20 09:38:52.573852 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566530    2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 20 09:38:52.573852 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566535    2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 20 09:38:52.573852 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566538    2566 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 20 09:38:52.573852 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566541    2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 20 09:38:52.573852 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566550    2566 flags.go:64] FLAG: --node-status-max-images="50"
Jul 20 09:38:52.573852 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566553    2566 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 20 09:38:52.573852 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566556    2566 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 20 09:38:52.573852 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566558    2566 flags.go:64] FLAG: --pod-cidr=""
Jul 20 09:38:52.574439 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566561    2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 20 09:38:52.574439 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566567    2566 flags.go:64] FLAG: --pod-manifest-path=""
Jul 20 09:38:52.574439 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566570    2566 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 20 09:38:52.574439 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566573    2566 flags.go:64] FLAG: --pods-per-core="0"
Jul 20 09:38:52.574439 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566575    2566 flags.go:64] FLAG: --port="10250"
Jul 20 09:38:52.574439 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566578    2566 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 20 09:38:52.574439 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566581    2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0229200d18616b8b5"
Jul 20 09:38:52.574439 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566584    2566 flags.go:64] FLAG: --qos-reserved=""
Jul 20 09:38:52.574439 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566587    2566 flags.go:64] FLAG: --read-only-port="10255"
Jul 20 09:38:52.574439 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566590    2566 flags.go:64] FLAG: --register-node="true"
Jul 20 09:38:52.574439 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566593    2566 flags.go:64] FLAG: --register-with-taints=""
Jul 20 09:38:52.574439 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566597    2566 flags.go:64] FLAG: --registry-burst="10"
Jul 20 09:38:52.574439 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566600    2566 flags.go:64] FLAG: --registry-qps="5"
Jul 20 09:38:52.574439 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566603    2566 flags.go:64] FLAG: --reserved-cpus=""
Jul 20 09:38:52.574439 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566606    2566 flags.go:64] FLAG: --reserved-memory=""
Jul 20 09:38:52.574439 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566609    2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 20 09:38:52.574439 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566613    2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 20 09:38:52.574439 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566616    2566 flags.go:64] FLAG: --rotate-certificates="false"
Jul 20 09:38:52.574439 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566619    2566 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 20 09:38:52.574439 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566621    2566 flags.go:64] FLAG: --runonce="false"
Jul 20 09:38:52.574439 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566625    2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 20 09:38:52.574439 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566627    2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 20 09:38:52.574439 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566630    2566 flags.go:64] FLAG: --seccomp-default="false"
Jul 20 09:38:52.574439 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566633    2566 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 20 09:38:52.574439 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566635    2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 20 09:38:52.575179 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566640    2566 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 20 09:38:52.575179 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566643    2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 20 09:38:52.575179 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566646    2566 flags.go:64] FLAG: --storage-driver-password="root"
Jul 20 09:38:52.575179 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566649    2566 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 20 09:38:52.575179 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566652    2566 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 20 09:38:52.575179 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566655    2566 flags.go:64] FLAG: --storage-driver-user="root"
Jul 20 09:38:52.575179 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566658    2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 20 09:38:52.575179 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566661    2566 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 20 09:38:52.575179 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566664    2566 flags.go:64] FLAG: --system-cgroups=""
Jul 20 09:38:52.575179 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566667    2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 20 09:38:52.575179 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566672    2566 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 20 09:38:52.575179 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566675    2566 flags.go:64] FLAG: --tls-cert-file=""
Jul 20 09:38:52.575179 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566678    2566 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 20 09:38:52.575179 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566682    2566 flags.go:64] FLAG: --tls-min-version=""
Jul 20 09:38:52.575179 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566685    2566 flags.go:64] FLAG: --tls-private-key-file=""
Jul 20 09:38:52.575179 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566687    2566 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 20 09:38:52.575179 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566691    2566 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 20 09:38:52.575179 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566693    2566 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 20 09:38:52.575179 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566696    2566 flags.go:64] FLAG: --v="2"
Jul 20 09:38:52.575179 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566700    2566 flags.go:64] FLAG: --version="false"
Jul 20 09:38:52.575179 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566705    2566 flags.go:64] FLAG: --vmodule=""
Jul 20 09:38:52.575179 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566709    2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 20 09:38:52.575179 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.566712    2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 20 09:38:52.575179 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566839    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 09:38:52.575179 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566843    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 09:38:52.575818 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566846    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 09:38:52.575818 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566849    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 09:38:52.575818 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566851    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 09:38:52.575818 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566854    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 09:38:52.575818 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566858    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 09:38:52.575818 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566860    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 09:38:52.575818 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566863    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 09:38:52.575818 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566868    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 09:38:52.575818 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566871    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 09:38:52.575818 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566873    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 09:38:52.575818 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566876    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 09:38:52.575818 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566878    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 09:38:52.575818 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566881    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 09:38:52.575818 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566883    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 09:38:52.575818 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566887    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 09:38:52.575818 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566891    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 09:38:52.575818 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566894    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 09:38:52.575818 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566897    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 09:38:52.575818 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566899    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 09:38:52.576302 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566902    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 09:38:52.576302 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566905    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 09:38:52.576302 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566907    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 09:38:52.576302 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566910    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 09:38:52.576302 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566912    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 09:38:52.576302 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566914    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 09:38:52.576302 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566917    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 09:38:52.576302 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566920    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 09:38:52.576302 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566922    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 09:38:52.576302 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566925    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 09:38:52.576302 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566928    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 09:38:52.576302 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566930    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 09:38:52.576302 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566933    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 09:38:52.576302 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566937    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 09:38:52.576302 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566939    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 09:38:52.576302 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566942    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 09:38:52.576302 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566944    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 09:38:52.576302 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566947    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 09:38:52.576302 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566949    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 09:38:52.576302 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566952    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 09:38:52.576854 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566955    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 09:38:52.576854 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566958    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 09:38:52.576854 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566960    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 09:38:52.576854 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566963    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 09:38:52.576854 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566965    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 09:38:52.576854 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566968    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 09:38:52.576854 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566970    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 09:38:52.576854 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566973    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 09:38:52.576854 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566975    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 09:38:52.576854 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566977    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 09:38:52.576854 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566980    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 09:38:52.576854 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566982    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 09:38:52.576854 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566985    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 09:38:52.576854 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566987    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 09:38:52.576854 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566991    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 09:38:52.576854 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566995    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 09:38:52.576854 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.566998    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 09:38:52.576854 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.567001    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 09:38:52.576854 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.567004    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 09:38:52.577335 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.567006    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 09:38:52.577335 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.567009    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 09:38:52.577335 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.567012    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 09:38:52.577335 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.567015    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 09:38:52.577335 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.567017    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 09:38:52.577335 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.567020    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 09:38:52.577335 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.567025    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 09:38:52.577335 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.567027    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 09:38:52.577335 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.567030    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 09:38:52.577335 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.567032    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 09:38:52.577335 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.567035    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 09:38:52.577335 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.567038    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 09:38:52.577335 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.567040    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 09:38:52.577335 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.567044    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 09:38:52.577335 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.567047    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 09:38:52.577335 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.567049    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 09:38:52.577335 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.567052    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 09:38:52.577335 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.567054    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 09:38:52.577335 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.567057    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 09:38:52.577335 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.567060    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 09:38:52.577938 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.567062    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 09:38:52.577938 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.567066    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 09:38:52.577938 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.567069    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 09:38:52.577938 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.567072    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 09:38:52.577938 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.567075    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 09:38:52.577938 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.567077    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 09:38:52.577938 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.567079    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 09:38:52.577938 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.567082    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 09:38:52.577938 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.567085    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 09:38:52.577938 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.567087    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 09:38:52.577938 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.567089    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 09:38:52.577938 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.567092    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 09:38:52.577938 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.567094    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 09:38:52.577938 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.567097    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 09:38:52.577938 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.567099    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 09:38:52.577938 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.567102    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 09:38:52.577938 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.567104    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 09:38:52.577938 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.567107    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 09:38:52.577938 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.567111    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 09:38:52.578498 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.567690    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 20 09:38:52.578498 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.575036    2566 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 20 09:38:52.578498 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.575059    2566 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 20 09:38:52.578498 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.575079    2566 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 20 09:38:52.578498 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.575085    2566 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 20 09:38:52.578498 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575146    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 09:38:52.578498 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575152    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 09:38:52.578498 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575155    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 09:38:52.578498 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575158    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 09:38:52.578498 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575161    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 09:38:52.578498 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575164    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 09:38:52.578498 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575167    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 09:38:52.578498 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575171    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 09:38:52.578498 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575174    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 09:38:52.578498 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575177    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 09:38:52.578897 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575180    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 09:38:52.578897 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575183    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 09:38:52.578897 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575185    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 09:38:52.578897 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575188    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 09:38:52.578897 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575190    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 09:38:52.578897 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575193    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 09:38:52.578897 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575196    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 09:38:52.578897 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575199    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 09:38:52.578897 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575203    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 09:38:52.578897 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575207    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 09:38:52.578897 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575210    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 09:38:52.578897 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575213    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 09:38:52.578897 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575216    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 09:38:52.578897 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575219    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 09:38:52.578897 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575223    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 09:38:52.578897 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575225    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 09:38:52.578897 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575228    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 09:38:52.578897 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575230    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 09:38:52.578897 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575233    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 09:38:52.579393 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575236    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 09:38:52.579393 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575240    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 09:38:52.579393 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575244    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 09:38:52.579393 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575246    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 09:38:52.579393 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575249    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 09:38:52.579393 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575253    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 09:38:52.579393 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575256    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 09:38:52.579393 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575259    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 09:38:52.579393 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575261    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 09:38:52.579393 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575264    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 09:38:52.579393 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575267    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 09:38:52.579393 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575269    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 09:38:52.579393 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575272    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 09:38:52.579393 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575275    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 09:38:52.579393 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575277    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 09:38:52.579393 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575279    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 09:38:52.579393 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575283    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 09:38:52.579393 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575285    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 09:38:52.579393 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575288    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 09:38:52.579895 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575291    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 09:38:52.579895 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575293    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 09:38:52.579895 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575295    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 09:38:52.579895 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575298    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 09:38:52.579895 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575301    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 09:38:52.579895 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575303    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 09:38:52.579895 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575306    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 09:38:52.579895 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575308    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 09:38:52.579895 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575311    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 09:38:52.579895 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575314    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 09:38:52.579895 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575316    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 09:38:52.579895 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575319    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 09:38:52.579895 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575321    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 09:38:52.579895 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575325    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 09:38:52.579895 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575327    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 09:38:52.579895 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575330    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 09:38:52.579895 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575333    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 09:38:52.579895 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575335    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 09:38:52.579895 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575338    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 09:38:52.579895 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575341    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 09:38:52.580399 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575343    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 09:38:52.580399 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575346    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 09:38:52.580399 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575348    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 09:38:52.580399 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575351    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 09:38:52.580399 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575353    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 09:38:52.580399 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575356    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 09:38:52.580399 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575358    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 09:38:52.580399 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575360    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 09:38:52.580399 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575363    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 09:38:52.580399 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575365    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 09:38:52.580399 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575368    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 09:38:52.580399 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575371    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 09:38:52.580399 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575373    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 09:38:52.580399 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575376    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 09:38:52.580399 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575378    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 09:38:52.580399 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575381    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 09:38:52.580399 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575384    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 09:38:52.580399 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575386    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 09:38:52.580399 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575388    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 09:38:52.580399 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575391    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 09:38:52.581032 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575393    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 09:38:52.581032 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575396    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 09:38:52.581032 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575399    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 09:38:52.581032 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575401    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 09:38:52.581032 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575404    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 09:38:52.581032 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575407    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 09:38:52.581032 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575410    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 09:38:52.581032 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575412    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 09:38:52.581032 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575415    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 09:38:52.581032 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575417    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 09:38:52.581032 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575420    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 09:38:52.581032 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.575426    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 20 09:38:52.581032 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575596    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 20 09:38:52.581032 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575602    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 20 09:38:52.581382 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575605    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 20 09:38:52.581382 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575608    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 20 09:38:52.581382 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575611    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 20 09:38:52.581382 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575614    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 20 09:38:52.581382 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575617    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 20 09:38:52.581382 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575619    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 20 09:38:52.581382 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575621    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 20 09:38:52.581382 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575624    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 20 09:38:52.581382 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575626    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 20 09:38:52.581382 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575629    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 20 09:38:52.581382 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575631    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 20 09:38:52.581382 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575634    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 20 09:38:52.581382 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575636    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 20 09:38:52.581382 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575639    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 20 09:38:52.581382 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575641    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 20 09:38:52.581382 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575645    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 20 09:38:52.581382 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575648    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 20 09:38:52.581382 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575651    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 20 09:38:52.581382 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575654    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 20 09:38:52.581382 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575656    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 20 09:38:52.581930 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575658    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 20 09:38:52.581930 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575664    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 20 09:38:52.581930 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575667    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 20 09:38:52.581930 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575670    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 20 09:38:52.581930 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575673    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 20 09:38:52.581930 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575676    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 20 09:38:52.581930 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575678    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 20 09:38:52.581930 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575681    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 20 09:38:52.581930 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575684    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 20 09:38:52.581930 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575686    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 20 09:38:52.581930 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575689    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 20 09:38:52.581930 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575691    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 20 09:38:52.581930 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575694    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 20 09:38:52.581930 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575696    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 20 09:38:52.581930 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575699    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 20 09:38:52.581930 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575702    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 20 09:38:52.581930 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575704    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 20 09:38:52.581930 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575707    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 20 09:38:52.581930 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575709    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 20 09:38:52.582404 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575712    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 20 09:38:52.582404 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575714    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 20 09:38:52.582404 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575717    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 20 09:38:52.582404 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575719    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 20 09:38:52.582404 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575721    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 20 09:38:52.582404 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575724    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 20 09:38:52.582404 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575726    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 20 09:38:52.582404 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575729    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 20 09:38:52.582404 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575731    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 20 09:38:52.582404 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575733    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 20 09:38:52.582404 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575736    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 20 09:38:52.582404 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575738    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 20 09:38:52.582404 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575741    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 20 09:38:52.582404 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575743    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 20 09:38:52.582404 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575746    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 20 09:38:52.582404 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575748    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 20 09:38:52.582404 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575751    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 20 09:38:52.582404 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575753    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 20 09:38:52.582404 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575756    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 20 09:38:52.582404 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575758    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 20 09:38:52.582917 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575761    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 20 09:38:52.582917 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575763    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 20 09:38:52.582917 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575766    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 20 09:38:52.582917 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575769    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 20 09:38:52.582917 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575772    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 20 09:38:52.582917 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575775    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 20 09:38:52.582917 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575779    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 20 09:38:52.582917 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575782    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 20 09:38:52.582917 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575786    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 20 09:38:52.582917 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575789    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 20 09:38:52.582917 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575792    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 20 09:38:52.582917 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575794    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 20 09:38:52.582917 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575797    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 20 09:38:52.582917 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575800    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 20 09:38:52.582917 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575802    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 20 09:38:52.582917 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575805    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 20 09:38:52.582917 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575808    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 20 09:38:52.582917 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575810    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 20 09:38:52.582917 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575812    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 20 09:38:52.583433 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575815    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 20 09:38:52.583433 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575817    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 20 09:38:52.583433 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575820    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 20 09:38:52.583433 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575822    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 20 09:38:52.583433 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575825    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 20 09:38:52.583433 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575828    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 20 09:38:52.583433 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575830    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 20 09:38:52.583433 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575833    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 20 09:38:52.583433 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575835    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 20 09:38:52.583433 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575838    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 20 09:38:52.583433 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575841    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 20 09:38:52.583433 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575843    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 20 09:38:52.583433 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575846    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 20 09:38:52.583433 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575848    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 20 09:38:52.583433 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575851    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 20 09:38:52.583433 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575853    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 20 09:38:52.583433 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575855    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 20 09:38:52.583433 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575858    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 20 09:38:52.583433 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:52.575860    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 20 09:38:52.583930 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.575865    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 20 09:38:52.583930 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.576002    2566 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 20 09:38:52.583930 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.578114    2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 20 09:38:52.583930 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.579438    2566 server.go:1019] "Starting client certificate rotation"
Jul 20 09:38:52.583930 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.579556    2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 20 09:38:52.583930 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.579594    2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 20 09:38:52.583930 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.581881    2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 20 09:38:52.583930 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.583741    2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 20 09:38:52.596369 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.596336    2566 log.go:25] "Validated CRI v1 runtime API"
Jul 20 09:38:52.598730 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.598704    2566 log.go:25] "Validated CRI v1 image API"
Jul 20 09:38:52.599660 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.599641    2566 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 20 09:38:52.602140 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.602117    2566 fs.go:135] Filesystem UUIDs: map[1d4c0d4b-71d4-4a23-a5dc-623c93f43fce:/dev/nvme0n1p3 29fcdfbe-cc74-44a5-84ce-81e829326491:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Jul 20 09:38:52.602221 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.602140    2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 20 09:38:52.607971 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.607839    2566 manager.go:216] Machine: {Timestamp:2026-07-20 09:38:52.605811691 +0000 UTC m=+0.302777837 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3095756 MemoryCapacity:32812167168 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29065e3c7fa5ebdbc976cf8f09dfbc SystemUUID:ec29065e-3c7f-a5eb-dbc9-76cf8f09dfbc BootID:131b515c-8bb6-4e21-91e1-dbb81d89f28c Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:5a:82:ea:e7:c1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:5a:82:ea:e7:c1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6a:e6:f9:7f:cb:0a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812167168 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 20 09:38:52.607971 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.607962    2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 20 09:38:52.608104 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.608067    2566 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 20 09:38:52.610644 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.610605    2566 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 20 09:38:52.610791 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.610644    2566 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-134.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 20 09:38:52.611112 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.611078    2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 20 09:38:52.611301 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.611289    2566 topology_manager.go:138] "Creating topology manager with none policy"
Jul 20 09:38:52.611340 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.611306    2566 container_manager_linux.go:309] "Creating device plugin manager"
Jul 20 09:38:52.611340 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.611314    2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 20 09:38:52.612056 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.612044    2566 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 20 09:38:52.613257 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.613238    2566 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 20 09:38:52.615947 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.615928    2566 state_mem.go:36] "Initialized new in-memory state store"
Jul 20 09:38:52.616069 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.616058    2566 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 20 09:38:52.616112 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.616086    2566 kubelet.go:486] "Attempting to sync node with API server"
Jul 20 09:38:52.616112 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.616096    2566 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 20 09:38:52.616112 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.616111    2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 20 09:38:52.616190 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.616120    2566 kubelet.go:398] "Adding apiserver pod source"
Jul 20 09:38:52.616190 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.616135    2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 20 09:38:52.616486 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.616474    2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 20 09:38:52.616552 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.616492    2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 20 09:38:52.620725 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.620704    2566 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 20 09:38:52.622231 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.622216    2566 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 20 09:38:52.622277 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.622249    2566 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 20 09:38:52.622844 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.622828    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 20 09:38:52.622875 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.622853    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 20 09:38:52.622875 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.622859    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 20 09:38:52.622875 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.622864    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 20 09:38:52.622875 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.622872    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 20 09:38:52.622979 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.622878    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 20 09:38:52.622979 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.622960    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 20 09:38:52.622979 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.622966    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 20 09:38:52.622979 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.622974    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 20 09:38:52.622979 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.622980    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 20 09:38:52.623099 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.622997    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 20 09:38:52.623099 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.623005    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 20 09:38:52.623824 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.623811    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 20 09:38:52.623824 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.623825    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 20 09:38:52.627342 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.627323    2566 server.go:1268] "Started kubelet"
Jul 20 09:38:52.627509 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.627423    2566 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 20 09:38:52.627597 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.627546    2566 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 20 09:38:52.628368 ip-10-0-140-134 systemd[1]: Started Kubernetes Kubelet.
Jul 20 09:38:52.629224 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.628336    2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 20 09:38:52.629224 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.628878    2566 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-134.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 20 09:38:52.629224 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:52.628977    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-134.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 20 09:38:52.629224 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:52.628983    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 20 09:38:52.632613 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.632577    2566 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 20 09:38:52.633863 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.633842    2566 server.go:310] "Adding debug handlers to kubelet server"
Jul 20 09:38:52.635286 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:52.635262    2566 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 20 09:38:52.637326 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.637306    2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jkqlv"
Jul 20 09:38:52.638526 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.638509    2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 20 09:38:52.638612 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.638538    2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 20 09:38:52.638674 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:52.638631    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-134.ec2.internal\" not found"
Jul 20 09:38:52.639095 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.639062    2566 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 20 09:38:52.639095 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.639076    2566 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 20 09:38:52.639390 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.639278    2566 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 20 09:38:52.639390 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.639342    2566 reconstruct.go:97] "Volume reconstruction finished"
Jul 20 09:38:52.639390 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.639350    2566 reconciler.go:29] "Reconciler: start to sync state"
Jul 20 09:38:52.639697 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.639675    2566 factory.go:55] Registering systemd factory
Jul 20 09:38:52.639762 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.639723    2566 factory.go:223] Registration of the systemd container factory successfully
Jul 20 09:38:52.639952 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.639932    2566 factory.go:153] Registering CRI-O factory
Jul 20 09:38:52.640010 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.639979    2566 factory.go:223] Registration of the crio container factory successfully
Jul 20 09:38:52.640061 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.640031    2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 20 09:38:52.640061 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.640059    2566 factory.go:103] Registering Raw factory
Jul 20 09:38:52.640163 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.640071    2566 manager.go:1196] Started watching for new ooms in manager
Jul 20 09:38:52.640680 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.640665    2566 manager.go:349] Starting recovery of all containers
Jul 20 09:38:52.642394 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:52.642354    2566 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-134.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 20 09:38:52.642541 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:52.642519    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 20 09:38:52.643237 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.643214    2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jkqlv"
Jul 20 09:38:52.643507 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:52.638916    2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-134.ec2.internal.18c3f5fa459b59d0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-134.ec2.internal,UID:ip-10-0-140-134.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-134.ec2.internal,},FirstTimestamp:2026-07-20 09:38:52.62728648 +0000 UTC m=+0.324252669,LastTimestamp:2026-07-20 09:38:52.62728648 +0000 UTC m=+0.324252669,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-134.ec2.internal,}"
Jul 20 09:38:52.648729 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.648701    2566 manager.go:354] Recovery completed
Jul 20 09:38:52.654029 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.654008    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 09:38:52.656243 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.656221    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 09:38:52.656353 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.656264    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 09:38:52.656353 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.656279    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientPID"
Jul 20 09:38:52.656933 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.656912    2566 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 20 09:38:52.657013 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.656934    2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 20 09:38:52.657013 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.656950    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 09:38:52.657013 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.656956    2566 state_mem.go:36] "Initialized new in-memory state store"
Jul 20 09:38:52.658099 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.658081    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 09:38:52.658172 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.658121    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 09:38:52.658172 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.658137    2566 policy_none.go:49] "None policy: Start"
Jul 20 09:38:52.658172 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.658162    2566 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 20 09:38:52.658303 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.658174    2566 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 20 09:38:52.658303 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.658138    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientPID"
Jul 20 09:38:52.659203 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.659190    2566 policy_none.go:47] "Start"
Jul 20 09:38:52.704500 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.704293    2566 manager.go:337] "Starting Device Plugin manager"
Jul 20 09:38:52.708525 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:52.704520    2566 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 20 09:38:52.708525 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.704532    2566 server.go:87] "Starting device plugin registration server"
Jul 20 09:38:52.708525 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.704805    2566 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 20 09:38:52.708525 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.704822    2566 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 20 09:38:52.708525 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.704940    2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 20 09:38:52.708525 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.705034    2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 20 09:38:52.708525 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.705043    2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 20 09:38:52.708525 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:52.705852    2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 20 09:38:52.708525 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:52.705894    2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-134.ec2.internal\" not found"
Jul 20 09:38:52.767069 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.767025    2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 20 09:38:52.768455 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.768424    2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 20 09:38:52.768597 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.768497    2566 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 20 09:38:52.768597 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.768528    2566 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 20 09:38:52.768693 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:52.768579    2566 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 20 09:38:52.770888 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.770863    2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 09:38:52.804972 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.804939    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 09:38:52.805986 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.805923    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 09:38:52.805986 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.805955    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 09:38:52.805986 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.805966    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientPID"
Jul 20 09:38:52.805986 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.805989    2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-134.ec2.internal"
Jul 20 09:38:52.816010 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.815988    2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-134.ec2.internal"
Jul 20 09:38:52.816092 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:52.816021    2566 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-134.ec2.internal\": node \"ip-10-0-140-134.ec2.internal\" not found"
Jul 20 09:38:52.838767 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:52.838732    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-134.ec2.internal\" not found"
Jul 20 09:38:52.869567 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.869518    2566 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-134.ec2.internal"]
Jul 20 09:38:52.869725 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.869635    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 09:38:52.870770 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.870752    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 09:38:52.870864 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.870784    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 09:38:52.870864 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.870794    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientPID"
Jul 20 09:38:52.872189 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.872175    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 09:38:52.872318 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.872303    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal"
Jul 20 09:38:52.872376 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.872340    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 09:38:52.873154 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.873137    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 09:38:52.873231 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.873161    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 09:38:52.873231 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.873170    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientPID"
Jul 20 09:38:52.873231 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.873183    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 09:38:52.873231 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.873201    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 09:38:52.873231 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.873210    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientPID"
Jul 20 09:38:52.874354 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.874340    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-134.ec2.internal"
Jul 20 09:38:52.874413 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.874365    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 20 09:38:52.875157 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.875140    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientMemory"
Jul 20 09:38:52.875231 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.875175    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasNoDiskPressure"
Jul 20 09:38:52.875231 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.875189    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeHasSufficientPID"
Jul 20 09:38:52.895191 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:52.895161    2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-134.ec2.internal\" not found" node="ip-10-0-140-134.ec2.internal"
Jul 20 09:38:52.898832 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:52.898812    2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-134.ec2.internal\" not found" node="ip-10-0-140-134.ec2.internal"
Jul 20 09:38:52.939749 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:52.939711    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-134.ec2.internal\" not found"
Jul 20 09:38:52.940896 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.940876    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/007fe94f9ffbab53bf60fc279d909e65-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal\" (UID: \"007fe94f9ffbab53bf60fc279d909e65\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal"
Jul 20 09:38:52.940951 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.940904    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/007fe94f9ffbab53bf60fc279d909e65-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal\" (UID: \"007fe94f9ffbab53bf60fc279d909e65\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal"
Jul 20 09:38:52.940951 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:52.940927    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e04307ae6e708dcc808dbfefa047d3b4-config\") pod \"kube-apiserver-proxy-ip-10-0-140-134.ec2.internal\" (UID: \"e04307ae6e708dcc808dbfefa047d3b4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-134.ec2.internal"
Jul 20 09:38:53.040732 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:53.040694    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-134.ec2.internal\" not found"
Jul 20 09:38:53.041847 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:53.041826    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e04307ae6e708dcc808dbfefa047d3b4-config\") pod \"kube-apiserver-proxy-ip-10-0-140-134.ec2.internal\" (UID: \"e04307ae6e708dcc808dbfefa047d3b4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-134.ec2.internal"
Jul 20 09:38:53.041894 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:53.041857    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/007fe94f9ffbab53bf60fc279d909e65-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal\" (UID: \"007fe94f9ffbab53bf60fc279d909e65\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal"
Jul 20 09:38:53.041942 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:53.041931    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/007fe94f9ffbab53bf60fc279d909e65-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal\" (UID: \"007fe94f9ffbab53bf60fc279d909e65\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal"
Jul 20 09:38:53.041989 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:53.041942    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e04307ae6e708dcc808dbfefa047d3b4-config\") pod \"kube-apiserver-proxy-ip-10-0-140-134.ec2.internal\" (UID: \"e04307ae6e708dcc808dbfefa047d3b4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-134.ec2.internal"
Jul 20 09:38:53.042024 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:53.041983    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/007fe94f9ffbab53bf60fc279d909e65-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal\" (UID: \"007fe94f9ffbab53bf60fc279d909e65\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal"
Jul 20 09:38:53.042063 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:53.042028    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/007fe94f9ffbab53bf60fc279d909e65-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal\" (UID: \"007fe94f9ffbab53bf60fc279d909e65\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal"
Jul 20 09:38:53.141518 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:53.141403    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-134.ec2.internal\" not found"
Jul 20 09:38:53.196628 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:53.196592    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal"
Jul 20 09:38:53.202639 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:53.202620    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-134.ec2.internal"
Jul 20 09:38:53.242516 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:53.242483    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-134.ec2.internal\" not found"
Jul 20 09:38:53.342992 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:53.342961    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-134.ec2.internal\" not found"
Jul 20 09:38:53.443383 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:53.443308    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-134.ec2.internal\" not found"
Jul 20 09:38:53.544013 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:53.543965    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-134.ec2.internal\" not found"
Jul 20 09:38:53.579151 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:53.579118    2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 20 09:38:53.579622 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:53.579269    2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 20 09:38:53.644305 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:53.644251    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-134.ec2.internal\" not found"
Jul 20 09:38:53.644601 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:53.644413    2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 09:38:53.647379 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:53.647349    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-19 09:33:53 +0000 UTC" deadline="2028-01-19 00:05:09.336299334 +0000 UTC"
Jul 20 09:38:53.647379 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:53.647376    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13142h26m15.688927601s"
Jul 20 09:38:53.704315 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:53.704242    2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 09:38:53.740824 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:53.740780    2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal"
Jul 20 09:38:53.751738 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:53.751712    2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 20 09:38:53.753507 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:53.753477    2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-134.ec2.internal"
Jul 20 09:38:53.753915 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:53.753890    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode04307ae6e708dcc808dbfefa047d3b4.slice/crio-893cf3937b4c59fdc377deb3e8782fb579621e0cba203d40c60b6b7282b6d457 WatchSource:0}: Error finding container 893cf3937b4c59fdc377deb3e8782fb579621e0cba203d40c60b6b7282b6d457: Status 404 returned error can't find the container with id 893cf3937b4c59fdc377deb3e8782fb579621e0cba203d40c60b6b7282b6d457
Jul 20 09:38:53.754412 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:53.754393    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod007fe94f9ffbab53bf60fc279d909e65.slice/crio-60c364711d9c058f60e2e3d5f87d89dc83930f0996f4bc7227dee34b392aafa9 WatchSource:0}: Error finding container 60c364711d9c058f60e2e3d5f87d89dc83930f0996f4bc7227dee34b392aafa9: Status 404 returned error can't find the container with id 60c364711d9c058f60e2e3d5f87d89dc83930f0996f4bc7227dee34b392aafa9
Jul 20 09:38:53.758141 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:53.758126    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 09:38:53.763709 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:53.763690    2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 20 09:38:53.772515 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:53.772447    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal" event={"ID":"007fe94f9ffbab53bf60fc279d909e65","Type":"ContainerStarted","Data":"60c364711d9c058f60e2e3d5f87d89dc83930f0996f4bc7227dee34b392aafa9"}
Jul 20 09:38:53.773486 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:53.773443    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-134.ec2.internal" event={"ID":"e04307ae6e708dcc808dbfefa047d3b4","Type":"ContainerStarted","Data":"893cf3937b4c59fdc377deb3e8782fb579621e0cba203d40c60b6b7282b6d457"}
Jul 20 09:38:55.262216 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.262179    2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 09:38:55.764081 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.764048    2566 apiserver.go:52] "Watching apiserver"
Jul 20 09:38:55.776558 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.776519    2566 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 20 09:38:55.777115 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.777088    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-vjr7g","openshift-cluster-node-tuning-operator/tuned-bk2gb","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal","openshift-multus/multus-rdqzb","openshift-multus/network-metrics-daemon-xdlfl","openshift-ovn-kubernetes/ovnkube-node-64r4x","kube-system/global-pull-secret-syncer-4lw87","kube-system/kube-apiserver-proxy-ip-10-0-140-134.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kdvx","openshift-dns/node-resolver-mqz47","openshift-image-registry/node-ca-6p7xh","openshift-multus/multus-additional-cni-plugins-gxttc","openshift-network-diagnostics/network-check-target-9kcgh","openshift-network-operator/iptables-alerter-wkmzk"]
Jul 20 09:38:55.778744 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.778712    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mqz47"
Jul 20 09:38:55.780454 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.780429    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9kcgh"
Jul 20 09:38:55.780562 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:55.780531    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9kcgh" podUID="f44bf7c3-e682-406b-92c7-b8ecfd6505f6"
Jul 20 09:38:55.781650 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.781625    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gxttc"
Jul 20 09:38:55.782413 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.782390    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 20 09:38:55.782508 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.782450    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 20 09:38:55.782666 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.782638    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-79h7l\""
Jul 20 09:38:55.782941 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.782924    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6p7xh"
Jul 20 09:38:55.784293 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.784271    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.784687 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.784667    2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 20 09:38:55.785893 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.785871    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.787251 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.787228    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lw87"
Jul 20 09:38:55.787346 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:55.787305    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lw87" podUID="d4a0ef09-53a6-47cb-a39f-15ae72031d8f"
Jul 20 09:38:55.788953 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.788922    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kdvx"
Jul 20 09:38:55.789064 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.789017    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xm7ld\""
Jul 20 09:38:55.789363 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.789344    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 20 09:38:55.789363 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.789360    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 20 09:38:55.789662 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.789648    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 20 09:38:55.789855 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.789839    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 20 09:38:55.789939 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.789923    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-x2ghp\""
Jul 20 09:38:55.791060 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.790139    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xdlfl"
Jul 20 09:38:55.791060 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:55.790203    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xdlfl" podUID="d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de"
Jul 20 09:38:55.791060 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.790650    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 20 09:38:55.791544 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.791093    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 20 09:38:55.791544 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.791307    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 20 09:38:55.791544 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.791511    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 20 09:38:55.791812 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.791782    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 20 09:38:55.792007 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.791955    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 20 09:38:55.792174 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.792160    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 20 09:38:55.792887 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.792869    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xtf86\""
Jul 20 09:38:55.793106 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.793092    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 20 09:38:55.793321 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.793306    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-8tnzg\""
Jul 20 09:38:55.793377 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.793338    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 20 09:38:55.793684 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.793660    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vjr7g"
Jul 20 09:38:55.793921 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.793905    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 20 09:38:55.794197 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.794174    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 20 09:38:55.794358 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.794335    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 20 09:38:55.794420 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.794180    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 20 09:38:55.794486 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.794467    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ttkbh\""
Jul 20 09:38:55.794556 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.794539    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 20 09:38:55.794785 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.794765    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wkmzk"
Jul 20 09:38:55.794860 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.794800    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:55.795372 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.795021    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5d536f0c-af6a-4250-a216-30973ed33030-hosts-file\") pod \"node-resolver-mqz47\" (UID: \"5d536f0c-af6a-4250-a216-30973ed33030\") " pod="openshift-dns/node-resolver-mqz47"
Jul 20 09:38:55.795372 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.795061    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4727c87a-fe93-4101-adcf-bc4795bfe808-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gxttc\" (UID: \"4727c87a-fe93-4101-adcf-bc4795bfe808\") " pod="openshift-multus/multus-additional-cni-plugins-gxttc"
Jul 20 09:38:55.795372 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.795090    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-multus-cni-dir\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.795372 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.795109    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-host-run-netns\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.795372 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.795134    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-host-var-lib-cni-bin\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.795372 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.795165    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-host-run-multus-certs\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.795372 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.795191    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-etc-kubernetes\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.795372 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.795219    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-host-kubelet\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.795372 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.795245    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5ndp\" (UniqueName: \"kubernetes.io/projected/f44bf7c3-e682-406b-92c7-b8ecfd6505f6-kube-api-access-c5ndp\") pod \"network-check-target-9kcgh\" (UID: \"f44bf7c3-e682-406b-92c7-b8ecfd6505f6\") " pod="openshift-network-diagnostics/network-check-target-9kcgh"
Jul 20 09:38:55.795372 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.795346    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-multus-conf-dir\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.795880 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.795404    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a962283d-f543-4d9f-97d5-87641b30b4c2-multus-daemon-config\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.795880 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.795442    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-systemd-units\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.795880 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.795466    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-host-slash\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.795880 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.795488    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-host-run-netns\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.795880 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.795535    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-run-systemd\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.795880 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.795567    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-var-lib-openvswitch\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.795880 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.795600    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4727c87a-fe93-4101-adcf-bc4795bfe808-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gxttc\" (UID: \"4727c87a-fe93-4101-adcf-bc4795bfe808\") " pod="openshift-multus/multus-additional-cni-plugins-gxttc"
Jul 20 09:38:55.795880 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.795624    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-os-release\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.795880 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.795669    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-host-run-k8s-cni-cncf-io\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.795880 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.795723    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-host-cni-bin\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.795880 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.795763    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.795880 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.795818    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5784af38-8a10-4cee-8510-64758ed45746-ovnkube-config\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.795880 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.795840    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbsqd\" (UniqueName: \"kubernetes.io/projected/5784af38-8a10-4cee-8510-64758ed45746-kube-api-access-fbsqd\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.795880 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.795884    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r644g\" (UniqueName: \"kubernetes.io/projected/5fffc7f3-af46-4089-9814-a3932da685a8-kube-api-access-r644g\") pod \"node-ca-6p7xh\" (UID: \"5fffc7f3-af46-4089-9814-a3932da685a8\") " pod="openshift-image-registry/node-ca-6p7xh"
Jul 20 09:38:55.796573 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.795906    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a962283d-f543-4d9f-97d5-87641b30b4c2-cni-binary-copy\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.796573 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.795929    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-host-var-lib-kubelet\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.796573 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.795992    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-etc-openvswitch\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.796573 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.796016    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-node-log\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.796573 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.796043    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-host-cni-netd\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.796573 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.796168    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5784af38-8a10-4cee-8510-64758ed45746-env-overrides\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.796573 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.796204    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5784af38-8a10-4cee-8510-64758ed45746-ovnkube-script-lib\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.796573 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.796257    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-host-var-lib-cni-multus\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.796573 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.796279    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4727c87a-fe93-4101-adcf-bc4795bfe808-system-cni-dir\") pod \"multus-additional-cni-plugins-gxttc\" (UID: \"4727c87a-fe93-4101-adcf-bc4795bfe808\") " pod="openshift-multus/multus-additional-cni-plugins-gxttc"
Jul 20 09:38:55.796573 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.796347    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4727c87a-fe93-4101-adcf-bc4795bfe808-cnibin\") pod \"multus-additional-cni-plugins-gxttc\" (UID: \"4727c87a-fe93-4101-adcf-bc4795bfe808\") " pod="openshift-multus/multus-additional-cni-plugins-gxttc"
Jul 20 09:38:55.796573 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.796374    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5fffc7f3-af46-4089-9814-a3932da685a8-host\") pod \"node-ca-6p7xh\" (UID: \"5fffc7f3-af46-4089-9814-a3932da685a8\") " pod="openshift-image-registry/node-ca-6p7xh"
Jul 20 09:38:55.796573 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.796399    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-cnibin\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.796573 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.796427    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-multus-socket-dir-parent\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.796573 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.796458    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2vt7\" (UniqueName: \"kubernetes.io/projected/a962283d-f543-4d9f-97d5-87641b30b4c2-kube-api-access-j2vt7\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.796573 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.796521    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-run-openvswitch\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.797280 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.796582    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d536f0c-af6a-4250-a216-30973ed33030-tmp-dir\") pod \"node-resolver-mqz47\" (UID: \"5d536f0c-af6a-4250-a216-30973ed33030\") " pod="openshift-dns/node-resolver-mqz47"
Jul 20 09:38:55.797280 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.796630    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4727c87a-fe93-4101-adcf-bc4795bfe808-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gxttc\" (UID: \"4727c87a-fe93-4101-adcf-bc4795bfe808\") " pod="openshift-multus/multus-additional-cni-plugins-gxttc"
Jul 20 09:38:55.797280 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.796652    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-run-ovn\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.797280 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.796654    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 20 09:38:55.797280 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.796690    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-log-socket\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.797280 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.796719    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-host-run-ovn-kubernetes\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.797280 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.796746    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqtfc\" (UniqueName: \"kubernetes.io/projected/5d536f0c-af6a-4250-a216-30973ed33030-kube-api-access-pqtfc\") pod \"node-resolver-mqz47\" (UID: \"5d536f0c-af6a-4250-a216-30973ed33030\") " pod="openshift-dns/node-resolver-mqz47"
Jul 20 09:38:55.797280 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.796766    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4727c87a-fe93-4101-adcf-bc4795bfe808-cni-binary-copy\") pod \"multus-additional-cni-plugins-gxttc\" (UID: \"4727c87a-fe93-4101-adcf-bc4795bfe808\") " pod="openshift-multus/multus-additional-cni-plugins-gxttc"
Jul 20 09:38:55.797280 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.796782    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5fffc7f3-af46-4089-9814-a3932da685a8-serviceca\") pod \"node-ca-6p7xh\" (UID: \"5fffc7f3-af46-4089-9814-a3932da685a8\") " pod="openshift-image-registry/node-ca-6p7xh"
Jul 20 09:38:55.797280 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.796795    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-hostroot\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.797280 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.796816    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5784af38-8a10-4cee-8510-64758ed45746-ovn-node-metrics-cert\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.797280 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.796830    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4727c87a-fe93-4101-adcf-bc4795bfe808-os-release\") pod \"multus-additional-cni-plugins-gxttc\" (UID: \"4727c87a-fe93-4101-adcf-bc4795bfe808\") " pod="openshift-multus/multus-additional-cni-plugins-gxttc"
Jul 20 09:38:55.797280 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.796845    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlwhm\" (UniqueName: \"kubernetes.io/projected/4727c87a-fe93-4101-adcf-bc4795bfe808-kube-api-access-wlwhm\") pod \"multus-additional-cni-plugins-gxttc\" (UID: \"4727c87a-fe93-4101-adcf-bc4795bfe808\") " pod="openshift-multus/multus-additional-cni-plugins-gxttc"
Jul 20 09:38:55.797280 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.796871    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-system-cni-dir\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.797280 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.797003    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-fwpzl\""
Jul 20 09:38:55.797280 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.797258    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 20 09:38:55.797932 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.797912    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 20 09:38:55.798044 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.798029    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 20 09:38:55.798586 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.798475    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 20 09:38:55.798586 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.798526    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jz444\""
Jul 20 09:38:55.798812 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.798795    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 20 09:38:55.798848 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.798835    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 20 09:38:55.799349 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.799333    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-kbvld\""
Jul 20 09:38:55.804931 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.804898    2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 20 09:38:55.825221 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.825195    2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-nbhw9"
Jul 20 09:38:55.835402 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.835374    2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-nbhw9"
Jul 20 09:38:55.886432 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.886401    2566 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 20 09:38:55.897732 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.897661    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/75a6adbe-72f8-4292-a8f6-605d496bffea-agent-certs\") pod \"konnectivity-agent-vjr7g\" (UID: \"75a6adbe-72f8-4292-a8f6-605d496bffea\") " pod="kube-system/konnectivity-agent-vjr7g"
Jul 20 09:38:55.897732 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.897713    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4727c87a-fe93-4101-adcf-bc4795bfe808-cnibin\") pod \"multus-additional-cni-plugins-gxttc\" (UID: \"4727c87a-fe93-4101-adcf-bc4795bfe808\") " pod="openshift-multus/multus-additional-cni-plugins-gxttc"
Jul 20 09:38:55.897967 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.897743    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2vt7\" (UniqueName: \"kubernetes.io/projected/a962283d-f543-4d9f-97d5-87641b30b4c2-kube-api-access-j2vt7\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.897967 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.897800    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-run-openvswitch\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.897967 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.897834    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/51f081b8-e74b-4ff6-b9c6-06a13e680d11-etc-selinux\") pod \"aws-ebs-csi-driver-node-6kdvx\" (UID: \"51f081b8-e74b-4ff6-b9c6-06a13e680d11\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kdvx"
Jul 20 09:38:55.897967 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.897859    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/78c743ae-071a-4620-ac25-5fb11708d229-etc-modprobe-d\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:55.897967 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.897884    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/75a6adbe-72f8-4292-a8f6-605d496bffea-konnectivity-ca\") pod \"konnectivity-agent-vjr7g\" (UID: \"75a6adbe-72f8-4292-a8f6-605d496bffea\") " pod="kube-system/konnectivity-agent-vjr7g"
Jul 20 09:38:55.897967 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.897917    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d536f0c-af6a-4250-a216-30973ed33030-tmp-dir\") pod \"node-resolver-mqz47\" (UID: \"5d536f0c-af6a-4250-a216-30973ed33030\") " pod="openshift-dns/node-resolver-mqz47"
Jul 20 09:38:55.898276 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.897964    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4727c87a-fe93-4101-adcf-bc4795bfe808-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gxttc\" (UID: \"4727c87a-fe93-4101-adcf-bc4795bfe808\") " pod="openshift-multus/multus-additional-cni-plugins-gxttc"
Jul 20 09:38:55.898276 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.898025    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-run-ovn\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.898276 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.898052    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-log-socket\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.898276 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.898079    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-host-run-ovn-kubernetes\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.898276 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.898110    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5784af38-8a10-4cee-8510-64758ed45746-ovn-node-metrics-cert\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.898276 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.898137    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dzmw\" (UniqueName: \"kubernetes.io/projected/51f081b8-e74b-4ff6-b9c6-06a13e680d11-kube-api-access-2dzmw\") pod \"aws-ebs-csi-driver-node-6kdvx\" (UID: \"51f081b8-e74b-4ff6-b9c6-06a13e680d11\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kdvx"
Jul 20 09:38:55.898276 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.898172    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/78c743ae-071a-4620-ac25-5fb11708d229-etc-sysconfig\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:55.898276 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.898201    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grx54\" (UniqueName: \"kubernetes.io/projected/78c743ae-071a-4620-ac25-5fb11708d229-kube-api-access-grx54\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:55.898276 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.898227    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bfe01381-3135-4dfe-9793-4f158e62303e-iptables-alerter-script\") pod \"iptables-alerter-wkmzk\" (UID: \"bfe01381-3135-4dfe-9793-4f158e62303e\") " pod="openshift-network-operator/iptables-alerter-wkmzk"
Jul 20 09:38:55.898276 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.898256    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bfe01381-3135-4dfe-9793-4f158e62303e-host-slash\") pod \"iptables-alerter-wkmzk\" (UID: \"bfe01381-3135-4dfe-9793-4f158e62303e\") " pod="openshift-network-operator/iptables-alerter-wkmzk"
Jul 20 09:38:55.898634 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.898283    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7x4tp\" (UniqueName: \"kubernetes.io/projected/bfe01381-3135-4dfe-9793-4f158e62303e-kube-api-access-7x4tp\") pod \"iptables-alerter-wkmzk\" (UID: \"bfe01381-3135-4dfe-9793-4f158e62303e\") " pod="openshift-network-operator/iptables-alerter-wkmzk"
Jul 20 09:38:55.898634 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.898306    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4727c87a-fe93-4101-adcf-bc4795bfe808-os-release\") pod \"multus-additional-cni-plugins-gxttc\" (UID: \"4727c87a-fe93-4101-adcf-bc4795bfe808\") " pod="openshift-multus/multus-additional-cni-plugins-gxttc"
Jul 20 09:38:55.898634 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.898333    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/78c743ae-071a-4620-ac25-5fb11708d229-var-lib-kubelet\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:55.898634 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.898362    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-host-run-netns\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.898634 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.898388    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-host-kubelet\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.898634 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.898404    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-run-systemd\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.898634 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.898418    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-var-lib-openvswitch\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.898634 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.898434    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-etc-openvswitch\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.898634 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.898450    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/51f081b8-e74b-4ff6-b9c6-06a13e680d11-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6kdvx\" (UID: \"51f081b8-e74b-4ff6-b9c6-06a13e680d11\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kdvx"
Jul 20 09:38:55.898634 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.898465    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/78c743ae-071a-4620-ac25-5fb11708d229-etc-sysctl-conf\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:55.898634 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.898504    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5ndp\" (UniqueName: \"kubernetes.io/projected/f44bf7c3-e682-406b-92c7-b8ecfd6505f6-kube-api-access-c5ndp\") pod \"network-check-target-9kcgh\" (UID: \"f44bf7c3-e682-406b-92c7-b8ecfd6505f6\") " pod="openshift-network-diagnostics/network-check-target-9kcgh"
Jul 20 09:38:55.898634 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.898523    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-multus-conf-dir\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.898634 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.898556    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/51f081b8-e74b-4ff6-b9c6-06a13e680d11-registration-dir\") pod \"aws-ebs-csi-driver-node-6kdvx\" (UID: \"51f081b8-e74b-4ff6-b9c6-06a13e680d11\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kdvx"
Jul 20 09:38:55.898634 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.898582    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-host-run-k8s-cni-cncf-io\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.898634 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.898605    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-host-cni-bin\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.898634 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.898632    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5784af38-8a10-4cee-8510-64758ed45746-ovnkube-config\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.899093 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.898669    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbsqd\" (UniqueName: \"kubernetes.io/projected/5784af38-8a10-4cee-8510-64758ed45746-kube-api-access-fbsqd\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.899093 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.898695    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de-metrics-certs\") pod \"network-metrics-daemon-xdlfl\" (UID: \"d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de\") " pod="openshift-multus/network-metrics-daemon-xdlfl"
Jul 20 09:38:55.899093 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.898714    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfccm\" (UniqueName: \"kubernetes.io/projected/d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de-kube-api-access-vfccm\") pod \"network-metrics-daemon-xdlfl\" (UID: \"d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de\") " pod="openshift-multus/network-metrics-daemon-xdlfl"
Jul 20 09:38:55.899093 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.898736    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/78c743ae-071a-4620-ac25-5fb11708d229-host\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:55.899093 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.898760    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a962283d-f543-4d9f-97d5-87641b30b4c2-cni-binary-copy\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.899093 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.898781    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-host-var-lib-kubelet\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.899093 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.898806    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5784af38-8a10-4cee-8510-64758ed45746-env-overrides\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.899093 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.898830    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/51f081b8-e74b-4ff6-b9c6-06a13e680d11-sys-fs\") pod \"aws-ebs-csi-driver-node-6kdvx\" (UID: \"51f081b8-e74b-4ff6-b9c6-06a13e680d11\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kdvx"
Jul 20 09:38:55.899093 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.898856    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/78c743ae-071a-4620-ac25-5fb11708d229-etc-kubernetes\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:55.899093 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.898875    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-host-var-lib-cni-multus\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.899093 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.898900    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4727c87a-fe93-4101-adcf-bc4795bfe808-system-cni-dir\") pod \"multus-additional-cni-plugins-gxttc\" (UID: \"4727c87a-fe93-4101-adcf-bc4795bfe808\") " pod="openshift-multus/multus-additional-cni-plugins-gxttc"
Jul 20 09:38:55.899093 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.898919    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5fffc7f3-af46-4089-9814-a3932da685a8-host\") pod \"node-ca-6p7xh\" (UID: \"5fffc7f3-af46-4089-9814-a3932da685a8\") " pod="openshift-image-registry/node-ca-6p7xh"
Jul 20 09:38:55.899093 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.898941    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-cnibin\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.899093 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.898984    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-multus-socket-dir-parent\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.899093 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.899002    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d4a0ef09-53a6-47cb-a39f-15ae72031d8f-original-pull-secret\") pod \"global-pull-secret-syncer-4lw87\" (UID: \"d4a0ef09-53a6-47cb-a39f-15ae72031d8f\") " pod="kube-system/global-pull-secret-syncer-4lw87"
Jul 20 09:38:55.899093 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.899018    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/78c743ae-071a-4620-ac25-5fb11708d229-lib-modules\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:55.899093 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.899036    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqtfc\" (UniqueName: \"kubernetes.io/projected/5d536f0c-af6a-4250-a216-30973ed33030-kube-api-access-pqtfc\") pod \"node-resolver-mqz47\" (UID: \"5d536f0c-af6a-4250-a216-30973ed33030\") " pod="openshift-dns/node-resolver-mqz47"
Jul 20 09:38:55.899536 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.899055    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4727c87a-fe93-4101-adcf-bc4795bfe808-cni-binary-copy\") pod \"multus-additional-cni-plugins-gxttc\" (UID: \"4727c87a-fe93-4101-adcf-bc4795bfe808\") " pod="openshift-multus/multus-additional-cni-plugins-gxttc"
Jul 20 09:38:55.899536 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.899077    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5fffc7f3-af46-4089-9814-a3932da685a8-serviceca\") pod \"node-ca-6p7xh\" (UID: \"5fffc7f3-af46-4089-9814-a3932da685a8\") " pod="openshift-image-registry/node-ca-6p7xh"
Jul 20 09:38:55.899536 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.899094    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-hostroot\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.899536 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.899111    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlwhm\" (UniqueName: \"kubernetes.io/projected/4727c87a-fe93-4101-adcf-bc4795bfe808-kube-api-access-wlwhm\") pod \"multus-additional-cni-plugins-gxttc\" (UID: \"4727c87a-fe93-4101-adcf-bc4795bfe808\") " pod="openshift-multus/multus-additional-cni-plugins-gxttc"
Jul 20 09:38:55.899536 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.899132    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-system-cni-dir\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.899536 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.899156    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/51f081b8-e74b-4ff6-b9c6-06a13e680d11-device-dir\") pod \"aws-ebs-csi-driver-node-6kdvx\" (UID: \"51f081b8-e74b-4ff6-b9c6-06a13e680d11\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kdvx"
Jul 20 09:38:55.899536 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.899177    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/78c743ae-071a-4620-ac25-5fb11708d229-etc-sysctl-d\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:55.899536 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.899200    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/78c743ae-071a-4620-ac25-5fb11708d229-etc-systemd\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:55.899536 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.899226    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5d536f0c-af6a-4250-a216-30973ed33030-hosts-file\") pod \"node-resolver-mqz47\" (UID: \"5d536f0c-af6a-4250-a216-30973ed33030\") " pod="openshift-dns/node-resolver-mqz47"
Jul 20 09:38:55.899536 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.899250    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4727c87a-fe93-4101-adcf-bc4795bfe808-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gxttc\" (UID: \"4727c87a-fe93-4101-adcf-bc4795bfe808\") " pod="openshift-multus/multus-additional-cni-plugins-gxttc"
Jul 20 09:38:55.899536 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.899275    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-multus-cni-dir\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.899536 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.899299    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-host-var-lib-cni-bin\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.899536 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.899326    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-host-run-multus-certs\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.899536 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.899350    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-etc-kubernetes\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.899536 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.899366    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-host-slash\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.899536 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.899381    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/51f081b8-e74b-4ff6-b9c6-06a13e680d11-socket-dir\") pod \"aws-ebs-csi-driver-node-6kdvx\" (UID: \"51f081b8-e74b-4ff6-b9c6-06a13e680d11\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kdvx"
Jul 20 09:38:55.899536 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.899398    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a962283d-f543-4d9f-97d5-87641b30b4c2-multus-daemon-config\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.900007 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.899463    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-systemd-units\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.900007 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.899484    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-host-run-netns\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.900007 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.899503    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.900007 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.899537    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/78c743ae-071a-4620-ac25-5fb11708d229-run\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:55.900007 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.899553    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/78c743ae-071a-4620-ac25-5fb11708d229-etc-tuned\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:55.900007 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.899582    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4727c87a-fe93-4101-adcf-bc4795bfe808-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gxttc\" (UID: \"4727c87a-fe93-4101-adcf-bc4795bfe808\") " pod="openshift-multus/multus-additional-cni-plugins-gxttc"
Jul 20 09:38:55.900007 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.899615    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-os-release\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.900007 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.899642    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d4a0ef09-53a6-47cb-a39f-15ae72031d8f-kubelet-config\") pod \"global-pull-secret-syncer-4lw87\" (UID: \"d4a0ef09-53a6-47cb-a39f-15ae72031d8f\") " pod="kube-system/global-pull-secret-syncer-4lw87"
Jul 20 09:38:55.900007 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.899663    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d4a0ef09-53a6-47cb-a39f-15ae72031d8f-dbus\") pod \"global-pull-secret-syncer-4lw87\" (UID: \"d4a0ef09-53a6-47cb-a39f-15ae72031d8f\") " pod="kube-system/global-pull-secret-syncer-4lw87"
Jul 20 09:38:55.900007 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.899687    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/78c743ae-071a-4620-ac25-5fb11708d229-sys\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:55.900007 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.899706    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r644g\" (UniqueName: \"kubernetes.io/projected/5fffc7f3-af46-4089-9814-a3932da685a8-kube-api-access-r644g\") pod \"node-ca-6p7xh\" (UID: \"5fffc7f3-af46-4089-9814-a3932da685a8\") " pod="openshift-image-registry/node-ca-6p7xh"
Jul 20 09:38:55.900007 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.899733    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-node-log\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.900007 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.899761    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-host-cni-netd\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.900007 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.899785    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5784af38-8a10-4cee-8510-64758ed45746-ovnkube-script-lib\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.900007 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.899803    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/78c743ae-071a-4620-ac25-5fb11708d229-tmp\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:55.900007 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.899903    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4727c87a-fe93-4101-adcf-bc4795bfe808-cnibin\") pod \"multus-additional-cni-plugins-gxttc\" (UID: \"4727c87a-fe93-4101-adcf-bc4795bfe808\") " pod="openshift-multus/multus-additional-cni-plugins-gxttc"
Jul 20 09:38:55.900525 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.900089    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-run-systemd\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.900525 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.900125    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-host-cni-bin\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.900525 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.900094    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-var-lib-openvswitch\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.900525 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.900292    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-host-var-lib-cni-multus\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.900525 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.900344    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4727c87a-fe93-4101-adcf-bc4795bfe808-system-cni-dir\") pod \"multus-additional-cni-plugins-gxttc\" (UID: \"4727c87a-fe93-4101-adcf-bc4795bfe808\") " pod="openshift-multus/multus-additional-cni-plugins-gxttc"
Jul 20 09:38:55.900525 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.900393    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-multus-conf-dir\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.900525 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.900449    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-cnibin\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.900525 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.900482    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-multus-socket-dir-parent\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.900525 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.900506    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-host-run-k8s-cni-cncf-io\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.900525 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.900508    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5d536f0c-af6a-4250-a216-30973ed33030-tmp-dir\") pod \"node-resolver-mqz47\" (UID: \"5d536f0c-af6a-4250-a216-30973ed33030\") " pod="openshift-dns/node-resolver-mqz47"
Jul 20 09:38:55.900862 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.900765    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5784af38-8a10-4cee-8510-64758ed45746-ovnkube-config\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.900862 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.900829    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-host-kubelet\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.900862 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.900094    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-run-openvswitch\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.900987 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.900861    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-etc-openvswitch\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.901181 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.901159    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4727c87a-fe93-4101-adcf-bc4795bfe808-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gxttc\" (UID: \"4727c87a-fe93-4101-adcf-bc4795bfe808\") " pod="openshift-multus/multus-additional-cni-plugins-gxttc"
Jul 20 09:38:55.901215 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.901193    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5fffc7f3-af46-4089-9814-a3932da685a8-serviceca\") pod \"node-ca-6p7xh\" (UID: \"5fffc7f3-af46-4089-9814-a3932da685a8\") " pod="openshift-image-registry/node-ca-6p7xh"
Jul 20 09:38:55.901215 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.901200    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4727c87a-fe93-4101-adcf-bc4795bfe808-cni-binary-copy\") pod \"multus-additional-cni-plugins-gxttc\" (UID: \"4727c87a-fe93-4101-adcf-bc4795bfe808\") " pod="openshift-multus/multus-additional-cni-plugins-gxttc"
Jul 20 09:38:55.901277 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.901233    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-hostroot\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.901277 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.901235    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-log-socket\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.901332 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.901297    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-system-cni-dir\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.901362 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.901348    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-host-run-ovn-kubernetes\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.901362 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.901358    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5d536f0c-af6a-4250-a216-30973ed33030-hosts-file\") pod \"node-resolver-mqz47\" (UID: \"5d536f0c-af6a-4250-a216-30973ed33030\") " pod="openshift-dns/node-resolver-mqz47"
Jul 20 09:38:55.901443 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.901427    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a962283d-f543-4d9f-97d5-87641b30b4c2-cni-binary-copy\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.901477 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.901458    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4727c87a-fe93-4101-adcf-bc4795bfe808-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gxttc\" (UID: \"4727c87a-fe93-4101-adcf-bc4795bfe808\") " pod="openshift-multus/multus-additional-cni-plugins-gxttc"
Jul 20 09:38:55.901508 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.901475    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5fffc7f3-af46-4089-9814-a3932da685a8-host\") pod \"node-ca-6p7xh\" (UID: \"5fffc7f3-af46-4089-9814-a3932da685a8\") " pod="openshift-image-registry/node-ca-6p7xh"
Jul 20 09:38:55.901508 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.901483    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4727c87a-fe93-4101-adcf-bc4795bfe808-os-release\") pod \"multus-additional-cni-plugins-gxttc\" (UID: \"4727c87a-fe93-4101-adcf-bc4795bfe808\") " pod="openshift-multus/multus-additional-cni-plugins-gxttc"
Jul 20 09:38:55.901571 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.901525    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-host-run-multus-certs\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.901571 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.901528    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a962283d-f543-4d9f-97d5-87641b30b4c2-multus-daemon-config\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.901571 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.901561    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-systemd-units\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.901652 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.901595    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-host-run-netns\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.901652 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.901620    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.901720 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.901681    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-multus-cni-dir\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.901791 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.901774    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-host-var-lib-cni-bin\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.901824 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.901776    2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 20 09:38:55.901824 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.901818    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-etc-kubernetes\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.901889 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.901833    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5784af38-8a10-4cee-8510-64758ed45746-env-overrides\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.901889 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.901850    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-host-slash\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.901889 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.901871    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-host-var-lib-kubelet\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.902015 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.901896    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-host-run-netns\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.902015 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.901882    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-run-ovn\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.902103 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.902043    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-node-log\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.902103 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.902084    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5784af38-8a10-4cee-8510-64758ed45746-host-cni-netd\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.902103 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.902090    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4727c87a-fe93-4101-adcf-bc4795bfe808-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gxttc\" (UID: \"4727c87a-fe93-4101-adcf-bc4795bfe808\") " pod="openshift-multus/multus-additional-cni-plugins-gxttc"
Jul 20 09:38:55.902273 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.902245    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a962283d-f543-4d9f-97d5-87641b30b4c2-os-release\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.902744 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.902717    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5784af38-8a10-4cee-8510-64758ed45746-ovnkube-script-lib\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.906221 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.906197    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5784af38-8a10-4cee-8510-64758ed45746-ovn-node-metrics-cert\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.907207 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:55.907182    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 09:38:55.907207 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:55.907216    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 09:38:55.907356 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:55.907230    2566 projected.go:196] Error preparing data for projected volume kube-api-access-c5ndp for pod openshift-network-diagnostics/network-check-target-9kcgh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 09:38:55.907356 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:55.907305    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f44bf7c3-e682-406b-92c7-b8ecfd6505f6-kube-api-access-c5ndp podName:f44bf7c3-e682-406b-92c7-b8ecfd6505f6 nodeName:}" failed. No retries permitted until 2026-07-20 09:38:56.407273385 +0000 UTC m=+2.958727846 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-c5ndp" (UniqueName: "kubernetes.io/projected/f44bf7c3-e682-406b-92c7-b8ecfd6505f6-kube-api-access-c5ndp") pod "network-check-target-9kcgh" (UID: "f44bf7c3-e682-406b-92c7-b8ecfd6505f6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 09:38:55.911419 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.911391    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqtfc\" (UniqueName: \"kubernetes.io/projected/5d536f0c-af6a-4250-a216-30973ed33030-kube-api-access-pqtfc\") pod \"node-resolver-mqz47\" (UID: \"5d536f0c-af6a-4250-a216-30973ed33030\") " pod="openshift-dns/node-resolver-mqz47"
Jul 20 09:38:55.911610 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.911392    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2vt7\" (UniqueName: \"kubernetes.io/projected/a962283d-f543-4d9f-97d5-87641b30b4c2-kube-api-access-j2vt7\") pod \"multus-rdqzb\" (UID: \"a962283d-f543-4d9f-97d5-87641b30b4c2\") " pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:55.911610 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.911404    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbsqd\" (UniqueName: \"kubernetes.io/projected/5784af38-8a10-4cee-8510-64758ed45746-kube-api-access-fbsqd\") pod \"ovnkube-node-64r4x\" (UID: \"5784af38-8a10-4cee-8510-64758ed45746\") " pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:55.913236 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.913205    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlwhm\" (UniqueName: \"kubernetes.io/projected/4727c87a-fe93-4101-adcf-bc4795bfe808-kube-api-access-wlwhm\") pod \"multus-additional-cni-plugins-gxttc\" (UID: \"4727c87a-fe93-4101-adcf-bc4795bfe808\") " pod="openshift-multus/multus-additional-cni-plugins-gxttc"
Jul 20 09:38:55.914018 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:55.913997    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r644g\" (UniqueName: \"kubernetes.io/projected/5fffc7f3-af46-4089-9814-a3932da685a8-kube-api-access-r644g\") pod \"node-ca-6p7xh\" (UID: \"5fffc7f3-af46-4089-9814-a3932da685a8\") " pod="openshift-image-registry/node-ca-6p7xh"
Jul 20 09:38:56.000101 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.000065    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/51f081b8-e74b-4ff6-b9c6-06a13e680d11-device-dir\") pod \"aws-ebs-csi-driver-node-6kdvx\" (UID: \"51f081b8-e74b-4ff6-b9c6-06a13e680d11\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kdvx"
Jul 20 09:38:56.000101 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.000104    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/78c743ae-071a-4620-ac25-5fb11708d229-etc-sysctl-d\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:56.000353 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.000122    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/78c743ae-071a-4620-ac25-5fb11708d229-etc-systemd\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:56.000353 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.000207    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/51f081b8-e74b-4ff6-b9c6-06a13e680d11-device-dir\") pod \"aws-ebs-csi-driver-node-6kdvx\" (UID: \"51f081b8-e74b-4ff6-b9c6-06a13e680d11\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kdvx"
Jul 20 09:38:56.000353 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.000236    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/51f081b8-e74b-4ff6-b9c6-06a13e680d11-socket-dir\") pod \"aws-ebs-csi-driver-node-6kdvx\" (UID: \"51f081b8-e74b-4ff6-b9c6-06a13e680d11\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kdvx"
Jul 20 09:38:56.000353 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.000251    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/78c743ae-071a-4620-ac25-5fb11708d229-etc-sysctl-d\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:56.000353 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.000272    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/78c743ae-071a-4620-ac25-5fb11708d229-run\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:56.000353 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.000281    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/78c743ae-071a-4620-ac25-5fb11708d229-etc-systemd\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:56.000353 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.000294    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/78c743ae-071a-4620-ac25-5fb11708d229-etc-tuned\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:56.000353 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.000338    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/78c743ae-071a-4620-ac25-5fb11708d229-run\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:56.000651 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.000369    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d4a0ef09-53a6-47cb-a39f-15ae72031d8f-kubelet-config\") pod \"global-pull-secret-syncer-4lw87\" (UID: \"d4a0ef09-53a6-47cb-a39f-15ae72031d8f\") " pod="kube-system/global-pull-secret-syncer-4lw87"
Jul 20 09:38:56.000651 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.000391    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/51f081b8-e74b-4ff6-b9c6-06a13e680d11-socket-dir\") pod \"aws-ebs-csi-driver-node-6kdvx\" (UID: \"51f081b8-e74b-4ff6-b9c6-06a13e680d11\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kdvx"
Jul 20 09:38:56.000651 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.000396    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d4a0ef09-53a6-47cb-a39f-15ae72031d8f-dbus\") pod \"global-pull-secret-syncer-4lw87\" (UID: \"d4a0ef09-53a6-47cb-a39f-15ae72031d8f\") " pod="kube-system/global-pull-secret-syncer-4lw87"
Jul 20 09:38:56.000651 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.000420    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/78c743ae-071a-4620-ac25-5fb11708d229-sys\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:56.000651 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.000467    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/78c743ae-071a-4620-ac25-5fb11708d229-sys\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:56.000651 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.000511    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d4a0ef09-53a6-47cb-a39f-15ae72031d8f-kubelet-config\") pod \"global-pull-secret-syncer-4lw87\" (UID: \"d4a0ef09-53a6-47cb-a39f-15ae72031d8f\") " pod="kube-system/global-pull-secret-syncer-4lw87"
Jul 20 09:38:56.000651 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.000521    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d4a0ef09-53a6-47cb-a39f-15ae72031d8f-dbus\") pod \"global-pull-secret-syncer-4lw87\" (UID: \"d4a0ef09-53a6-47cb-a39f-15ae72031d8f\") " pod="kube-system/global-pull-secret-syncer-4lw87"
Jul 20 09:38:56.000651 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.000551    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/78c743ae-071a-4620-ac25-5fb11708d229-tmp\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:56.000651 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.000575    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/75a6adbe-72f8-4292-a8f6-605d496bffea-agent-certs\") pod \"konnectivity-agent-vjr7g\" (UID: \"75a6adbe-72f8-4292-a8f6-605d496bffea\") " pod="kube-system/konnectivity-agent-vjr7g"
Jul 20 09:38:56.000651 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.000601    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/51f081b8-e74b-4ff6-b9c6-06a13e680d11-etc-selinux\") pod \"aws-ebs-csi-driver-node-6kdvx\" (UID: \"51f081b8-e74b-4ff6-b9c6-06a13e680d11\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kdvx"
Jul 20 09:38:56.000651 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.000624    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/78c743ae-071a-4620-ac25-5fb11708d229-etc-modprobe-d\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:56.000651 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.000649    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/75a6adbe-72f8-4292-a8f6-605d496bffea-konnectivity-ca\") pod \"konnectivity-agent-vjr7g\" (UID: \"75a6adbe-72f8-4292-a8f6-605d496bffea\") " pod="kube-system/konnectivity-agent-vjr7g"
Jul 20 09:38:56.001173 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.000681    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dzmw\" (UniqueName: \"kubernetes.io/projected/51f081b8-e74b-4ff6-b9c6-06a13e680d11-kube-api-access-2dzmw\") pod \"aws-ebs-csi-driver-node-6kdvx\" (UID: \"51f081b8-e74b-4ff6-b9c6-06a13e680d11\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kdvx"
Jul 20 09:38:56.001173 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.000707    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/78c743ae-071a-4620-ac25-5fb11708d229-etc-sysconfig\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:56.001173 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.000730    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grx54\" (UniqueName: \"kubernetes.io/projected/78c743ae-071a-4620-ac25-5fb11708d229-kube-api-access-grx54\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:56.001173 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.000753    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bfe01381-3135-4dfe-9793-4f158e62303e-iptables-alerter-script\") pod \"iptables-alerter-wkmzk\" (UID: \"bfe01381-3135-4dfe-9793-4f158e62303e\") " pod="openshift-network-operator/iptables-alerter-wkmzk"
Jul 20 09:38:56.001173 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.000758    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/51f081b8-e74b-4ff6-b9c6-06a13e680d11-etc-selinux\") pod \"aws-ebs-csi-driver-node-6kdvx\" (UID: \"51f081b8-e74b-4ff6-b9c6-06a13e680d11\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kdvx"
Jul 20 09:38:56.001173 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.000777    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bfe01381-3135-4dfe-9793-4f158e62303e-host-slash\") pod \"iptables-alerter-wkmzk\" (UID: \"bfe01381-3135-4dfe-9793-4f158e62303e\") " pod="openshift-network-operator/iptables-alerter-wkmzk"
Jul 20 09:38:56.001173 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.000801    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x4tp\" (UniqueName: \"kubernetes.io/projected/bfe01381-3135-4dfe-9793-4f158e62303e-kube-api-access-7x4tp\") pod \"iptables-alerter-wkmzk\" (UID: \"bfe01381-3135-4dfe-9793-4f158e62303e\") " pod="openshift-network-operator/iptables-alerter-wkmzk"
Jul 20 09:38:56.001173 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.000831    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/78c743ae-071a-4620-ac25-5fb11708d229-var-lib-kubelet\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:56.001173 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.000860    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/51f081b8-e74b-4ff6-b9c6-06a13e680d11-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6kdvx\" (UID: \"51f081b8-e74b-4ff6-b9c6-06a13e680d11\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kdvx"
Jul 20 09:38:56.001173 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.000883    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/78c743ae-071a-4620-ac25-5fb11708d229-etc-sysctl-conf\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:56.001173 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.000922    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/51f081b8-e74b-4ff6-b9c6-06a13e680d11-registration-dir\") pod \"aws-ebs-csi-driver-node-6kdvx\" (UID: \"51f081b8-e74b-4ff6-b9c6-06a13e680d11\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kdvx"
Jul 20 09:38:56.001173 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.000952    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de-metrics-certs\") pod \"network-metrics-daemon-xdlfl\" (UID: \"d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de\") " pod="openshift-multus/network-metrics-daemon-xdlfl"
Jul 20 09:38:56.001173 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.000995    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfccm\" (UniqueName: \"kubernetes.io/projected/d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de-kube-api-access-vfccm\") pod \"network-metrics-daemon-xdlfl\" (UID: \"d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de\") " pod="openshift-multus/network-metrics-daemon-xdlfl"
Jul 20 09:38:56.001173 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.001039    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/78c743ae-071a-4620-ac25-5fb11708d229-host\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:56.001173 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.001066    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/51f081b8-e74b-4ff6-b9c6-06a13e680d11-sys-fs\") pod \"aws-ebs-csi-driver-node-6kdvx\" (UID: \"51f081b8-e74b-4ff6-b9c6-06a13e680d11\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kdvx"
Jul 20 09:38:56.001173 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.001089    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/78c743ae-071a-4620-ac25-5fb11708d229-etc-kubernetes\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:56.001173 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.001122    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d4a0ef09-53a6-47cb-a39f-15ae72031d8f-original-pull-secret\") pod \"global-pull-secret-syncer-4lw87\" (UID: \"d4a0ef09-53a6-47cb-a39f-15ae72031d8f\") " pod="kube-system/global-pull-secret-syncer-4lw87"
Jul 20 09:38:56.001991 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.001145    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/78c743ae-071a-4620-ac25-5fb11708d229-lib-modules\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:56.001991 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.001286    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/78c743ae-071a-4620-ac25-5fb11708d229-lib-modules\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:56.001991 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.001358    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/78c743ae-071a-4620-ac25-5fb11708d229-etc-sysconfig\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:56.001991 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.001425    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/78c743ae-071a-4620-ac25-5fb11708d229-etc-sysctl-conf\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:56.001991 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.001427    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bfe01381-3135-4dfe-9793-4f158e62303e-host-slash\") pod \"iptables-alerter-wkmzk\" (UID: \"bfe01381-3135-4dfe-9793-4f158e62303e\") " pod="openshift-network-operator/iptables-alerter-wkmzk"
Jul 20 09:38:56.001991 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.001488    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/78c743ae-071a-4620-ac25-5fb11708d229-etc-modprobe-d\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:56.001991 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:56.001508    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 09:38:56.001991 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.001526    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/78c743ae-071a-4620-ac25-5fb11708d229-var-lib-kubelet\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:56.001991 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:56.001573    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de-metrics-certs podName:d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de nodeName:}" failed. No retries permitted until 2026-07-20 09:38:56.501553853 +0000 UTC m=+3.053008314 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de-metrics-certs") pod "network-metrics-daemon-xdlfl" (UID: "d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 09:38:56.001991 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.001916    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/78c743ae-071a-4620-ac25-5fb11708d229-etc-kubernetes\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:56.001991 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.001963    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/78c743ae-071a-4620-ac25-5fb11708d229-host\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:56.002470 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:56.002031    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 09:38:56.002470 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.002066    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bfe01381-3135-4dfe-9793-4f158e62303e-iptables-alerter-script\") pod \"iptables-alerter-wkmzk\" (UID: \"bfe01381-3135-4dfe-9793-4f158e62303e\") " pod="openshift-network-operator/iptables-alerter-wkmzk"
Jul 20 09:38:56.002470 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.002031    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/51f081b8-e74b-4ff6-b9c6-06a13e680d11-sys-fs\") pod \"aws-ebs-csi-driver-node-6kdvx\" (UID: \"51f081b8-e74b-4ff6-b9c6-06a13e680d11\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kdvx"
Jul 20 09:38:56.002470 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:56.002077    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4a0ef09-53a6-47cb-a39f-15ae72031d8f-original-pull-secret podName:d4a0ef09-53a6-47cb-a39f-15ae72031d8f nodeName:}" failed. No retries permitted until 2026-07-20 09:38:56.502062702 +0000 UTC m=+3.053517151 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d4a0ef09-53a6-47cb-a39f-15ae72031d8f-original-pull-secret") pod "global-pull-secret-syncer-4lw87" (UID: "d4a0ef09-53a6-47cb-a39f-15ae72031d8f") : object "kube-system"/"original-pull-secret" not registered
Jul 20 09:38:56.002470 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.001490    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/51f081b8-e74b-4ff6-b9c6-06a13e680d11-registration-dir\") pod \"aws-ebs-csi-driver-node-6kdvx\" (UID: \"51f081b8-e74b-4ff6-b9c6-06a13e680d11\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kdvx"
Jul 20 09:38:56.002470 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.002123    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/75a6adbe-72f8-4292-a8f6-605d496bffea-konnectivity-ca\") pod \"konnectivity-agent-vjr7g\" (UID: \"75a6adbe-72f8-4292-a8f6-605d496bffea\") " pod="kube-system/konnectivity-agent-vjr7g"
Jul 20 09:38:56.002470 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.002138    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/51f081b8-e74b-4ff6-b9c6-06a13e680d11-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6kdvx\" (UID: \"51f081b8-e74b-4ff6-b9c6-06a13e680d11\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kdvx"
Jul 20 09:38:56.003456 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.003425    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/78c743ae-071a-4620-ac25-5fb11708d229-etc-tuned\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:56.003746 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.003725    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/78c743ae-071a-4620-ac25-5fb11708d229-tmp\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:56.004279 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.004256    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/75a6adbe-72f8-4292-a8f6-605d496bffea-agent-certs\") pod \"konnectivity-agent-vjr7g\" (UID: \"75a6adbe-72f8-4292-a8f6-605d496bffea\") " pod="kube-system/konnectivity-agent-vjr7g"
Jul 20 09:38:56.013178 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.013145    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7x4tp\" (UniqueName: \"kubernetes.io/projected/bfe01381-3135-4dfe-9793-4f158e62303e-kube-api-access-7x4tp\") pod \"iptables-alerter-wkmzk\" (UID: \"bfe01381-3135-4dfe-9793-4f158e62303e\") " pod="openshift-network-operator/iptables-alerter-wkmzk"
Jul 20 09:38:56.013925 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.013896    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dzmw\" (UniqueName: \"kubernetes.io/projected/51f081b8-e74b-4ff6-b9c6-06a13e680d11-kube-api-access-2dzmw\") pod \"aws-ebs-csi-driver-node-6kdvx\" (UID: \"51f081b8-e74b-4ff6-b9c6-06a13e680d11\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kdvx"
Jul 20 09:38:56.014079 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.014052    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grx54\" (UniqueName: \"kubernetes.io/projected/78c743ae-071a-4620-ac25-5fb11708d229-kube-api-access-grx54\") pod \"tuned-bk2gb\" (UID: \"78c743ae-071a-4620-ac25-5fb11708d229\") " pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:56.014428 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.014373    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfccm\" (UniqueName: \"kubernetes.io/projected/d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de-kube-api-access-vfccm\") pod \"network-metrics-daemon-xdlfl\" (UID: \"d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de\") " pod="openshift-multus/network-metrics-daemon-xdlfl"
Jul 20 09:38:56.094546 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.094502    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mqz47"
Jul 20 09:38:56.104623 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.104590    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gxttc"
Jul 20 09:38:56.115865 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.115821    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6p7xh"
Jul 20 09:38:56.122934 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.122902    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rdqzb"
Jul 20 09:38:56.129030 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.128997    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:38:56.136184 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.136152    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kdvx"
Jul 20 09:38:56.143327 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.143300    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vjr7g"
Jul 20 09:38:56.152183 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.152141    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wkmzk"
Jul 20 09:38:56.157263 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.157238    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bk2gb"
Jul 20 09:38:56.301225 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.301142    2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 20 09:38:56.505032 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.504997    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5ndp\" (UniqueName: \"kubernetes.io/projected/f44bf7c3-e682-406b-92c7-b8ecfd6505f6-kube-api-access-c5ndp\") pod \"network-check-target-9kcgh\" (UID: \"f44bf7c3-e682-406b-92c7-b8ecfd6505f6\") " pod="openshift-network-diagnostics/network-check-target-9kcgh"
Jul 20 09:38:56.505032 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.505037    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de-metrics-certs\") pod \"network-metrics-daemon-xdlfl\" (UID: \"d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de\") " pod="openshift-multus/network-metrics-daemon-xdlfl"
Jul 20 09:38:56.505272 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:56.505147    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 09:38:56.505272 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:56.505161    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 09:38:56.505272 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:56.505189    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 09:38:56.505272 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:56.505203    2566 projected.go:196] Error preparing data for projected volume kube-api-access-c5ndp for pod openshift-network-diagnostics/network-check-target-9kcgh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 09:38:56.505272 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:56.505211    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de-metrics-certs podName:d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de nodeName:}" failed. No retries permitted until 2026-07-20 09:38:57.505193035 +0000 UTC m=+4.056647481 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de-metrics-certs") pod "network-metrics-daemon-xdlfl" (UID: "d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 09:38:56.505272 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:56.505237    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 09:38:56.505272 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.505161    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d4a0ef09-53a6-47cb-a39f-15ae72031d8f-original-pull-secret\") pod \"global-pull-secret-syncer-4lw87\" (UID: \"d4a0ef09-53a6-47cb-a39f-15ae72031d8f\") " pod="kube-system/global-pull-secret-syncer-4lw87"
Jul 20 09:38:56.505272 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:56.505254    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f44bf7c3-e682-406b-92c7-b8ecfd6505f6-kube-api-access-c5ndp podName:f44bf7c3-e682-406b-92c7-b8ecfd6505f6 nodeName:}" failed. No retries permitted until 2026-07-20 09:38:57.505239711 +0000 UTC m=+4.056694172 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-c5ndp" (UniqueName: "kubernetes.io/projected/f44bf7c3-e682-406b-92c7-b8ecfd6505f6-kube-api-access-c5ndp") pod "network-check-target-9kcgh" (UID: "f44bf7c3-e682-406b-92c7-b8ecfd6505f6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 09:38:56.505272 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:56.505277    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4a0ef09-53a6-47cb-a39f-15ae72031d8f-original-pull-secret podName:d4a0ef09-53a6-47cb-a39f-15ae72031d8f nodeName:}" failed. No retries permitted until 2026-07-20 09:38:57.505265461 +0000 UTC m=+4.056719919 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d4a0ef09-53a6-47cb-a39f-15ae72031d8f-original-pull-secret") pod "global-pull-secret-syncer-4lw87" (UID: "d4a0ef09-53a6-47cb-a39f-15ae72031d8f") : object "kube-system"/"original-pull-secret" not registered
Jul 20 09:38:56.837814 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.837754    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 09:33:55 +0000 UTC" deadline="2028-05-05 13:11:08.704702322 +0000 UTC"
Jul 20 09:38:56.837814 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:56.837802    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15723h32m11.866904244s"
Jul 20 09:38:56.953225 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:56.953189    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d536f0c_af6a_4250_a216_30973ed33030.slice/crio-f0cbe3fb7176ac3517b933ec898ed228fd9d460b1a2dedf5008b8fd09a05cd6d WatchSource:0}: Error finding container f0cbe3fb7176ac3517b933ec898ed228fd9d460b1a2dedf5008b8fd09a05cd6d: Status 404 returned error can't find the container with id f0cbe3fb7176ac3517b933ec898ed228fd9d460b1a2dedf5008b8fd09a05cd6d
Jul 20 09:38:56.953503 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:56.953476    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51f081b8_e74b_4ff6_b9c6_06a13e680d11.slice/crio-d6c2bffd34b41ff768c23082137185599d3be687e1457d43536f505fea99e279 WatchSource:0}: Error finding container d6c2bffd34b41ff768c23082137185599d3be687e1457d43536f505fea99e279: Status 404 returned error can't find the container with id d6c2bffd34b41ff768c23082137185599d3be687e1457d43536f505fea99e279
Jul 20 09:38:56.956368 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:56.956345    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5784af38_8a10_4cee_8510_64758ed45746.slice/crio-b9b8607db480a22d9c0dfaad3aff344b88b910119a149ed617c278423742fe2f WatchSource:0}: Error finding container b9b8607db480a22d9c0dfaad3aff344b88b910119a149ed617c278423742fe2f: Status 404 returned error can't find the container with id b9b8607db480a22d9c0dfaad3aff344b88b910119a149ed617c278423742fe2f
Jul 20 09:38:56.958644 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:56.958541    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfe01381_3135_4dfe_9793_4f158e62303e.slice/crio-37a699cff16a7ecd0fc0593b5562eb2da37fc6231f48b870b6492b63ec0e0cf2 WatchSource:0}: Error finding container 37a699cff16a7ecd0fc0593b5562eb2da37fc6231f48b870b6492b63ec0e0cf2: Status 404 returned error can't find the container with id 37a699cff16a7ecd0fc0593b5562eb2da37fc6231f48b870b6492b63ec0e0cf2
Jul 20 09:38:56.959474 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:56.959456    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fffc7f3_af46_4089_9814_a3932da685a8.slice/crio-ba6b2cec6f443e6daa274fc95d71d84fcbc9fa6a47974a1ffe1da86a3d4d2b6f WatchSource:0}: Error finding container ba6b2cec6f443e6daa274fc95d71d84fcbc9fa6a47974a1ffe1da86a3d4d2b6f: Status 404 returned error can't find the container with id ba6b2cec6f443e6daa274fc95d71d84fcbc9fa6a47974a1ffe1da86a3d4d2b6f
Jul 20 09:38:56.961024 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:56.960894    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4727c87a_fe93_4101_adcf_bc4795bfe808.slice/crio-b70c97db3a74d7ac0403ab66f3a202ccec84fa5cc47674779fc8a84ffb956f10 WatchSource:0}: Error finding container b70c97db3a74d7ac0403ab66f3a202ccec84fa5cc47674779fc8a84ffb956f10: Status 404 returned error can't find the container with id b70c97db3a74d7ac0403ab66f3a202ccec84fa5cc47674779fc8a84ffb956f10
Jul 20 09:38:56.963078 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:38:56.963052    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78c743ae_071a_4620_ac25_5fb11708d229.slice/crio-36b625d500020140c087c28d027edaa6875f122a230fc9531c51f42794bd2f50 WatchSource:0}: Error finding container 36b625d500020140c087c28d027edaa6875f122a230fc9531c51f42794bd2f50: Status 404 returned error can't find the container with id 36b625d500020140c087c28d027edaa6875f122a230fc9531c51f42794bd2f50
Jul 20 09:38:57.513520 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:57.513092    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5ndp\" (UniqueName: \"kubernetes.io/projected/f44bf7c3-e682-406b-92c7-b8ecfd6505f6-kube-api-access-c5ndp\") pod \"network-check-target-9kcgh\" (UID: \"f44bf7c3-e682-406b-92c7-b8ecfd6505f6\") " pod="openshift-network-diagnostics/network-check-target-9kcgh"
Jul 20 09:38:57.513520 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:57.513297    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de-metrics-certs\") pod \"network-metrics-daemon-xdlfl\" (UID: \"d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de\") " pod="openshift-multus/network-metrics-daemon-xdlfl"
Jul 20 09:38:57.513520 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:57.513322    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d4a0ef09-53a6-47cb-a39f-15ae72031d8f-original-pull-secret\") pod \"global-pull-secret-syncer-4lw87\" (UID: \"d4a0ef09-53a6-47cb-a39f-15ae72031d8f\") " pod="kube-system/global-pull-secret-syncer-4lw87"
Jul 20 09:38:57.513520 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:57.513379    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 09:38:57.513520 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:57.513407    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 09:38:57.513520 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:57.513410    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 09:38:57.513520 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:57.513426    2566 projected.go:196] Error preparing data for projected volume kube-api-access-c5ndp for pod openshift-network-diagnostics/network-check-target-9kcgh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 09:38:57.513520 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:57.513454    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4a0ef09-53a6-47cb-a39f-15ae72031d8f-original-pull-secret podName:d4a0ef09-53a6-47cb-a39f-15ae72031d8f nodeName:}" failed. No retries permitted until 2026-07-20 09:38:59.513441692 +0000 UTC m=+6.064896134 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d4a0ef09-53a6-47cb-a39f-15ae72031d8f-original-pull-secret") pod "global-pull-secret-syncer-4lw87" (UID: "d4a0ef09-53a6-47cb-a39f-15ae72031d8f") : object "kube-system"/"original-pull-secret" not registered
Jul 20 09:38:57.513520 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:57.513473    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f44bf7c3-e682-406b-92c7-b8ecfd6505f6-kube-api-access-c5ndp podName:f44bf7c3-e682-406b-92c7-b8ecfd6505f6 nodeName:}" failed. No retries permitted until 2026-07-20 09:38:59.513459164 +0000 UTC m=+6.064913606 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-c5ndp" (UniqueName: "kubernetes.io/projected/f44bf7c3-e682-406b-92c7-b8ecfd6505f6-kube-api-access-c5ndp") pod "network-check-target-9kcgh" (UID: "f44bf7c3-e682-406b-92c7-b8ecfd6505f6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 09:38:57.514125 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:57.513540    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 09:38:57.514125 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:57.513583    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de-metrics-certs podName:d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de nodeName:}" failed. No retries permitted until 2026-07-20 09:38:59.513567226 +0000 UTC m=+6.065021668 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de-metrics-certs") pod "network-metrics-daemon-xdlfl" (UID: "d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 09:38:57.838996 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:57.838828    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 09:33:55 +0000 UTC" deadline="2028-03-05 12:24:26.334434845 +0000 UTC"
Jul 20 09:38:57.838996 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:57.838893    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14258h45m28.495545802s"
Jul 20 09:38:57.916622 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:57.916589    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lw87"
Jul 20 09:38:57.916787 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:57.916734    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lw87" podUID="d4a0ef09-53a6-47cb-a39f-15ae72031d8f"
Jul 20 09:38:57.917157 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:57.917138    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xdlfl"
Jul 20 09:38:57.917298 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:57.917248    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xdlfl" podUID="d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de"
Jul 20 09:38:57.917359 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:57.917323    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9kcgh"
Jul 20 09:38:57.917410 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:57.917389    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9kcgh" podUID="f44bf7c3-e682-406b-92c7-b8ecfd6505f6"
Jul 20 09:38:57.934816 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:57.933900    2566 generic.go:356] "Generic (PLEG): container finished" podID="007fe94f9ffbab53bf60fc279d909e65" containerID="46eeba2261c8b9f69705f22868659b3da88835cddfcc310ef5bb148cf6d3b80c" exitCode=0
Jul 20 09:38:57.934816 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:57.934028    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal" event={"ID":"007fe94f9ffbab53bf60fc279d909e65","Type":"ContainerDied","Data":"46eeba2261c8b9f69705f22868659b3da88835cddfcc310ef5bb148cf6d3b80c"}
Jul 20 09:38:57.942114 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:57.942023    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6p7xh" event={"ID":"5fffc7f3-af46-4089-9814-a3932da685a8","Type":"ContainerStarted","Data":"ba6b2cec6f443e6daa274fc95d71d84fcbc9fa6a47974a1ffe1da86a3d4d2b6f"}
Jul 20 09:38:57.945446 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:57.945350    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mqz47" event={"ID":"5d536f0c-af6a-4250-a216-30973ed33030","Type":"ContainerStarted","Data":"f0cbe3fb7176ac3517b933ec898ed228fd9d460b1a2dedf5008b8fd09a05cd6d"}
Jul 20 09:38:57.947816 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:57.947735    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bk2gb" event={"ID":"78c743ae-071a-4620-ac25-5fb11708d229","Type":"ContainerStarted","Data":"36b625d500020140c087c28d027edaa6875f122a230fc9531c51f42794bd2f50"}
Jul 20 09:38:57.969176 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:57.969045    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gxttc" event={"ID":"4727c87a-fe93-4101-adcf-bc4795bfe808","Type":"ContainerStarted","Data":"b70c97db3a74d7ac0403ab66f3a202ccec84fa5cc47674779fc8a84ffb956f10"}
Jul 20 09:38:57.978924 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:57.978791    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-64r4x" event={"ID":"5784af38-8a10-4cee-8510-64758ed45746","Type":"ContainerStarted","Data":"b9b8607db480a22d9c0dfaad3aff344b88b910119a149ed617c278423742fe2f"}
Jul 20 09:38:57.982952 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:57.982910    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vjr7g" event={"ID":"75a6adbe-72f8-4292-a8f6-605d496bffea","Type":"ContainerStarted","Data":"58980361f7d8c7eb3483ec85c88b769d4614dc69354406339b9c88dc1f6d28c2"}
Jul 20 09:38:57.985768 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:57.985730    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kdvx" event={"ID":"51f081b8-e74b-4ff6-b9c6-06a13e680d11","Type":"ContainerStarted","Data":"d6c2bffd34b41ff768c23082137185599d3be687e1457d43536f505fea99e279"}
Jul 20 09:38:57.996128 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:57.996084    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wkmzk" event={"ID":"bfe01381-3135-4dfe-9793-4f158e62303e","Type":"ContainerStarted","Data":"37a699cff16a7ecd0fc0593b5562eb2da37fc6231f48b870b6492b63ec0e0cf2"}
Jul 20 09:38:58.008587 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:58.008545    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rdqzb" event={"ID":"a962283d-f543-4d9f-97d5-87641b30b4c2","Type":"ContainerStarted","Data":"538f5408de93b8d12962d6ec4b285648499e990556291007ce4accdfd2ce75f0"}
Jul 20 09:38:58.022564 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:58.022522    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-134.ec2.internal" event={"ID":"e04307ae6e708dcc808dbfefa047d3b4","Type":"ContainerStarted","Data":"a0b0eb57b69497701470560c7a8725368a98afc0e8ecb8b44f38c186d2f38134"}
Jul 20 09:38:58.039895 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:58.039529    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-134.ec2.internal" podStartSLOduration=4.039504687 podStartE2EDuration="4.039504687s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 09:38:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:38:58.03922903 +0000 UTC m=+4.590683495" watchObservedRunningTime="2026-07-20 09:38:58.039504687 +0000 UTC m=+4.590959152"
Jul 20 09:38:59.052781 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:59.052720    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal" event={"ID":"007fe94f9ffbab53bf60fc279d909e65","Type":"ContainerStarted","Data":"9dcca66f55ab82baec62ba2f06e4904b02aeff6391d2683d85951f6cc9264258"}
Jul 20 09:38:59.533382 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:59.533274    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5ndp\" (UniqueName: \"kubernetes.io/projected/f44bf7c3-e682-406b-92c7-b8ecfd6505f6-kube-api-access-c5ndp\") pod \"network-check-target-9kcgh\" (UID: \"f44bf7c3-e682-406b-92c7-b8ecfd6505f6\") " pod="openshift-network-diagnostics/network-check-target-9kcgh"
Jul 20 09:38:59.533382 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:59.533347    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de-metrics-certs\") pod \"network-metrics-daemon-xdlfl\" (UID: \"d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de\") " pod="openshift-multus/network-metrics-daemon-xdlfl"
Jul 20 09:38:59.533382 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:59.533380    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d4a0ef09-53a6-47cb-a39f-15ae72031d8f-original-pull-secret\") pod \"global-pull-secret-syncer-4lw87\" (UID: \"d4a0ef09-53a6-47cb-a39f-15ae72031d8f\") " pod="kube-system/global-pull-secret-syncer-4lw87"
Jul 20 09:38:59.533686 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:59.533548    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 09:38:59.533686 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:59.533612    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4a0ef09-53a6-47cb-a39f-15ae72031d8f-original-pull-secret podName:d4a0ef09-53a6-47cb-a39f-15ae72031d8f nodeName:}" failed. No retries permitted until 2026-07-20 09:39:03.533592547 +0000 UTC m=+10.085046992 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d4a0ef09-53a6-47cb-a39f-15ae72031d8f-original-pull-secret") pod "global-pull-secret-syncer-4lw87" (UID: "d4a0ef09-53a6-47cb-a39f-15ae72031d8f") : object "kube-system"/"original-pull-secret" not registered
Jul 20 09:38:59.534062 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:59.534040    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 09:38:59.534062 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:59.534066    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 09:38:59.534231 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:59.534079    2566 projected.go:196] Error preparing data for projected volume kube-api-access-c5ndp for pod openshift-network-diagnostics/network-check-target-9kcgh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 09:38:59.534231 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:59.534128    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f44bf7c3-e682-406b-92c7-b8ecfd6505f6-kube-api-access-c5ndp podName:f44bf7c3-e682-406b-92c7-b8ecfd6505f6 nodeName:}" failed. No retries permitted until 2026-07-20 09:39:03.534112257 +0000 UTC m=+10.085566720 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-c5ndp" (UniqueName: "kubernetes.io/projected/f44bf7c3-e682-406b-92c7-b8ecfd6505f6-kube-api-access-c5ndp") pod "network-check-target-9kcgh" (UID: "f44bf7c3-e682-406b-92c7-b8ecfd6505f6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 09:38:59.534231 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:59.534195    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 09:38:59.534231 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:59.534227    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de-metrics-certs podName:d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de nodeName:}" failed. No retries permitted until 2026-07-20 09:39:03.534215412 +0000 UTC m=+10.085669868 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de-metrics-certs") pod "network-metrics-daemon-xdlfl" (UID: "d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 09:38:59.917259 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:59.917161    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9kcgh"
Jul 20 09:38:59.917259 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:59.917213    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xdlfl"
Jul 20 09:38:59.926195 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:59.926136    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9kcgh" podUID="f44bf7c3-e682-406b-92c7-b8ecfd6505f6"
Jul 20 09:38:59.926766 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:59.926739    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xdlfl" podUID="d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de"
Jul 20 09:38:59.926870 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:38:59.926828    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lw87"
Jul 20 09:38:59.927008 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:38:59.926986    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lw87" podUID="d4a0ef09-53a6-47cb-a39f-15ae72031d8f"
Jul 20 09:39:01.915556 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:01.915084    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lw87"
Jul 20 09:39:01.915556 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:01.915105    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xdlfl"
Jul 20 09:39:01.915556 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:01.915216    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lw87" podUID="d4a0ef09-53a6-47cb-a39f-15ae72031d8f"
Jul 20 09:39:01.915556 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:01.915357    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xdlfl" podUID="d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de"
Jul 20 09:39:01.915556 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:01.915423    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9kcgh"
Jul 20 09:39:01.915556 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:01.915492    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9kcgh" podUID="f44bf7c3-e682-406b-92c7-b8ecfd6505f6"
Jul 20 09:39:03.567172 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:03.567128    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d4a0ef09-53a6-47cb-a39f-15ae72031d8f-original-pull-secret\") pod \"global-pull-secret-syncer-4lw87\" (UID: \"d4a0ef09-53a6-47cb-a39f-15ae72031d8f\") " pod="kube-system/global-pull-secret-syncer-4lw87"
Jul 20 09:39:03.567562 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:03.567357    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5ndp\" (UniqueName: \"kubernetes.io/projected/f44bf7c3-e682-406b-92c7-b8ecfd6505f6-kube-api-access-c5ndp\") pod \"network-check-target-9kcgh\" (UID: \"f44bf7c3-e682-406b-92c7-b8ecfd6505f6\") " pod="openshift-network-diagnostics/network-check-target-9kcgh"
Jul 20 09:39:03.567562 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:03.567382    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de-metrics-certs\") pod \"network-metrics-daemon-xdlfl\" (UID: \"d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de\") " pod="openshift-multus/network-metrics-daemon-xdlfl"
Jul 20 09:39:03.567562 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:03.567503    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 09:39:03.567562 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:03.567554    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de-metrics-certs podName:d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de nodeName:}" failed. No retries permitted until 2026-07-20 09:39:11.56753939 +0000 UTC m=+18.118993834 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de-metrics-certs") pod "network-metrics-daemon-xdlfl" (UID: "d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 09:39:03.567708 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:03.567599    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 09:39:03.567708 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:03.567620    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4a0ef09-53a6-47cb-a39f-15ae72031d8f-original-pull-secret podName:d4a0ef09-53a6-47cb-a39f-15ae72031d8f nodeName:}" failed. No retries permitted until 2026-07-20 09:39:11.5676136 +0000 UTC m=+18.119068042 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d4a0ef09-53a6-47cb-a39f-15ae72031d8f-original-pull-secret") pod "global-pull-secret-syncer-4lw87" (UID: "d4a0ef09-53a6-47cb-a39f-15ae72031d8f") : object "kube-system"/"original-pull-secret" not registered
Jul 20 09:39:03.567708 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:03.567680    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 09:39:03.567708 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:03.567696    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 09:39:03.567708 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:03.567706    2566 projected.go:196] Error preparing data for projected volume kube-api-access-c5ndp for pod openshift-network-diagnostics/network-check-target-9kcgh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 09:39:03.567867 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:03.567731    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f44bf7c3-e682-406b-92c7-b8ecfd6505f6-kube-api-access-c5ndp podName:f44bf7c3-e682-406b-92c7-b8ecfd6505f6 nodeName:}" failed. No retries permitted until 2026-07-20 09:39:11.567724129 +0000 UTC m=+18.119178571 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-c5ndp" (UniqueName: "kubernetes.io/projected/f44bf7c3-e682-406b-92c7-b8ecfd6505f6-kube-api-access-c5ndp") pod "network-check-target-9kcgh" (UID: "f44bf7c3-e682-406b-92c7-b8ecfd6505f6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 09:39:03.916866 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:03.915077    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lw87"
Jul 20 09:39:03.916866 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:03.915206    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lw87" podUID="d4a0ef09-53a6-47cb-a39f-15ae72031d8f"
Jul 20 09:39:03.916866 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:03.915578    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xdlfl"
Jul 20 09:39:03.916866 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:03.915674    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xdlfl" podUID="d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de"
Jul 20 09:39:03.917201 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:03.916905    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9kcgh"
Jul 20 09:39:03.917201 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:03.917028    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9kcgh" podUID="f44bf7c3-e682-406b-92c7-b8ecfd6505f6"
Jul 20 09:39:05.917133 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:05.917095    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lw87"
Jul 20 09:39:05.917589 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:05.917237    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lw87" podUID="d4a0ef09-53a6-47cb-a39f-15ae72031d8f"
Jul 20 09:39:05.917657 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:05.917636    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xdlfl"
Jul 20 09:39:05.917925 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:05.917725    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xdlfl" podUID="d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de"
Jul 20 09:39:05.917925 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:05.917787    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9kcgh"
Jul 20 09:39:05.917925 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:05.917855    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9kcgh" podUID="f44bf7c3-e682-406b-92c7-b8ecfd6505f6"
Jul 20 09:39:07.914441 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:07.914397    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lw87"
Jul 20 09:39:07.914441 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:07.914438    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9kcgh"
Jul 20 09:39:07.914890 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:07.914408    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xdlfl"
Jul 20 09:39:07.914890 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:07.914542    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lw87" podUID="d4a0ef09-53a6-47cb-a39f-15ae72031d8f"
Jul 20 09:39:07.914890 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:07.914628    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xdlfl" podUID="d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de"
Jul 20 09:39:07.914890 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:07.914690    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9kcgh" podUID="f44bf7c3-e682-406b-92c7-b8ecfd6505f6"
Jul 20 09:39:09.914419 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:09.914375    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lw87"
Jul 20 09:39:09.914886 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:09.914382    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xdlfl"
Jul 20 09:39:09.914886 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:09.914504    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9kcgh"
Jul 20 09:39:09.914886 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:09.914509    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lw87" podUID="d4a0ef09-53a6-47cb-a39f-15ae72031d8f"
Jul 20 09:39:09.914886 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:09.914582    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xdlfl" podUID="d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de"
Jul 20 09:39:09.914886 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:09.914643    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9kcgh" podUID="f44bf7c3-e682-406b-92c7-b8ecfd6505f6"
Jul 20 09:39:11.642869 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:11.642823    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5ndp\" (UniqueName: \"kubernetes.io/projected/f44bf7c3-e682-406b-92c7-b8ecfd6505f6-kube-api-access-c5ndp\") pod \"network-check-target-9kcgh\" (UID: \"f44bf7c3-e682-406b-92c7-b8ecfd6505f6\") " pod="openshift-network-diagnostics/network-check-target-9kcgh"
Jul 20 09:39:11.643450 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:11.642882    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de-metrics-certs\") pod \"network-metrics-daemon-xdlfl\" (UID: \"d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de\") " pod="openshift-multus/network-metrics-daemon-xdlfl"
Jul 20 09:39:11.643450 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:11.642918    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d4a0ef09-53a6-47cb-a39f-15ae72031d8f-original-pull-secret\") pod \"global-pull-secret-syncer-4lw87\" (UID: \"d4a0ef09-53a6-47cb-a39f-15ae72031d8f\") " pod="kube-system/global-pull-secret-syncer-4lw87"
Jul 20 09:39:11.643450 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:11.643041    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 09:39:11.643450 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:11.643057    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 09:39:11.643450 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:11.643074    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 09:39:11.643450 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:11.643089    2566 projected.go:196] Error preparing data for projected volume kube-api-access-c5ndp for pod openshift-network-diagnostics/network-check-target-9kcgh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 09:39:11.643450 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:11.643121    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4a0ef09-53a6-47cb-a39f-15ae72031d8f-original-pull-secret podName:d4a0ef09-53a6-47cb-a39f-15ae72031d8f nodeName:}" failed. No retries permitted until 2026-07-20 09:39:27.643102357 +0000 UTC m=+34.194556806 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d4a0ef09-53a6-47cb-a39f-15ae72031d8f-original-pull-secret") pod "global-pull-secret-syncer-4lw87" (UID: "d4a0ef09-53a6-47cb-a39f-15ae72031d8f") : object "kube-system"/"original-pull-secret" not registered
Jul 20 09:39:11.643450 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:11.643057    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 09:39:11.643450 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:11.643148    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f44bf7c3-e682-406b-92c7-b8ecfd6505f6-kube-api-access-c5ndp podName:f44bf7c3-e682-406b-92c7-b8ecfd6505f6 nodeName:}" failed. No retries permitted until 2026-07-20 09:39:27.643130134 +0000 UTC m=+34.194584599 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-c5ndp" (UniqueName: "kubernetes.io/projected/f44bf7c3-e682-406b-92c7-b8ecfd6505f6-kube-api-access-c5ndp") pod "network-check-target-9kcgh" (UID: "f44bf7c3-e682-406b-92c7-b8ecfd6505f6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 09:39:11.643450 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:11.643194    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de-metrics-certs podName:d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de nodeName:}" failed. No retries permitted until 2026-07-20 09:39:27.643178078 +0000 UTC m=+34.194632523 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de-metrics-certs") pod "network-metrics-daemon-xdlfl" (UID: "d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 09:39:11.914387 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:11.914297    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xdlfl"
Jul 20 09:39:11.914387 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:11.914322    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9kcgh"
Jul 20 09:39:11.914387 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:11.914302    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lw87"
Jul 20 09:39:11.914640 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:11.914451    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xdlfl" podUID="d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de"
Jul 20 09:39:11.914640 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:11.914496    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lw87" podUID="d4a0ef09-53a6-47cb-a39f-15ae72031d8f"
Jul 20 09:39:11.914640 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:11.914539    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9kcgh" podUID="f44bf7c3-e682-406b-92c7-b8ecfd6505f6"
Jul 20 09:39:13.915603 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:13.915566    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9kcgh"
Jul 20 09:39:13.916094 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:13.915649    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lw87"
Jul 20 09:39:13.916094 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:13.915667    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9kcgh" podUID="f44bf7c3-e682-406b-92c7-b8ecfd6505f6"
Jul 20 09:39:13.916094 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:13.915765    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lw87" podUID="d4a0ef09-53a6-47cb-a39f-15ae72031d8f"
Jul 20 09:39:13.916094 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:13.915794    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xdlfl"
Jul 20 09:39:13.916094 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:13.915846    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xdlfl" podUID="d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de"
Jul 20 09:39:15.914707 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:15.914674    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9kcgh"
Jul 20 09:39:15.915259 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:15.914726    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xdlfl"
Jul 20 09:39:15.915259 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:15.914741    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lw87"
Jul 20 09:39:15.915259 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:15.914820    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xdlfl" podUID="d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de"
Jul 20 09:39:15.915259 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:15.914914    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9kcgh" podUID="f44bf7c3-e682-406b-92c7-b8ecfd6505f6"
Jul 20 09:39:15.915259 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:15.915034    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lw87" podUID="d4a0ef09-53a6-47cb-a39f-15ae72031d8f"
Jul 20 09:39:17.098200 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:17.097644    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bk2gb" event={"ID":"78c743ae-071a-4620-ac25-5fb11708d229","Type":"ContainerStarted","Data":"97b1905d3c7d1d83e10024a512cc94f2aca54cf62a07ddc9c06116f4e10af449"}
Jul 20 09:39:17.099454 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:17.099423    2566 generic.go:356] "Generic (PLEG): container finished" podID="4727c87a-fe93-4101-adcf-bc4795bfe808" containerID="4ad1c5befa04ee76d1c1ed7f7e3524ffbc015380136b56822610528dd3f0c32e" exitCode=0
Jul 20 09:39:17.099583 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:17.099514    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gxttc" event={"ID":"4727c87a-fe93-4101-adcf-bc4795bfe808","Type":"ContainerDied","Data":"4ad1c5befa04ee76d1c1ed7f7e3524ffbc015380136b56822610528dd3f0c32e"}
Jul 20 09:39:17.105905 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:17.105874    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-64r4x" event={"ID":"5784af38-8a10-4cee-8510-64758ed45746","Type":"ContainerStarted","Data":"834cbc45488a2859cf4e3039d3b6026987baf48306aed24ed9c47cee1cce240f"}
Jul 20 09:39:17.106067 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:17.105962    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-64r4x" event={"ID":"5784af38-8a10-4cee-8510-64758ed45746","Type":"ContainerStarted","Data":"3c599b3399dfa379ce96b634d99f2c3f7ac38a11629633217db684a381660534"}
Jul 20 09:39:17.106067 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:17.106025    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-64r4x" event={"ID":"5784af38-8a10-4cee-8510-64758ed45746","Type":"ContainerStarted","Data":"16e82a8c3cb0f8b8d0ce2aaeac7be21df1aae146d7bbefbe0c8fe6f4cd7ffcc6"}
Jul 20 09:39:17.106184 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:17.106077    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-64r4x" event={"ID":"5784af38-8a10-4cee-8510-64758ed45746","Type":"ContainerStarted","Data":"3feed3d33dd1801af2c9df482a28d99fdad5f2ce1c3ca4335b615850d2cdd60c"}
Jul 20 09:39:17.106184 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:17.106125    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-64r4x" event={"ID":"5784af38-8a10-4cee-8510-64758ed45746","Type":"ContainerStarted","Data":"b5cd93015e9a40912e60624fff71f834cbcb4fff1c782c489fa471adfdbe5007"}
Jul 20 09:39:17.106184 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:17.106170    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-64r4x" event={"ID":"5784af38-8a10-4cee-8510-64758ed45746","Type":"ContainerStarted","Data":"2fe0730d8b0f6f5ba6622ed641f5aa817aff24789fe051e9677d16bdc82eabb0"}
Jul 20 09:39:17.107547 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:17.107516    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vjr7g" event={"ID":"75a6adbe-72f8-4292-a8f6-605d496bffea","Type":"ContainerStarted","Data":"3bc5c87970e210af57aaeb85f30dcd10569a78bc328422bafcb00b0cf51132de"}
Jul 20 09:39:17.109094 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:17.109060    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kdvx" event={"ID":"51f081b8-e74b-4ff6-b9c6-06a13e680d11","Type":"ContainerStarted","Data":"9cda4b94a24cf4db4dca35da6d5a937630b22e520fdc98c12f7bb0d8cd2bce6c"}
Jul 20 09:39:17.110649 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:17.110624    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rdqzb" event={"ID":"a962283d-f543-4d9f-97d5-87641b30b4c2","Type":"ContainerStarted","Data":"a60cc559d812099fd3139165088255ce65e7d81c0b0d3fc3bc69d69c86dd7741"}
Jul 20 09:39:17.112123 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:17.112101    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6p7xh" event={"ID":"5fffc7f3-af46-4089-9814-a3932da685a8","Type":"ContainerStarted","Data":"64a34c32f5ad0891e2f8b2e9c65ba2c27afb4aa9105bbd758c5aa417281f5c69"}
Jul 20 09:39:17.113604 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:17.113583    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mqz47" event={"ID":"5d536f0c-af6a-4250-a216-30973ed33030","Type":"ContainerStarted","Data":"d3ce49ab2dd3dc5a1a7e7dde410e5d8fa5c229d306b31bce8d4dd09616fac916"}
Jul 20 09:39:17.137212 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:17.137149    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-bk2gb" podStartSLOduration=3.977994408 podStartE2EDuration="23.137129107s" totalImagesPullingTime="19.159134699s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 09:38:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:39:17.136708136 +0000 UTC m=+23.688162601" watchObservedRunningTime="2026-07-20 09:39:17.137129107 +0000 UTC m=+23.688583573"
Jul 20 09:39:17.137414 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:17.137286    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-134.ec2.internal" podStartSLOduration=23.13727937 podStartE2EDuration="23.13727937s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 09:38:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:38:59.069368497 +0000 UTC m=+5.620822962" watchObservedRunningTime="2026-07-20 09:39:17.13727937 +0000 UTC m=+23.688733834"
Jul 20 09:39:17.169026 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:17.168951    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mqz47" podStartSLOduration=9.379333946 podStartE2EDuration="24.168934334s" totalImagesPullingTime="14.789600388s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 09:38:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:39:17.163725162 +0000 UTC m=+23.715179626" watchObservedRunningTime="2026-07-20 09:39:17.168934334 +0000 UTC m=+23.720388798"
Jul 20 09:39:17.213683 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:17.213628    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-rdqzb" podStartSLOduration=3.98887758 podStartE2EDuration="23.213614104s" totalImagesPullingTime="19.224736524s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 09:38:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:39:17.212927659 +0000 UTC m=+23.764382137" watchObservedRunningTime="2026-07-20 09:39:17.213614104 +0000 UTC m=+23.765068568"
Jul 20 09:39:17.243109 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:17.243053    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vjr7g" podStartSLOduration=8.450511781 podStartE2EDuration="23.243039137s" totalImagesPullingTime="14.792527356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 09:38:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:39:17.24269574 +0000 UTC m=+23.794150204" watchObservedRunningTime="2026-07-20 09:39:17.243039137 +0000 UTC m=+23.794493601"
Jul 20 09:39:17.300906 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:17.300848    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6p7xh" podStartSLOduration=5.14169005 podStartE2EDuration="24.300827904s" totalImagesPullingTime="19.159137854s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 09:38:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:39:17.300342834 +0000 UTC m=+23.851797298" watchObservedRunningTime="2026-07-20 09:39:17.300827904 +0000 UTC m=+23.852282368"
Jul 20 09:39:17.419041 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:17.419008    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vjr7g"
Jul 20 09:39:17.419822 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:17.419807    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vjr7g"
Jul 20 09:39:17.670619 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:17.670545    2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 20 09:39:17.864607 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:17.864458    2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-20T09:39:17.670567838Z","UUID":"ca598477-d37c-464a-8ed3-3029199a0363","Handler":null,"Name":"","Endpoint":""}
Jul 20 09:39:17.865947 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:17.865915    2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 20 09:39:17.865947 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:17.865952    2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 20 09:39:17.914600 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:17.914568    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lw87"
Jul 20 09:39:17.914600 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:17.914590    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xdlfl"
Jul 20 09:39:17.914840 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:17.914692    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lw87" podUID="d4a0ef09-53a6-47cb-a39f-15ae72031d8f"
Jul 20 09:39:17.914894 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:17.914841    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xdlfl" podUID="d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de"
Jul 20 09:39:17.914942 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:17.914888    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9kcgh"
Jul 20 09:39:17.915038 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:17.915011    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9kcgh" podUID="f44bf7c3-e682-406b-92c7-b8ecfd6505f6"
Jul 20 09:39:18.118352 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:18.118246    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kdvx" event={"ID":"51f081b8-e74b-4ff6-b9c6-06a13e680d11","Type":"ContainerStarted","Data":"9b6a820c71387059ecaea7f82d0ed507a10e1f0d3d362426759880b9414d48fc"}
Jul 20 09:39:18.119930 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:18.119890    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wkmzk" event={"ID":"bfe01381-3135-4dfe-9793-4f158e62303e","Type":"ContainerStarted","Data":"b67ad6d15eae79be9b3a5c808ad0d214228e62063323395119f492889d41b770"}
Jul 20 09:39:18.120863 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:18.120831    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vjr7g"
Jul 20 09:39:18.121005 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:18.120893    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vjr7g"
Jul 20 09:39:18.142381 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:18.142325    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wkmzk" podStartSLOduration=4.983575536 podStartE2EDuration="24.142292967s" totalImagesPullingTime="19.158717431s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 09:38:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:39:18.141531671 +0000 UTC m=+24.692986154" watchObservedRunningTime="2026-07-20 09:39:18.142292967 +0000 UTC m=+24.693747430"
Jul 20 09:39:19.127667 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:19.127626    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-64r4x" event={"ID":"5784af38-8a10-4cee-8510-64758ed45746","Type":"ContainerStarted","Data":"d65f20d3f578d8f62bff3264385b4d8a98c2c5f7070fb6edd4f23f01b41d7127"}
Jul 20 09:39:19.915392 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:19.915351    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9kcgh"
Jul 20 09:39:19.915392 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:19.915381    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xdlfl"
Jul 20 09:39:19.915751 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:19.915347    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lw87"
Jul 20 09:39:19.915751 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:19.915491    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9kcgh" podUID="f44bf7c3-e682-406b-92c7-b8ecfd6505f6"
Jul 20 09:39:19.915751 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:19.915560    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xdlfl" podUID="d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de"
Jul 20 09:39:19.915751 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:19.915645    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lw87" podUID="d4a0ef09-53a6-47cb-a39f-15ae72031d8f"
Jul 20 09:39:20.132427 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:20.132382    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kdvx" event={"ID":"51f081b8-e74b-4ff6-b9c6-06a13e680d11","Type":"ContainerStarted","Data":"2399a78cd89832ba8e3c21c7603d4f5a2dff978520b87606f18f4941bfc73125"}
Jul 20 09:39:20.161986 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:20.161918    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kdvx" podStartSLOduration=4.4545233920000005 podStartE2EDuration="26.161901598s" totalImagesPullingTime="21.707378206s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 09:38:54 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:39:20.161389132 +0000 UTC m=+26.712843597" watchObservedRunningTime="2026-07-20 09:39:20.161901598 +0000 UTC m=+26.713356097"
Jul 20 09:39:21.914776 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:21.914587    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9kcgh"
Jul 20 09:39:21.915580 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:21.914592    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lw87"
Jul 20 09:39:21.915580 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:21.914857    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9kcgh" podUID="f44bf7c3-e682-406b-92c7-b8ecfd6505f6"
Jul 20 09:39:21.915580 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:21.914627    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xdlfl"
Jul 20 09:39:21.915580 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:21.914951    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lw87" podUID="d4a0ef09-53a6-47cb-a39f-15ae72031d8f"
Jul 20 09:39:21.915580 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:21.915101    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xdlfl" podUID="d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de"
Jul 20 09:39:22.139852 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:22.139757    2566 generic.go:356] "Generic (PLEG): container finished" podID="4727c87a-fe93-4101-adcf-bc4795bfe808" containerID="9cdf73e0174f1ddb643dcd3fde054c31799fc7e3aa42710e64233506c8feb0fb" exitCode=0
Jul 20 09:39:22.139852 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:22.139805    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gxttc" event={"ID":"4727c87a-fe93-4101-adcf-bc4795bfe808","Type":"ContainerDied","Data":"9cdf73e0174f1ddb643dcd3fde054c31799fc7e3aa42710e64233506c8feb0fb"}
Jul 20 09:39:22.143847 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:22.143820    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-64r4x" event={"ID":"5784af38-8a10-4cee-8510-64758ed45746","Type":"ContainerStarted","Data":"000b39c6b6e6d6c33d83dfce97374494f803b41c032cd9a8e2e01947d9d668da"}
Jul 20 09:39:22.144125 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:22.144104    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:39:22.144199 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:22.144182    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:39:22.161160 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:22.161123    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:39:22.188449 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:22.188365    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-64r4x" podStartSLOduration=8.948729257 podStartE2EDuration="28.188343459s" totalImagesPullingTime="19.239614202s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 09:38:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:39:22.185691011 +0000 UTC m=+28.737145477" watchObservedRunningTime="2026-07-20 09:39:22.188343459 +0000 UTC m=+28.739797924"
Jul 20 09:39:23.147015 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:23.146786    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:39:23.163827 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:23.163801    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:39:23.449170 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:23.449069    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9kcgh"]
Jul 20 09:39:23.449334 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:23.449196    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9kcgh"
Jul 20 09:39:23.449334 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:23.449280    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9kcgh" podUID="f44bf7c3-e682-406b-92c7-b8ecfd6505f6"
Jul 20 09:39:23.452252 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:23.452192    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4lw87"]
Jul 20 09:39:23.452400 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:23.452336    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lw87"
Jul 20 09:39:23.452462 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:23.452442    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lw87" podUID="d4a0ef09-53a6-47cb-a39f-15ae72031d8f"
Jul 20 09:39:23.452871 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:23.452847    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xdlfl"]
Jul 20 09:39:23.453017 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:23.453004    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xdlfl"
Jul 20 09:39:23.453142 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:23.453118    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xdlfl" podUID="d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de"
Jul 20 09:39:24.150668 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:24.150629    2566 generic.go:356] "Generic (PLEG): container finished" podID="4727c87a-fe93-4101-adcf-bc4795bfe808" containerID="a4afb069d9201abaf0c4cee6978ac33e7eef085a4feeab9126f64a43721627c4" exitCode=0
Jul 20 09:39:24.151206 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:24.150728    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gxttc" event={"ID":"4727c87a-fe93-4101-adcf-bc4795bfe808","Type":"ContainerDied","Data":"a4afb069d9201abaf0c4cee6978ac33e7eef085a4feeab9126f64a43721627c4"}
Jul 20 09:39:24.915045 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:24.915000    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9kcgh"
Jul 20 09:39:24.915045 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:24.915039    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lw87"
Jul 20 09:39:24.915301 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:24.915000    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xdlfl"
Jul 20 09:39:24.915301 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:24.915134    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9kcgh" podUID="f44bf7c3-e682-406b-92c7-b8ecfd6505f6"
Jul 20 09:39:24.915301 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:24.915218    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xdlfl" podUID="d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de"
Jul 20 09:39:24.915301 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:24.915295    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lw87" podUID="d4a0ef09-53a6-47cb-a39f-15ae72031d8f"
Jul 20 09:39:26.157613 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:26.157411    2566 generic.go:356] "Generic (PLEG): container finished" podID="4727c87a-fe93-4101-adcf-bc4795bfe808" containerID="b913dc1ffecd965efc5835951e38f0071aa9b6819811a2242ae6306eefc8fd67" exitCode=0
Jul 20 09:39:26.158062 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:26.157488    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gxttc" event={"ID":"4727c87a-fe93-4101-adcf-bc4795bfe808","Type":"ContainerDied","Data":"b913dc1ffecd965efc5835951e38f0071aa9b6819811a2242ae6306eefc8fd67"}
Jul 20 09:39:26.915301 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:26.915262    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xdlfl"
Jul 20 09:39:26.915484 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:26.915272    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lw87"
Jul 20 09:39:26.915484 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:26.915406    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9kcgh"
Jul 20 09:39:26.915484 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:26.915459    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lw87" podUID="d4a0ef09-53a6-47cb-a39f-15ae72031d8f"
Jul 20 09:39:26.915484 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:26.915375    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xdlfl" podUID="d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de"
Jul 20 09:39:26.915671 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:26.915520    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9kcgh" podUID="f44bf7c3-e682-406b-92c7-b8ecfd6505f6"
Jul 20 09:39:27.658168 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:27.658133    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5ndp\" (UniqueName: \"kubernetes.io/projected/f44bf7c3-e682-406b-92c7-b8ecfd6505f6-kube-api-access-c5ndp\") pod \"network-check-target-9kcgh\" (UID: \"f44bf7c3-e682-406b-92c7-b8ecfd6505f6\") " pod="openshift-network-diagnostics/network-check-target-9kcgh"
Jul 20 09:39:27.658619 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:27.658176    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de-metrics-certs\") pod \"network-metrics-daemon-xdlfl\" (UID: \"d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de\") " pod="openshift-multus/network-metrics-daemon-xdlfl"
Jul 20 09:39:27.658619 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:27.658204    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d4a0ef09-53a6-47cb-a39f-15ae72031d8f-original-pull-secret\") pod \"global-pull-secret-syncer-4lw87\" (UID: \"d4a0ef09-53a6-47cb-a39f-15ae72031d8f\") " pod="kube-system/global-pull-secret-syncer-4lw87"
Jul 20 09:39:27.658619 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:27.658323    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 20 09:39:27.658619 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:27.658336    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 09:39:27.658619 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:27.658346    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 20 09:39:27.658619 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:27.658376    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 20 09:39:27.658619 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:27.658380    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4a0ef09-53a6-47cb-a39f-15ae72031d8f-original-pull-secret podName:d4a0ef09-53a6-47cb-a39f-15ae72031d8f nodeName:}" failed. No retries permitted until 2026-07-20 09:39:59.658367508 +0000 UTC m=+66.209821950 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d4a0ef09-53a6-47cb-a39f-15ae72031d8f-original-pull-secret") pod "global-pull-secret-syncer-4lw87" (UID: "d4a0ef09-53a6-47cb-a39f-15ae72031d8f") : object "kube-system"/"original-pull-secret" not registered
Jul 20 09:39:27.658619 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:27.658390    2566 projected.go:196] Error preparing data for projected volume kube-api-access-c5ndp for pod openshift-network-diagnostics/network-check-target-9kcgh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 09:39:27.658619 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:27.658406    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de-metrics-certs podName:d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de nodeName:}" failed. No retries permitted until 2026-07-20 09:39:59.6583874 +0000 UTC m=+66.209841860 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de-metrics-certs") pod "network-metrics-daemon-xdlfl" (UID: "d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 20 09:39:27.658619 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:27.658437    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f44bf7c3-e682-406b-92c7-b8ecfd6505f6-kube-api-access-c5ndp podName:f44bf7c3-e682-406b-92c7-b8ecfd6505f6 nodeName:}" failed. No retries permitted until 2026-07-20 09:39:59.658423476 +0000 UTC m=+66.209877934 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-c5ndp" (UniqueName: "kubernetes.io/projected/f44bf7c3-e682-406b-92c7-b8ecfd6505f6-kube-api-access-c5ndp") pod "network-check-target-9kcgh" (UID: "f44bf7c3-e682-406b-92c7-b8ecfd6505f6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 20 09:39:27.698391 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:27.698364    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mqz47_5d536f0c-af6a-4250-a216-30973ed33030/dns-node-resolver/0.log"
Jul 20 09:39:28.479117 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.479083    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6p7xh_5fffc7f3-af46-4089-9814-a3932da685a8/node-ca/0.log"
Jul 20 09:39:28.790085 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.790005    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-134.ec2.internal" event="NodeReady"
Jul 20 09:39:28.790585 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.790175    2566 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 20 09:39:28.824046 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.824006    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-57589fb79f-2hrv7"]
Jul 20 09:39:28.825828 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.825802    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57589fb79f-2hrv7"
Jul 20 09:39:28.828789 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.828767    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 20 09:39:28.828948 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.828931    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 20 09:39:28.829351 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.829331    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-ppbr5\""
Jul 20 09:39:28.829411 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.829346    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 20 09:39:28.837408 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.837371    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-tt5ld"]
Jul 20 09:39:28.838361 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.838324    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 20 09:39:28.840646 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.840620    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tt5ld"
Jul 20 09:39:28.841784 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.841723    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-57589fb79f-2hrv7"]
Jul 20 09:39:28.843074 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.843048    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-kttx7"]
Jul 20 09:39:28.844416 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.844360    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-v7wvr\""
Jul 20 09:39:28.844537 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.844360    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 20 09:39:28.844537 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.844455    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 20 09:39:28.844727 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.844707    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 20 09:39:28.846273 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.845180    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kttx7"
Jul 20 09:39:28.847247 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.847223    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tt5ld"]
Jul 20 09:39:28.847908 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.847887    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vz98l\""
Jul 20 09:39:28.848017 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.847921    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 20 09:39:28.848087 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.847887    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 20 09:39:28.848444 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.848427    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 20 09:39:28.848687 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.848671    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 20 09:39:28.857225 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.857199    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9fsst"]
Jul 20 09:39:28.859183 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.859145    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9fsst"
Jul 20 09:39:28.862535 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.862494    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 20 09:39:28.862791 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.862774    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 20 09:39:28.863002 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.862963    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6tr7c\""
Jul 20 09:39:28.863169 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.863146    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kttx7"]
Jul 20 09:39:28.872616 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.872584    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9fsst"]
Jul 20 09:39:28.914542 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.914504    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xdlfl"
Jul 20 09:39:28.914542 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.914532    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9kcgh"
Jul 20 09:39:28.914774 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.914535    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lw87"
Jul 20 09:39:28.917361 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.917331    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 20 09:39:28.917520 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.917455    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 20 09:39:28.917607 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.917588    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 20 09:39:28.917669 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.917648    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pgl5c\""
Jul 20 09:39:28.917723 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.917702    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hssrx\""
Jul 20 09:39:28.917774 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.917753    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 20 09:39:28.968292 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.968259    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/efd4b7ad-96ab-4ecf-8082-97598b6fa971-registry-certificates\") pod \"image-registry-57589fb79f-2hrv7\" (UID: \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\") " pod="openshift-image-registry/image-registry-57589fb79f-2hrv7"
Jul 20 09:39:28.968292 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.968301    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/efd4b7ad-96ab-4ecf-8082-97598b6fa971-image-registry-private-configuration\") pod \"image-registry-57589fb79f-2hrv7\" (UID: \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\") " pod="openshift-image-registry/image-registry-57589fb79f-2hrv7"
Jul 20 09:39:28.968550 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.968332    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/efd4b7ad-96ab-4ecf-8082-97598b6fa971-ca-trust-extracted\") pod \"image-registry-57589fb79f-2hrv7\" (UID: \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\") " pod="openshift-image-registry/image-registry-57589fb79f-2hrv7"
Jul 20 09:39:28.968550 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.968372    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5k6d6\" (UniqueName: \"kubernetes.io/projected/76f390a5-3fce-422a-b2af-10096745a490-kube-api-access-5k6d6\") pod \"dns-default-9fsst\" (UID: \"76f390a5-3fce-422a-b2af-10096745a490\") " pod="openshift-dns/dns-default-9fsst"
Jul 20 09:39:28.968550 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.968396    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/76f390a5-3fce-422a-b2af-10096745a490-tmp-dir\") pod \"dns-default-9fsst\" (UID: \"76f390a5-3fce-422a-b2af-10096745a490\") " pod="openshift-dns/dns-default-9fsst"
Jul 20 09:39:28.968550 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.968419    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/851041f2-912c-462e-92c1-c2ac3d7022da-data-volume\") pod \"insights-runtime-extractor-kttx7\" (UID: \"851041f2-912c-462e-92c1-c2ac3d7022da\") " pod="openshift-insights/insights-runtime-extractor-kttx7"
Jul 20 09:39:28.968550 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.968444    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkmn5\" (UniqueName: \"kubernetes.io/projected/851041f2-912c-462e-92c1-c2ac3d7022da-kube-api-access-tkmn5\") pod \"insights-runtime-extractor-kttx7\" (UID: \"851041f2-912c-462e-92c1-c2ac3d7022da\") " pod="openshift-insights/insights-runtime-extractor-kttx7"
Jul 20 09:39:28.968550 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.968474    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7pqv\" (UniqueName: \"kubernetes.io/projected/e8908aa8-e9e1-4c71-b064-d230e2dcf434-kube-api-access-v7pqv\") pod \"ingress-canary-tt5ld\" (UID: \"e8908aa8-e9e1-4c71-b064-d230e2dcf434\") " pod="openshift-ingress-canary/ingress-canary-tt5ld"
Jul 20 09:39:28.968550 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.968494    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/efd4b7ad-96ab-4ecf-8082-97598b6fa971-registry-tls\") pod \"image-registry-57589fb79f-2hrv7\" (UID: \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\") " pod="openshift-image-registry/image-registry-57589fb79f-2hrv7"
Jul 20 09:39:28.968550 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.968510    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/76f390a5-3fce-422a-b2af-10096745a490-metrics-tls\") pod \"dns-default-9fsst\" (UID: \"76f390a5-3fce-422a-b2af-10096745a490\") " pod="openshift-dns/dns-default-9fsst"
Jul 20 09:39:28.968550 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.968536    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/76f390a5-3fce-422a-b2af-10096745a490-config-volume\") pod \"dns-default-9fsst\" (UID: \"76f390a5-3fce-422a-b2af-10096745a490\") " pod="openshift-dns/dns-default-9fsst"
Jul 20 09:39:28.969010 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.968558    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/851041f2-912c-462e-92c1-c2ac3d7022da-crio-socket\") pod \"insights-runtime-extractor-kttx7\" (UID: \"851041f2-912c-462e-92c1-c2ac3d7022da\") " pod="openshift-insights/insights-runtime-extractor-kttx7"
Jul 20 09:39:28.969010 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.968611    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/851041f2-912c-462e-92c1-c2ac3d7022da-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kttx7\" (UID: \"851041f2-912c-462e-92c1-c2ac3d7022da\") " pod="openshift-insights/insights-runtime-extractor-kttx7"
Jul 20 09:39:28.969010 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.968704    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/efd4b7ad-96ab-4ecf-8082-97598b6fa971-installation-pull-secrets\") pod \"image-registry-57589fb79f-2hrv7\" (UID: \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\") " pod="openshift-image-registry/image-registry-57589fb79f-2hrv7"
Jul 20 09:39:28.969010 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.968730    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/efd4b7ad-96ab-4ecf-8082-97598b6fa971-bound-sa-token\") pod \"image-registry-57589fb79f-2hrv7\" (UID: \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\") " pod="openshift-image-registry/image-registry-57589fb79f-2hrv7"
Jul 20 09:39:28.969010 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.968756    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/efd4b7ad-96ab-4ecf-8082-97598b6fa971-trusted-ca\") pod \"image-registry-57589fb79f-2hrv7\" (UID: \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\") " pod="openshift-image-registry/image-registry-57589fb79f-2hrv7"
Jul 20 09:39:28.969010 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.968771    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpf2q\" (UniqueName: \"kubernetes.io/projected/efd4b7ad-96ab-4ecf-8082-97598b6fa971-kube-api-access-bpf2q\") pod \"image-registry-57589fb79f-2hrv7\" (UID: \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\") " pod="openshift-image-registry/image-registry-57589fb79f-2hrv7"
Jul 20 09:39:28.969010 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.968794    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e8908aa8-e9e1-4c71-b064-d230e2dcf434-cert\") pod \"ingress-canary-tt5ld\" (UID: \"e8908aa8-e9e1-4c71-b064-d230e2dcf434\") " pod="openshift-ingress-canary/ingress-canary-tt5ld"
Jul 20 09:39:28.969010 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:28.968823    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/851041f2-912c-462e-92c1-c2ac3d7022da-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kttx7\" (UID: \"851041f2-912c-462e-92c1-c2ac3d7022da\") " pod="openshift-insights/insights-runtime-extractor-kttx7"
Jul 20 09:39:29.069651 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.069562    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/76f390a5-3fce-422a-b2af-10096745a490-config-volume\") pod \"dns-default-9fsst\" (UID: \"76f390a5-3fce-422a-b2af-10096745a490\") " pod="openshift-dns/dns-default-9fsst"
Jul 20 09:39:29.069651 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.069602    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/851041f2-912c-462e-92c1-c2ac3d7022da-crio-socket\") pod \"insights-runtime-extractor-kttx7\" (UID: \"851041f2-912c-462e-92c1-c2ac3d7022da\") " pod="openshift-insights/insights-runtime-extractor-kttx7"
Jul 20 09:39:29.069651 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.069618    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/851041f2-912c-462e-92c1-c2ac3d7022da-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kttx7\" (UID: \"851041f2-912c-462e-92c1-c2ac3d7022da\") " pod="openshift-insights/insights-runtime-extractor-kttx7"
Jul 20 09:39:29.069651 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.069637    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/efd4b7ad-96ab-4ecf-8082-97598b6fa971-installation-pull-secrets\") pod \"image-registry-57589fb79f-2hrv7\" (UID: \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\") " pod="openshift-image-registry/image-registry-57589fb79f-2hrv7"
Jul 20 09:39:29.069651 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.069652    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/efd4b7ad-96ab-4ecf-8082-97598b6fa971-bound-sa-token\") pod \"image-registry-57589fb79f-2hrv7\" (UID: \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\") " pod="openshift-image-registry/image-registry-57589fb79f-2hrv7"
Jul 20 09:39:29.070034 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.069669    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/efd4b7ad-96ab-4ecf-8082-97598b6fa971-trusted-ca\") pod \"image-registry-57589fb79f-2hrv7\" (UID: \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\") " pod="openshift-image-registry/image-registry-57589fb79f-2hrv7"
Jul 20 09:39:29.070034 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:29.069719    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 20 09:39:29.070034 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:29.069828    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/851041f2-912c-462e-92c1-c2ac3d7022da-kube-rbac-proxy-cm podName:851041f2-912c-462e-92c1-c2ac3d7022da nodeName:}" failed. No retries permitted until 2026-07-20 09:39:29.569804227 +0000 UTC m=+36.121258685 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/851041f2-912c-462e-92c1-c2ac3d7022da-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kttx7" (UID: "851041f2-912c-462e-92c1-c2ac3d7022da") : configmap "kube-rbac-proxy" not found
Jul 20 09:39:29.070034 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.069721    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpf2q\" (UniqueName: \"kubernetes.io/projected/efd4b7ad-96ab-4ecf-8082-97598b6fa971-kube-api-access-bpf2q\") pod \"image-registry-57589fb79f-2hrv7\" (UID: \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\") " pod="openshift-image-registry/image-registry-57589fb79f-2hrv7"
Jul 20 09:39:29.070034 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.069947    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e8908aa8-e9e1-4c71-b064-d230e2dcf434-cert\") pod \"ingress-canary-tt5ld\" (UID: \"e8908aa8-e9e1-4c71-b064-d230e2dcf434\") " pod="openshift-ingress-canary/ingress-canary-tt5ld"
Jul 20 09:39:29.070034 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.070001    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/851041f2-912c-462e-92c1-c2ac3d7022da-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kttx7\" (UID: \"851041f2-912c-462e-92c1-c2ac3d7022da\") " pod="openshift-insights/insights-runtime-extractor-kttx7"
Jul 20 09:39:29.070397 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.070089    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/efd4b7ad-96ab-4ecf-8082-97598b6fa971-registry-certificates\") pod \"image-registry-57589fb79f-2hrv7\" (UID: \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\") " pod="openshift-image-registry/image-registry-57589fb79f-2hrv7"
Jul 20 09:39:29.070397 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.070123    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/efd4b7ad-96ab-4ecf-8082-97598b6fa971-image-registry-private-configuration\") pod \"image-registry-57589fb79f-2hrv7\" (UID: \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\") " pod="openshift-image-registry/image-registry-57589fb79f-2hrv7"
Jul 20 09:39:29.070397 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.070152    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/efd4b7ad-96ab-4ecf-8082-97598b6fa971-ca-trust-extracted\") pod \"image-registry-57589fb79f-2hrv7\" (UID: \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\") " pod="openshift-image-registry/image-registry-57589fb79f-2hrv7"
Jul 20 09:39:29.070397 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.070195    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5k6d6\" (UniqueName: \"kubernetes.io/projected/76f390a5-3fce-422a-b2af-10096745a490-kube-api-access-5k6d6\") pod \"dns-default-9fsst\" (UID: \"76f390a5-3fce-422a-b2af-10096745a490\") " pod="openshift-dns/dns-default-9fsst"
Jul 20 09:39:29.070397 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.070221    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/76f390a5-3fce-422a-b2af-10096745a490-tmp-dir\") pod \"dns-default-9fsst\" (UID: \"76f390a5-3fce-422a-b2af-10096745a490\") " pod="openshift-dns/dns-default-9fsst"
Jul 20 09:39:29.070397 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.070245    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/851041f2-912c-462e-92c1-c2ac3d7022da-data-volume\") pod \"insights-runtime-extractor-kttx7\" (UID: \"851041f2-912c-462e-92c1-c2ac3d7022da\") " pod="openshift-insights/insights-runtime-extractor-kttx7"
Jul 20 09:39:29.070397 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.070257    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/76f390a5-3fce-422a-b2af-10096745a490-config-volume\") pod \"dns-default-9fsst\" (UID: \"76f390a5-3fce-422a-b2af-10096745a490\") " pod="openshift-dns/dns-default-9fsst"
Jul 20 09:39:29.070397 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.070268    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkmn5\" (UniqueName: \"kubernetes.io/projected/851041f2-912c-462e-92c1-c2ac3d7022da-kube-api-access-tkmn5\") pod \"insights-runtime-extractor-kttx7\" (UID: \"851041f2-912c-462e-92c1-c2ac3d7022da\") " pod="openshift-insights/insights-runtime-extractor-kttx7"
Jul 20 09:39:29.070397 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.070325    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7pqv\" (UniqueName: \"kubernetes.io/projected/e8908aa8-e9e1-4c71-b064-d230e2dcf434-kube-api-access-v7pqv\") pod \"ingress-canary-tt5ld\" (UID: \"e8908aa8-e9e1-4c71-b064-d230e2dcf434\") " pod="openshift-ingress-canary/ingress-canary-tt5ld"
Jul 20 09:39:29.070397 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.070359    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/efd4b7ad-96ab-4ecf-8082-97598b6fa971-registry-tls\") pod \"image-registry-57589fb79f-2hrv7\" (UID: \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\") " pod="openshift-image-registry/image-registry-57589fb79f-2hrv7"
Jul 20 09:39:29.070397 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.070386    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/76f390a5-3fce-422a-b2af-10096745a490-metrics-tls\") pod \"dns-default-9fsst\" (UID: \"76f390a5-3fce-422a-b2af-10096745a490\") " pod="openshift-dns/dns-default-9fsst"
Jul 20 09:39:29.070901 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:29.070523    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 09:39:29.070901 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:29.070568    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76f390a5-3fce-422a-b2af-10096745a490-metrics-tls podName:76f390a5-3fce-422a-b2af-10096745a490 nodeName:}" failed. No retries permitted until 2026-07-20 09:39:29.570553514 +0000 UTC m=+36.122007973 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/76f390a5-3fce-422a-b2af-10096745a490-metrics-tls") pod "dns-default-9fsst" (UID: "76f390a5-3fce-422a-b2af-10096745a490") : secret "dns-default-metrics-tls" not found
Jul 20 09:39:29.070901 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.069896    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/851041f2-912c-462e-92c1-c2ac3d7022da-crio-socket\") pod \"insights-runtime-extractor-kttx7\" (UID: \"851041f2-912c-462e-92c1-c2ac3d7022da\") " pod="openshift-insights/insights-runtime-extractor-kttx7"
Jul 20 09:39:29.070901 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:29.070697    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 09:39:29.070901 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.070705    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/efd4b7ad-96ab-4ecf-8082-97598b6fa971-ca-trust-extracted\") pod \"image-registry-57589fb79f-2hrv7\" (UID: \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\") " pod="openshift-image-registry/image-registry-57589fb79f-2hrv7"
Jul 20 09:39:29.070901 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:29.070723    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 20 09:39:29.070901 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:29.070735    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57589fb79f-2hrv7: secret "image-registry-tls" not found
Jul 20 09:39:29.070901 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.070747    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/efd4b7ad-96ab-4ecf-8082-97598b6fa971-trusted-ca\") pod \"image-registry-57589fb79f-2hrv7\" (UID: \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\") " pod="openshift-image-registry/image-registry-57589fb79f-2hrv7"
Jul 20 09:39:29.070901 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:29.070769    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/851041f2-912c-462e-92c1-c2ac3d7022da-insights-runtime-extractor-tls podName:851041f2-912c-462e-92c1-c2ac3d7022da nodeName:}" failed. No retries permitted until 2026-07-20 09:39:29.570752107 +0000 UTC m=+36.122206553 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/851041f2-912c-462e-92c1-c2ac3d7022da-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kttx7" (UID: "851041f2-912c-462e-92c1-c2ac3d7022da") : secret "insights-runtime-extractor-tls" not found
Jul 20 09:39:29.070901 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:29.070788    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/efd4b7ad-96ab-4ecf-8082-97598b6fa971-registry-tls podName:efd4b7ad-96ab-4ecf-8082-97598b6fa971 nodeName:}" failed. No retries permitted until 2026-07-20 09:39:29.570777978 +0000 UTC m=+36.122232426 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/efd4b7ad-96ab-4ecf-8082-97598b6fa971-registry-tls") pod "image-registry-57589fb79f-2hrv7" (UID: "efd4b7ad-96ab-4ecf-8082-97598b6fa971") : secret "image-registry-tls" not found
Jul 20 09:39:29.071278 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.070997    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/76f390a5-3fce-422a-b2af-10096745a490-tmp-dir\") pod \"dns-default-9fsst\" (UID: \"76f390a5-3fce-422a-b2af-10096745a490\") " pod="openshift-dns/dns-default-9fsst"
Jul 20 09:39:29.071278 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.071046    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/851041f2-912c-462e-92c1-c2ac3d7022da-data-volume\") pod \"insights-runtime-extractor-kttx7\" (UID: \"851041f2-912c-462e-92c1-c2ac3d7022da\") " pod="openshift-insights/insights-runtime-extractor-kttx7"
Jul 20 09:39:29.071278 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.071185    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/efd4b7ad-96ab-4ecf-8082-97598b6fa971-registry-certificates\") pod \"image-registry-57589fb79f-2hrv7\" (UID: \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\") " pod="openshift-image-registry/image-registry-57589fb79f-2hrv7"
Jul 20 09:39:29.074628 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.074594    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/efd4b7ad-96ab-4ecf-8082-97598b6fa971-image-registry-private-configuration\") pod \"image-registry-57589fb79f-2hrv7\" (UID: \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\") " pod="openshift-image-registry/image-registry-57589fb79f-2hrv7"
Jul 20 09:39:29.074831 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.074811    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/efd4b7ad-96ab-4ecf-8082-97598b6fa971-installation-pull-secrets\") pod \"image-registry-57589fb79f-2hrv7\" (UID: \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\") " pod="openshift-image-registry/image-registry-57589fb79f-2hrv7"
Jul 20 09:39:29.075043 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.075018    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e8908aa8-e9e1-4c71-b064-d230e2dcf434-cert\") pod \"ingress-canary-tt5ld\" (UID: \"e8908aa8-e9e1-4c71-b064-d230e2dcf434\") " pod="openshift-ingress-canary/ingress-canary-tt5ld"
Jul 20 09:39:29.080036 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.080004    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkmn5\" (UniqueName: \"kubernetes.io/projected/851041f2-912c-462e-92c1-c2ac3d7022da-kube-api-access-tkmn5\") pod \"insights-runtime-extractor-kttx7\" (UID: \"851041f2-912c-462e-92c1-c2ac3d7022da\") " pod="openshift-insights/insights-runtime-extractor-kttx7"
Jul 20 09:39:29.080291 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.080096    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpf2q\" (UniqueName: \"kubernetes.io/projected/efd4b7ad-96ab-4ecf-8082-97598b6fa971-kube-api-access-bpf2q\") pod \"image-registry-57589fb79f-2hrv7\" (UID: \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\") " pod="openshift-image-registry/image-registry-57589fb79f-2hrv7"
Jul 20 09:39:29.080491 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.080468    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5k6d6\" (UniqueName: \"kubernetes.io/projected/76f390a5-3fce-422a-b2af-10096745a490-kube-api-access-5k6d6\") pod \"dns-default-9fsst\" (UID: \"76f390a5-3fce-422a-b2af-10096745a490\") " pod="openshift-dns/dns-default-9fsst"
Jul 20 09:39:29.080670 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.080620    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7pqv\" (UniqueName: \"kubernetes.io/projected/e8908aa8-e9e1-4c71-b064-d230e2dcf434-kube-api-access-v7pqv\") pod \"ingress-canary-tt5ld\" (UID: \"e8908aa8-e9e1-4c71-b064-d230e2dcf434\") " pod="openshift-ingress-canary/ingress-canary-tt5ld"
Jul 20 09:39:29.081528 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.081504    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/efd4b7ad-96ab-4ecf-8082-97598b6fa971-bound-sa-token\") pod \"image-registry-57589fb79f-2hrv7\" (UID: \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\") " pod="openshift-image-registry/image-registry-57589fb79f-2hrv7"
Jul 20 09:39:29.156391 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.156353    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tt5ld"
Jul 20 09:39:29.297650 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.297615    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-bvb2x"]
Jul 20 09:39:29.300361 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.300331    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-bvb2x"
Jul 20 09:39:29.302822 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.302798    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 20 09:39:29.302822 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.302809    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 20 09:39:29.304716 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.303629    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 20 09:39:29.304716 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.303740    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 20 09:39:29.304716 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.303631    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-clkgk\""
Jul 20 09:39:29.309108 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.309080    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-bvb2x"]
Jul 20 09:39:29.321673 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.321596    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tt5ld"]
Jul 20 09:39:29.326229 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:39:29.325962    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8908aa8_e9e1_4c71_b064_d230e2dcf434.slice/crio-22425041c0115036c0232efcce647ae0d24418eb2d1df27ceb1d4ae82624f76a WatchSource:0}: Error finding container 22425041c0115036c0232efcce647ae0d24418eb2d1df27ceb1d4ae82624f76a: Status 404 returned error can't find the container with id 22425041c0115036c0232efcce647ae0d24418eb2d1df27ceb1d4ae82624f76a
Jul 20 09:39:29.373940 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.373901    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hghqd\" (UniqueName: \"kubernetes.io/projected/40ffebfe-8f50-4bae-a063-138c58d741cc-kube-api-access-hghqd\") pod \"service-ca-58598695f7-bvb2x\" (UID: \"40ffebfe-8f50-4bae-a063-138c58d741cc\") " pod="openshift-service-ca/service-ca-58598695f7-bvb2x"
Jul 20 09:39:29.374142 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.374059    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/40ffebfe-8f50-4bae-a063-138c58d741cc-signing-key\") pod \"service-ca-58598695f7-bvb2x\" (UID: \"40ffebfe-8f50-4bae-a063-138c58d741cc\") " pod="openshift-service-ca/service-ca-58598695f7-bvb2x"
Jul 20 09:39:29.374142 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.374120    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/40ffebfe-8f50-4bae-a063-138c58d741cc-signing-cabundle\") pod \"service-ca-58598695f7-bvb2x\" (UID: \"40ffebfe-8f50-4bae-a063-138c58d741cc\") " pod="openshift-service-ca/service-ca-58598695f7-bvb2x"
Jul 20 09:39:29.474628 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.474591    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/40ffebfe-8f50-4bae-a063-138c58d741cc-signing-key\") pod \"service-ca-58598695f7-bvb2x\" (UID: \"40ffebfe-8f50-4bae-a063-138c58d741cc\") " pod="openshift-service-ca/service-ca-58598695f7-bvb2x"
Jul 20 09:39:29.474628 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.474637    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/40ffebfe-8f50-4bae-a063-138c58d741cc-signing-cabundle\") pod \"service-ca-58598695f7-bvb2x\" (UID: \"40ffebfe-8f50-4bae-a063-138c58d741cc\") " pod="openshift-service-ca/service-ca-58598695f7-bvb2x"
Jul 20 09:39:29.474909 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.474692    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hghqd\" (UniqueName: \"kubernetes.io/projected/40ffebfe-8f50-4bae-a063-138c58d741cc-kube-api-access-hghqd\") pod \"service-ca-58598695f7-bvb2x\" (UID: \"40ffebfe-8f50-4bae-a063-138c58d741cc\") " pod="openshift-service-ca/service-ca-58598695f7-bvb2x"
Jul 20 09:39:29.475486 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.475456    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/40ffebfe-8f50-4bae-a063-138c58d741cc-signing-cabundle\") pod \"service-ca-58598695f7-bvb2x\" (UID: \"40ffebfe-8f50-4bae-a063-138c58d741cc\") " pod="openshift-service-ca/service-ca-58598695f7-bvb2x"
Jul 20 09:39:29.478619 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.478580    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/40ffebfe-8f50-4bae-a063-138c58d741cc-signing-key\") pod \"service-ca-58598695f7-bvb2x\" (UID: \"40ffebfe-8f50-4bae-a063-138c58d741cc\") " pod="openshift-service-ca/service-ca-58598695f7-bvb2x"
Jul 20 09:39:29.483481 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.483457    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hghqd\" (UniqueName: \"kubernetes.io/projected/40ffebfe-8f50-4bae-a063-138c58d741cc-kube-api-access-hghqd\") pod \"service-ca-58598695f7-bvb2x\" (UID: \"40ffebfe-8f50-4bae-a063-138c58d741cc\") " pod="openshift-service-ca/service-ca-58598695f7-bvb2x"
Jul 20 09:39:29.575545 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.575411    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/efd4b7ad-96ab-4ecf-8082-97598b6fa971-registry-tls\") pod \"image-registry-57589fb79f-2hrv7\" (UID: \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\") " pod="openshift-image-registry/image-registry-57589fb79f-2hrv7"
Jul 20 09:39:29.575545 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.575447    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/76f390a5-3fce-422a-b2af-10096745a490-metrics-tls\") pod \"dns-default-9fsst\" (UID: \"76f390a5-3fce-422a-b2af-10096745a490\") " pod="openshift-dns/dns-default-9fsst"
Jul 20 09:39:29.575545 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.575498    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/851041f2-912c-462e-92c1-c2ac3d7022da-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kttx7\" (UID: \"851041f2-912c-462e-92c1-c2ac3d7022da\") " pod="openshift-insights/insights-runtime-extractor-kttx7"
Jul 20 09:39:29.575545 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.575544    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/851041f2-912c-462e-92c1-c2ac3d7022da-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kttx7\" (UID: \"851041f2-912c-462e-92c1-c2ac3d7022da\") " pod="openshift-insights/insights-runtime-extractor-kttx7"
Jul 20 09:39:29.575865 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:29.575571    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 20 09:39:29.575865 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:29.575588    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57589fb79f-2hrv7: secret "image-registry-tls" not found
Jul 20 09:39:29.575865 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:29.575618    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 20 09:39:29.575865 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:29.575645    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/efd4b7ad-96ab-4ecf-8082-97598b6fa971-registry-tls podName:efd4b7ad-96ab-4ecf-8082-97598b6fa971 nodeName:}" failed. No retries permitted until 2026-07-20 09:39:30.575627768 +0000 UTC m=+37.127082228 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/efd4b7ad-96ab-4ecf-8082-97598b6fa971-registry-tls") pod "image-registry-57589fb79f-2hrv7" (UID: "efd4b7ad-96ab-4ecf-8082-97598b6fa971") : secret "image-registry-tls" not found
Jul 20 09:39:29.575865 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:29.575662    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 09:39:29.575865 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:29.575687    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/851041f2-912c-462e-92c1-c2ac3d7022da-kube-rbac-proxy-cm podName:851041f2-912c-462e-92c1-c2ac3d7022da nodeName:}" failed. No retries permitted until 2026-07-20 09:39:30.575666173 +0000 UTC m=+37.127120615 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/851041f2-912c-462e-92c1-c2ac3d7022da-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kttx7" (UID: "851041f2-912c-462e-92c1-c2ac3d7022da") : configmap "kube-rbac-proxy" not found
Jul 20 09:39:29.575865 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:29.575619    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 09:39:29.575865 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:29.575709    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/851041f2-912c-462e-92c1-c2ac3d7022da-insights-runtime-extractor-tls podName:851041f2-912c-462e-92c1-c2ac3d7022da nodeName:}" failed. No retries permitted until 2026-07-20 09:39:30.575698277 +0000 UTC m=+37.127152721 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/851041f2-912c-462e-92c1-c2ac3d7022da-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kttx7" (UID: "851041f2-912c-462e-92c1-c2ac3d7022da") : secret "insights-runtime-extractor-tls" not found
Jul 20 09:39:29.575865 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:29.575730    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76f390a5-3fce-422a-b2af-10096745a490-metrics-tls podName:76f390a5-3fce-422a-b2af-10096745a490 nodeName:}" failed. No retries permitted until 2026-07-20 09:39:30.57571989 +0000 UTC m=+37.127174332 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/76f390a5-3fce-422a-b2af-10096745a490-metrics-tls") pod "dns-default-9fsst" (UID: "76f390a5-3fce-422a-b2af-10096745a490") : secret "dns-default-metrics-tls" not found
Jul 20 09:39:29.614607 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:29.614569    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-bvb2x"
Jul 20 09:39:30.168271 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:30.168233    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tt5ld" event={"ID":"e8908aa8-e9e1-4c71-b064-d230e2dcf434","Type":"ContainerStarted","Data":"22425041c0115036c0232efcce647ae0d24418eb2d1df27ceb1d4ae82624f76a"}
Jul 20 09:39:30.585409 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:30.585323    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/efd4b7ad-96ab-4ecf-8082-97598b6fa971-registry-tls\") pod \"image-registry-57589fb79f-2hrv7\" (UID: \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\") " pod="openshift-image-registry/image-registry-57589fb79f-2hrv7"
Jul 20 09:39:30.585409 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:30.585365    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/76f390a5-3fce-422a-b2af-10096745a490-metrics-tls\") pod \"dns-default-9fsst\" (UID: \"76f390a5-3fce-422a-b2af-10096745a490\") " pod="openshift-dns/dns-default-9fsst"
Jul 20 09:39:30.585409 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:30.585398    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/851041f2-912c-462e-92c1-c2ac3d7022da-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kttx7\" (UID: \"851041f2-912c-462e-92c1-c2ac3d7022da\") " pod="openshift-insights/insights-runtime-extractor-kttx7"
Jul 20 09:39:30.585673 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:30.585446    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/851041f2-912c-462e-92c1-c2ac3d7022da-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kttx7\" (UID: \"851041f2-912c-462e-92c1-c2ac3d7022da\") " pod="openshift-insights/insights-runtime-extractor-kttx7"
Jul 20 09:39:30.585673 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:30.585506    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 09:39:30.585673 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:30.585574    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76f390a5-3fce-422a-b2af-10096745a490-metrics-tls podName:76f390a5-3fce-422a-b2af-10096745a490 nodeName:}" failed. No retries permitted until 2026-07-20 09:39:32.585555033 +0000 UTC m=+39.137009501 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/76f390a5-3fce-422a-b2af-10096745a490-metrics-tls") pod "dns-default-9fsst" (UID: "76f390a5-3fce-422a-b2af-10096745a490") : secret "dns-default-metrics-tls" not found
Jul 20 09:39:30.585673 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:30.585584    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 09:39:30.585673 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:30.585617    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/851041f2-912c-462e-92c1-c2ac3d7022da-insights-runtime-extractor-tls podName:851041f2-912c-462e-92c1-c2ac3d7022da nodeName:}" failed. No retries permitted until 2026-07-20 09:39:32.58560738 +0000 UTC m=+39.137061821 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/851041f2-912c-462e-92c1-c2ac3d7022da-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kttx7" (UID: "851041f2-912c-462e-92c1-c2ac3d7022da") : secret "insights-runtime-extractor-tls" not found
Jul 20 09:39:30.585673 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:30.585504    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 20 09:39:30.585673 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:30.585633    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57589fb79f-2hrv7: secret "image-registry-tls" not found
Jul 20 09:39:30.585673 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:30.585662    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/efd4b7ad-96ab-4ecf-8082-97598b6fa971-registry-tls podName:efd4b7ad-96ab-4ecf-8082-97598b6fa971 nodeName:}" failed. No retries permitted until 2026-07-20 09:39:32.585653383 +0000 UTC m=+39.137107824 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/efd4b7ad-96ab-4ecf-8082-97598b6fa971-registry-tls") pod "image-registry-57589fb79f-2hrv7" (UID: "efd4b7ad-96ab-4ecf-8082-97598b6fa971") : secret "image-registry-tls" not found
Jul 20 09:39:30.586612 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:30.586586    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/851041f2-912c-462e-92c1-c2ac3d7022da-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kttx7\" (UID: \"851041f2-912c-462e-92c1-c2ac3d7022da\") " pod="openshift-insights/insights-runtime-extractor-kttx7"
Jul 20 09:39:31.014604 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:31.014570    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-bvb2x"]
Jul 20 09:39:31.020109 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:39:31.020074    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40ffebfe_8f50_4bae_a063_138c58d741cc.slice/crio-ca00093ac6fe4774ae555c301736ac4dc16f7ddb72a9590360691391be2071d7 WatchSource:0}: Error finding container ca00093ac6fe4774ae555c301736ac4dc16f7ddb72a9590360691391be2071d7: Status 404 returned error can't find the container with id ca00093ac6fe4774ae555c301736ac4dc16f7ddb72a9590360691391be2071d7
Jul 20 09:39:31.172071 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:31.172035    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-bvb2x" event={"ID":"40ffebfe-8f50-4bae-a063-138c58d741cc","Type":"ContainerStarted","Data":"ca00093ac6fe4774ae555c301736ac4dc16f7ddb72a9590360691391be2071d7"}
Jul 20 09:39:31.175536 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:31.175500    2566 generic.go:356] "Generic (PLEG): container finished" podID="4727c87a-fe93-4101-adcf-bc4795bfe808" containerID="193a3deed6611ccc1c36a0e48b0670bcad31f2bb07efb72e396ac7cf91bc17ee" exitCode=0
Jul 20 09:39:31.175677 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:31.175547    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gxttc" event={"ID":"4727c87a-fe93-4101-adcf-bc4795bfe808","Type":"ContainerDied","Data":"193a3deed6611ccc1c36a0e48b0670bcad31f2bb07efb72e396ac7cf91bc17ee"}
Jul 20 09:39:32.188553 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:32.188458    2566 generic.go:356] "Generic (PLEG): container finished" podID="4727c87a-fe93-4101-adcf-bc4795bfe808" containerID="5cff1a16126dc9da22bd98d1ea1983247277a5aac6dba841d8a1f4319bf1423c" exitCode=0
Jul 20 09:39:32.188553 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:32.188540    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gxttc" event={"ID":"4727c87a-fe93-4101-adcf-bc4795bfe808","Type":"ContainerDied","Data":"5cff1a16126dc9da22bd98d1ea1983247277a5aac6dba841d8a1f4319bf1423c"}
Jul 20 09:39:32.606084 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:32.606036    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/efd4b7ad-96ab-4ecf-8082-97598b6fa971-registry-tls\") pod \"image-registry-57589fb79f-2hrv7\" (UID: \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\") " pod="openshift-image-registry/image-registry-57589fb79f-2hrv7"
Jul 20 09:39:32.606084 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:32.606093    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/76f390a5-3fce-422a-b2af-10096745a490-metrics-tls\") pod \"dns-default-9fsst\" (UID: \"76f390a5-3fce-422a-b2af-10096745a490\") " pod="openshift-dns/dns-default-9fsst"
Jul 20 09:39:32.606351 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:32.606157    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/851041f2-912c-462e-92c1-c2ac3d7022da-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kttx7\" (UID: \"851041f2-912c-462e-92c1-c2ac3d7022da\") " pod="openshift-insights/insights-runtime-extractor-kttx7"
Jul 20 09:39:32.606351 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:32.606214    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 20 09:39:32.606351 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:32.606236    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57589fb79f-2hrv7: secret "image-registry-tls" not found
Jul 20 09:39:32.606351 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:32.606252    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 09:39:32.606351 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:32.606272    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 09:39:32.606351 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:32.606314    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/851041f2-912c-462e-92c1-c2ac3d7022da-insights-runtime-extractor-tls podName:851041f2-912c-462e-92c1-c2ac3d7022da nodeName:}" failed. No retries permitted until 2026-07-20 09:39:36.60629339 +0000 UTC m=+43.157747831 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/851041f2-912c-462e-92c1-c2ac3d7022da-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kttx7" (UID: "851041f2-912c-462e-92c1-c2ac3d7022da") : secret "insights-runtime-extractor-tls" not found
Jul 20 09:39:32.606351 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:32.606328    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/efd4b7ad-96ab-4ecf-8082-97598b6fa971-registry-tls podName:efd4b7ad-96ab-4ecf-8082-97598b6fa971 nodeName:}" failed. No retries permitted until 2026-07-20 09:39:36.606322379 +0000 UTC m=+43.157776821 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/efd4b7ad-96ab-4ecf-8082-97598b6fa971-registry-tls") pod "image-registry-57589fb79f-2hrv7" (UID: "efd4b7ad-96ab-4ecf-8082-97598b6fa971") : secret "image-registry-tls" not found
Jul 20 09:39:32.606351 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:32.606338    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76f390a5-3fce-422a-b2af-10096745a490-metrics-tls podName:76f390a5-3fce-422a-b2af-10096745a490 nodeName:}" failed. No retries permitted until 2026-07-20 09:39:36.60633246 +0000 UTC m=+43.157786902 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/76f390a5-3fce-422a-b2af-10096745a490-metrics-tls") pod "dns-default-9fsst" (UID: "76f390a5-3fce-422a-b2af-10096745a490") : secret "dns-default-metrics-tls" not found
Jul 20 09:39:34.195616 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:34.195578    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tt5ld" event={"ID":"e8908aa8-e9e1-4c71-b064-d230e2dcf434","Type":"ContainerStarted","Data":"6d232e53aef46e86091d7b545416d13aea78ab35b94cf5d01cc9b3e24b4c9704"}
Jul 20 09:39:34.202740 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:34.202710    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gxttc" event={"ID":"4727c87a-fe93-4101-adcf-bc4795bfe808","Type":"ContainerStarted","Data":"afb506ade5424792d5a10263d5e8b2bfd2c624771c0628fea6fe4a0af88871fd"}
Jul 20 09:39:34.204150 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:34.204124    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-bvb2x" event={"ID":"40ffebfe-8f50-4bae-a063-138c58d741cc","Type":"ContainerStarted","Data":"8ef0db5eba82495564bfae5fd25841c773bd55d7201926eef5ab79d40782e7a1"}
Jul 20 09:39:34.215157 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:34.215106    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-tt5ld" podStartSLOduration=1.919256321 podStartE2EDuration="6.215091649s" totalImagesPullingTime="4.295835328s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 09:39:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:39:34.214718829 +0000 UTC m=+40.766173294" watchObservedRunningTime="2026-07-20 09:39:34.215091649 +0000 UTC m=+40.766546112"
Jul 20 09:39:34.237129 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:34.237073    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-gxttc" podStartSLOduration=10.838099797 podStartE2EDuration="41.237057452s" totalImagesPullingTime="30.398957655s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 09:38:53 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:39:34.236880899 +0000 UTC m=+40.788335364" watchObservedRunningTime="2026-07-20 09:39:34.237057452 +0000 UTC m=+40.788511916"
Jul 20 09:39:34.252813 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:34.252759    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-bvb2x" podStartSLOduration=2.646099181 podStartE2EDuration="5.252740583s" totalImagesPullingTime="2.606641402s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 09:39:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:39:34.252309662 +0000 UTC m=+40.803764126" watchObservedRunningTime="2026-07-20 09:39:34.252740583 +0000 UTC m=+40.804195055"
Jul 20 09:39:36.643389 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:36.643339    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/efd4b7ad-96ab-4ecf-8082-97598b6fa971-registry-tls\") pod \"image-registry-57589fb79f-2hrv7\" (UID: \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\") " pod="openshift-image-registry/image-registry-57589fb79f-2hrv7"
Jul 20 09:39:36.643389 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:36.643394    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/76f390a5-3fce-422a-b2af-10096745a490-metrics-tls\") pod \"dns-default-9fsst\" (UID: \"76f390a5-3fce-422a-b2af-10096745a490\") " pod="openshift-dns/dns-default-9fsst"
Jul 20 09:39:36.643831 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:36.643448    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/851041f2-912c-462e-92c1-c2ac3d7022da-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kttx7\" (UID: \"851041f2-912c-462e-92c1-c2ac3d7022da\") " pod="openshift-insights/insights-runtime-extractor-kttx7"
Jul 20 09:39:36.643831 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:36.643518    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 20 09:39:36.643831 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:36.643541    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57589fb79f-2hrv7: secret "image-registry-tls" not found
Jul 20 09:39:36.643831 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:36.643561    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 20 09:39:36.643831 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:36.643582    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 20 09:39:36.643831 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:36.643606    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/efd4b7ad-96ab-4ecf-8082-97598b6fa971-registry-tls podName:efd4b7ad-96ab-4ecf-8082-97598b6fa971 nodeName:}" failed. No retries permitted until 2026-07-20 09:39:44.643586585 +0000 UTC m=+51.195041032 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/efd4b7ad-96ab-4ecf-8082-97598b6fa971-registry-tls") pod "image-registry-57589fb79f-2hrv7" (UID: "efd4b7ad-96ab-4ecf-8082-97598b6fa971") : secret "image-registry-tls" not found
Jul 20 09:39:36.643831 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:36.643627    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76f390a5-3fce-422a-b2af-10096745a490-metrics-tls podName:76f390a5-3fce-422a-b2af-10096745a490 nodeName:}" failed. No retries permitted until 2026-07-20 09:39:44.643614527 +0000 UTC m=+51.195068969 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/76f390a5-3fce-422a-b2af-10096745a490-metrics-tls") pod "dns-default-9fsst" (UID: "76f390a5-3fce-422a-b2af-10096745a490") : secret "dns-default-metrics-tls" not found
Jul 20 09:39:36.643831 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:39:36.643641    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/851041f2-912c-462e-92c1-c2ac3d7022da-insights-runtime-extractor-tls podName:851041f2-912c-462e-92c1-c2ac3d7022da nodeName:}" failed. No retries permitted until 2026-07-20 09:39:44.643634715 +0000 UTC m=+51.195089157 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/851041f2-912c-462e-92c1-c2ac3d7022da-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kttx7" (UID: "851041f2-912c-462e-92c1-c2ac3d7022da") : secret "insights-runtime-extractor-tls" not found
Jul 20 09:39:44.710666 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:44.710623    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/851041f2-912c-462e-92c1-c2ac3d7022da-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kttx7\" (UID: \"851041f2-912c-462e-92c1-c2ac3d7022da\") " pod="openshift-insights/insights-runtime-extractor-kttx7"
Jul 20 09:39:44.711300 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:44.710727    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/efd4b7ad-96ab-4ecf-8082-97598b6fa971-registry-tls\") pod \"image-registry-57589fb79f-2hrv7\" (UID: \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\") " pod="openshift-image-registry/image-registry-57589fb79f-2hrv7"
Jul 20 09:39:44.711300 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:44.710753    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/76f390a5-3fce-422a-b2af-10096745a490-metrics-tls\") pod \"dns-default-9fsst\" (UID: \"76f390a5-3fce-422a-b2af-10096745a490\") " pod="openshift-dns/dns-default-9fsst"
Jul 20 09:39:44.714263 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:44.714239    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/76f390a5-3fce-422a-b2af-10096745a490-metrics-tls\") pod \"dns-default-9fsst\" (UID: \"76f390a5-3fce-422a-b2af-10096745a490\") " pod="openshift-dns/dns-default-9fsst"
Jul 20 09:39:44.714263 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:44.714253    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/851041f2-912c-462e-92c1-c2ac3d7022da-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kttx7\" (UID: \"851041f2-912c-462e-92c1-c2ac3d7022da\") " pod="openshift-insights/insights-runtime-extractor-kttx7"
Jul 20 09:39:44.714439 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:44.714297    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/efd4b7ad-96ab-4ecf-8082-97598b6fa971-registry-tls\") pod \"image-registry-57589fb79f-2hrv7\" (UID: \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\") " pod="openshift-image-registry/image-registry-57589fb79f-2hrv7"
Jul 20 09:39:44.743461 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:44.743410    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57589fb79f-2hrv7"
Jul 20 09:39:44.763682 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:44.763644    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kttx7"
Jul 20 09:39:44.774442 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:44.774400    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9fsst"
Jul 20 09:39:44.920230 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:44.920196    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-57589fb79f-2hrv7"]
Jul 20 09:39:44.924338 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:39:44.924303    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefd4b7ad_96ab_4ecf_8082_97598b6fa971.slice/crio-0d719befdab083e609c7fd63b06a533cd76dc1663805d3c209c981dcc8aa6436 WatchSource:0}: Error finding container 0d719befdab083e609c7fd63b06a533cd76dc1663805d3c209c981dcc8aa6436: Status 404 returned error can't find the container with id 0d719befdab083e609c7fd63b06a533cd76dc1663805d3c209c981dcc8aa6436
Jul 20 09:39:44.940307 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:44.940268    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kttx7"]
Jul 20 09:39:44.942741 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:39:44.942707    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod851041f2_912c_462e_92c1_c2ac3d7022da.slice/crio-f1f436f792f6dc485348ceed8dc3f3da88bc33d8ca25273757821fedcfea4f2d WatchSource:0}: Error finding container f1f436f792f6dc485348ceed8dc3f3da88bc33d8ca25273757821fedcfea4f2d: Status 404 returned error can't find the container with id f1f436f792f6dc485348ceed8dc3f3da88bc33d8ca25273757821fedcfea4f2d
Jul 20 09:39:44.962715 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:44.962628    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9fsst"]
Jul 20 09:39:44.966083 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:39:44.966057    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76f390a5_3fce_422a_b2af_10096745a490.slice/crio-4fb721d0a5c9c1b200610e189154e2db7a8a29de57417ce1331fd89feeac077a WatchSource:0}: Error finding container 4fb721d0a5c9c1b200610e189154e2db7a8a29de57417ce1331fd89feeac077a: Status 404 returned error can't find the container with id 4fb721d0a5c9c1b200610e189154e2db7a8a29de57417ce1331fd89feeac077a
Jul 20 09:39:45.234462 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:45.234360    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-57589fb79f-2hrv7" event={"ID":"efd4b7ad-96ab-4ecf-8082-97598b6fa971","Type":"ContainerStarted","Data":"f5ce90221a1b864482fb23b6fc237dead7a8cbc2fbaccb1dc71bce09617ccf40"}
Jul 20 09:39:45.234462 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:45.234419    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-57589fb79f-2hrv7" event={"ID":"efd4b7ad-96ab-4ecf-8082-97598b6fa971","Type":"ContainerStarted","Data":"0d719befdab083e609c7fd63b06a533cd76dc1663805d3c209c981dcc8aa6436"}
Jul 20 09:39:45.234671 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:45.234480    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-57589fb79f-2hrv7"
Jul 20 09:39:45.235684 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:45.235643    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9fsst" event={"ID":"76f390a5-3fce-422a-b2af-10096745a490","Type":"ContainerStarted","Data":"4fb721d0a5c9c1b200610e189154e2db7a8a29de57417ce1331fd89feeac077a"}
Jul 20 09:39:45.236875 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:45.236840    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kttx7" event={"ID":"851041f2-912c-462e-92c1-c2ac3d7022da","Type":"ContainerStarted","Data":"f1f436f792f6dc485348ceed8dc3f3da88bc33d8ca25273757821fedcfea4f2d"}
Jul 20 09:39:45.259043 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:45.258988    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-57589fb79f-2hrv7" podStartSLOduration=41.258952795 podStartE2EDuration="41.258952795s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 09:39:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:39:45.258519176 +0000 UTC m=+51.809973662" watchObservedRunningTime="2026-07-20 09:39:45.258952795 +0000 UTC m=+51.810407258"
Jul 20 09:39:51.256867 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:51.256824    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9fsst" event={"ID":"76f390a5-3fce-422a-b2af-10096745a490","Type":"ContainerStarted","Data":"58a93b3d27859b8fa181e57c591edd766df09c3383855d30ec047813fdc18ce6"}
Jul 20 09:39:51.257280 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:51.256891    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9fsst" event={"ID":"76f390a5-3fce-422a-b2af-10096745a490","Type":"ContainerStarted","Data":"453cd4d06bcc2d6e5dd0b11e384bf792683fafc8ca30c2f767e027c2f048ab91"}
Jul 20 09:39:51.257280 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:51.256925    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9fsst"
Jul 20 09:39:51.259008 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:51.258960    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kttx7" event={"ID":"851041f2-912c-462e-92c1-c2ac3d7022da","Type":"ContainerStarted","Data":"aafd2d4013220736f57c75a0109e94304826ce9455d2fa8840448469ef63cf39"}
Jul 20 09:39:51.274778 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:51.274714    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9fsst" podStartSLOduration=18.045548981 podStartE2EDuration="23.274690002s" totalImagesPullingTime="5.229141021s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 09:39:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:39:51.274661041 +0000 UTC m=+57.826115508" watchObservedRunningTime="2026-07-20 09:39:51.274690002 +0000 UTC m=+57.826144461"
Jul 20 09:39:53.027938 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.027890    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57456855cb-4dxkr"]
Jul 20 09:39:53.031469 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.031436    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57456855cb-4dxkr"
Jul 20 09:39:53.035327 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.035297    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 20 09:39:53.037009 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.036656    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-jtdhx\""
Jul 20 09:39:53.037009 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.036680    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 20 09:39:53.037009 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.036691    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 20 09:39:53.037009 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.036691    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 20 09:39:53.043580 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.042341    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c6bc7f849-xxmgg"]
Jul 20 09:39:53.043580 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.045655    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57456855cb-4dxkr"]
Jul 20 09:39:53.043580 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.045765    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c6bc7f849-xxmgg"
Jul 20 09:39:53.048718 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.048693    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 20 09:39:53.055087 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.055060    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b48cd4c44-z4rj4"]
Jul 20 09:39:53.059738 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.058598    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b48cd4c44-z4rj4"
Jul 20 09:39:53.061000 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.060952    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c6bc7f849-xxmgg"]
Jul 20 09:39:53.062113 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.061717    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 20 09:39:53.062113 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.061900    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 20 09:39:53.062113 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.061999    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 20 09:39:53.062113 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.061900    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 20 09:39:53.074151 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.074118    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b48cd4c44-z4rj4"]
Jul 20 09:39:53.086509 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.086472    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/900cb6f0-059f-46fa-96cf-0cafd8182dee-ca\") pod \"cluster-proxy-proxy-agent-7b48cd4c44-z4rj4\" (UID: \"900cb6f0-059f-46fa-96cf-0cafd8182dee\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b48cd4c44-z4rj4"
Jul 20 09:39:53.086679 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.086572    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/4a5a1749-a594-4d29-a3cc-068900e83589-klusterlet-config\") pod \"klusterlet-addon-workmgr-6c6bc7f849-xxmgg\" (UID: \"4a5a1749-a594-4d29-a3cc-068900e83589\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c6bc7f849-xxmgg"
Jul 20 09:39:53.086679 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.086602    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e9ea8f40-f761-45dc-acc1-638ad8015f0d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-57456855cb-4dxkr\" (UID: \"e9ea8f40-f761-45dc-acc1-638ad8015f0d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57456855cb-4dxkr"
Jul 20 09:39:53.086679 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.086624    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gmzp\" (UniqueName: \"kubernetes.io/projected/e9ea8f40-f761-45dc-acc1-638ad8015f0d-kube-api-access-7gmzp\") pod \"managed-serviceaccount-addon-agent-57456855cb-4dxkr\" (UID: \"e9ea8f40-f761-45dc-acc1-638ad8015f0d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57456855cb-4dxkr"
Jul 20 09:39:53.086679 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.086649    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a5a1749-a594-4d29-a3cc-068900e83589-tmp\") pod \"klusterlet-addon-workmgr-6c6bc7f849-xxmgg\" (UID: \"4a5a1749-a594-4d29-a3cc-068900e83589\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c6bc7f849-xxmgg"
Jul 20 09:39:53.086679 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.086666    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wb44m\" (UniqueName: \"kubernetes.io/projected/4a5a1749-a594-4d29-a3cc-068900e83589-kube-api-access-wb44m\") pod \"klusterlet-addon-workmgr-6c6bc7f849-xxmgg\" (UID: \"4a5a1749-a594-4d29-a3cc-068900e83589\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c6bc7f849-xxmgg"
Jul 20 09:39:53.086937 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.086687    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sxm6\" (UniqueName: \"kubernetes.io/projected/900cb6f0-059f-46fa-96cf-0cafd8182dee-kube-api-access-2sxm6\") pod \"cluster-proxy-proxy-agent-7b48cd4c44-z4rj4\" (UID: \"900cb6f0-059f-46fa-96cf-0cafd8182dee\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b48cd4c44-z4rj4"
Jul 20 09:39:53.086937 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.086800    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/900cb6f0-059f-46fa-96cf-0cafd8182dee-hub\") pod \"cluster-proxy-proxy-agent-7b48cd4c44-z4rj4\" (UID: \"900cb6f0-059f-46fa-96cf-0cafd8182dee\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b48cd4c44-z4rj4"
Jul 20 09:39:53.086937 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.086840    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/900cb6f0-059f-46fa-96cf-0cafd8182dee-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7b48cd4c44-z4rj4\" (UID: \"900cb6f0-059f-46fa-96cf-0cafd8182dee\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b48cd4c44-z4rj4"
Jul 20 09:39:53.086937 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.086885    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/900cb6f0-059f-46fa-96cf-0cafd8182dee-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7b48cd4c44-z4rj4\" (UID: \"900cb6f0-059f-46fa-96cf-0cafd8182dee\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b48cd4c44-z4rj4"
Jul 20 09:39:53.087134 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.086941    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/900cb6f0-059f-46fa-96cf-0cafd8182dee-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7b48cd4c44-z4rj4\" (UID: \"900cb6f0-059f-46fa-96cf-0cafd8182dee\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b48cd4c44-z4rj4"
Jul 20 09:39:53.114194 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.114156    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-57589fb79f-2hrv7"]
Jul 20 09:39:53.188108 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.188066    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/4a5a1749-a594-4d29-a3cc-068900e83589-klusterlet-config\") pod \"klusterlet-addon-workmgr-6c6bc7f849-xxmgg\" (UID: \"4a5a1749-a594-4d29-a3cc-068900e83589\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c6bc7f849-xxmgg"
Jul 20 09:39:53.188291 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.188114    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e9ea8f40-f761-45dc-acc1-638ad8015f0d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-57456855cb-4dxkr\" (UID: \"e9ea8f40-f761-45dc-acc1-638ad8015f0d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57456855cb-4dxkr"
Jul 20 09:39:53.188291 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.188250    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gmzp\" (UniqueName: \"kubernetes.io/projected/e9ea8f40-f761-45dc-acc1-638ad8015f0d-kube-api-access-7gmzp\") pod \"managed-serviceaccount-addon-agent-57456855cb-4dxkr\" (UID: \"e9ea8f40-f761-45dc-acc1-638ad8015f0d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57456855cb-4dxkr"
Jul 20 09:39:53.188394 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.188303    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a5a1749-a594-4d29-a3cc-068900e83589-tmp\") pod \"klusterlet-addon-workmgr-6c6bc7f849-xxmgg\" (UID: \"4a5a1749-a594-4d29-a3cc-068900e83589\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c6bc7f849-xxmgg"
Jul 20 09:39:53.188394 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.188338    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wb44m\" (UniqueName: \"kubernetes.io/projected/4a5a1749-a594-4d29-a3cc-068900e83589-kube-api-access-wb44m\") pod \"klusterlet-addon-workmgr-6c6bc7f849-xxmgg\" (UID: \"4a5a1749-a594-4d29-a3cc-068900e83589\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c6bc7f849-xxmgg"
Jul 20 09:39:53.188394 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.188367    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sxm6\" (UniqueName: \"kubernetes.io/projected/900cb6f0-059f-46fa-96cf-0cafd8182dee-kube-api-access-2sxm6\") pod \"cluster-proxy-proxy-agent-7b48cd4c44-z4rj4\" (UID: \"900cb6f0-059f-46fa-96cf-0cafd8182dee\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b48cd4c44-z4rj4"
Jul 20 09:39:53.188542 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.188411    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/900cb6f0-059f-46fa-96cf-0cafd8182dee-hub\") pod \"cluster-proxy-proxy-agent-7b48cd4c44-z4rj4\" (UID: \"900cb6f0-059f-46fa-96cf-0cafd8182dee\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b48cd4c44-z4rj4"
Jul 20 09:39:53.188542 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.188442    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/900cb6f0-059f-46fa-96cf-0cafd8182dee-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7b48cd4c44-z4rj4\" (UID: \"900cb6f0-059f-46fa-96cf-0cafd8182dee\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b48cd4c44-z4rj4"
Jul 20 09:39:53.188542 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.188481    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/900cb6f0-059f-46fa-96cf-0cafd8182dee-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7b48cd4c44-z4rj4\" (UID: \"900cb6f0-059f-46fa-96cf-0cafd8182dee\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b48cd4c44-z4rj4"
Jul 20 09:39:53.188668 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.188540    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/900cb6f0-059f-46fa-96cf-0cafd8182dee-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7b48cd4c44-z4rj4\" (UID: \"900cb6f0-059f-46fa-96cf-0cafd8182dee\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b48cd4c44-z4rj4"
Jul 20 09:39:53.188668 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.188581    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/900cb6f0-059f-46fa-96cf-0cafd8182dee-ca\") pod \"cluster-proxy-proxy-agent-7b48cd4c44-z4rj4\" (UID: \"900cb6f0-059f-46fa-96cf-0cafd8182dee\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b48cd4c44-z4rj4"
Jul 20 09:39:53.189324 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.188883    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a5a1749-a594-4d29-a3cc-068900e83589-tmp\") pod \"klusterlet-addon-workmgr-6c6bc7f849-xxmgg\" (UID: \"4a5a1749-a594-4d29-a3cc-068900e83589\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c6bc7f849-xxmgg"
Jul 20 09:39:53.189697 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.189621    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/900cb6f0-059f-46fa-96cf-0cafd8182dee-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7b48cd4c44-z4rj4\" (UID: \"900cb6f0-059f-46fa-96cf-0cafd8182dee\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b48cd4c44-z4rj4"
Jul 20 09:39:53.191796 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.191766    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/900cb6f0-059f-46fa-96cf-0cafd8182dee-ca\") pod \"cluster-proxy-proxy-agent-7b48cd4c44-z4rj4\" (UID: \"900cb6f0-059f-46fa-96cf-0cafd8182dee\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b48cd4c44-z4rj4"
Jul 20 09:39:53.191905 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.191790    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/900cb6f0-059f-46fa-96cf-0cafd8182dee-hub\") pod \"cluster-proxy-proxy-agent-7b48cd4c44-z4rj4\" (UID: \"900cb6f0-059f-46fa-96cf-0cafd8182dee\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b48cd4c44-z4rj4"
Jul 20 09:39:53.191999 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.191902    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/4a5a1749-a594-4d29-a3cc-068900e83589-klusterlet-config\") pod \"klusterlet-addon-workmgr-6c6bc7f849-xxmgg\" (UID: \"4a5a1749-a594-4d29-a3cc-068900e83589\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c6bc7f849-xxmgg"
Jul 20 09:39:53.192376 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.192349    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e9ea8f40-f761-45dc-acc1-638ad8015f0d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-57456855cb-4dxkr\" (UID: \"e9ea8f40-f761-45dc-acc1-638ad8015f0d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57456855cb-4dxkr"
Jul 20 09:39:53.192480 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.192463    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/900cb6f0-059f-46fa-96cf-0cafd8182dee-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7b48cd4c44-z4rj4\" (UID: \"900cb6f0-059f-46fa-96cf-0cafd8182dee\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b48cd4c44-z4rj4"
Jul 20 09:39:53.192906 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.192880    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/900cb6f0-059f-46fa-96cf-0cafd8182dee-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7b48cd4c44-z4rj4\" (UID: \"900cb6f0-059f-46fa-96cf-0cafd8182dee\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b48cd4c44-z4rj4"
Jul 20 09:39:53.201309 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.201280    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gmzp\" (UniqueName: \"kubernetes.io/projected/e9ea8f40-f761-45dc-acc1-638ad8015f0d-kube-api-access-7gmzp\") pod \"managed-serviceaccount-addon-agent-57456855cb-4dxkr\" (UID: \"e9ea8f40-f761-45dc-acc1-638ad8015f0d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57456855cb-4dxkr"
Jul 20 09:39:53.203653 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.203625    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wb44m\" (UniqueName: \"kubernetes.io/projected/4a5a1749-a594-4d29-a3cc-068900e83589-kube-api-access-wb44m\") pod \"klusterlet-addon-workmgr-6c6bc7f849-xxmgg\" (UID: \"4a5a1749-a594-4d29-a3cc-068900e83589\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c6bc7f849-xxmgg"
Jul 20 09:39:53.204578 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.204553    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sxm6\" (UniqueName: \"kubernetes.io/projected/900cb6f0-059f-46fa-96cf-0cafd8182dee-kube-api-access-2sxm6\") pod \"cluster-proxy-proxy-agent-7b48cd4c44-z4rj4\" (UID: \"900cb6f0-059f-46fa-96cf-0cafd8182dee\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b48cd4c44-z4rj4"
Jul 20 09:39:53.233589 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.233551    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-558df75cb7-vz7f9"]
Jul 20 09:39:53.239028 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.238998    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-558df75cb7-vz7f9"
Jul 20 09:39:53.252921 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.252881    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-558df75cb7-vz7f9"]
Jul 20 09:39:53.289366 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.289281    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/56a1e57c-315d-463e-9e9a-92a3ce040736-trusted-ca\") pod \"image-registry-558df75cb7-vz7f9\" (UID: \"56a1e57c-315d-463e-9e9a-92a3ce040736\") " pod="openshift-image-registry/image-registry-558df75cb7-vz7f9"
Jul 20 09:39:53.289366 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.289357    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/56a1e57c-315d-463e-9e9a-92a3ce040736-bound-sa-token\") pod \"image-registry-558df75cb7-vz7f9\" (UID: \"56a1e57c-315d-463e-9e9a-92a3ce040736\") " pod="openshift-image-registry/image-registry-558df75cb7-vz7f9"
Jul 20 09:39:53.289560 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.289458    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/56a1e57c-315d-463e-9e9a-92a3ce040736-registry-tls\") pod \"image-registry-558df75cb7-vz7f9\" (UID: \"56a1e57c-315d-463e-9e9a-92a3ce040736\") " pod="openshift-image-registry/image-registry-558df75cb7-vz7f9"
Jul 20 09:39:53.289560 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.289484    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/56a1e57c-315d-463e-9e9a-92a3ce040736-installation-pull-secrets\") pod \"image-registry-558df75cb7-vz7f9\" (UID: \"56a1e57c-315d-463e-9e9a-92a3ce040736\") " pod="openshift-image-registry/image-registry-558df75cb7-vz7f9"
Jul 20 09:39:53.289560 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.289520    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/56a1e57c-315d-463e-9e9a-92a3ce040736-ca-trust-extracted\") pod \"image-registry-558df75cb7-vz7f9\" (UID: \"56a1e57c-315d-463e-9e9a-92a3ce040736\") " pod="openshift-image-registry/image-registry-558df75cb7-vz7f9"
Jul 20 09:39:53.289560 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.289539    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmv9m\" (UniqueName: \"kubernetes.io/projected/56a1e57c-315d-463e-9e9a-92a3ce040736-kube-api-access-kmv9m\") pod \"image-registry-558df75cb7-vz7f9\" (UID: \"56a1e57c-315d-463e-9e9a-92a3ce040736\") " pod="openshift-image-registry/image-registry-558df75cb7-vz7f9"
Jul 20 09:39:53.289710 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.289573    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/56a1e57c-315d-463e-9e9a-92a3ce040736-registry-certificates\") pod \"image-registry-558df75cb7-vz7f9\" (UID: \"56a1e57c-315d-463e-9e9a-92a3ce040736\") " pod="openshift-image-registry/image-registry-558df75cb7-vz7f9"
Jul 20 09:39:53.289710 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.289604    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/56a1e57c-315d-463e-9e9a-92a3ce040736-image-registry-private-configuration\") pod \"image-registry-558df75cb7-vz7f9\" (UID: \"56a1e57c-315d-463e-9e9a-92a3ce040736\") " pod="openshift-image-registry/image-registry-558df75cb7-vz7f9"
Jul 20 09:39:53.355202 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.355170    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57456855cb-4dxkr"
Jul 20 09:39:53.369355 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.369321    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c6bc7f849-xxmgg"
Jul 20 09:39:53.379539 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.379506    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b48cd4c44-z4rj4"
Jul 20 09:39:53.390759 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.390725    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/56a1e57c-315d-463e-9e9a-92a3ce040736-bound-sa-token\") pod \"image-registry-558df75cb7-vz7f9\" (UID: \"56a1e57c-315d-463e-9e9a-92a3ce040736\") " pod="openshift-image-registry/image-registry-558df75cb7-vz7f9"
Jul 20 09:39:53.390938 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.390820    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/56a1e57c-315d-463e-9e9a-92a3ce040736-registry-tls\") pod \"image-registry-558df75cb7-vz7f9\" (UID: \"56a1e57c-315d-463e-9e9a-92a3ce040736\") " pod="openshift-image-registry/image-registry-558df75cb7-vz7f9"
Jul 20 09:39:53.390938 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.390849    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/56a1e57c-315d-463e-9e9a-92a3ce040736-installation-pull-secrets\") pod \"image-registry-558df75cb7-vz7f9\" (UID: \"56a1e57c-315d-463e-9e9a-92a3ce040736\") " pod="openshift-image-registry/image-registry-558df75cb7-vz7f9"
Jul 20 09:39:53.390938 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.390880    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/56a1e57c-315d-463e-9e9a-92a3ce040736-ca-trust-extracted\") pod \"image-registry-558df75cb7-vz7f9\" (UID: \"56a1e57c-315d-463e-9e9a-92a3ce040736\") " pod="openshift-image-registry/image-registry-558df75cb7-vz7f9"
Jul 20 09:39:53.390938 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.390906    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmv9m\" (UniqueName: \"kubernetes.io/projected/56a1e57c-315d-463e-9e9a-92a3ce040736-kube-api-access-kmv9m\") pod \"image-registry-558df75cb7-vz7f9\" (UID: \"56a1e57c-315d-463e-9e9a-92a3ce040736\") " pod="openshift-image-registry/image-registry-558df75cb7-vz7f9"
Jul 20 09:39:53.391180 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.391065    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/56a1e57c-315d-463e-9e9a-92a3ce040736-registry-certificates\") pod \"image-registry-558df75cb7-vz7f9\" (UID: \"56a1e57c-315d-463e-9e9a-92a3ce040736\") " pod="openshift-image-registry/image-registry-558df75cb7-vz7f9"
Jul 20 09:39:53.391180 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.391115    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/56a1e57c-315d-463e-9e9a-92a3ce040736-image-registry-private-configuration\") pod \"image-registry-558df75cb7-vz7f9\" (UID: \"56a1e57c-315d-463e-9e9a-92a3ce040736\") " pod="openshift-image-registry/image-registry-558df75cb7-vz7f9"
Jul 20 09:39:53.391180 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.391150    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/56a1e57c-315d-463e-9e9a-92a3ce040736-trusted-ca\") pod \"image-registry-558df75cb7-vz7f9\" (UID: \"56a1e57c-315d-463e-9e9a-92a3ce040736\") " pod="openshift-image-registry/image-registry-558df75cb7-vz7f9"
Jul 20 09:39:53.391364 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.391333    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/56a1e57c-315d-463e-9e9a-92a3ce040736-ca-trust-extracted\") pod \"image-registry-558df75cb7-vz7f9\" (UID: \"56a1e57c-315d-463e-9e9a-92a3ce040736\") " pod="openshift-image-registry/image-registry-558df75cb7-vz7f9"
Jul 20 09:39:53.392355 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.392320    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/56a1e57c-315d-463e-9e9a-92a3ce040736-registry-certificates\") pod \"image-registry-558df75cb7-vz7f9\" (UID: \"56a1e57c-315d-463e-9e9a-92a3ce040736\") " pod="openshift-image-registry/image-registry-558df75cb7-vz7f9"
Jul 20 09:39:53.393247 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.393221    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/56a1e57c-315d-463e-9e9a-92a3ce040736-trusted-ca\") pod \"image-registry-558df75cb7-vz7f9\" (UID: \"56a1e57c-315d-463e-9e9a-92a3ce040736\") " pod="openshift-image-registry/image-registry-558df75cb7-vz7f9"
Jul 20 09:39:53.393889 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.393860    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/56a1e57c-315d-463e-9e9a-92a3ce040736-image-registry-private-configuration\") pod \"image-registry-558df75cb7-vz7f9\" (UID: \"56a1e57c-315d-463e-9e9a-92a3ce040736\") " pod="openshift-image-registry/image-registry-558df75cb7-vz7f9"
Jul 20 09:39:53.394048 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.393886    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/56a1e57c-315d-463e-9e9a-92a3ce040736-registry-tls\") pod \"image-registry-558df75cb7-vz7f9\" (UID: \"56a1e57c-315d-463e-9e9a-92a3ce040736\") " pod="openshift-image-registry/image-registry-558df75cb7-vz7f9"
Jul 20 09:39:53.394048 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.393893    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/56a1e57c-315d-463e-9e9a-92a3ce040736-installation-pull-secrets\") pod \"image-registry-558df75cb7-vz7f9\" (UID: \"56a1e57c-315d-463e-9e9a-92a3ce040736\") " pod="openshift-image-registry/image-registry-558df75cb7-vz7f9"
Jul 20 09:39:53.402491 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.402463    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/56a1e57c-315d-463e-9e9a-92a3ce040736-bound-sa-token\") pod \"image-registry-558df75cb7-vz7f9\" (UID: \"56a1e57c-315d-463e-9e9a-92a3ce040736\") " pod="openshift-image-registry/image-registry-558df75cb7-vz7f9"
Jul 20 09:39:53.402887 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.402863    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmv9m\" (UniqueName: \"kubernetes.io/projected/56a1e57c-315d-463e-9e9a-92a3ce040736-kube-api-access-kmv9m\") pod \"image-registry-558df75cb7-vz7f9\" (UID: \"56a1e57c-315d-463e-9e9a-92a3ce040736\") " pod="openshift-image-registry/image-registry-558df75cb7-vz7f9"
Jul 20 09:39:53.562737 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.562703    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-558df75cb7-vz7f9"
Jul 20 09:39:53.808279 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.807078    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57456855cb-4dxkr"]
Jul 20 09:39:53.822697 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.822669    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c6bc7f849-xxmgg"]
Jul 20 09:39:53.824864 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:39:53.824829    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a5a1749_a594_4d29_a3cc_068900e83589.slice/crio-7b91879a1e9c5a9a12c1ebb2f56fa0c2fbd437cd964c682c17ad3aa6e9e6c876 WatchSource:0}: Error finding container 7b91879a1e9c5a9a12c1ebb2f56fa0c2fbd437cd964c682c17ad3aa6e9e6c876: Status 404 returned error can't find the container with id 7b91879a1e9c5a9a12c1ebb2f56fa0c2fbd437cd964c682c17ad3aa6e9e6c876
Jul 20 09:39:53.840965 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.840926    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b48cd4c44-z4rj4"]
Jul 20 09:39:53.843731 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:53.843655    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-558df75cb7-vz7f9"]
Jul 20 09:39:53.844600 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:39:53.844572    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod900cb6f0_059f_46fa_96cf_0cafd8182dee.slice/crio-a4758025d6de0c9b9a43bb28382b41ec5b5649e02c9e5b36312873c8095d9f48 WatchSource:0}: Error finding container a4758025d6de0c9b9a43bb28382b41ec5b5649e02c9e5b36312873c8095d9f48: Status 404 returned error can't find the container with id a4758025d6de0c9b9a43bb28382b41ec5b5649e02c9e5b36312873c8095d9f48
Jul 20 09:39:53.846165 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:39:53.846141    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56a1e57c_315d_463e_9e9a_92a3ce040736.slice/crio-226507758c928b5b61c57bd0313546e5f0221a4f20d4e0ce36d495fbb50f20a1 WatchSource:0}: Error finding container 226507758c928b5b61c57bd0313546e5f0221a4f20d4e0ce36d495fbb50f20a1: Status 404 returned error can't find the container with id 226507758c928b5b61c57bd0313546e5f0221a4f20d4e0ce36d495fbb50f20a1
Jul 20 09:39:54.274928 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:54.274889    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kttx7" event={"ID":"851041f2-912c-462e-92c1-c2ac3d7022da","Type":"ContainerStarted","Data":"42b74a4c0e2b8bacef09edf226647730ea00c40fe2c9ed11d523c1fba2ce059f"}
Jul 20 09:39:54.276849 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:54.276748    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-558df75cb7-vz7f9" event={"ID":"56a1e57c-315d-463e-9e9a-92a3ce040736","Type":"ContainerStarted","Data":"cde952d546cdc43c58147ba1b6e71c202f7b5209fa86cfd88b1f2ab001bac8fd"}
Jul 20 09:39:54.276849 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:54.276805    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-558df75cb7-vz7f9" event={"ID":"56a1e57c-315d-463e-9e9a-92a3ce040736","Type":"ContainerStarted","Data":"226507758c928b5b61c57bd0313546e5f0221a4f20d4e0ce36d495fbb50f20a1"}
Jul 20 09:39:54.277097 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:54.276890    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-558df75cb7-vz7f9"
Jul 20 09:39:54.278456 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:54.278393    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c6bc7f849-xxmgg" event={"ID":"4a5a1749-a594-4d29-a3cc-068900e83589","Type":"ContainerStarted","Data":"7b91879a1e9c5a9a12c1ebb2f56fa0c2fbd437cd964c682c17ad3aa6e9e6c876"}
Jul 20 09:39:54.279794 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:54.279760    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b48cd4c44-z4rj4" event={"ID":"900cb6f0-059f-46fa-96cf-0cafd8182dee","Type":"ContainerStarted","Data":"a4758025d6de0c9b9a43bb28382b41ec5b5649e02c9e5b36312873c8095d9f48"}
Jul 20 09:39:54.288120 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:54.288071    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57456855cb-4dxkr" event={"ID":"e9ea8f40-f761-45dc-acc1-638ad8015f0d","Type":"ContainerStarted","Data":"3bb7354688ef2df89a184bf5bd3662d53a9b204677af9952785970d28090e2d1"}
Jul 20 09:39:54.302508 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:54.302436    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-558df75cb7-vz7f9" podStartSLOduration=1.302414566 podStartE2EDuration="1.302414566s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 09:39:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:39:54.301439417 +0000 UTC m=+60.852893910" watchObservedRunningTime="2026-07-20 09:39:54.302414566 +0000 UTC m=+60.853869032"
Jul 20 09:39:55.186129 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:55.185436    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-64r4x"
Jul 20 09:39:59.756786 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:59.756732    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d4a0ef09-53a6-47cb-a39f-15ae72031d8f-original-pull-secret\") pod \"global-pull-secret-syncer-4lw87\" (UID: \"d4a0ef09-53a6-47cb-a39f-15ae72031d8f\") " pod="kube-system/global-pull-secret-syncer-4lw87"
Jul 20 09:39:59.757406 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:59.756834    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5ndp\" (UniqueName: \"kubernetes.io/projected/f44bf7c3-e682-406b-92c7-b8ecfd6505f6-kube-api-access-c5ndp\") pod \"network-check-target-9kcgh\" (UID: \"f44bf7c3-e682-406b-92c7-b8ecfd6505f6\") " pod="openshift-network-diagnostics/network-check-target-9kcgh"
Jul 20 09:39:59.757406 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:59.756864    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de-metrics-certs\") pod \"network-metrics-daemon-xdlfl\" (UID: \"d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de\") " pod="openshift-multus/network-metrics-daemon-xdlfl"
Jul 20 09:39:59.760114 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:59.759829    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 20 09:39:59.760114 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:59.759888    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 20 09:39:59.761273 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:59.761247    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 20 09:39:59.770027 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:59.769959    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 20 09:39:59.770508 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:59.770484    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d4a0ef09-53a6-47cb-a39f-15ae72031d8f-original-pull-secret\") pod \"global-pull-secret-syncer-4lw87\" (UID: \"d4a0ef09-53a6-47cb-a39f-15ae72031d8f\") " pod="kube-system/global-pull-secret-syncer-4lw87"
Jul 20 09:39:59.770508 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:59.770496    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de-metrics-certs\") pod \"network-metrics-daemon-xdlfl\" (UID: \"d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de\") " pod="openshift-multus/network-metrics-daemon-xdlfl"
Jul 20 09:39:59.781773 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:59.781741    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5ndp\" (UniqueName: \"kubernetes.io/projected/f44bf7c3-e682-406b-92c7-b8ecfd6505f6-kube-api-access-c5ndp\") pod \"network-check-target-9kcgh\" (UID: \"f44bf7c3-e682-406b-92c7-b8ecfd6505f6\") " pod="openshift-network-diagnostics/network-check-target-9kcgh"
Jul 20 09:39:59.828763 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:59.828729    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hssrx\""
Jul 20 09:39:59.835333 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:59.835296    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pgl5c\""
Jul 20 09:39:59.836254 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:59.836221    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xdlfl"
Jul 20 09:39:59.840222 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:59.840198    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lw87"
Jul 20 09:39:59.842987 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:39:59.842953    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9kcgh"
Jul 20 09:40:01.268479 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:01.268443    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9fsst"
Jul 20 09:40:02.500335 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:02.500243    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9kcgh"]
Jul 20 09:40:02.510632 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:40:02.510587    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf44bf7c3_e682_406b_92c7_b8ecfd6505f6.slice/crio-8f48046bb6a40d74154414d92518f0fbd63fb90741aee834e95d8d70d0c58ebb WatchSource:0}: Error finding container 8f48046bb6a40d74154414d92518f0fbd63fb90741aee834e95d8d70d0c58ebb: Status 404 returned error can't find the container with id 8f48046bb6a40d74154414d92518f0fbd63fb90741aee834e95d8d70d0c58ebb
Jul 20 09:40:02.527780 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:02.527661    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xdlfl"]
Jul 20 09:40:02.532058 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:40:02.532020    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd242fbc4_edf9_46b0_aa2f_4c8f3e8ef9de.slice/crio-64fb3535d3166af905a94dc48ba5522b222580d90ef3fdb3cc63b9fa7e181e27 WatchSource:0}: Error finding container 64fb3535d3166af905a94dc48ba5522b222580d90ef3fdb3cc63b9fa7e181e27: Status 404 returned error can't find the container with id 64fb3535d3166af905a94dc48ba5522b222580d90ef3fdb3cc63b9fa7e181e27
Jul 20 09:40:02.734795 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:02.734755    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4lw87"]
Jul 20 09:40:02.738996 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:40:02.738903    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4a0ef09_53a6_47cb_a39f_15ae72031d8f.slice/crio-cef047922a48c4de9108d1aae094256ffad13de83c2bb7d5f3827d2b80573d2c WatchSource:0}: Error finding container cef047922a48c4de9108d1aae094256ffad13de83c2bb7d5f3827d2b80573d2c: Status 404 returned error can't find the container with id cef047922a48c4de9108d1aae094256ffad13de83c2bb7d5f3827d2b80573d2c
Jul 20 09:40:03.121083 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:03.121033    2566 patch_prober.go:28] interesting pod/image-registry-57589fb79f-2hrv7 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 20 09:40:03.121285 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:03.121095    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-57589fb79f-2hrv7" podUID="efd4b7ad-96ab-4ecf-8082-97598b6fa971" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 20 09:40:03.327120 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:03.327076    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9kcgh" event={"ID":"f44bf7c3-e682-406b-92c7-b8ecfd6505f6","Type":"ContainerStarted","Data":"8f48046bb6a40d74154414d92518f0fbd63fb90741aee834e95d8d70d0c58ebb"}
Jul 20 09:40:03.328685 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:03.328646    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xdlfl" event={"ID":"d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de","Type":"ContainerStarted","Data":"64fb3535d3166af905a94dc48ba5522b222580d90ef3fdb3cc63b9fa7e181e27"}
Jul 20 09:40:03.331256 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:03.331219    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kttx7" event={"ID":"851041f2-912c-462e-92c1-c2ac3d7022da","Type":"ContainerStarted","Data":"fe17aca6bb3f132799872f658770086bfbf4f592051ef6ecb43f820c16737898"}
Jul 20 09:40:03.333033 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:03.333002    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c6bc7f849-xxmgg" event={"ID":"4a5a1749-a594-4d29-a3cc-068900e83589","Type":"ContainerStarted","Data":"436e98111953df10b53580a98fc6a1077c1305831f57553ea9d74c3163a4667f"}
Jul 20 09:40:03.333425 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:03.333408    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c6bc7f849-xxmgg"
Jul 20 09:40:03.335130 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:03.335104    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c6bc7f849-xxmgg"
Jul 20 09:40:03.335570 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:03.335545    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b48cd4c44-z4rj4" event={"ID":"900cb6f0-059f-46fa-96cf-0cafd8182dee","Type":"ContainerStarted","Data":"477b89243abe4562da8265027cc3ea73e56b9ea7f21022af0df7b19fca564dfb"}
Jul 20 09:40:03.335695 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:03.335583    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b48cd4c44-z4rj4" event={"ID":"900cb6f0-059f-46fa-96cf-0cafd8182dee","Type":"ContainerStarted","Data":"6af3444867e1f3feae728ff89354fe0f171f35277386ab94f6407232cccf25fc"}
Jul 20 09:40:03.335695 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:03.335603    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b48cd4c44-z4rj4" event={"ID":"900cb6f0-059f-46fa-96cf-0cafd8182dee","Type":"ContainerStarted","Data":"01066402b5fcacd8866cb7a1a28de2749aa8c43b825aec160527bda6c1689e33"}
Jul 20 09:40:03.337569 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:03.337541    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57456855cb-4dxkr" event={"ID":"e9ea8f40-f761-45dc-acc1-638ad8015f0d","Type":"ContainerStarted","Data":"487b97303fdf7602c33904aa31ec9466daeb6f52953a3294d427d0ab0313d4a2"}
Jul 20 09:40:03.338826 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:03.338802    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4lw87" event={"ID":"d4a0ef09-53a6-47cb-a39f-15ae72031d8f","Type":"ContainerStarted","Data":"cef047922a48c4de9108d1aae094256ffad13de83c2bb7d5f3827d2b80573d2c"}
Jul 20 09:40:03.352369 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:03.352275    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-kttx7" podStartSLOduration=18.143027021 podStartE2EDuration="35.352249942s" totalImagesPullingTime="17.209222921s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 09:39:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:40:03.351188849 +0000 UTC m=+69.902643325" watchObservedRunningTime="2026-07-20 09:40:03.352249942 +0000 UTC m=+69.903704403"
Jul 20 09:40:03.369695 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:03.369642    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-57456855cb-4dxkr" podStartSLOduration=1.885165403 podStartE2EDuration="10.369625153s" totalImagesPullingTime="8.48445975s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 09:39:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:40:03.368524907 +0000 UTC m=+69.919979374" watchObservedRunningTime="2026-07-20 09:40:03.369625153 +0000 UTC m=+69.921079616"
Jul 20 09:40:03.390720 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:03.390608    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6c6bc7f849-xxmgg" podStartSLOduration=1.901232262 podStartE2EDuration="10.390590754s" totalImagesPullingTime="8.489358492s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 09:39:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:40:03.389761695 +0000 UTC m=+69.941216173" watchObservedRunningTime="2026-07-20 09:40:03.390590754 +0000 UTC m=+69.942045217"
Jul 20 09:40:03.416521 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:03.416453    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b48cd4c44-z4rj4" podStartSLOduration=1.9619198359999999 podStartE2EDuration="10.416432288s" totalImagesPullingTime="8.454512452s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 09:39:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:40:03.41543472 +0000 UTC m=+69.966889185" watchObservedRunningTime="2026-07-20 09:40:03.416432288 +0000 UTC m=+69.967886759"
Jul 20 09:40:05.354184 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:05.354119    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xdlfl" event={"ID":"d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de","Type":"ContainerStarted","Data":"8dab1b86cf07384fb8ba6e2e1454aa842053457f3bae66425295b65b46f42230"}
Jul 20 09:40:05.354184 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:05.354189    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xdlfl" event={"ID":"d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de","Type":"ContainerStarted","Data":"c97109094af2dc864bddef16cafbc6dfa2066793fb76e03a0e88f6d8d999231f"}
Jul 20 09:40:05.375360 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:05.375291    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xdlfl" podStartSLOduration=69.257387424 podStartE2EDuration="1m11.375272981s" totalImagesPullingTime="2.117885557s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 09:38:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:40:05.373700074 +0000 UTC m=+71.925154539" watchObservedRunningTime="2026-07-20 09:40:05.375272981 +0000 UTC m=+71.926727445"
Jul 20 09:40:08.369094 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:08.369052    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4lw87" event={"ID":"d4a0ef09-53a6-47cb-a39f-15ae72031d8f","Type":"ContainerStarted","Data":"95c337bbc89babe091ddc12559a0776fa677081810cec8c0997eb1a98aeff2d6"}
Jul 20 09:40:08.370713 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:08.370679    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9kcgh" event={"ID":"f44bf7c3-e682-406b-92c7-b8ecfd6505f6","Type":"ContainerStarted","Data":"f4189acb5165f88cba1711d779f5f490b492963ecebecc1f19566edfa4e3f171"}
Jul 20 09:40:08.370869 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:08.370780    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9kcgh"
Jul 20 09:40:08.389030 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:08.388950    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-4lw87" podStartSLOduration=69.101801846 podStartE2EDuration="1m14.38893451s" totalImagesPullingTime="5.287132664s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 09:38:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:40:08.388553752 +0000 UTC m=+74.940008217" watchObservedRunningTime="2026-07-20 09:40:08.38893451 +0000 UTC m=+74.940388974"
Jul 20 09:40:08.406948 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:08.406885    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9kcgh" podStartSLOduration=69.896038022 podStartE2EDuration="1m15.406865234s" totalImagesPullingTime="5.510827212s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 09:38:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:40:08.406340225 +0000 UTC m=+74.957794689" watchObservedRunningTime="2026-07-20 09:40:08.406865234 +0000 UTC m=+74.958319698"
Jul 20 09:40:12.447375 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:12.447342    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-qkqmv"]
Jul 20 09:40:12.455019 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:12.454963    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qkqmv"
Jul 20 09:40:12.460370 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:12.460325    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 20 09:40:12.460577 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:12.460388    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 20 09:40:12.460577 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:12.460330    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 20 09:40:12.460577 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:12.460325    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 20 09:40:12.462165 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:12.462119    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 20 09:40:12.462514 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:12.462274    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 20 09:40:12.462624 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:12.462598    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-9vmvl\""
Jul 20 09:40:12.571114 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:12.571077    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6b15d8a2-2161-4e9c-b21a-5bc43495db76-node-exporter-textfile\") pod \"node-exporter-qkqmv\" (UID: \"6b15d8a2-2161-4e9c-b21a-5bc43495db76\") " pod="openshift-monitoring/node-exporter-qkqmv"
Jul 20 09:40:12.571114 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:12.571115    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6b15d8a2-2161-4e9c-b21a-5bc43495db76-metrics-client-ca\") pod \"node-exporter-qkqmv\" (UID: \"6b15d8a2-2161-4e9c-b21a-5bc43495db76\") " pod="openshift-monitoring/node-exporter-qkqmv"
Jul 20 09:40:12.571350 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:12.571143    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6b15d8a2-2161-4e9c-b21a-5bc43495db76-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qkqmv\" (UID: \"6b15d8a2-2161-4e9c-b21a-5bc43495db76\") " pod="openshift-monitoring/node-exporter-qkqmv"
Jul 20 09:40:12.571350 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:12.571160    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkrc2\" (UniqueName: \"kubernetes.io/projected/6b15d8a2-2161-4e9c-b21a-5bc43495db76-kube-api-access-tkrc2\") pod \"node-exporter-qkqmv\" (UID: \"6b15d8a2-2161-4e9c-b21a-5bc43495db76\") " pod="openshift-monitoring/node-exporter-qkqmv"
Jul 20 09:40:12.571350 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:12.571230    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6b15d8a2-2161-4e9c-b21a-5bc43495db76-node-exporter-tls\") pod \"node-exporter-qkqmv\" (UID: \"6b15d8a2-2161-4e9c-b21a-5bc43495db76\") " pod="openshift-monitoring/node-exporter-qkqmv"
Jul 20 09:40:12.571350 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:12.571285    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b15d8a2-2161-4e9c-b21a-5bc43495db76-sys\") pod \"node-exporter-qkqmv\" (UID: \"6b15d8a2-2161-4e9c-b21a-5bc43495db76\") " pod="openshift-monitoring/node-exporter-qkqmv"
Jul 20 09:40:12.571350 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:12.571308    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6b15d8a2-2161-4e9c-b21a-5bc43495db76-node-exporter-accelerators-collector-config\") pod \"node-exporter-qkqmv\" (UID: \"6b15d8a2-2161-4e9c-b21a-5bc43495db76\") " pod="openshift-monitoring/node-exporter-qkqmv"
Jul 20 09:40:12.571350 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:12.571337    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6b15d8a2-2161-4e9c-b21a-5bc43495db76-root\") pod \"node-exporter-qkqmv\" (UID: \"6b15d8a2-2161-4e9c-b21a-5bc43495db76\") " pod="openshift-monitoring/node-exporter-qkqmv"
Jul 20 09:40:12.571541 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:12.571378    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6b15d8a2-2161-4e9c-b21a-5bc43495db76-node-exporter-wtmp\") pod \"node-exporter-qkqmv\" (UID: \"6b15d8a2-2161-4e9c-b21a-5bc43495db76\") " pod="openshift-monitoring/node-exporter-qkqmv"
Jul 20 09:40:12.672552 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:12.672516    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6b15d8a2-2161-4e9c-b21a-5bc43495db76-node-exporter-textfile\") pod \"node-exporter-qkqmv\" (UID: \"6b15d8a2-2161-4e9c-b21a-5bc43495db76\") " pod="openshift-monitoring/node-exporter-qkqmv"
Jul 20 09:40:12.672552 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:12.672555    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6b15d8a2-2161-4e9c-b21a-5bc43495db76-metrics-client-ca\") pod \"node-exporter-qkqmv\" (UID: \"6b15d8a2-2161-4e9c-b21a-5bc43495db76\") " pod="openshift-monitoring/node-exporter-qkqmv"
Jul 20 09:40:12.672826 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:12.672583    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6b15d8a2-2161-4e9c-b21a-5bc43495db76-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qkqmv\" (UID: \"6b15d8a2-2161-4e9c-b21a-5bc43495db76\") " pod="openshift-monitoring/node-exporter-qkqmv"
Jul 20 09:40:12.672826 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:12.672725    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkrc2\" (UniqueName: \"kubernetes.io/projected/6b15d8a2-2161-4e9c-b21a-5bc43495db76-kube-api-access-tkrc2\") pod \"node-exporter-qkqmv\" (UID: \"6b15d8a2-2161-4e9c-b21a-5bc43495db76\") " pod="openshift-monitoring/node-exporter-qkqmv"
Jul 20 09:40:12.672826 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:12.672776    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6b15d8a2-2161-4e9c-b21a-5bc43495db76-node-exporter-tls\") pod \"node-exporter-qkqmv\" (UID: \"6b15d8a2-2161-4e9c-b21a-5bc43495db76\") " pod="openshift-monitoring/node-exporter-qkqmv"
Jul 20 09:40:12.673009 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:12.672828    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b15d8a2-2161-4e9c-b21a-5bc43495db76-sys\") pod \"node-exporter-qkqmv\" (UID: \"6b15d8a2-2161-4e9c-b21a-5bc43495db76\") " pod="openshift-monitoring/node-exporter-qkqmv"
Jul 20 09:40:12.673009 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:12.672867    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6b15d8a2-2161-4e9c-b21a-5bc43495db76-node-exporter-accelerators-collector-config\") pod \"node-exporter-qkqmv\" (UID: \"6b15d8a2-2161-4e9c-b21a-5bc43495db76\") " pod="openshift-monitoring/node-exporter-qkqmv"
Jul 20 09:40:12.673009 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:12.672899    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6b15d8a2-2161-4e9c-b21a-5bc43495db76-root\") pod \"node-exporter-qkqmv\" (UID: \"6b15d8a2-2161-4e9c-b21a-5bc43495db76\") " pod="openshift-monitoring/node-exporter-qkqmv"
Jul 20 09:40:12.673009 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:12.672929    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b15d8a2-2161-4e9c-b21a-5bc43495db76-sys\") pod \"node-exporter-qkqmv\" (UID: \"6b15d8a2-2161-4e9c-b21a-5bc43495db76\") " pod="openshift-monitoring/node-exporter-qkqmv"
Jul 20 09:40:12.673009 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:12.672951    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6b15d8a2-2161-4e9c-b21a-5bc43495db76-node-exporter-wtmp\") pod \"node-exporter-qkqmv\" (UID: \"6b15d8a2-2161-4e9c-b21a-5bc43495db76\") " pod="openshift-monitoring/node-exporter-qkqmv"
Jul 20 09:40:12.673009 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:12.672967    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6b15d8a2-2161-4e9c-b21a-5bc43495db76-node-exporter-textfile\") pod \"node-exporter-qkqmv\" (UID: \"6b15d8a2-2161-4e9c-b21a-5bc43495db76\") " pod="openshift-monitoring/node-exporter-qkqmv"
Jul 20 09:40:12.673277 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:12.673088    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6b15d8a2-2161-4e9c-b21a-5bc43495db76-root\") pod \"node-exporter-qkqmv\" (UID: \"6b15d8a2-2161-4e9c-b21a-5bc43495db76\") " pod="openshift-monitoring/node-exporter-qkqmv"
Jul 20 09:40:12.673277 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:12.673199    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6b15d8a2-2161-4e9c-b21a-5bc43495db76-node-exporter-wtmp\") pod \"node-exporter-qkqmv\" (UID: \"6b15d8a2-2161-4e9c-b21a-5bc43495db76\") " pod="openshift-monitoring/node-exporter-qkqmv"
Jul 20 09:40:12.673345 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:12.673292    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6b15d8a2-2161-4e9c-b21a-5bc43495db76-metrics-client-ca\") pod \"node-exporter-qkqmv\" (UID: \"6b15d8a2-2161-4e9c-b21a-5bc43495db76\") " pod="openshift-monitoring/node-exporter-qkqmv"
Jul 20 09:40:12.673471 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:12.673445    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6b15d8a2-2161-4e9c-b21a-5bc43495db76-node-exporter-accelerators-collector-config\") pod \"node-exporter-qkqmv\" (UID: \"6b15d8a2-2161-4e9c-b21a-5bc43495db76\") " pod="openshift-monitoring/node-exporter-qkqmv"
Jul 20 09:40:12.675262 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:12.675226    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6b15d8a2-2161-4e9c-b21a-5bc43495db76-node-exporter-tls\") pod \"node-exporter-qkqmv\" (UID: \"6b15d8a2-2161-4e9c-b21a-5bc43495db76\") " pod="openshift-monitoring/node-exporter-qkqmv"
Jul 20 09:40:12.675368 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:12.675324    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6b15d8a2-2161-4e9c-b21a-5bc43495db76-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qkqmv\" (UID: \"6b15d8a2-2161-4e9c-b21a-5bc43495db76\") " pod="openshift-monitoring/node-exporter-qkqmv"
Jul 20 09:40:12.682591 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:12.682555    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkrc2\" (UniqueName: \"kubernetes.io/projected/6b15d8a2-2161-4e9c-b21a-5bc43495db76-kube-api-access-tkrc2\") pod \"node-exporter-qkqmv\" (UID: \"6b15d8a2-2161-4e9c-b21a-5bc43495db76\") " pod="openshift-monitoring/node-exporter-qkqmv"
Jul 20 09:40:12.765035 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:12.764891    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qkqmv"
Jul 20 09:40:12.775899 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:40:12.775860    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b15d8a2_2161_4e9c_b21a_5bc43495db76.slice/crio-ee2a82aa6547779e48226566b9df7e4d0e2db4c86cad51648b72c8baae2416d4 WatchSource:0}: Error finding container ee2a82aa6547779e48226566b9df7e4d0e2db4c86cad51648b72c8baae2416d4: Status 404 returned error can't find the container with id ee2a82aa6547779e48226566b9df7e4d0e2db4c86cad51648b72c8baae2416d4
Jul 20 09:40:13.119730 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:13.119701    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-57589fb79f-2hrv7"
Jul 20 09:40:13.392518 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:13.392429    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qkqmv" event={"ID":"6b15d8a2-2161-4e9c-b21a-5bc43495db76","Type":"ContainerStarted","Data":"ee2a82aa6547779e48226566b9df7e4d0e2db4c86cad51648b72c8baae2416d4"}
Jul 20 09:40:14.397673 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:14.397630    2566 generic.go:356] "Generic (PLEG): container finished" podID="6b15d8a2-2161-4e9c-b21a-5bc43495db76" containerID="842e1f55268cc8b19bf19c9984e4025f122a51a1705ea9aa4e328737864466c8" exitCode=0
Jul 20 09:40:14.398074 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:14.397683    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qkqmv" event={"ID":"6b15d8a2-2161-4e9c-b21a-5bc43495db76","Type":"ContainerDied","Data":"842e1f55268cc8b19bf19c9984e4025f122a51a1705ea9aa4e328737864466c8"}
Jul 20 09:40:15.306212 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:15.306178    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-558df75cb7-vz7f9"
Jul 20 09:40:15.404146 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:15.404107    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qkqmv" event={"ID":"6b15d8a2-2161-4e9c-b21a-5bc43495db76","Type":"ContainerStarted","Data":"c929867a33d639135727929ad109dd7364f88c1dd0efca32065f1c1511bed379"}
Jul 20 09:40:15.404654 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:15.404165    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qkqmv" event={"ID":"6b15d8a2-2161-4e9c-b21a-5bc43495db76","Type":"ContainerStarted","Data":"c573dbee55d3ae1dfd732360fab529140b5ce8f92666d68f4465a8366f983ba4"}
Jul 20 09:40:15.424994 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:15.424910    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-qkqmv" podStartSLOduration=2.364675033 podStartE2EDuration="3.424892807s" totalImagesPullingTime="981.272775ms" totalInitContainerRuntime="78.944999ms" isStatefulPod=true podCreationTimestamp="2026-07-20 09:40:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:40:15.424355475 +0000 UTC m=+81.975809952" watchObservedRunningTime="2026-07-20 09:40:15.424892807 +0000 UTC m=+81.976347271"
Jul 20 09:40:18.139514 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:18.139472    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-57589fb79f-2hrv7" podUID="efd4b7ad-96ab-4ecf-8082-97598b6fa971" containerName="registry" containerID="cri-o://f5ce90221a1b864482fb23b6fc237dead7a8cbc2fbaccb1dc71bce09617ccf40" gracePeriod=30
Jul 20 09:40:18.386275 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:18.386243    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57589fb79f-2hrv7"
Jul 20 09:40:18.417781 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:18.417684    2566 generic.go:356] "Generic (PLEG): container finished" podID="efd4b7ad-96ab-4ecf-8082-97598b6fa971" containerID="f5ce90221a1b864482fb23b6fc237dead7a8cbc2fbaccb1dc71bce09617ccf40" exitCode=0
Jul 20 09:40:18.417781 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:18.417735    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-57589fb79f-2hrv7" event={"ID":"efd4b7ad-96ab-4ecf-8082-97598b6fa971","Type":"ContainerDied","Data":"f5ce90221a1b864482fb23b6fc237dead7a8cbc2fbaccb1dc71bce09617ccf40"}
Jul 20 09:40:18.417781 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:18.417777    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57589fb79f-2hrv7"
Jul 20 09:40:18.418080 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:18.417793    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-57589fb79f-2hrv7" event={"ID":"efd4b7ad-96ab-4ecf-8082-97598b6fa971","Type":"ContainerDied","Data":"0d719befdab083e609c7fd63b06a533cd76dc1663805d3c209c981dcc8aa6436"}
Jul 20 09:40:18.418080 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:18.417813    2566 scope.go:117] "RemoveContainer" containerID="f5ce90221a1b864482fb23b6fc237dead7a8cbc2fbaccb1dc71bce09617ccf40"
Jul 20 09:40:18.420208 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:18.420187    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/efd4b7ad-96ab-4ecf-8082-97598b6fa971-bound-sa-token\") pod \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\" (UID: \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\") "
Jul 20 09:40:18.420328 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:18.420217    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/efd4b7ad-96ab-4ecf-8082-97598b6fa971-installation-pull-secrets\") pod \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\" (UID: \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\") "
Jul 20 09:40:18.420328 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:18.420263    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/efd4b7ad-96ab-4ecf-8082-97598b6fa971-image-registry-private-configuration\") pod \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\" (UID: \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\") "
Jul 20 09:40:18.420328 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:18.420303    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/efd4b7ad-96ab-4ecf-8082-97598b6fa971-registry-certificates\") pod \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\" (UID: \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\") "
Jul 20 09:40:18.420489 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:18.420342    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/efd4b7ad-96ab-4ecf-8082-97598b6fa971-trusted-ca\") pod \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\" (UID: \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\") "
Jul 20 09:40:18.420594 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:18.420401    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bpf2q\" (UniqueName: \"kubernetes.io/projected/efd4b7ad-96ab-4ecf-8082-97598b6fa971-kube-api-access-bpf2q\") pod \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\" (UID: \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\") "
Jul 20 09:40:18.420656 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:18.420621    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/efd4b7ad-96ab-4ecf-8082-97598b6fa971-ca-trust-extracted\") pod \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\" (UID: \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\") "
Jul 20 09:40:18.420656 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:18.420649    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/efd4b7ad-96ab-4ecf-8082-97598b6fa971-registry-tls\") pod \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\" (UID: \"efd4b7ad-96ab-4ecf-8082-97598b6fa971\") "
Jul 20 09:40:18.420776 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:18.420748    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efd4b7ad-96ab-4ecf-8082-97598b6fa971-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "efd4b7ad-96ab-4ecf-8082-97598b6fa971" (UID: "efd4b7ad-96ab-4ecf-8082-97598b6fa971"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 09:40:18.421986 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:18.421260    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efd4b7ad-96ab-4ecf-8082-97598b6fa971-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "efd4b7ad-96ab-4ecf-8082-97598b6fa971" (UID: "efd4b7ad-96ab-4ecf-8082-97598b6fa971"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 09:40:18.422127 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:18.422000    2566 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/efd4b7ad-96ab-4ecf-8082-97598b6fa971-registry-certificates\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 09:40:18.423512 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:18.423425    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efd4b7ad-96ab-4ecf-8082-97598b6fa971-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "efd4b7ad-96ab-4ecf-8082-97598b6fa971" (UID: "efd4b7ad-96ab-4ecf-8082-97598b6fa971"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 09:40:18.424052 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:18.424019    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efd4b7ad-96ab-4ecf-8082-97598b6fa971-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "efd4b7ad-96ab-4ecf-8082-97598b6fa971" (UID: "efd4b7ad-96ab-4ecf-8082-97598b6fa971"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 09:40:18.424573 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:18.424548    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efd4b7ad-96ab-4ecf-8082-97598b6fa971-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "efd4b7ad-96ab-4ecf-8082-97598b6fa971" (UID: "efd4b7ad-96ab-4ecf-8082-97598b6fa971"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 09:40:18.424944 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:18.424909    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efd4b7ad-96ab-4ecf-8082-97598b6fa971-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "efd4b7ad-96ab-4ecf-8082-97598b6fa971" (UID: "efd4b7ad-96ab-4ecf-8082-97598b6fa971"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 09:40:18.425053 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:18.424996    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efd4b7ad-96ab-4ecf-8082-97598b6fa971-kube-api-access-bpf2q" (OuterVolumeSpecName: "kube-api-access-bpf2q") pod "efd4b7ad-96ab-4ecf-8082-97598b6fa971" (UID: "efd4b7ad-96ab-4ecf-8082-97598b6fa971"). InnerVolumeSpecName "kube-api-access-bpf2q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 09:40:18.430415 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:18.430364    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efd4b7ad-96ab-4ecf-8082-97598b6fa971-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "efd4b7ad-96ab-4ecf-8082-97598b6fa971" (UID: "efd4b7ad-96ab-4ecf-8082-97598b6fa971"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 09:40:18.440106 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:18.439964    2566 scope.go:117] "RemoveContainer" containerID="f5ce90221a1b864482fb23b6fc237dead7a8cbc2fbaccb1dc71bce09617ccf40"
Jul 20 09:40:18.440589 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:40:18.440563    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5ce90221a1b864482fb23b6fc237dead7a8cbc2fbaccb1dc71bce09617ccf40\": container with ID starting with f5ce90221a1b864482fb23b6fc237dead7a8cbc2fbaccb1dc71bce09617ccf40 not found: ID does not exist" containerID="f5ce90221a1b864482fb23b6fc237dead7a8cbc2fbaccb1dc71bce09617ccf40"
Jul 20 09:40:18.440695 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:18.440601    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5ce90221a1b864482fb23b6fc237dead7a8cbc2fbaccb1dc71bce09617ccf40"} err="failed to get container status \"f5ce90221a1b864482fb23b6fc237dead7a8cbc2fbaccb1dc71bce09617ccf40\": rpc error: code = NotFound desc = could not find container \"f5ce90221a1b864482fb23b6fc237dead7a8cbc2fbaccb1dc71bce09617ccf40\": container with ID starting with f5ce90221a1b864482fb23b6fc237dead7a8cbc2fbaccb1dc71bce09617ccf40 not found: ID does not exist"
Jul 20 09:40:18.522737 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:18.522698    2566 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/efd4b7ad-96ab-4ecf-8082-97598b6fa971-image-registry-private-configuration\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 09:40:18.522737 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:18.522730    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/efd4b7ad-96ab-4ecf-8082-97598b6fa971-trusted-ca\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 09:40:18.522737 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:18.522739    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bpf2q\" (UniqueName: \"kubernetes.io/projected/efd4b7ad-96ab-4ecf-8082-97598b6fa971-kube-api-access-bpf2q\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 09:40:18.522737 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:18.522747    2566 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/efd4b7ad-96ab-4ecf-8082-97598b6fa971-ca-trust-extracted\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 09:40:18.522967 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:18.522756    2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/efd4b7ad-96ab-4ecf-8082-97598b6fa971-registry-tls\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 09:40:18.522967 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:18.522764    2566 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/efd4b7ad-96ab-4ecf-8082-97598b6fa971-bound-sa-token\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 09:40:18.522967 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:18.522772    2566 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/efd4b7ad-96ab-4ecf-8082-97598b6fa971-installation-pull-secrets\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 09:40:18.741238 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:18.741190    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-57589fb79f-2hrv7"]
Jul 20 09:40:18.743277 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:18.743249    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-57589fb79f-2hrv7"]
Jul 20 09:40:19.919418 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:19.919374    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efd4b7ad-96ab-4ecf-8082-97598b6fa971" path="/var/lib/kubelet/pods/efd4b7ad-96ab-4ecf-8082-97598b6fa971/volumes"
Jul 20 09:40:39.376905 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:40:39.376870    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9kcgh"
Jul 20 09:43:53.801850 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:43:53.801812    2566 kubelet.go:1648] "Image garbage collection succeeded"
Jul 20 09:44:03.947308 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:03.947260    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n"]
Jul 20 09:44:03.948808 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:03.947506    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efd4b7ad-96ab-4ecf-8082-97598b6fa971" containerName="registry"
Jul 20 09:44:03.948808 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:03.947517    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="efd4b7ad-96ab-4ecf-8082-97598b6fa971" containerName="registry"
Jul 20 09:44:03.948808 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:03.947570    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="efd4b7ad-96ab-4ecf-8082-97598b6fa971" containerName="registry"
Jul 20 09:44:03.949339 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:03.949318    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n"
Jul 20 09:44:03.955755 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:03.955714    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 20 09:44:03.955994 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:03.955954    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 20 09:44:03.956129 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:03.956002    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 20 09:44:03.956129 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:03.956069    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 20 09:44:03.963090 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:03.963053    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-6kkln\""
Jul 20 09:44:03.982035 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:03.981995    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n"]
Jul 20 09:44:04.029518 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:04.029470    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/06374749-5b32-4b20-8752-34af82ef1d16-tmp\") pod \"opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n\" (UID: \"06374749-5b32-4b20-8752-34af82ef1d16\") " pod="opendatahub/opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n"
Jul 20 09:44:04.029697 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:04.029533    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/06374749-5b32-4b20-8752-34af82ef1d16-manifests\") pod \"opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n\" (UID: \"06374749-5b32-4b20-8752-34af82ef1d16\") " pod="opendatahub/opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n"
Jul 20 09:44:04.029697 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:04.029564    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/06374749-5b32-4b20-8752-34af82ef1d16-webhook-cert\") pod \"opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n\" (UID: \"06374749-5b32-4b20-8752-34af82ef1d16\") " pod="opendatahub/opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n"
Jul 20 09:44:04.029697 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:04.029636    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b77xp\" (UniqueName: \"kubernetes.io/projected/06374749-5b32-4b20-8752-34af82ef1d16-kube-api-access-b77xp\") pod \"opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n\" (UID: \"06374749-5b32-4b20-8752-34af82ef1d16\") " pod="opendatahub/opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n"
Jul 20 09:44:04.029697 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:04.029671    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/06374749-5b32-4b20-8752-34af82ef1d16-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n\" (UID: \"06374749-5b32-4b20-8752-34af82ef1d16\") " pod="opendatahub/opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n"
Jul 20 09:44:04.130362 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:04.130313    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/06374749-5b32-4b20-8752-34af82ef1d16-webhook-cert\") pod \"opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n\" (UID: \"06374749-5b32-4b20-8752-34af82ef1d16\") " pod="opendatahub/opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n"
Jul 20 09:44:04.130362 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:04.130367    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b77xp\" (UniqueName: \"kubernetes.io/projected/06374749-5b32-4b20-8752-34af82ef1d16-kube-api-access-b77xp\") pod \"opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n\" (UID: \"06374749-5b32-4b20-8752-34af82ef1d16\") " pod="opendatahub/opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n"
Jul 20 09:44:04.130623 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:04.130439    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/06374749-5b32-4b20-8752-34af82ef1d16-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n\" (UID: \"06374749-5b32-4b20-8752-34af82ef1d16\") " pod="opendatahub/opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n"
Jul 20 09:44:04.130623 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:04.130496    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/06374749-5b32-4b20-8752-34af82ef1d16-tmp\") pod \"opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n\" (UID: \"06374749-5b32-4b20-8752-34af82ef1d16\") " pod="opendatahub/opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n"
Jul 20 09:44:04.130623 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:04.130564    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/06374749-5b32-4b20-8752-34af82ef1d16-manifests\") pod \"opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n\" (UID: \"06374749-5b32-4b20-8752-34af82ef1d16\") " pod="opendatahub/opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n"
Jul 20 09:44:04.130968 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:04.130947    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/06374749-5b32-4b20-8752-34af82ef1d16-manifests\") pod \"opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n\" (UID: \"06374749-5b32-4b20-8752-34af82ef1d16\") " pod="opendatahub/opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n"
Jul 20 09:44:04.133267 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:04.133231    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/06374749-5b32-4b20-8752-34af82ef1d16-tmp\") pod \"opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n\" (UID: \"06374749-5b32-4b20-8752-34af82ef1d16\") " pod="opendatahub/opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n"
Jul 20 09:44:04.133434 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:04.133364    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/06374749-5b32-4b20-8752-34af82ef1d16-webhook-cert\") pod \"opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n\" (UID: \"06374749-5b32-4b20-8752-34af82ef1d16\") " pod="opendatahub/opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n"
Jul 20 09:44:04.133498 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:04.133446    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/06374749-5b32-4b20-8752-34af82ef1d16-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n\" (UID: \"06374749-5b32-4b20-8752-34af82ef1d16\") " pod="opendatahub/opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n"
Jul 20 09:44:04.141829 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:04.141797    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b77xp\" (UniqueName: \"kubernetes.io/projected/06374749-5b32-4b20-8752-34af82ef1d16-kube-api-access-b77xp\") pod \"opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n\" (UID: \"06374749-5b32-4b20-8752-34af82ef1d16\") " pod="opendatahub/opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n"
Jul 20 09:44:04.261619 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:04.261512    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n"
Jul 20 09:44:04.418938 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:04.418658    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n"]
Jul 20 09:44:04.421193 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:44:04.421162    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06374749_5b32_4b20_8752_34af82ef1d16.slice/crio-4bbe6422f9a9531c23f5d34c0a5c571920c2af0c3faa4fd90d0f9e12badc5d81 WatchSource:0}: Error finding container 4bbe6422f9a9531c23f5d34c0a5c571920c2af0c3faa4fd90d0f9e12badc5d81: Status 404 returned error can't find the container with id 4bbe6422f9a9531c23f5d34c0a5c571920c2af0c3faa4fd90d0f9e12badc5d81
Jul 20 09:44:04.423041 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:04.423022    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 09:44:05.178671 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:05.178624    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n" event={"ID":"06374749-5b32-4b20-8752-34af82ef1d16","Type":"ContainerStarted","Data":"4bbe6422f9a9531c23f5d34c0a5c571920c2af0c3faa4fd90d0f9e12badc5d81"}
Jul 20 09:44:08.192308 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:08.192259    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n" event={"ID":"06374749-5b32-4b20-8752-34af82ef1d16","Type":"ContainerStarted","Data":"3a70f08ce8993c4985b9eb4c66311cf5e1cbf62f887a3694615eb2000b61a86c"}
Jul 20 09:44:09.198199 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:09.198164    2566 generic.go:356] "Generic (PLEG): container finished" podID="06374749-5b32-4b20-8752-34af82ef1d16" containerID="3a70f08ce8993c4985b9eb4c66311cf5e1cbf62f887a3694615eb2000b61a86c" exitCode=0
Jul 20 09:44:09.198600 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:09.198207    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n" event={"ID":"06374749-5b32-4b20-8752-34af82ef1d16","Type":"ContainerDied","Data":"3a70f08ce8993c4985b9eb4c66311cf5e1cbf62f887a3694615eb2000b61a86c"}
Jul 20 09:44:10.203677 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:10.203632    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n" event={"ID":"06374749-5b32-4b20-8752-34af82ef1d16","Type":"ContainerStarted","Data":"7e8a87aad88409770af90243fb22fba28f0f67aa266f9300c9061e300bb9b954"}
Jul 20 09:44:10.204101 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:10.203755    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n"
Jul 20 09:44:10.231079 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:10.230996    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n" podStartSLOduration=2.797446352 podStartE2EDuration="7.230954382s" totalImagesPullingTime="3.364545802s" totalInitContainerRuntime="1.068962228s" isStatefulPod=false podCreationTimestamp="2026-07-20 09:44:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:44:10.227672546 +0000 UTC m=+316.779127022" watchObservedRunningTime="2026-07-20 09:44:10.230954382 +0000 UTC m=+316.782408846"
Jul 20 09:44:21.209132 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:21.209100    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n"
Jul 20 09:44:24.036227 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:24.036190    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7b6cfc896b-8t64n"]
Jul 20 09:44:24.046771 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:24.046738    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7b6cfc896b-8t64n"
Jul 20 09:44:24.053305 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:24.053271    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 20 09:44:24.053305 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:24.053272    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 20 09:44:24.053947 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:24.053740    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 20 09:44:24.054636 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:24.054611    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 20 09:44:24.054636 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:24.054624    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-cksps\""
Jul 20 09:44:24.061020 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:24.060953    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7b6cfc896b-8t64n"]
Jul 20 09:44:24.191196 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:24.191148    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14906f6c-ca75-4f3a-8455-ac29c6cefd38-tls-certs\") pod \"kube-auth-proxy-7b6cfc896b-8t64n\" (UID: \"14906f6c-ca75-4f3a-8455-ac29c6cefd38\") " pod="openshift-ingress/kube-auth-proxy-7b6cfc896b-8t64n"
Jul 20 09:44:24.191402 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:24.191208    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pq6gv\" (UniqueName: \"kubernetes.io/projected/14906f6c-ca75-4f3a-8455-ac29c6cefd38-kube-api-access-pq6gv\") pod \"kube-auth-proxy-7b6cfc896b-8t64n\" (UID: \"14906f6c-ca75-4f3a-8455-ac29c6cefd38\") " pod="openshift-ingress/kube-auth-proxy-7b6cfc896b-8t64n"
Jul 20 09:44:24.191402 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:24.191243    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/14906f6c-ca75-4f3a-8455-ac29c6cefd38-tmp\") pod \"kube-auth-proxy-7b6cfc896b-8t64n\" (UID: \"14906f6c-ca75-4f3a-8455-ac29c6cefd38\") " pod="openshift-ingress/kube-auth-proxy-7b6cfc896b-8t64n"
Jul 20 09:44:24.292615 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:24.292518    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14906f6c-ca75-4f3a-8455-ac29c6cefd38-tls-certs\") pod \"kube-auth-proxy-7b6cfc896b-8t64n\" (UID: \"14906f6c-ca75-4f3a-8455-ac29c6cefd38\") " pod="openshift-ingress/kube-auth-proxy-7b6cfc896b-8t64n"
Jul 20 09:44:24.292615 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:24.292577    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pq6gv\" (UniqueName: \"kubernetes.io/projected/14906f6c-ca75-4f3a-8455-ac29c6cefd38-kube-api-access-pq6gv\") pod \"kube-auth-proxy-7b6cfc896b-8t64n\" (UID: \"14906f6c-ca75-4f3a-8455-ac29c6cefd38\") " pod="openshift-ingress/kube-auth-proxy-7b6cfc896b-8t64n"
Jul 20 09:44:24.292793 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:24.292622    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/14906f6c-ca75-4f3a-8455-ac29c6cefd38-tmp\") pod \"kube-auth-proxy-7b6cfc896b-8t64n\" (UID: \"14906f6c-ca75-4f3a-8455-ac29c6cefd38\") " pod="openshift-ingress/kube-auth-proxy-7b6cfc896b-8t64n"
Jul 20 09:44:24.295227 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:24.295199    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/14906f6c-ca75-4f3a-8455-ac29c6cefd38-tmp\") pod \"kube-auth-proxy-7b6cfc896b-8t64n\" (UID: \"14906f6c-ca75-4f3a-8455-ac29c6cefd38\") " pod="openshift-ingress/kube-auth-proxy-7b6cfc896b-8t64n"
Jul 20 09:44:24.295361 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:24.295328    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14906f6c-ca75-4f3a-8455-ac29c6cefd38-tls-certs\") pod \"kube-auth-proxy-7b6cfc896b-8t64n\" (UID: \"14906f6c-ca75-4f3a-8455-ac29c6cefd38\") " pod="openshift-ingress/kube-auth-proxy-7b6cfc896b-8t64n"
Jul 20 09:44:24.315727 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:24.315689    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pq6gv\" (UniqueName: \"kubernetes.io/projected/14906f6c-ca75-4f3a-8455-ac29c6cefd38-kube-api-access-pq6gv\") pod \"kube-auth-proxy-7b6cfc896b-8t64n\" (UID: \"14906f6c-ca75-4f3a-8455-ac29c6cefd38\") " pod="openshift-ingress/kube-auth-proxy-7b6cfc896b-8t64n"
Jul 20 09:44:24.357629 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:24.357585    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7b6cfc896b-8t64n"
Jul 20 09:44:24.374928 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:24.374894    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-rkmvr"]
Jul 20 09:44:24.377294 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:24.377271    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-rkmvr"
Jul 20 09:44:24.381679 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:24.381644    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 20 09:44:24.381851 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:24.381776    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-59vqq\""
Jul 20 09:44:24.416098 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:24.414269    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-rkmvr"]
Jul 20 09:44:24.494672 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:24.494640    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c047ca14-af23-48e7-84f8-a2b51edec8bf-cert\") pod \"odh-model-controller-7998c67dc4-rkmvr\" (UID: \"c047ca14-af23-48e7-84f8-a2b51edec8bf\") " pod="opendatahub/odh-model-controller-7998c67dc4-rkmvr"
Jul 20 09:44:24.494845 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:24.494716    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh9dg\" (UniqueName: \"kubernetes.io/projected/c047ca14-af23-48e7-84f8-a2b51edec8bf-kube-api-access-qh9dg\") pod \"odh-model-controller-7998c67dc4-rkmvr\" (UID: \"c047ca14-af23-48e7-84f8-a2b51edec8bf\") " pod="opendatahub/odh-model-controller-7998c67dc4-rkmvr"
Jul 20 09:44:24.516337 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:24.516257    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7b6cfc896b-8t64n"]
Jul 20 09:44:24.518765 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:44:24.518731    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14906f6c_ca75_4f3a_8455_ac29c6cefd38.slice/crio-ae368bb9d09ee39a2422669868dbe3711b4f7c836180c8ef8291d90c9a016e9b WatchSource:0}: Error finding container ae368bb9d09ee39a2422669868dbe3711b4f7c836180c8ef8291d90c9a016e9b: Status 404 returned error can't find the container with id ae368bb9d09ee39a2422669868dbe3711b4f7c836180c8ef8291d90c9a016e9b
Jul 20 09:44:24.595879 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:24.595839    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh9dg\" (UniqueName: \"kubernetes.io/projected/c047ca14-af23-48e7-84f8-a2b51edec8bf-kube-api-access-qh9dg\") pod \"odh-model-controller-7998c67dc4-rkmvr\" (UID: \"c047ca14-af23-48e7-84f8-a2b51edec8bf\") " pod="opendatahub/odh-model-controller-7998c67dc4-rkmvr"
Jul 20 09:44:24.596126 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:24.595902    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c047ca14-af23-48e7-84f8-a2b51edec8bf-cert\") pod \"odh-model-controller-7998c67dc4-rkmvr\" (UID: \"c047ca14-af23-48e7-84f8-a2b51edec8bf\") " pod="opendatahub/odh-model-controller-7998c67dc4-rkmvr"
Jul 20 09:44:24.596126 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:44:24.596027    2566 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 20 09:44:24.596126 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:44:24.596093    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c047ca14-af23-48e7-84f8-a2b51edec8bf-cert podName:c047ca14-af23-48e7-84f8-a2b51edec8bf nodeName:}" failed. No retries permitted until 2026-07-20 09:44:25.096076148 +0000 UTC m=+331.647530590 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c047ca14-af23-48e7-84f8-a2b51edec8bf-cert") pod "odh-model-controller-7998c67dc4-rkmvr" (UID: "c047ca14-af23-48e7-84f8-a2b51edec8bf") : secret "odh-model-controller-webhook-cert" not found
Jul 20 09:44:24.609384 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:24.609353    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh9dg\" (UniqueName: \"kubernetes.io/projected/c047ca14-af23-48e7-84f8-a2b51edec8bf-kube-api-access-qh9dg\") pod \"odh-model-controller-7998c67dc4-rkmvr\" (UID: \"c047ca14-af23-48e7-84f8-a2b51edec8bf\") " pod="opendatahub/odh-model-controller-7998c67dc4-rkmvr"
Jul 20 09:44:25.099519 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.099475    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c047ca14-af23-48e7-84f8-a2b51edec8bf-cert\") pod \"odh-model-controller-7998c67dc4-rkmvr\" (UID: \"c047ca14-af23-48e7-84f8-a2b51edec8bf\") " pod="opendatahub/odh-model-controller-7998c67dc4-rkmvr"
Jul 20 09:44:25.102114 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.102092    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c047ca14-af23-48e7-84f8-a2b51edec8bf-cert\") pod \"odh-model-controller-7998c67dc4-rkmvr\" (UID: \"c047ca14-af23-48e7-84f8-a2b51edec8bf\") " pod="opendatahub/odh-model-controller-7998c67dc4-rkmvr"
Jul 20 09:44:25.265622 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.265581    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7b6cfc896b-8t64n" event={"ID":"14906f6c-ca75-4f3a-8455-ac29c6cefd38","Type":"ContainerStarted","Data":"ae368bb9d09ee39a2422669868dbe3711b4f7c836180c8ef8291d90c9a016e9b"}
Jul 20 09:44:25.291588 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.291543    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-rkmvr"
Jul 20 09:44:25.316689 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.316649    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-rhthj"]
Jul 20 09:44:25.320505 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.320460    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-rhthj"
Jul 20 09:44:25.324129 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.324101    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-vfrlx\""
Jul 20 09:44:25.324291 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.324099    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 20 09:44:25.336987 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.336891    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-rhthj"]
Jul 20 09:44:25.402320 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.402279    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vw8hr\" (UniqueName: \"kubernetes.io/projected/ec4dee2b-b4be-4eff-b5a5-d07f9e9d27fc-kube-api-access-vw8hr\") pod \"model-serving-api-54d89bd5fc-rhthj\" (UID: \"ec4dee2b-b4be-4eff-b5a5-d07f9e9d27fc\") " pod="opendatahub/model-serving-api-54d89bd5fc-rhthj"
Jul 20 09:44:25.402320 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.402335    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec4dee2b-b4be-4eff-b5a5-d07f9e9d27fc-tls-certs\") pod \"model-serving-api-54d89bd5fc-rhthj\" (UID: \"ec4dee2b-b4be-4eff-b5a5-d07f9e9d27fc\") " pod="opendatahub/model-serving-api-54d89bd5fc-rhthj"
Jul 20 09:44:25.475583 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.475551    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-rkmvr"]
Jul 20 09:44:25.478024 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:44:25.477966    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc047ca14_af23_48e7_84f8_a2b51edec8bf.slice/crio-da57cd046d01c5b5f0c2ac7bde23b30a881d9905f00677ff8b224fb3247a8147 WatchSource:0}: Error finding container da57cd046d01c5b5f0c2ac7bde23b30a881d9905f00677ff8b224fb3247a8147: Status 404 returned error can't find the container with id da57cd046d01c5b5f0c2ac7bde23b30a881d9905f00677ff8b224fb3247a8147
Jul 20 09:44:25.503659 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.503622    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vw8hr\" (UniqueName: \"kubernetes.io/projected/ec4dee2b-b4be-4eff-b5a5-d07f9e9d27fc-kube-api-access-vw8hr\") pod \"model-serving-api-54d89bd5fc-rhthj\" (UID: \"ec4dee2b-b4be-4eff-b5a5-d07f9e9d27fc\") " pod="opendatahub/model-serving-api-54d89bd5fc-rhthj"
Jul 20 09:44:25.503659 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.503668    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec4dee2b-b4be-4eff-b5a5-d07f9e9d27fc-tls-certs\") pod \"model-serving-api-54d89bd5fc-rhthj\" (UID: \"ec4dee2b-b4be-4eff-b5a5-d07f9e9d27fc\") " pod="opendatahub/model-serving-api-54d89bd5fc-rhthj"
Jul 20 09:44:25.503923 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:44:25.503819    2566 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 20 09:44:25.503923 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:44:25.503912    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec4dee2b-b4be-4eff-b5a5-d07f9e9d27fc-tls-certs podName:ec4dee2b-b4be-4eff-b5a5-d07f9e9d27fc nodeName:}" failed. No retries permitted until 2026-07-20 09:44:26.003881295 +0000 UTC m=+332.555335757 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/ec4dee2b-b4be-4eff-b5a5-d07f9e9d27fc-tls-certs") pod "model-serving-api-54d89bd5fc-rhthj" (UID: "ec4dee2b-b4be-4eff-b5a5-d07f9e9d27fc") : secret "model-serving-api-tls" not found
Jul 20 09:44:25.516533 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.516502    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vw8hr\" (UniqueName: \"kubernetes.io/projected/ec4dee2b-b4be-4eff-b5a5-d07f9e9d27fc-kube-api-access-vw8hr\") pod \"model-serving-api-54d89bd5fc-rhthj\" (UID: \"ec4dee2b-b4be-4eff-b5a5-d07f9e9d27fc\") " pod="opendatahub/model-serving-api-54d89bd5fc-rhthj"
Jul 20 09:44:25.645921 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.645823    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-vv6w4"]
Jul 20 09:44:25.649684 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.649651    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv6w4"
Jul 20 09:44:25.652679 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.652454    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 20 09:44:25.652679 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.652535    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-924jw\""
Jul 20 09:44:25.652935 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.652788    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 20 09:44:25.653211 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.653189    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 20 09:44:25.653292 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.653193    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 20 09:44:25.665186 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.665157    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-vv6w4"]
Jul 20 09:44:25.806407 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.806367    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-vv6w4\" (UID: \"8d241b72-f123-4dc2-8aa7-c51b7c696d1b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv6w4"
Jul 20 09:44:25.806407 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.806409    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-local-certs\") pod \"istiod-openshift-gateway-7574459656-vv6w4\" (UID: \"8d241b72-f123-4dc2-8aa7-c51b7c696d1b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv6w4"
Jul 20 09:44:25.806658 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.806440    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-cacerts\") pod \"istiod-openshift-gateway-7574459656-vv6w4\" (UID: \"8d241b72-f123-4dc2-8aa7-c51b7c696d1b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv6w4"
Jul 20 09:44:25.806658 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.806499    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-vv6w4\" (UID: \"8d241b72-f123-4dc2-8aa7-c51b7c696d1b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv6w4"
Jul 20 09:44:25.806658 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.806551    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-vv6w4\" (UID: \"8d241b72-f123-4dc2-8aa7-c51b7c696d1b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv6w4"
Jul 20 09:44:25.806658 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.806630    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-istio-token\") pod \"istiod-openshift-gateway-7574459656-vv6w4\" (UID: \"8d241b72-f123-4dc2-8aa7-c51b7c696d1b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv6w4"
Jul 20 09:44:25.806819 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.806681    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99qf2\" (UniqueName: \"kubernetes.io/projected/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-kube-api-access-99qf2\") pod \"istiod-openshift-gateway-7574459656-vv6w4\" (UID: \"8d241b72-f123-4dc2-8aa7-c51b7c696d1b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv6w4"
Jul 20 09:44:25.907350 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.907257    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99qf2\" (UniqueName: \"kubernetes.io/projected/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-kube-api-access-99qf2\") pod \"istiod-openshift-gateway-7574459656-vv6w4\" (UID: \"8d241b72-f123-4dc2-8aa7-c51b7c696d1b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv6w4"
Jul 20 09:44:25.907350 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.907309    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-vv6w4\" (UID: \"8d241b72-f123-4dc2-8aa7-c51b7c696d1b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv6w4"
Jul 20 09:44:25.907350 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.907337    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-local-certs\") pod \"istiod-openshift-gateway-7574459656-vv6w4\" (UID: \"8d241b72-f123-4dc2-8aa7-c51b7c696d1b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv6w4"
Jul 20 09:44:25.907690 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.907371    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-cacerts\") pod \"istiod-openshift-gateway-7574459656-vv6w4\" (UID: \"8d241b72-f123-4dc2-8aa7-c51b7c696d1b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv6w4"
Jul 20 09:44:25.907690 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.907402    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-vv6w4\" (UID: \"8d241b72-f123-4dc2-8aa7-c51b7c696d1b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv6w4"
Jul 20 09:44:25.907690 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.907470    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-vv6w4\" (UID: \"8d241b72-f123-4dc2-8aa7-c51b7c696d1b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv6w4"
Jul 20 09:44:25.907690 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.907530    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-istio-token\") pod \"istiod-openshift-gateway-7574459656-vv6w4\" (UID: \"8d241b72-f123-4dc2-8aa7-c51b7c696d1b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv6w4"
Jul 20 09:44:25.908229 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.908192    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-vv6w4\" (UID: \"8d241b72-f123-4dc2-8aa7-c51b7c696d1b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv6w4"
Jul 20 09:44:25.910312 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.910283    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-local-certs\") pod \"istiod-openshift-gateway-7574459656-vv6w4\" (UID: \"8d241b72-f123-4dc2-8aa7-c51b7c696d1b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv6w4"
Jul 20 09:44:25.910436 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.910391    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-vv6w4\" (UID: \"8d241b72-f123-4dc2-8aa7-c51b7c696d1b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv6w4"
Jul 20 09:44:25.910549 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.910528    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-vv6w4\" (UID: \"8d241b72-f123-4dc2-8aa7-c51b7c696d1b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv6w4"
Jul 20 09:44:25.910597 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.910529    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-cacerts\") pod \"istiod-openshift-gateway-7574459656-vv6w4\" (UID: \"8d241b72-f123-4dc2-8aa7-c51b7c696d1b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv6w4"
Jul 20 09:44:25.916890 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.916855    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-istio-token\") pod \"istiod-openshift-gateway-7574459656-vv6w4\" (UID: \"8d241b72-f123-4dc2-8aa7-c51b7c696d1b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv6w4"
Jul 20 09:44:25.917186 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.917157    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99qf2\" (UniqueName: \"kubernetes.io/projected/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-kube-api-access-99qf2\") pod \"istiod-openshift-gateway-7574459656-vv6w4\" (UID: \"8d241b72-f123-4dc2-8aa7-c51b7c696d1b\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv6w4"
Jul 20 09:44:25.960817 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:25.960776    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv6w4"
Jul 20 09:44:26.008796 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:26.008631    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec4dee2b-b4be-4eff-b5a5-d07f9e9d27fc-tls-certs\") pod \"model-serving-api-54d89bd5fc-rhthj\" (UID: \"ec4dee2b-b4be-4eff-b5a5-d07f9e9d27fc\") " pod="opendatahub/model-serving-api-54d89bd5fc-rhthj"
Jul 20 09:44:26.011866 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:26.011809    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec4dee2b-b4be-4eff-b5a5-d07f9e9d27fc-tls-certs\") pod \"model-serving-api-54d89bd5fc-rhthj\" (UID: \"ec4dee2b-b4be-4eff-b5a5-d07f9e9d27fc\") " pod="opendatahub/model-serving-api-54d89bd5fc-rhthj"
Jul 20 09:44:26.127534 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:26.127489    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-vv6w4"]
Jul 20 09:44:26.130333 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:44:26.130301    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d241b72_f123_4dc2_8aa7_c51b7c696d1b.slice/crio-5252a5efbca31aa4f69fdf696227516c9e0eae3d20bcd29d79c1e8034ec7167a WatchSource:0}: Error finding container 5252a5efbca31aa4f69fdf696227516c9e0eae3d20bcd29d79c1e8034ec7167a: Status 404 returned error can't find the container with id 5252a5efbca31aa4f69fdf696227516c9e0eae3d20bcd29d79c1e8034ec7167a
Jul 20 09:44:26.237071 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:26.236909    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-rhthj"
Jul 20 09:44:26.273027 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:26.272936    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-rkmvr" event={"ID":"c047ca14-af23-48e7-84f8-a2b51edec8bf","Type":"ContainerStarted","Data":"da57cd046d01c5b5f0c2ac7bde23b30a881d9905f00677ff8b224fb3247a8147"}
Jul 20 09:44:26.274836 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:26.274800    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv6w4" event={"ID":"8d241b72-f123-4dc2-8aa7-c51b7c696d1b","Type":"ContainerStarted","Data":"5252a5efbca31aa4f69fdf696227516c9e0eae3d20bcd29d79c1e8034ec7167a"}
Jul 20 09:44:26.406040 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:26.405782    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-rhthj"]
Jul 20 09:44:27.971550 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:44:27.971512    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec4dee2b_b4be_4eff_b5a5_d07f9e9d27fc.slice/crio-44778d1a8d83b4be610ded4067ee32ca757749a07aac158d88a56e8331fc4ab6 WatchSource:0}: Error finding container 44778d1a8d83b4be610ded4067ee32ca757749a07aac158d88a56e8331fc4ab6: Status 404 returned error can't find the container with id 44778d1a8d83b4be610ded4067ee32ca757749a07aac158d88a56e8331fc4ab6
Jul 20 09:44:28.285854 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:28.285752    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-rhthj" event={"ID":"ec4dee2b-b4be-4eff-b5a5-d07f9e9d27fc","Type":"ContainerStarted","Data":"44778d1a8d83b4be610ded4067ee32ca757749a07aac158d88a56e8331fc4ab6"}
Jul 20 09:44:30.175101 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:30.175048    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 20 09:44:30.175412 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:30.175173    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892156Ki","pods":"250"}
Jul 20 09:44:30.295565 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:30.295527    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7b6cfc896b-8t64n" event={"ID":"14906f6c-ca75-4f3a-8455-ac29c6cefd38","Type":"ContainerStarted","Data":"011d93d85426dabec63d8676bd7d7ef1b01bebe122a6d0520d2f20a3e55fe6f3"}
Jul 20 09:44:30.297520 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:30.297490    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-rkmvr" event={"ID":"c047ca14-af23-48e7-84f8-a2b51edec8bf","Type":"ContainerStarted","Data":"24ab13e766a12776425a81b4831100fb55c26d4d6ae9ba066592ed162459a776"}
Jul 20 09:44:30.297684 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:30.297550    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-rkmvr"
Jul 20 09:44:30.299277 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:30.299247    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-rhthj" event={"ID":"ec4dee2b-b4be-4eff-b5a5-d07f9e9d27fc","Type":"ContainerStarted","Data":"9044bb86f376b4dff1258e2d1050c9747b85dd6b41d754aa82243f25147cc92f"}
Jul 20 09:44:30.299415 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:30.299321    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-rhthj"
Jul 20 09:44:30.301542 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:30.301506    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv6w4" event={"ID":"8d241b72-f123-4dc2-8aa7-c51b7c696d1b","Type":"ContainerStarted","Data":"6d510a904685806cdd480e030639e1901cbba380e0a5d7056568535c880a2423"}
Jul 20 09:44:30.302104 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:30.302066    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv6w4"
Jul 20 09:44:30.317863 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:30.317803    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7b6cfc896b-8t64n" podStartSLOduration=2.071595161 podStartE2EDuration="7.317782394s" totalImagesPullingTime="5.246187233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 09:44:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:44:30.317362473 +0000 UTC m=+336.868816940" watchObservedRunningTime="2026-07-20 09:44:30.317782394 +0000 UTC m=+336.869236860"
Jul 20 09:44:30.340356 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:30.340304    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv6w4" podStartSLOduration=1.2980225220000001 podStartE2EDuration="5.340285788s" totalImagesPullingTime="4.042263266s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 09:44:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:44:30.339186114 +0000 UTC m=+336.890640609" watchObservedRunningTime="2026-07-20 09:44:30.340285788 +0000 UTC m=+336.891740231"
Jul 20 09:44:30.360188 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:30.360133    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-rkmvr" podStartSLOduration=2.060356405 podStartE2EDuration="6.360116185s" totalImagesPullingTime="4.29975978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 09:44:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:44:30.358724778 +0000 UTC m=+336.910179244" watchObservedRunningTime="2026-07-20 09:44:30.360116185 +0000 UTC m=+336.911570648"
Jul 20 09:44:30.384548 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:30.384492    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-rhthj" podStartSLOduration=3.163813384 podStartE2EDuration="5.384477374s" totalImagesPullingTime="2.22066399s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 09:44:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:44:30.383045281 +0000 UTC m=+336.934499748" watchObservedRunningTime="2026-07-20 09:44:30.384477374 +0000 UTC m=+336.935931873"
Jul 20 09:44:31.307019 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:31.306968    2566 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-vv6w4 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 20 09:44:31.307425 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:31.307045    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv6w4" podUID="8d241b72-f123-4dc2-8aa7-c51b7c696d1b" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 20 09:44:32.309606 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:32.309572    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv6w4"
Jul 20 09:44:35.520912 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:35.520875    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-z8w5d"]
Jul 20 09:44:35.523915 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:35.523889    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-z8w5d"
Jul 20 09:44:35.526654 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:35.526629    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-qhbgp\""
Jul 20 09:44:35.526654 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:35.526640    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 20 09:44:35.533870 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:35.533838    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-r5fbn"]
Jul 20 09:44:35.537128 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:35.537100    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r5fbn"
Jul 20 09:44:35.537539 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:35.537517    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-z8w5d"]
Jul 20 09:44:35.541582 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:35.541549    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-pqz9w\""
Jul 20 09:44:35.541875 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:35.541857    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 20 09:44:35.554470 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:35.554428    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-r5fbn"]
Jul 20 09:44:35.695357 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:35.695316    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95l29\" (UniqueName: \"kubernetes.io/projected/04c45b36-fe41-4276-8b2b-0991fd92123b-kube-api-access-95l29\") pod \"kserve-controller-manager-d6f4b65b8-z8w5d\" (UID: \"04c45b36-fe41-4276-8b2b-0991fd92123b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-z8w5d"
Jul 20 09:44:35.695542 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:35.695382    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/04c45b36-fe41-4276-8b2b-0991fd92123b-cert\") pod \"kserve-controller-manager-d6f4b65b8-z8w5d\" (UID: \"04c45b36-fe41-4276-8b2b-0991fd92123b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-z8w5d"
Jul 20 09:44:35.695542 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:35.695409    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4f5d4b58-2e51-4be3-b5f3-22a5e75b273a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-r5fbn\" (UID: \"4f5d4b58-2e51-4be3-b5f3-22a5e75b273a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r5fbn"
Jul 20 09:44:35.695542 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:35.695473    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmqgm\" (UniqueName: \"kubernetes.io/projected/4f5d4b58-2e51-4be3-b5f3-22a5e75b273a-kube-api-access-qmqgm\") pod \"llmisvc-controller-manager-777c6bd7d5-r5fbn\" (UID: \"4f5d4b58-2e51-4be3-b5f3-22a5e75b273a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r5fbn"
Jul 20 09:44:35.795990 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:35.795879    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmqgm\" (UniqueName: \"kubernetes.io/projected/4f5d4b58-2e51-4be3-b5f3-22a5e75b273a-kube-api-access-qmqgm\") pod \"llmisvc-controller-manager-777c6bd7d5-r5fbn\" (UID: \"4f5d4b58-2e51-4be3-b5f3-22a5e75b273a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r5fbn"
Jul 20 09:44:35.795990 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:35.795966    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95l29\" (UniqueName: \"kubernetes.io/projected/04c45b36-fe41-4276-8b2b-0991fd92123b-kube-api-access-95l29\") pod \"kserve-controller-manager-d6f4b65b8-z8w5d\" (UID: \"04c45b36-fe41-4276-8b2b-0991fd92123b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-z8w5d"
Jul 20 09:44:35.796189 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:35.796030    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/04c45b36-fe41-4276-8b2b-0991fd92123b-cert\") pod \"kserve-controller-manager-d6f4b65b8-z8w5d\" (UID: \"04c45b36-fe41-4276-8b2b-0991fd92123b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-z8w5d"
Jul 20 09:44:35.796189 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:44:35.796124    2566 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 20 09:44:35.796189 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:44:35.796180    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04c45b36-fe41-4276-8b2b-0991fd92123b-cert podName:04c45b36-fe41-4276-8b2b-0991fd92123b nodeName:}" failed. No retries permitted until 2026-07-20 09:44:36.296161806 +0000 UTC m=+342.847616248 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/04c45b36-fe41-4276-8b2b-0991fd92123b-cert") pod "kserve-controller-manager-d6f4b65b8-z8w5d" (UID: "04c45b36-fe41-4276-8b2b-0991fd92123b") : secret "kserve-webhook-server-cert" not found
Jul 20 09:44:35.796294 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:35.796220    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4f5d4b58-2e51-4be3-b5f3-22a5e75b273a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-r5fbn\" (UID: \"4f5d4b58-2e51-4be3-b5f3-22a5e75b273a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r5fbn"
Jul 20 09:44:35.796385 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:44:35.796369    2566 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 20 09:44:35.796453 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:44:35.796443    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f5d4b58-2e51-4be3-b5f3-22a5e75b273a-cert podName:4f5d4b58-2e51-4be3-b5f3-22a5e75b273a nodeName:}" failed. No retries permitted until 2026-07-20 09:44:36.296425636 +0000 UTC m=+342.847880081 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4f5d4b58-2e51-4be3-b5f3-22a5e75b273a-cert") pod "llmisvc-controller-manager-777c6bd7d5-r5fbn" (UID: "4f5d4b58-2e51-4be3-b5f3-22a5e75b273a") : secret "llmisvc-webhook-server-cert" not found
Jul 20 09:44:35.809764 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:35.809725    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95l29\" (UniqueName: \"kubernetes.io/projected/04c45b36-fe41-4276-8b2b-0991fd92123b-kube-api-access-95l29\") pod \"kserve-controller-manager-d6f4b65b8-z8w5d\" (UID: \"04c45b36-fe41-4276-8b2b-0991fd92123b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-z8w5d"
Jul 20 09:44:35.813385 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:35.813352    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmqgm\" (UniqueName: \"kubernetes.io/projected/4f5d4b58-2e51-4be3-b5f3-22a5e75b273a-kube-api-access-qmqgm\") pod \"llmisvc-controller-manager-777c6bd7d5-r5fbn\" (UID: \"4f5d4b58-2e51-4be3-b5f3-22a5e75b273a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r5fbn"
Jul 20 09:44:36.300420 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:36.300372    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/04c45b36-fe41-4276-8b2b-0991fd92123b-cert\") pod \"kserve-controller-manager-d6f4b65b8-z8w5d\" (UID: \"04c45b36-fe41-4276-8b2b-0991fd92123b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-z8w5d"
Jul 20 09:44:36.300420 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:36.300413    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4f5d4b58-2e51-4be3-b5f3-22a5e75b273a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-r5fbn\" (UID: \"4f5d4b58-2e51-4be3-b5f3-22a5e75b273a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r5fbn"
Jul 20 09:44:36.303130 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:36.303089    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/04c45b36-fe41-4276-8b2b-0991fd92123b-cert\") pod \"kserve-controller-manager-d6f4b65b8-z8w5d\" (UID: \"04c45b36-fe41-4276-8b2b-0991fd92123b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-z8w5d"
Jul 20 09:44:36.303288 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:36.303161    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4f5d4b58-2e51-4be3-b5f3-22a5e75b273a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-r5fbn\" (UID: \"4f5d4b58-2e51-4be3-b5f3-22a5e75b273a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r5fbn"
Jul 20 09:44:36.438742 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:36.438704    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-z8w5d"
Jul 20 09:44:36.449164 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:36.449119    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r5fbn"
Jul 20 09:44:36.597837 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:36.597621    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-z8w5d"]
Jul 20 09:44:36.600645 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:44:36.600615    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04c45b36_fe41_4276_8b2b_0991fd92123b.slice/crio-f1daeec6a0601e060c80074c5af6dd4083a07a84c12a21fa634f94714e8c1b08 WatchSource:0}: Error finding container f1daeec6a0601e060c80074c5af6dd4083a07a84c12a21fa634f94714e8c1b08: Status 404 returned error can't find the container with id f1daeec6a0601e060c80074c5af6dd4083a07a84c12a21fa634f94714e8c1b08
Jul 20 09:44:36.617118 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:36.617095    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-r5fbn"]
Jul 20 09:44:36.619787 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:44:36.619746    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod4f5d4b58_2e51_4be3_b5f3_22a5e75b273a.slice/crio-36d3c957bfcb5926069bc38f109c107d7a61f7e69a96cf7956c7773d72cda85a WatchSource:0}: Error finding container 36d3c957bfcb5926069bc38f109c107d7a61f7e69a96cf7956c7773d72cda85a: Status 404 returned error can't find the container with id 36d3c957bfcb5926069bc38f109c107d7a61f7e69a96cf7956c7773d72cda85a
Jul 20 09:44:36.883642 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:36.883544    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-vv6w4"]
Jul 20 09:44:36.883801 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:36.883734    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv6w4" podUID="8d241b72-f123-4dc2-8aa7-c51b7c696d1b" containerName="discovery" containerID="cri-o://6d510a904685806cdd480e030639e1901cbba380e0a5d7056568535c880a2423" gracePeriod=30
Jul 20 09:44:37.139409 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:37.139325    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv6w4"
Jul 20 09:44:37.313798 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:37.313757    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-local-certs\") pod \"8d241b72-f123-4dc2-8aa7-c51b7c696d1b\" (UID: \"8d241b72-f123-4dc2-8aa7-c51b7c696d1b\") "
Jul 20 09:44:37.314006 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:37.313838    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-istio-kubeconfig\") pod \"8d241b72-f123-4dc2-8aa7-c51b7c696d1b\" (UID: \"8d241b72-f123-4dc2-8aa7-c51b7c696d1b\") "
Jul 20 09:44:37.314006 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:37.313872    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-istio-csr-dns-cert\") pod \"8d241b72-f123-4dc2-8aa7-c51b7c696d1b\" (UID: \"8d241b72-f123-4dc2-8aa7-c51b7c696d1b\") "
Jul 20 09:44:37.314006 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:37.313911    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-istio-csr-ca-configmap\") pod \"8d241b72-f123-4dc2-8aa7-c51b7c696d1b\" (UID: \"8d241b72-f123-4dc2-8aa7-c51b7c696d1b\") "
Jul 20 09:44:37.314006 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:37.313940    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-99qf2\" (UniqueName: \"kubernetes.io/projected/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-kube-api-access-99qf2\") pod \"8d241b72-f123-4dc2-8aa7-c51b7c696d1b\" (UID: \"8d241b72-f123-4dc2-8aa7-c51b7c696d1b\") "
Jul 20 09:44:37.314006 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:37.313999    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-istio-token\") pod \"8d241b72-f123-4dc2-8aa7-c51b7c696d1b\" (UID: \"8d241b72-f123-4dc2-8aa7-c51b7c696d1b\") "
Jul 20 09:44:37.314274 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:37.314044    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-cacerts\") pod \"8d241b72-f123-4dc2-8aa7-c51b7c696d1b\" (UID: \"8d241b72-f123-4dc2-8aa7-c51b7c696d1b\") "
Jul 20 09:44:37.315036 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:37.314952    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "8d241b72-f123-4dc2-8aa7-c51b7c696d1b" (UID: "8d241b72-f123-4dc2-8aa7-c51b7c696d1b"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 09:44:37.317087 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:37.317013    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "8d241b72-f123-4dc2-8aa7-c51b7c696d1b" (UID: "8d241b72-f123-4dc2-8aa7-c51b7c696d1b"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 09:44:37.317349 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:37.317313    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-local-certs" (OuterVolumeSpecName: "local-certs") pod "8d241b72-f123-4dc2-8aa7-c51b7c696d1b" (UID: "8d241b72-f123-4dc2-8aa7-c51b7c696d1b"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 09:44:37.317434 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:37.317414    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "8d241b72-f123-4dc2-8aa7-c51b7c696d1b" (UID: "8d241b72-f123-4dc2-8aa7-c51b7c696d1b"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 09:44:37.317597 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:37.317572    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-cacerts" (OuterVolumeSpecName: "cacerts") pod "8d241b72-f123-4dc2-8aa7-c51b7c696d1b" (UID: "8d241b72-f123-4dc2-8aa7-c51b7c696d1b"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 09:44:37.318133 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:37.317899    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-kube-api-access-99qf2" (OuterVolumeSpecName: "kube-api-access-99qf2") pod "8d241b72-f123-4dc2-8aa7-c51b7c696d1b" (UID: "8d241b72-f123-4dc2-8aa7-c51b7c696d1b"). InnerVolumeSpecName "kube-api-access-99qf2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 09:44:37.318133 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:37.317914    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-istio-token" (OuterVolumeSpecName: "istio-token") pod "8d241b72-f123-4dc2-8aa7-c51b7c696d1b" (UID: "8d241b72-f123-4dc2-8aa7-c51b7c696d1b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 09:44:37.329152 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:37.329109    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r5fbn" event={"ID":"4f5d4b58-2e51-4be3-b5f3-22a5e75b273a","Type":"ContainerStarted","Data":"36d3c957bfcb5926069bc38f109c107d7a61f7e69a96cf7956c7773d72cda85a"}
Jul 20 09:44:37.330696 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:37.330667    2566 generic.go:356] "Generic (PLEG): container finished" podID="8d241b72-f123-4dc2-8aa7-c51b7c696d1b" containerID="6d510a904685806cdd480e030639e1901cbba380e0a5d7056568535c880a2423" exitCode=0
Jul 20 09:44:37.330844 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:37.330724    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv6w4"
Jul 20 09:44:37.330844 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:37.330758    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv6w4" event={"ID":"8d241b72-f123-4dc2-8aa7-c51b7c696d1b","Type":"ContainerDied","Data":"6d510a904685806cdd480e030639e1901cbba380e0a5d7056568535c880a2423"}
Jul 20 09:44:37.330844 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:37.330810    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv6w4" event={"ID":"8d241b72-f123-4dc2-8aa7-c51b7c696d1b","Type":"ContainerDied","Data":"5252a5efbca31aa4f69fdf696227516c9e0eae3d20bcd29d79c1e8034ec7167a"}
Jul 20 09:44:37.330844 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:37.330841    2566 scope.go:117] "RemoveContainer" containerID="6d510a904685806cdd480e030639e1901cbba380e0a5d7056568535c880a2423"
Jul 20 09:44:37.332091 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:37.332059    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-z8w5d" event={"ID":"04c45b36-fe41-4276-8b2b-0991fd92123b","Type":"ContainerStarted","Data":"f1daeec6a0601e060c80074c5af6dd4083a07a84c12a21fa634f94714e8c1b08"}
Jul 20 09:44:37.372469 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:37.372431    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-vv6w4"]
Jul 20 09:44:37.383436 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:37.383352    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-vv6w4"]
Jul 20 09:44:37.393334 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:37.393312    2566 scope.go:117] "RemoveContainer" containerID="6d510a904685806cdd480e030639e1901cbba380e0a5d7056568535c880a2423"
Jul 20 09:44:37.393721 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:44:37.393688    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d510a904685806cdd480e030639e1901cbba380e0a5d7056568535c880a2423\": container with ID starting with 6d510a904685806cdd480e030639e1901cbba380e0a5d7056568535c880a2423 not found: ID does not exist" containerID="6d510a904685806cdd480e030639e1901cbba380e0a5d7056568535c880a2423"
Jul 20 09:44:37.393869 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:37.393732    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d510a904685806cdd480e030639e1901cbba380e0a5d7056568535c880a2423"} err="failed to get container status \"6d510a904685806cdd480e030639e1901cbba380e0a5d7056568535c880a2423\": rpc error: code = NotFound desc = could not find container \"6d510a904685806cdd480e030639e1901cbba380e0a5d7056568535c880a2423\": container with ID starting with 6d510a904685806cdd480e030639e1901cbba380e0a5d7056568535c880a2423 not found: ID does not exist"
Jul 20 09:44:37.415231 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:37.415199    2566 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-local-certs\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 09:44:37.415231 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:37.415231    2566 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-istio-kubeconfig\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 09:44:37.415424 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:37.415244    2566 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-istio-csr-dns-cert\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 09:44:37.415424 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:37.415256    2566 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-istio-csr-ca-configmap\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 09:44:37.415424 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:37.415268    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-99qf2\" (UniqueName: \"kubernetes.io/projected/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-kube-api-access-99qf2\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 09:44:37.415424 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:37.415279    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-istio-token\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 09:44:37.415424 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:37.415290    2566 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8d241b72-f123-4dc2-8aa7-c51b7c696d1b-cacerts\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 09:44:37.924366 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:37.924324    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d241b72-f123-4dc2-8aa7-c51b7c696d1b" path="/var/lib/kubelet/pods/8d241b72-f123-4dc2-8aa7-c51b7c696d1b/volumes"
Jul 20 09:44:39.345999 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:39.345950    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-z8w5d" event={"ID":"04c45b36-fe41-4276-8b2b-0991fd92123b","Type":"ContainerStarted","Data":"b52c1b1c47bf4c5690dd68813ad79cbfff750dc960a586b1fd2a0d01ac4d97ea"}
Jul 20 09:44:39.346422 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:39.346040    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-z8w5d"
Jul 20 09:44:39.368921 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:39.368856    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-z8w5d" podStartSLOduration=2.049735663 podStartE2EDuration="4.368837633s" totalImagesPullingTime="2.31910197s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 09:44:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:44:39.368013681 +0000 UTC m=+345.919468142" watchObservedRunningTime="2026-07-20 09:44:39.368837633 +0000 UTC m=+345.920292149"
Jul 20 09:44:40.352152 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:40.352112    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r5fbn" event={"ID":"4f5d4b58-2e51-4be3-b5f3-22a5e75b273a","Type":"ContainerStarted","Data":"2a7cf5dc1769182846c0b14047c87bd365017584f3d8d3506ef99d98efbebeb0"}
Jul 20 09:44:40.374603 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:40.374542    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r5fbn" podStartSLOduration=2.377657918 podStartE2EDuration="5.374523508s" totalImagesPullingTime="2.99686559s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 09:44:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:44:40.373779323 +0000 UTC m=+346.925233787" watchObservedRunningTime="2026-07-20 09:44:40.374523508 +0000 UTC m=+346.925978000"
Jul 20 09:44:41.309236 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:41.309203    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-rhthj"
Jul 20 09:44:41.353606 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:41.353573    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-rkmvr"
Jul 20 09:44:41.357424 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:44:41.357392    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r5fbn"
Jul 20 09:45:10.357482 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:10.357437    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-z8w5d"
Jul 20 09:45:12.363429 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:12.363396    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-r5fbn"
Jul 20 09:45:24.936817 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:24.936771    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9t5ww"]
Jul 20 09:45:24.937264 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:24.937094    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d241b72-f123-4dc2-8aa7-c51b7c696d1b" containerName="discovery"
Jul 20 09:45:24.937264 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:24.937107    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d241b72-f123-4dc2-8aa7-c51b7c696d1b" containerName="discovery"
Jul 20 09:45:24.937264 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:24.937164    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d241b72-f123-4dc2-8aa7-c51b7c696d1b" containerName="discovery"
Jul 20 09:45:24.939124 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:24.939100    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-9t5ww"
Jul 20 09:45:24.941810 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:24.941783    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 20 09:45:24.942890 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:24.942860    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-kkfkx\""
Jul 20 09:45:24.942890 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:24.942890    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 20 09:45:24.951172 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:24.951125    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9t5ww"]
Jul 20 09:45:24.990839 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:24.990797    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgfjg\" (UniqueName: \"kubernetes.io/projected/3f0e057a-47d6-4cef-9ff0-4d312ddc2a18-kube-api-access-tgfjg\") pod \"kuadrant-operator-catalog-9t5ww\" (UID: \"3f0e057a-47d6-4cef-9ff0-4d312ddc2a18\") " pod="kuadrant-system/kuadrant-operator-catalog-9t5ww"
Jul 20 09:45:25.092007 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:25.091933    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgfjg\" (UniqueName: \"kubernetes.io/projected/3f0e057a-47d6-4cef-9ff0-4d312ddc2a18-kube-api-access-tgfjg\") pod \"kuadrant-operator-catalog-9t5ww\" (UID: \"3f0e057a-47d6-4cef-9ff0-4d312ddc2a18\") " pod="kuadrant-system/kuadrant-operator-catalog-9t5ww"
Jul 20 09:45:25.101032 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:25.101000    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgfjg\" (UniqueName: \"kubernetes.io/projected/3f0e057a-47d6-4cef-9ff0-4d312ddc2a18-kube-api-access-tgfjg\") pod \"kuadrant-operator-catalog-9t5ww\" (UID: \"3f0e057a-47d6-4cef-9ff0-4d312ddc2a18\") " pod="kuadrant-system/kuadrant-operator-catalog-9t5ww"
Jul 20 09:45:25.250329 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:25.250237    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-9t5ww"
Jul 20 09:45:25.287427 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:25.287394    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9t5ww"]
Jul 20 09:45:25.390373 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:25.390337    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9t5ww"]
Jul 20 09:45:25.394200 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:45:25.394172    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f0e057a_47d6_4cef_9ff0_4d312ddc2a18.slice/crio-6b4aed3692362cca0f01160de95ca9f808b4039e8780ec74ea6c5ee29a0fbb68 WatchSource:0}: Error finding container 6b4aed3692362cca0f01160de95ca9f808b4039e8780ec74ea6c5ee29a0fbb68: Status 404 returned error can't find the container with id 6b4aed3692362cca0f01160de95ca9f808b4039e8780ec74ea6c5ee29a0fbb68
Jul 20 09:45:25.497651 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:25.497611    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n4ljw"]
Jul 20 09:45:25.500318 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:25.500299    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-n4ljw"
Jul 20 09:45:25.508999 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:25.508949    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n4ljw"]
Jul 20 09:45:25.524938 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:25.524893    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-9t5ww" event={"ID":"3f0e057a-47d6-4cef-9ff0-4d312ddc2a18","Type":"ContainerStarted","Data":"6b4aed3692362cca0f01160de95ca9f808b4039e8780ec74ea6c5ee29a0fbb68"}
Jul 20 09:45:25.595631 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:25.595585    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7lvb\" (UniqueName: \"kubernetes.io/projected/93704a5a-df36-40b8-9f69-a7b5342ecc36-kube-api-access-w7lvb\") pod \"kuadrant-operator-catalog-n4ljw\" (UID: \"93704a5a-df36-40b8-9f69-a7b5342ecc36\") " pod="kuadrant-system/kuadrant-operator-catalog-n4ljw"
Jul 20 09:45:25.696484 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:25.696447    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7lvb\" (UniqueName: \"kubernetes.io/projected/93704a5a-df36-40b8-9f69-a7b5342ecc36-kube-api-access-w7lvb\") pod \"kuadrant-operator-catalog-n4ljw\" (UID: \"93704a5a-df36-40b8-9f69-a7b5342ecc36\") " pod="kuadrant-system/kuadrant-operator-catalog-n4ljw"
Jul 20 09:45:25.705028 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:25.704964    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7lvb\" (UniqueName: \"kubernetes.io/projected/93704a5a-df36-40b8-9f69-a7b5342ecc36-kube-api-access-w7lvb\") pod \"kuadrant-operator-catalog-n4ljw\" (UID: \"93704a5a-df36-40b8-9f69-a7b5342ecc36\") " pod="kuadrant-system/kuadrant-operator-catalog-n4ljw"
Jul 20 09:45:25.810477 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:25.810430    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-n4ljw"
Jul 20 09:45:25.960078 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:25.960015    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n4ljw"]
Jul 20 09:45:25.962240 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:45:25.962211    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93704a5a_df36_40b8_9f69_a7b5342ecc36.slice/crio-cd7a83ea44bd100fa1e7c072d05eaf52b2c53cd4713bd550ce3b242e13809b52 WatchSource:0}: Error finding container cd7a83ea44bd100fa1e7c072d05eaf52b2c53cd4713bd550ce3b242e13809b52: Status 404 returned error can't find the container with id cd7a83ea44bd100fa1e7c072d05eaf52b2c53cd4713bd550ce3b242e13809b52
Jul 20 09:45:26.530591 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:26.530555    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-n4ljw" event={"ID":"93704a5a-df36-40b8-9f69-a7b5342ecc36","Type":"ContainerStarted","Data":"cd7a83ea44bd100fa1e7c072d05eaf52b2c53cd4713bd550ce3b242e13809b52"}
Jul 20 09:45:28.541575 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:28.541539    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-n4ljw" event={"ID":"93704a5a-df36-40b8-9f69-a7b5342ecc36","Type":"ContainerStarted","Data":"56f7cc71b7e37b114489ab1290180410b6270183badb001da9acea17624bf0ce"}
Jul 20 09:45:28.543076 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:28.543044    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-9t5ww" event={"ID":"3f0e057a-47d6-4cef-9ff0-4d312ddc2a18","Type":"ContainerStarted","Data":"f2f6814726aa7836d1d81c7315ddb63a838cdfe5e4679e0c8d123e8cf9695c86"}
Jul 20 09:45:28.543229 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:28.543140    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-9t5ww" podUID="3f0e057a-47d6-4cef-9ff0-4d312ddc2a18" containerName="registry-server" containerID="cri-o://f2f6814726aa7836d1d81c7315ddb63a838cdfe5e4679e0c8d123e8cf9695c86" gracePeriod=2
Jul 20 09:45:28.558815 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:28.558767    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-n4ljw" podStartSLOduration=1.896113173 podStartE2EDuration="3.558752111s" totalImagesPullingTime="1.662638938s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 09:45:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:45:28.557393671 +0000 UTC m=+395.108848134" watchObservedRunningTime="2026-07-20 09:45:28.558752111 +0000 UTC m=+395.110206609"
Jul 20 09:45:28.575916 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:28.575859    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-9t5ww" podStartSLOduration=2.372003975 podStartE2EDuration="4.575842072s" totalImagesPullingTime="2.203838097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 09:45:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:45:28.574665091 +0000 UTC m=+395.126119557" watchObservedRunningTime="2026-07-20 09:45:28.575842072 +0000 UTC m=+395.127296549"
Jul 20 09:45:28.782374 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:28.782343    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-9t5ww"
Jul 20 09:45:28.824163 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:28.824125    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tgfjg\" (UniqueName: \"kubernetes.io/projected/3f0e057a-47d6-4cef-9ff0-4d312ddc2a18-kube-api-access-tgfjg\") pod \"3f0e057a-47d6-4cef-9ff0-4d312ddc2a18\" (UID: \"3f0e057a-47d6-4cef-9ff0-4d312ddc2a18\") "
Jul 20 09:45:28.826465 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:28.826421    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f0e057a-47d6-4cef-9ff0-4d312ddc2a18-kube-api-access-tgfjg" (OuterVolumeSpecName: "kube-api-access-tgfjg") pod "3f0e057a-47d6-4cef-9ff0-4d312ddc2a18" (UID: "3f0e057a-47d6-4cef-9ff0-4d312ddc2a18"). InnerVolumeSpecName "kube-api-access-tgfjg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 09:45:28.924739 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:28.924688    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tgfjg\" (UniqueName: \"kubernetes.io/projected/3f0e057a-47d6-4cef-9ff0-4d312ddc2a18-kube-api-access-tgfjg\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 09:45:29.548381 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:29.548344    2566 generic.go:356] "Generic (PLEG): container finished" podID="3f0e057a-47d6-4cef-9ff0-4d312ddc2a18" containerID="f2f6814726aa7836d1d81c7315ddb63a838cdfe5e4679e0c8d123e8cf9695c86" exitCode=0
Jul 20 09:45:29.548854 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:29.548392    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-9t5ww"
Jul 20 09:45:29.548854 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:29.548432    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-9t5ww" event={"ID":"3f0e057a-47d6-4cef-9ff0-4d312ddc2a18","Type":"ContainerDied","Data":"f2f6814726aa7836d1d81c7315ddb63a838cdfe5e4679e0c8d123e8cf9695c86"}
Jul 20 09:45:29.548854 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:29.548483    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-9t5ww" event={"ID":"3f0e057a-47d6-4cef-9ff0-4d312ddc2a18","Type":"ContainerDied","Data":"6b4aed3692362cca0f01160de95ca9f808b4039e8780ec74ea6c5ee29a0fbb68"}
Jul 20 09:45:29.548854 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:29.548503    2566 scope.go:117] "RemoveContainer" containerID="f2f6814726aa7836d1d81c7315ddb63a838cdfe5e4679e0c8d123e8cf9695c86"
Jul 20 09:45:29.557578 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:29.557559    2566 scope.go:117] "RemoveContainer" containerID="f2f6814726aa7836d1d81c7315ddb63a838cdfe5e4679e0c8d123e8cf9695c86"
Jul 20 09:45:29.557883 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:45:29.557866    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2f6814726aa7836d1d81c7315ddb63a838cdfe5e4679e0c8d123e8cf9695c86\": container with ID starting with f2f6814726aa7836d1d81c7315ddb63a838cdfe5e4679e0c8d123e8cf9695c86 not found: ID does not exist" containerID="f2f6814726aa7836d1d81c7315ddb63a838cdfe5e4679e0c8d123e8cf9695c86"
Jul 20 09:45:29.557921 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:29.557894    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2f6814726aa7836d1d81c7315ddb63a838cdfe5e4679e0c8d123e8cf9695c86"} err="failed to get container status \"f2f6814726aa7836d1d81c7315ddb63a838cdfe5e4679e0c8d123e8cf9695c86\": rpc error: code = NotFound desc = could not find container \"f2f6814726aa7836d1d81c7315ddb63a838cdfe5e4679e0c8d123e8cf9695c86\": container with ID starting with f2f6814726aa7836d1d81c7315ddb63a838cdfe5e4679e0c8d123e8cf9695c86 not found: ID does not exist"
Jul 20 09:45:29.572736 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:29.572700    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9t5ww"]
Jul 20 09:45:29.580221 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:29.580187    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9t5ww"]
Jul 20 09:45:29.919703 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:29.919624    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f0e057a-47d6-4cef-9ff0-4d312ddc2a18" path="/var/lib/kubelet/pods/3f0e057a-47d6-4cef-9ff0-4d312ddc2a18/volumes"
Jul 20 09:45:35.810931 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:35.810880    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-n4ljw"
Jul 20 09:45:35.811381 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:35.810958    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-n4ljw"
Jul 20 09:45:35.834685 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:35.834656    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-n4ljw"
Jul 20 09:45:36.599941 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:36.599912    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-n4ljw"
Jul 20 09:45:56.519077 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:56.519032    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-fdbb8"]
Jul 20 09:45:56.519482 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:56.519329    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f0e057a-47d6-4cef-9ff0-4d312ddc2a18" containerName="registry-server"
Jul 20 09:45:56.519482 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:56.519340    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f0e057a-47d6-4cef-9ff0-4d312ddc2a18" containerName="registry-server"
Jul 20 09:45:56.519482 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:56.519400    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f0e057a-47d6-4cef-9ff0-4d312ddc2a18" containerName="registry-server"
Jul 20 09:45:56.524090 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:56.524064    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-fdbb8"
Jul 20 09:45:56.527035 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:56.527002    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-gtwtz\""
Jul 20 09:45:56.527205 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:56.527076    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 20 09:45:56.535123 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:56.535089    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-fdbb8"]
Jul 20 09:45:56.651060 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:56.651008    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghjd5\" (UniqueName: \"kubernetes.io/projected/f772c922-ff2c-428f-85d6-6cd11bf9f591-kube-api-access-ghjd5\") pod \"dns-operator-controller-manager-864dd78c4c-fdbb8\" (UID: \"f772c922-ff2c-428f-85d6-6cd11bf9f591\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-fdbb8"
Jul 20 09:45:56.752515 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:56.752468    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghjd5\" (UniqueName: \"kubernetes.io/projected/f772c922-ff2c-428f-85d6-6cd11bf9f591-kube-api-access-ghjd5\") pod \"dns-operator-controller-manager-864dd78c4c-fdbb8\" (UID: \"f772c922-ff2c-428f-85d6-6cd11bf9f591\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-fdbb8"
Jul 20 09:45:56.762937 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:56.762903    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghjd5\" (UniqueName: \"kubernetes.io/projected/f772c922-ff2c-428f-85d6-6cd11bf9f591-kube-api-access-ghjd5\") pod \"dns-operator-controller-manager-864dd78c4c-fdbb8\" (UID: \"f772c922-ff2c-428f-85d6-6cd11bf9f591\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-fdbb8"
Jul 20 09:45:56.834885 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:56.834852    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-fdbb8"
Jul 20 09:45:56.988873 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:56.988835    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-fdbb8"]
Jul 20 09:45:56.991693 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:45:56.991653    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf772c922_ff2c_428f_85d6_6cd11bf9f591.slice/crio-d2430a55a4af9e278bd8772d599c169d7839ba7b40e0d1f1df50aa73a75ad96c WatchSource:0}: Error finding container d2430a55a4af9e278bd8772d599c169d7839ba7b40e0d1f1df50aa73a75ad96c: Status 404 returned error can't find the container with id d2430a55a4af9e278bd8772d599c169d7839ba7b40e0d1f1df50aa73a75ad96c
Jul 20 09:45:57.662652 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:57.662616    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-fdbb8" event={"ID":"f772c922-ff2c-428f-85d6-6cd11bf9f591","Type":"ContainerStarted","Data":"d2430a55a4af9e278bd8772d599c169d7839ba7b40e0d1f1df50aa73a75ad96c"}
Jul 20 09:45:59.679151 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:59.679105    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-fdbb8" event={"ID":"f772c922-ff2c-428f-85d6-6cd11bf9f591","Type":"ContainerStarted","Data":"1ed190b76adcd8398b8457e3976c702c0fceb7c7af6b9ad647d360da447a7f36"}
Jul 20 09:45:59.679563 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:59.679285    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-fdbb8"
Jul 20 09:45:59.697577 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:45:59.697516    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-fdbb8" podStartSLOduration=1.9482268999999999 podStartE2EDuration="3.697498969s" totalImagesPullingTime="1.749272069s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 09:45:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:45:59.696887549 +0000 UTC m=+426.248342014" watchObservedRunningTime="2026-07-20 09:45:59.697498969 +0000 UTC m=+426.248953530"
Jul 20 09:46:00.672966 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:00.672926    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-4mff6"]
Jul 20 09:46:00.676329 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:00.676306    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-4mff6"
Jul 20 09:46:00.681377 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:00.681336    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-lvprg\""
Jul 20 09:46:00.701320 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:00.701280    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-4mff6"]
Jul 20 09:46:00.788533 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:00.788476    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgtb9\" (UniqueName: \"kubernetes.io/projected/ca3131a0-9844-4886-a94d-502f10b00b75-kube-api-access-vgtb9\") pod \"authorino-operator-75cc8d5cb7-4mff6\" (UID: \"ca3131a0-9844-4886-a94d-502f10b00b75\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-4mff6"
Jul 20 09:46:00.890014 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:00.889942    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgtb9\" (UniqueName: \"kubernetes.io/projected/ca3131a0-9844-4886-a94d-502f10b00b75-kube-api-access-vgtb9\") pod \"authorino-operator-75cc8d5cb7-4mff6\" (UID: \"ca3131a0-9844-4886-a94d-502f10b00b75\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-4mff6"
Jul 20 09:46:00.899140 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:00.899102    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgtb9\" (UniqueName: \"kubernetes.io/projected/ca3131a0-9844-4886-a94d-502f10b00b75-kube-api-access-vgtb9\") pod \"authorino-operator-75cc8d5cb7-4mff6\" (UID: \"ca3131a0-9844-4886-a94d-502f10b00b75\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-4mff6"
Jul 20 09:46:00.987233 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:00.987134    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-4mff6"
Jul 20 09:46:01.142876 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:01.142845    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-4mff6"]
Jul 20 09:46:01.145637 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:46:01.145603    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca3131a0_9844_4886_a94d_502f10b00b75.slice/crio-8ed7b804c838b1a8a48b3ae63ba111615af552a9de40e7797dfd8ce9c8743dad WatchSource:0}: Error finding container 8ed7b804c838b1a8a48b3ae63ba111615af552a9de40e7797dfd8ce9c8743dad: Status 404 returned error can't find the container with id 8ed7b804c838b1a8a48b3ae63ba111615af552a9de40e7797dfd8ce9c8743dad
Jul 20 09:46:01.697072 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:01.697037    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-4mff6" event={"ID":"ca3131a0-9844-4886-a94d-502f10b00b75","Type":"ContainerStarted","Data":"8ed7b804c838b1a8a48b3ae63ba111615af552a9de40e7797dfd8ce9c8743dad"}
Jul 20 09:46:03.707670 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:03.707570    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-4mff6" event={"ID":"ca3131a0-9844-4886-a94d-502f10b00b75","Type":"ContainerStarted","Data":"9be34b0f82cf1b546f5eae45f93b5a08976b5da833c9c5ed0e36f2529d68bb6a"}
Jul 20 09:46:03.708071 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:03.707685    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-4mff6"
Jul 20 09:46:03.727716 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:03.727661    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-4mff6" podStartSLOduration=1.5091262570000001 podStartE2EDuration="3.727642383s" totalImagesPullingTime="2.218516126s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 09:46:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:46:03.724679705 +0000 UTC m=+430.276134168" watchObservedRunningTime="2026-07-20 09:46:03.727642383 +0000 UTC m=+430.279096921"
Jul 20 09:46:05.757772 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:05.757733    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-plzn8"]
Jul 20 09:46:05.761212 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:05.761188    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-plzn8"
Jul 20 09:46:05.764130 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:05.764099    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-xctf2\""
Jul 20 09:46:05.777144 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:05.777111    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-plzn8"]
Jul 20 09:46:05.936060 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:05.935951    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnvt4\" (UniqueName: \"kubernetes.io/projected/9269faf3-2bba-4db5-9344-26d619abce52-kube-api-access-tnvt4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-plzn8\" (UID: \"9269faf3-2bba-4db5-9344-26d619abce52\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-plzn8"
Jul 20 09:46:05.936326 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:05.936082    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9269faf3-2bba-4db5-9344-26d619abce52-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-plzn8\" (UID: \"9269faf3-2bba-4db5-9344-26d619abce52\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-plzn8"
Jul 20 09:46:06.036870 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:06.036781    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnvt4\" (UniqueName: \"kubernetes.io/projected/9269faf3-2bba-4db5-9344-26d619abce52-kube-api-access-tnvt4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-plzn8\" (UID: \"9269faf3-2bba-4db5-9344-26d619abce52\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-plzn8"
Jul 20 09:46:06.036870 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:06.036830    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9269faf3-2bba-4db5-9344-26d619abce52-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-plzn8\" (UID: \"9269faf3-2bba-4db5-9344-26d619abce52\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-plzn8"
Jul 20 09:46:06.037183 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:06.037165    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9269faf3-2bba-4db5-9344-26d619abce52-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-plzn8\" (UID: \"9269faf3-2bba-4db5-9344-26d619abce52\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-plzn8"
Jul 20 09:46:06.049979 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:06.049947    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnvt4\" (UniqueName: \"kubernetes.io/projected/9269faf3-2bba-4db5-9344-26d619abce52-kube-api-access-tnvt4\") pod \"kuadrant-operator-controller-manager-8684c9ff49-plzn8\" (UID: \"9269faf3-2bba-4db5-9344-26d619abce52\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-plzn8"
Jul 20 09:46:06.070713 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:06.070670    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-plzn8"
Jul 20 09:46:06.243954 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:06.243930    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-plzn8"]
Jul 20 09:46:06.246319 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:46:06.246285    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9269faf3_2bba_4db5_9344_26d619abce52.slice/crio-a73580100d4a56d4f56758683956137ececd3bb75d57eb48feefcda598ac9321 WatchSource:0}: Error finding container a73580100d4a56d4f56758683956137ececd3bb75d57eb48feefcda598ac9321: Status 404 returned error can't find the container with id a73580100d4a56d4f56758683956137ececd3bb75d57eb48feefcda598ac9321
Jul 20 09:46:06.720655 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:06.720614    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-plzn8" event={"ID":"9269faf3-2bba-4db5-9344-26d619abce52","Type":"ContainerStarted","Data":"a73580100d4a56d4f56758683956137ececd3bb75d57eb48feefcda598ac9321"}
Jul 20 09:46:07.934036 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:07.933992    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phvv7"]
Jul 20 09:46:07.937565 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:07.937535    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phvv7"
Jul 20 09:46:07.943202 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:07.943172    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-gf8zq\""
Jul 20 09:46:07.949197 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:07.949159    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phvv7"]
Jul 20 09:46:07.952295 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:07.952263    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q99pl\" (UniqueName: \"kubernetes.io/projected/615d84ac-953a-4893-9bdd-307fb80356d8-kube-api-access-q99pl\") pod \"limitador-operator-controller-manager-859d7b78bd-phvv7\" (UID: \"615d84ac-953a-4893-9bdd-307fb80356d8\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phvv7"
Jul 20 09:46:08.053557 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:08.053517    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q99pl\" (UniqueName: \"kubernetes.io/projected/615d84ac-953a-4893-9bdd-307fb80356d8-kube-api-access-q99pl\") pod \"limitador-operator-controller-manager-859d7b78bd-phvv7\" (UID: \"615d84ac-953a-4893-9bdd-307fb80356d8\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phvv7"
Jul 20 09:46:08.074297 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:08.074266    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q99pl\" (UniqueName: \"kubernetes.io/projected/615d84ac-953a-4893-9bdd-307fb80356d8-kube-api-access-q99pl\") pod \"limitador-operator-controller-manager-859d7b78bd-phvv7\" (UID: \"615d84ac-953a-4893-9bdd-307fb80356d8\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phvv7"
Jul 20 09:46:08.251093 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:08.250920    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phvv7"
Jul 20 09:46:08.416854 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:08.416776    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phvv7"]
Jul 20 09:46:08.420273 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:46:08.420236    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod615d84ac_953a_4893_9bdd_307fb80356d8.slice/crio-984dd50a3a70ae73dd47a3f1d283059296e8d9861a3ec97c939c820e13a4654e WatchSource:0}: Error finding container 984dd50a3a70ae73dd47a3f1d283059296e8d9861a3ec97c939c820e13a4654e: Status 404 returned error can't find the container with id 984dd50a3a70ae73dd47a3f1d283059296e8d9861a3ec97c939c820e13a4654e
Jul 20 09:46:08.729838 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:08.729798    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phvv7" event={"ID":"615d84ac-953a-4893-9bdd-307fb80356d8","Type":"ContainerStarted","Data":"984dd50a3a70ae73dd47a3f1d283059296e8d9861a3ec97c939c820e13a4654e"}
Jul 20 09:46:10.686490 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:10.686457    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-fdbb8"
Jul 20 09:46:11.749696 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:11.749632    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phvv7" event={"ID":"615d84ac-953a-4893-9bdd-307fb80356d8","Type":"ContainerStarted","Data":"a6113c6a593ba351daf93b2dffb7d409a47f06ea8aa0dd28cfdad8b4f0afd4cb"}
Jul 20 09:46:11.750397 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:11.749939    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phvv7"
Jul 20 09:46:11.752542 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:11.752484    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-plzn8" event={"ID":"9269faf3-2bba-4db5-9344-26d619abce52","Type":"ContainerStarted","Data":"d98013460c4dfc861bc6a004f16858de47e3fa679e4b8a52a3385e0e4436543e"}
Jul 20 09:46:11.752798 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:11.752586    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-plzn8"
Jul 20 09:46:11.772701 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:11.772585    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phvv7" podStartSLOduration=2.056979696 podStartE2EDuration="4.772553909s" totalImagesPullingTime="2.715574213s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 09:46:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:46:11.771309249 +0000 UTC m=+438.322763738" watchObservedRunningTime="2026-07-20 09:46:11.772553909 +0000 UTC m=+438.324008374"
Jul 20 09:46:11.796560 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:11.796501    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-plzn8" podStartSLOduration=1.900899448 podStartE2EDuration="6.79648073s" totalImagesPullingTime="4.895581282s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 09:46:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:46:11.795373655 +0000 UTC m=+438.346828123" watchObservedRunningTime="2026-07-20 09:46:11.79648073 +0000 UTC m=+438.347935600"
Jul 20 09:46:14.017724 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:14.017678    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-6bwbt"]
Jul 20 09:46:14.021257 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:14.021232    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6bwbt"
Jul 20 09:46:14.026398 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:14.026365    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-6tgfg\""
Jul 20 09:46:14.028500 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:14.028469    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 20 09:46:14.028675 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:14.028599    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 20 09:46:14.032311 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:14.032281    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-6bwbt"]
Jul 20 09:46:14.107395 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:14.107353    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4659bf4f-c431-41b8-b8c5-1d6a5a1da700-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-6bwbt\" (UID: \"4659bf4f-c431-41b8-b8c5-1d6a5a1da700\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6bwbt"
Jul 20 09:46:14.107604 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:14.107416    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4659bf4f-c431-41b8-b8c5-1d6a5a1da700-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-6bwbt\" (UID: \"4659bf4f-c431-41b8-b8c5-1d6a5a1da700\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6bwbt"
Jul 20 09:46:14.107604 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:14.107540    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qb9lv\" (UniqueName: \"kubernetes.io/projected/4659bf4f-c431-41b8-b8c5-1d6a5a1da700-kube-api-access-qb9lv\") pod \"kuadrant-console-plugin-66b5774985-6bwbt\" (UID: \"4659bf4f-c431-41b8-b8c5-1d6a5a1da700\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6bwbt"
Jul 20 09:46:14.208512 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:14.208469    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qb9lv\" (UniqueName: \"kubernetes.io/projected/4659bf4f-c431-41b8-b8c5-1d6a5a1da700-kube-api-access-qb9lv\") pod \"kuadrant-console-plugin-66b5774985-6bwbt\" (UID: \"4659bf4f-c431-41b8-b8c5-1d6a5a1da700\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6bwbt"
Jul 20 09:46:14.208512 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:14.208517    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4659bf4f-c431-41b8-b8c5-1d6a5a1da700-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-6bwbt\" (UID: \"4659bf4f-c431-41b8-b8c5-1d6a5a1da700\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6bwbt"
Jul 20 09:46:14.208757 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:14.208567    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4659bf4f-c431-41b8-b8c5-1d6a5a1da700-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-6bwbt\" (UID: \"4659bf4f-c431-41b8-b8c5-1d6a5a1da700\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6bwbt"
Jul 20 09:46:14.209411 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:14.209389    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/4659bf4f-c431-41b8-b8c5-1d6a5a1da700-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-6bwbt\" (UID: \"4659bf4f-c431-41b8-b8c5-1d6a5a1da700\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6bwbt"
Jul 20 09:46:14.211437 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:14.211409    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/4659bf4f-c431-41b8-b8c5-1d6a5a1da700-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-6bwbt\" (UID: \"4659bf4f-c431-41b8-b8c5-1d6a5a1da700\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6bwbt"
Jul 20 09:46:14.222668 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:14.222633    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qb9lv\" (UniqueName: \"kubernetes.io/projected/4659bf4f-c431-41b8-b8c5-1d6a5a1da700-kube-api-access-qb9lv\") pod \"kuadrant-console-plugin-66b5774985-6bwbt\" (UID: \"4659bf4f-c431-41b8-b8c5-1d6a5a1da700\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6bwbt"
Jul 20 09:46:14.332358 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:14.332321    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6bwbt"
Jul 20 09:46:14.509630 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:14.509540    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-6bwbt"]
Jul 20 09:46:14.513328 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:46:14.513293    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4659bf4f_c431_41b8_b8c5_1d6a5a1da700.slice/crio-3ee8631481aefdccfa3a868336d102e1787eb60f28d8317ddfd8ff0e02f079c3 WatchSource:0}: Error finding container 3ee8631481aefdccfa3a868336d102e1787eb60f28d8317ddfd8ff0e02f079c3: Status 404 returned error can't find the container with id 3ee8631481aefdccfa3a868336d102e1787eb60f28d8317ddfd8ff0e02f079c3
Jul 20 09:46:14.713657 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:14.713562    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-4mff6"
Jul 20 09:46:14.774233 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:14.774193    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6bwbt" event={"ID":"4659bf4f-c431-41b8-b8c5-1d6a5a1da700","Type":"ContainerStarted","Data":"3ee8631481aefdccfa3a868336d102e1787eb60f28d8317ddfd8ff0e02f079c3"}
Jul 20 09:46:22.765843 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:22.765803    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phvv7"
Jul 20 09:46:22.766303 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:22.765906    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-plzn8"
Jul 20 09:46:24.594991 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:24.594944    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-plzn8"]
Jul 20 09:46:24.595556 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:24.595257    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-plzn8" podUID="9269faf3-2bba-4db5-9344-26d619abce52" containerName="manager" containerID="cri-o://d98013460c4dfc861bc6a004f16858de47e3fa679e4b8a52a3385e0e4436543e" gracePeriod=2
Jul 20 09:46:24.607677 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:24.607645    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-plzn8"]
Jul 20 09:46:24.611002 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:46:24.610926    2566 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"9269faf3-2bba-4db5-9344-26d619abce52\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-plzn8\": pods \"kuadrant-operator-controller-manager-8684c9ff49-plzn8\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-plzn8"
Jul 20 09:46:24.621610 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:24.621572    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phvv7"]
Jul 20 09:46:24.621881 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:24.621830    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phvv7" podUID="615d84ac-953a-4893-9bdd-307fb80356d8" containerName="manager" containerID="cri-o://a6113c6a593ba351daf93b2dffb7d409a47f06ea8aa0dd28cfdad8b4f0afd4cb" gracePeriod=2
Jul 20 09:46:24.628045 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:24.628011    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sf9zt"]
Jul 20 09:46:24.628479 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:24.628456    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9269faf3-2bba-4db5-9344-26d619abce52" containerName="manager"
Jul 20 09:46:24.628479 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:24.628482    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9269faf3-2bba-4db5-9344-26d619abce52" containerName="manager"
Jul 20 09:46:24.628657 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:24.628577    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9269faf3-2bba-4db5-9344-26d619abce52" containerName="manager"
Jul 20 09:46:24.634731 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:46:24.634685    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-plzn8\" is forbidden: User \"system:node:ip-10-0-140-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-134.ec2.internal' and this object" podUID="9269faf3-2bba-4db5-9344-26d619abce52" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-plzn8"
Jul 20 09:46:24.656712 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:24.656668    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phvv7"]
Jul 20 09:46:24.656712 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:24.656718    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sf9zt"]
Jul 20 09:46:24.657051 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:24.656747    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hv5mg"]
Jul 20 09:46:24.657051 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:24.656821    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sf9zt"
Jul 20 09:46:24.657191 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:24.657142    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="615d84ac-953a-4893-9bdd-307fb80356d8" containerName="manager"
Jul 20 09:46:24.657191 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:24.657157    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="615d84ac-953a-4893-9bdd-307fb80356d8" containerName="manager"
Jul 20 09:46:24.657296 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:24.657240    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="615d84ac-953a-4893-9bdd-307fb80356d8" containerName="manager"
Jul 20 09:46:24.659763 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:46:24.659620    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-phvv7\" is forbidden: User \"system:node:ip-10-0-140-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-134.ec2.internal' and this object" podUID="615d84ac-953a-4893-9bdd-307fb80356d8" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phvv7"
Jul 20 09:46:24.661916 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:24.661890    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hv5mg"
Jul 20 09:46:24.669111 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:24.669074    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hv5mg"]
Jul 20 09:46:24.688917 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:46:24.688870    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-plzn8\" is forbidden: User \"system:node:ip-10-0-140-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-134.ec2.internal' and this object" podUID="9269faf3-2bba-4db5-9344-26d619abce52" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-plzn8"
Jul 20 09:46:24.691417 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:46:24.691354    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-plzn8\" is forbidden: User \"system:node:ip-10-0-140-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-134.ec2.internal' and this object" podUID="9269faf3-2bba-4db5-9344-26d619abce52" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-plzn8"
Jul 20 09:46:24.693754 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:46:24.693711    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-phvv7\" is forbidden: User \"system:node:ip-10-0-140-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-134.ec2.internal' and this object" podUID="615d84ac-953a-4893-9bdd-307fb80356d8" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phvv7"
Jul 20 09:46:24.710412 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:24.710373    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dbwm\" (UniqueName: \"kubernetes.io/projected/d3401128-a458-4efc-8f7a-5ffb1bee1597-kube-api-access-2dbwm\") pod \"limitador-operator-controller-manager-859d7b78bd-hv5mg\" (UID: \"d3401128-a458-4efc-8f7a-5ffb1bee1597\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hv5mg"
Jul 20 09:46:24.710412 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:24.710418    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2feccc49-8b7d-45d0-a5c5-bc72071d98fa-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sf9zt\" (UID: \"2feccc49-8b7d-45d0-a5c5-bc72071d98fa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sf9zt"
Jul 20 09:46:24.710688 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:24.710446    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjw5x\" (UniqueName: \"kubernetes.io/projected/2feccc49-8b7d-45d0-a5c5-bc72071d98fa-kube-api-access-bjw5x\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sf9zt\" (UID: \"2feccc49-8b7d-45d0-a5c5-bc72071d98fa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sf9zt"
Jul 20 09:46:24.811866 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:24.811829    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dbwm\" (UniqueName: \"kubernetes.io/projected/d3401128-a458-4efc-8f7a-5ffb1bee1597-kube-api-access-2dbwm\") pod \"limitador-operator-controller-manager-859d7b78bd-hv5mg\" (UID: \"d3401128-a458-4efc-8f7a-5ffb1bee1597\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hv5mg"
Jul 20 09:46:24.812091 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:24.811881    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2feccc49-8b7d-45d0-a5c5-bc72071d98fa-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sf9zt\" (UID: \"2feccc49-8b7d-45d0-a5c5-bc72071d98fa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sf9zt"
Jul 20 09:46:24.812091 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:24.811906    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjw5x\" (UniqueName: \"kubernetes.io/projected/2feccc49-8b7d-45d0-a5c5-bc72071d98fa-kube-api-access-bjw5x\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sf9zt\" (UID: \"2feccc49-8b7d-45d0-a5c5-bc72071d98fa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sf9zt"
Jul 20 09:46:24.812381 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:24.812359    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2feccc49-8b7d-45d0-a5c5-bc72071d98fa-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sf9zt\" (UID: \"2feccc49-8b7d-45d0-a5c5-bc72071d98fa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sf9zt"
Jul 20 09:46:24.825941 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:24.825903    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjw5x\" (UniqueName: \"kubernetes.io/projected/2feccc49-8b7d-45d0-a5c5-bc72071d98fa-kube-api-access-bjw5x\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sf9zt\" (UID: \"2feccc49-8b7d-45d0-a5c5-bc72071d98fa\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sf9zt"
Jul 20 09:46:24.826134 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:24.825961    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dbwm\" (UniqueName: \"kubernetes.io/projected/d3401128-a458-4efc-8f7a-5ffb1bee1597-kube-api-access-2dbwm\") pod \"limitador-operator-controller-manager-859d7b78bd-hv5mg\" (UID: \"d3401128-a458-4efc-8f7a-5ffb1bee1597\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hv5mg"
Jul 20 09:46:25.062130 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:25.062089    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sf9zt"
Jul 20 09:46:25.069172 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:25.069136    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hv5mg"
Jul 20 09:46:36.091854 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:36.091825    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phvv7"
Jul 20 09:46:36.094464 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:36.094418    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-plzn8"
Jul 20 09:46:36.097550 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:46:36.097508    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-phvv7\" is forbidden: User \"system:node:ip-10-0-140-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-134.ec2.internal' and this object" podUID="615d84ac-953a-4893-9bdd-307fb80356d8" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phvv7"
Jul 20 09:46:36.101815 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:46:36.101369    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-plzn8\" is forbidden: User \"system:node:ip-10-0-140-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-134.ec2.internal' and this object" podUID="9269faf3-2bba-4db5-9344-26d619abce52" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-plzn8"
Jul 20 09:46:36.103648 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:36.103610    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sf9zt"]
Jul 20 09:46:36.103768 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:46:36.103685    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-phvv7\" is forbidden: User \"system:node:ip-10-0-140-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-134.ec2.internal' and this object" podUID="615d84ac-953a-4893-9bdd-307fb80356d8" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phvv7"
Jul 20 09:46:36.105102 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:46:36.105066    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2feccc49_8b7d_45d0_a5c5_bc72071d98fa.slice/crio-f018dcd07ee39ed90ba68c3b0badab5dd52c3879158a5174b152befd4af8de7c WatchSource:0}: Error finding container f018dcd07ee39ed90ba68c3b0badab5dd52c3879158a5174b152befd4af8de7c: Status 404 returned error can't find the container with id f018dcd07ee39ed90ba68c3b0badab5dd52c3879158a5174b152befd4af8de7c
Jul 20 09:46:36.124368 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:36.124335    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hv5mg"]
Jul 20 09:46:36.125082 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:46:36.125053    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3401128_a458_4efc_8f7a_5ffb1bee1597.slice/crio-a7aed180b5f2d7b23b3b1b90a5792b1f47cec9ab7d11f02c756f2e9993239763 WatchSource:0}: Error finding container a7aed180b5f2d7b23b3b1b90a5792b1f47cec9ab7d11f02c756f2e9993239763: Status 404 returned error can't find the container with id a7aed180b5f2d7b23b3b1b90a5792b1f47cec9ab7d11f02c756f2e9993239763
Jul 20 09:46:36.220548 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:36.220510    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tnvt4\" (UniqueName: \"kubernetes.io/projected/9269faf3-2bba-4db5-9344-26d619abce52-kube-api-access-tnvt4\") pod \"9269faf3-2bba-4db5-9344-26d619abce52\" (UID: \"9269faf3-2bba-4db5-9344-26d619abce52\") "
Jul 20 09:46:36.220755 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:36.220584    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q99pl\" (UniqueName: \"kubernetes.io/projected/615d84ac-953a-4893-9bdd-307fb80356d8-kube-api-access-q99pl\") pod \"615d84ac-953a-4893-9bdd-307fb80356d8\" (UID: \"615d84ac-953a-4893-9bdd-307fb80356d8\") "
Jul 20 09:46:36.220755 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:36.220701    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9269faf3-2bba-4db5-9344-26d619abce52-extensions-socket-volume\") pod \"9269faf3-2bba-4db5-9344-26d619abce52\" (UID: \"9269faf3-2bba-4db5-9344-26d619abce52\") "
Jul 20 09:46:36.221334 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:36.221234    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9269faf3-2bba-4db5-9344-26d619abce52-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9269faf3-2bba-4db5-9344-26d619abce52" (UID: "9269faf3-2bba-4db5-9344-26d619abce52"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 09:46:36.222962 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:36.222933    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/615d84ac-953a-4893-9bdd-307fb80356d8-kube-api-access-q99pl" (OuterVolumeSpecName: "kube-api-access-q99pl") pod "615d84ac-953a-4893-9bdd-307fb80356d8" (UID: "615d84ac-953a-4893-9bdd-307fb80356d8"). InnerVolumeSpecName "kube-api-access-q99pl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 09:46:36.222962 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:36.222940    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9269faf3-2bba-4db5-9344-26d619abce52-kube-api-access-tnvt4" (OuterVolumeSpecName: "kube-api-access-tnvt4") pod "9269faf3-2bba-4db5-9344-26d619abce52" (UID: "9269faf3-2bba-4db5-9344-26d619abce52"). InnerVolumeSpecName "kube-api-access-tnvt4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 09:46:36.321378 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:36.321333    2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9269faf3-2bba-4db5-9344-26d619abce52-extensions-socket-volume\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 09:46:36.321378 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:36.321367    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tnvt4\" (UniqueName: \"kubernetes.io/projected/9269faf3-2bba-4db5-9344-26d619abce52-kube-api-access-tnvt4\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 09:46:36.321378 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:36.321376    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q99pl\" (UniqueName: \"kubernetes.io/projected/615d84ac-953a-4893-9bdd-307fb80356d8-kube-api-access-q99pl\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 09:46:36.895281 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:36.895237    2566 generic.go:356] "Generic (PLEG): container finished" podID="615d84ac-953a-4893-9bdd-307fb80356d8" containerID="a6113c6a593ba351daf93b2dffb7d409a47f06ea8aa0dd28cfdad8b4f0afd4cb" exitCode=0
Jul 20 09:46:36.895489 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:36.895291    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phvv7"
Jul 20 09:46:36.895489 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:36.895343    2566 scope.go:117] "RemoveContainer" containerID="a6113c6a593ba351daf93b2dffb7d409a47f06ea8aa0dd28cfdad8b4f0afd4cb"
Jul 20 09:46:36.897077 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:36.897043    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6bwbt" event={"ID":"4659bf4f-c431-41b8-b8c5-1d6a5a1da700","Type":"ContainerStarted","Data":"5959c75d5d5493513d3e319f234c16c5465057e2524b67b85ec210ef69781881"}
Jul 20 09:46:36.898670 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:46:36.898637    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-plzn8\" is forbidden: User \"system:node:ip-10-0-140-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-134.ec2.internal' and this object" podUID="9269faf3-2bba-4db5-9344-26d619abce52" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-plzn8"
Jul 20 09:46:36.898842 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:36.898819    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sf9zt" event={"ID":"2feccc49-8b7d-45d0-a5c5-bc72071d98fa","Type":"ContainerStarted","Data":"b59aab0f011798fee6cab0604b22c79f4164f311d098a5c7d4fe00c85ea87f51"}
Jul 20 09:46:36.898937 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:36.898876    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sf9zt" event={"ID":"2feccc49-8b7d-45d0-a5c5-bc72071d98fa","Type":"ContainerStarted","Data":"f018dcd07ee39ed90ba68c3b0badab5dd52c3879158a5174b152befd4af8de7c"}
Jul 20 09:46:36.899022 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:36.898987    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sf9zt"
Jul 20 09:46:36.900687 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:36.900661    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hv5mg" event={"ID":"d3401128-a458-4efc-8f7a-5ffb1bee1597","Type":"ContainerStarted","Data":"a02b068e02390f4c6abde0839745a994178af444125d3e46ab6ce7872bfba299"}
Jul 20 09:46:36.900815 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:36.900715    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hv5mg" event={"ID":"d3401128-a458-4efc-8f7a-5ffb1bee1597","Type":"ContainerStarted","Data":"a7aed180b5f2d7b23b3b1b90a5792b1f47cec9ab7d11f02c756f2e9993239763"}
Jul 20 09:46:36.900815 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:36.900747    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hv5mg"
Jul 20 09:46:36.901901 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:46:36.901871    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-phvv7\" is forbidden: User \"system:node:ip-10-0-140-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-134.ec2.internal' and this object" podUID="615d84ac-953a-4893-9bdd-307fb80356d8" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phvv7"
Jul 20 09:46:36.902174 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:36.902151    2566 generic.go:356] "Generic (PLEG): container finished" podID="9269faf3-2bba-4db5-9344-26d619abce52" containerID="d98013460c4dfc861bc6a004f16858de47e3fa679e4b8a52a3385e0e4436543e" exitCode=0
Jul 20 09:46:36.902249 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:36.902199    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-plzn8"
Jul 20 09:46:36.905262 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:46:36.905235    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-phvv7\" is forbidden: User \"system:node:ip-10-0-140-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-134.ec2.internal' and this object" podUID="615d84ac-953a-4893-9bdd-307fb80356d8" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phvv7"
Jul 20 09:46:36.906762 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:36.906741    2566 scope.go:117] "RemoveContainer" containerID="a6113c6a593ba351daf93b2dffb7d409a47f06ea8aa0dd28cfdad8b4f0afd4cb"
Jul 20 09:46:36.907148 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:46:36.907127    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6113c6a593ba351daf93b2dffb7d409a47f06ea8aa0dd28cfdad8b4f0afd4cb\": container with ID starting with a6113c6a593ba351daf93b2dffb7d409a47f06ea8aa0dd28cfdad8b4f0afd4cb not found: ID does not exist" containerID="a6113c6a593ba351daf93b2dffb7d409a47f06ea8aa0dd28cfdad8b4f0afd4cb"
Jul 20 09:46:36.907203 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:36.907158    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6113c6a593ba351daf93b2dffb7d409a47f06ea8aa0dd28cfdad8b4f0afd4cb"} err="failed to get container status \"a6113c6a593ba351daf93b2dffb7d409a47f06ea8aa0dd28cfdad8b4f0afd4cb\": rpc error: code = NotFound desc = could not find container \"a6113c6a593ba351daf93b2dffb7d409a47f06ea8aa0dd28cfdad8b4f0afd4cb\": container with ID starting with a6113c6a593ba351daf93b2dffb7d409a47f06ea8aa0dd28cfdad8b4f0afd4cb not found: ID does not exist"
Jul 20 09:46:36.907203 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:36.907176    2566 scope.go:117] "RemoveContainer" containerID="d98013460c4dfc861bc6a004f16858de47e3fa679e4b8a52a3385e0e4436543e"
Jul 20 09:46:36.924158 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:36.924016    2566 scope.go:117] "RemoveContainer" containerID="d98013460c4dfc861bc6a004f16858de47e3fa679e4b8a52a3385e0e4436543e"
Jul 20 09:46:36.924399 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:46:36.924378    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d98013460c4dfc861bc6a004f16858de47e3fa679e4b8a52a3385e0e4436543e\": container with ID starting with d98013460c4dfc861bc6a004f16858de47e3fa679e4b8a52a3385e0e4436543e not found: ID does not exist" containerID="d98013460c4dfc861bc6a004f16858de47e3fa679e4b8a52a3385e0e4436543e"
Jul 20 09:46:36.924455 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:36.924409    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d98013460c4dfc861bc6a004f16858de47e3fa679e4b8a52a3385e0e4436543e"} err="failed to get container status \"d98013460c4dfc861bc6a004f16858de47e3fa679e4b8a52a3385e0e4436543e\": rpc error: code = NotFound desc = could not find container \"d98013460c4dfc861bc6a004f16858de47e3fa679e4b8a52a3385e0e4436543e\": container with ID starting with d98013460c4dfc861bc6a004f16858de47e3fa679e4b8a52a3385e0e4436543e not found: ID does not exist"
Jul 20 09:46:36.936375 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:36.936283    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hv5mg" podStartSLOduration=12.936264545 podStartE2EDuration="12.936264545s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 09:46:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:46:36.93531213 +0000 UTC m=+463.486766596" watchObservedRunningTime="2026-07-20 09:46:36.936264545 +0000 UTC m=+463.487719009"
Jul 20 09:46:36.968028 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:36.967933    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-6bwbt" podStartSLOduration=2.469761757 podStartE2EDuration="23.967913157s" totalImagesPullingTime="21.4981514s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 09:46:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:46:36.966547137 +0000 UTC m=+463.518001601" watchObservedRunningTime="2026-07-20 09:46:36.967913157 +0000 UTC m=+463.519367623"
Jul 20 09:46:36.998106 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:36.998029    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sf9zt" podStartSLOduration=12.998000744 podStartE2EDuration="12.998000744s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 09:46:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:46:36.995016933 +0000 UTC m=+463.546471397" watchObservedRunningTime="2026-07-20 09:46:36.998000744 +0000 UTC m=+463.549455208"
Jul 20 09:46:37.001917 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:46:37.001876    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-plzn8\" is forbidden: User \"system:node:ip-10-0-140-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-134.ec2.internal' and this object" podUID="9269faf3-2bba-4db5-9344-26d619abce52" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-plzn8"
Jul 20 09:46:37.004268 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:46:37.004234    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-plzn8\" is forbidden: User \"system:node:ip-10-0-140-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-134.ec2.internal' and this object" podUID="9269faf3-2bba-4db5-9344-26d619abce52" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-plzn8"
Jul 20 09:46:37.007094 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:46:37.007062    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-phvv7\" is forbidden: User \"system:node:ip-10-0-140-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-134.ec2.internal' and this object" podUID="615d84ac-953a-4893-9bdd-307fb80356d8" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-phvv7"
Jul 20 09:46:37.921179 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:37.921143    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="615d84ac-953a-4893-9bdd-307fb80356d8" path="/var/lib/kubelet/pods/615d84ac-953a-4893-9bdd-307fb80356d8/volumes"
Jul 20 09:46:37.921557 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:37.921504    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9269faf3-2bba-4db5-9344-26d619abce52" path="/var/lib/kubelet/pods/9269faf3-2bba-4db5-9344-26d619abce52/volumes"
Jul 20 09:46:41.543384 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:41.543342    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sf9zt"]
Jul 20 09:46:41.543789 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:41.543555    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sf9zt" podUID="2feccc49-8b7d-45d0-a5c5-bc72071d98fa" containerName="manager" containerID="cri-o://b59aab0f011798fee6cab0604b22c79f4164f311d098a5c7d4fe00c85ea87f51" gracePeriod=10
Jul 20 09:46:41.545693 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:41.545671    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sf9zt"
Jul 20 09:46:41.792303 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:41.792259    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7bdl"]
Jul 20 09:46:41.795909 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:41.795838    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7bdl"
Jul 20 09:46:41.804145 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:41.804115    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sf9zt"
Jul 20 09:46:41.822073 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:41.822024    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7bdl"]
Jul 20 09:46:41.874690 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:41.874656    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wpmw\" (UniqueName: \"kubernetes.io/projected/a1c0d635-044b-46db-a732-57eda1799859-kube-api-access-8wpmw\") pod \"kuadrant-operator-controller-manager-86878c769f-c7bdl\" (UID: \"a1c0d635-044b-46db-a732-57eda1799859\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7bdl"
Jul 20 09:46:41.874888 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:41.874781    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a1c0d635-044b-46db-a732-57eda1799859-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-c7bdl\" (UID: \"a1c0d635-044b-46db-a732-57eda1799859\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7bdl"
Jul 20 09:46:41.928755 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:41.928721    2566 generic.go:356] "Generic (PLEG): container finished" podID="2feccc49-8b7d-45d0-a5c5-bc72071d98fa" containerID="b59aab0f011798fee6cab0604b22c79f4164f311d098a5c7d4fe00c85ea87f51" exitCode=0
Jul 20 09:46:41.928932 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:41.928763    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sf9zt"
Jul 20 09:46:41.928932 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:41.928806    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sf9zt" event={"ID":"2feccc49-8b7d-45d0-a5c5-bc72071d98fa","Type":"ContainerDied","Data":"b59aab0f011798fee6cab0604b22c79f4164f311d098a5c7d4fe00c85ea87f51"}
Jul 20 09:46:41.928932 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:41.928865    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sf9zt" event={"ID":"2feccc49-8b7d-45d0-a5c5-bc72071d98fa","Type":"ContainerDied","Data":"f018dcd07ee39ed90ba68c3b0badab5dd52c3879158a5174b152befd4af8de7c"}
Jul 20 09:46:41.928932 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:41.928894    2566 scope.go:117] "RemoveContainer" containerID="b59aab0f011798fee6cab0604b22c79f4164f311d098a5c7d4fe00c85ea87f51"
Jul 20 09:46:41.937897 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:41.937867    2566 scope.go:117] "RemoveContainer" containerID="b59aab0f011798fee6cab0604b22c79f4164f311d098a5c7d4fe00c85ea87f51"
Jul 20 09:46:41.938263 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:46:41.938244    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b59aab0f011798fee6cab0604b22c79f4164f311d098a5c7d4fe00c85ea87f51\": container with ID starting with b59aab0f011798fee6cab0604b22c79f4164f311d098a5c7d4fe00c85ea87f51 not found: ID does not exist" containerID="b59aab0f011798fee6cab0604b22c79f4164f311d098a5c7d4fe00c85ea87f51"
Jul 20 09:46:41.938343 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:41.938273    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b59aab0f011798fee6cab0604b22c79f4164f311d098a5c7d4fe00c85ea87f51"} err="failed to get container status \"b59aab0f011798fee6cab0604b22c79f4164f311d098a5c7d4fe00c85ea87f51\": rpc error: code = NotFound desc = could not find container \"b59aab0f011798fee6cab0604b22c79f4164f311d098a5c7d4fe00c85ea87f51\": container with ID starting with b59aab0f011798fee6cab0604b22c79f4164f311d098a5c7d4fe00c85ea87f51 not found: ID does not exist"
Jul 20 09:46:41.975756 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:41.975712    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bjw5x\" (UniqueName: \"kubernetes.io/projected/2feccc49-8b7d-45d0-a5c5-bc72071d98fa-kube-api-access-bjw5x\") pod \"2feccc49-8b7d-45d0-a5c5-bc72071d98fa\" (UID: \"2feccc49-8b7d-45d0-a5c5-bc72071d98fa\") "
Jul 20 09:46:41.975756 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:41.975759    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2feccc49-8b7d-45d0-a5c5-bc72071d98fa-extensions-socket-volume\") pod \"2feccc49-8b7d-45d0-a5c5-bc72071d98fa\" (UID: \"2feccc49-8b7d-45d0-a5c5-bc72071d98fa\") "
Jul 20 09:46:41.976038 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:41.975912    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a1c0d635-044b-46db-a732-57eda1799859-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-c7bdl\" (UID: \"a1c0d635-044b-46db-a732-57eda1799859\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7bdl"
Jul 20 09:46:41.976038 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:41.975946    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wpmw\" (UniqueName: \"kubernetes.io/projected/a1c0d635-044b-46db-a732-57eda1799859-kube-api-access-8wpmw\") pod \"kuadrant-operator-controller-manager-86878c769f-c7bdl\" (UID: \"a1c0d635-044b-46db-a732-57eda1799859\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7bdl"
Jul 20 09:46:41.976277 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:41.976245    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2feccc49-8b7d-45d0-a5c5-bc72071d98fa-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2feccc49-8b7d-45d0-a5c5-bc72071d98fa" (UID: "2feccc49-8b7d-45d0-a5c5-bc72071d98fa"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 09:46:41.976347 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:41.976334    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a1c0d635-044b-46db-a732-57eda1799859-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-c7bdl\" (UID: \"a1c0d635-044b-46db-a732-57eda1799859\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7bdl"
Jul 20 09:46:41.978127 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:41.978098    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2feccc49-8b7d-45d0-a5c5-bc72071d98fa-kube-api-access-bjw5x" (OuterVolumeSpecName: "kube-api-access-bjw5x") pod "2feccc49-8b7d-45d0-a5c5-bc72071d98fa" (UID: "2feccc49-8b7d-45d0-a5c5-bc72071d98fa"). InnerVolumeSpecName "kube-api-access-bjw5x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 09:46:41.992177 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:41.992139    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wpmw\" (UniqueName: \"kubernetes.io/projected/a1c0d635-044b-46db-a732-57eda1799859-kube-api-access-8wpmw\") pod \"kuadrant-operator-controller-manager-86878c769f-c7bdl\" (UID: \"a1c0d635-044b-46db-a732-57eda1799859\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7bdl"
Jul 20 09:46:42.077180 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:42.077143    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bjw5x\" (UniqueName: \"kubernetes.io/projected/2feccc49-8b7d-45d0-a5c5-bc72071d98fa-kube-api-access-bjw5x\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 09:46:42.077180 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:42.077174    2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2feccc49-8b7d-45d0-a5c5-bc72071d98fa-extensions-socket-volume\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 09:46:42.110237 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:42.110205    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7bdl"
Jul 20 09:46:42.264083 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:42.264044    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sf9zt"]
Jul 20 09:46:42.270963 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:42.270925    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sf9zt"]
Jul 20 09:46:42.279203 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:46:42.279161    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1c0d635_044b_46db_a732_57eda1799859.slice/crio-1def6fe671b9ac9ceb2aac72fb13b1d67a7ba09a403e059f1cdfcb09f0113b6b WatchSource:0}: Error finding container 1def6fe671b9ac9ceb2aac72fb13b1d67a7ba09a403e059f1cdfcb09f0113b6b: Status 404 returned error can't find the container with id 1def6fe671b9ac9ceb2aac72fb13b1d67a7ba09a403e059f1cdfcb09f0113b6b
Jul 20 09:46:42.279595 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:42.279572    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7bdl"]
Jul 20 09:46:42.936610 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:42.936571    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7bdl" event={"ID":"a1c0d635-044b-46db-a732-57eda1799859","Type":"ContainerStarted","Data":"33eef42ec4bb4648cf2c8a118540bb781ddd6fc1210531cadaf26355c6ef6581"}
Jul 20 09:46:42.937134 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:42.936629    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7bdl" event={"ID":"a1c0d635-044b-46db-a732-57eda1799859","Type":"ContainerStarted","Data":"1def6fe671b9ac9ceb2aac72fb13b1d67a7ba09a403e059f1cdfcb09f0113b6b"}
Jul 20 09:46:42.937134 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:42.936658    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7bdl"
Jul 20 09:46:42.983655 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:42.983598    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7bdl" podStartSLOduration=1.983577328 podStartE2EDuration="1.983577328s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 09:46:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:46:42.983167783 +0000 UTC m=+469.534622273" watchObservedRunningTime="2026-07-20 09:46:42.983577328 +0000 UTC m=+469.535031792"
Jul 20 09:46:43.920683 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:43.920646    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2feccc49-8b7d-45d0-a5c5-bc72071d98fa" path="/var/lib/kubelet/pods/2feccc49-8b7d-45d0-a5c5-bc72071d98fa/volumes"
Jul 20 09:46:47.913445 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:47.913418    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hv5mg"
Jul 20 09:46:53.943580 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:46:53.943547    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7bdl"
Jul 20 09:47:12.575213 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:12.575176    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 09:47:12.575642 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:12.575488    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2feccc49-8b7d-45d0-a5c5-bc72071d98fa" containerName="manager"
Jul 20 09:47:12.575642 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:12.575532    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2feccc49-8b7d-45d0-a5c5-bc72071d98fa" containerName="manager"
Jul 20 09:47:12.575642 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:12.575599    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2feccc49-8b7d-45d0-a5c5-bc72071d98fa" containerName="manager"
Jul 20 09:47:12.599624 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:12.599585    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 09:47:12.599785 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:12.599715    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"
Jul 20 09:47:12.602678 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:12.602643    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 20 09:47:12.609566 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:12.609532    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 09:47:12.630291 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:12.630246    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bqd5\" (UniqueName: \"kubernetes.io/projected/e7c22a59-967f-4e99-b2a6-5bf80adfa210-kube-api-access-9bqd5\") pod \"limitador-limitador-54d6d988d5-6kpnm\" (UID: \"e7c22a59-967f-4e99-b2a6-5bf80adfa210\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"
Jul 20 09:47:12.630512 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:12.630309    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e7c22a59-967f-4e99-b2a6-5bf80adfa210-config-file\") pod \"limitador-limitador-54d6d988d5-6kpnm\" (UID: \"e7c22a59-967f-4e99-b2a6-5bf80adfa210\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"
Jul 20 09:47:12.731460 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:12.731422    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bqd5\" (UniqueName: \"kubernetes.io/projected/e7c22a59-967f-4e99-b2a6-5bf80adfa210-kube-api-access-9bqd5\") pod \"limitador-limitador-54d6d988d5-6kpnm\" (UID: \"e7c22a59-967f-4e99-b2a6-5bf80adfa210\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"
Jul 20 09:47:12.731681 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:12.731502    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e7c22a59-967f-4e99-b2a6-5bf80adfa210-config-file\") pod \"limitador-limitador-54d6d988d5-6kpnm\" (UID: \"e7c22a59-967f-4e99-b2a6-5bf80adfa210\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"
Jul 20 09:47:12.732261 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:12.732237    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e7c22a59-967f-4e99-b2a6-5bf80adfa210-config-file\") pod \"limitador-limitador-54d6d988d5-6kpnm\" (UID: \"e7c22a59-967f-4e99-b2a6-5bf80adfa210\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"
Jul 20 09:47:12.741180 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:12.741148    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bqd5\" (UniqueName: \"kubernetes.io/projected/e7c22a59-967f-4e99-b2a6-5bf80adfa210-kube-api-access-9bqd5\") pod \"limitador-limitador-54d6d988d5-6kpnm\" (UID: \"e7c22a59-967f-4e99-b2a6-5bf80adfa210\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"
Jul 20 09:47:12.913148 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:12.913033    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"
Jul 20 09:47:13.068294 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:13.068254    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 09:47:13.074288 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:47:13.074251    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7c22a59_967f_4e99_b2a6_5bf80adfa210.slice/crio-03465a667bbd1b2b30dd52aa2801641f89097c2dbf11dfbd85c72b00b8741f3b WatchSource:0}: Error finding container 03465a667bbd1b2b30dd52aa2801641f89097c2dbf11dfbd85c72b00b8741f3b: Status 404 returned error can't find the container with id 03465a667bbd1b2b30dd52aa2801641f89097c2dbf11dfbd85c72b00b8741f3b
Jul 20 09:47:13.155887 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:13.155849    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-bxxd6"]
Jul 20 09:47:13.160686 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:13.160654    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-bxxd6"
Jul 20 09:47:13.164931 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:13.164853    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-69txb\""
Jul 20 09:47:13.168871 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:13.168707    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-bxxd6"]
Jul 20 09:47:13.236057 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:13.235964    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-685vp\" (UniqueName: \"kubernetes.io/projected/c1428ce9-478b-4367-8e57-00dcda1906c5-kube-api-access-685vp\") pod \"authorino-75d556bd77-bxxd6\" (UID: \"c1428ce9-478b-4367-8e57-00dcda1906c5\") " pod="kuadrant-system/authorino-75d556bd77-bxxd6"
Jul 20 09:47:13.336874 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:13.336830    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-685vp\" (UniqueName: \"kubernetes.io/projected/c1428ce9-478b-4367-8e57-00dcda1906c5-kube-api-access-685vp\") pod \"authorino-75d556bd77-bxxd6\" (UID: \"c1428ce9-478b-4367-8e57-00dcda1906c5\") " pod="kuadrant-system/authorino-75d556bd77-bxxd6"
Jul 20 09:47:13.347180 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:13.347150    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-685vp\" (UniqueName: \"kubernetes.io/projected/c1428ce9-478b-4367-8e57-00dcda1906c5-kube-api-access-685vp\") pod \"authorino-75d556bd77-bxxd6\" (UID: \"c1428ce9-478b-4367-8e57-00dcda1906c5\") " pod="kuadrant-system/authorino-75d556bd77-bxxd6"
Jul 20 09:47:13.472324 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:13.472217    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-bxxd6"
Jul 20 09:47:13.663346 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:13.663302    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-bxxd6"]
Jul 20 09:47:13.668653 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:47:13.668610    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc1428ce9_478b_4367_8e57_00dcda1906c5.slice/crio-ef7426131bd3aff95a574833d070ac43e37e04d98702e39b12fc1c523e2ea8e0 WatchSource:0}: Error finding container ef7426131bd3aff95a574833d070ac43e37e04d98702e39b12fc1c523e2ea8e0: Status 404 returned error can't find the container with id ef7426131bd3aff95a574833d070ac43e37e04d98702e39b12fc1c523e2ea8e0
Jul 20 09:47:14.076947 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:14.076902    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-bxxd6" event={"ID":"c1428ce9-478b-4367-8e57-00dcda1906c5","Type":"ContainerStarted","Data":"ef7426131bd3aff95a574833d070ac43e37e04d98702e39b12fc1c523e2ea8e0"}
Jul 20 09:47:14.078547 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:14.078502    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-6kpnm" event={"ID":"e7c22a59-967f-4e99-b2a6-5bf80adfa210","Type":"ContainerStarted","Data":"03465a667bbd1b2b30dd52aa2801641f89097c2dbf11dfbd85c72b00b8741f3b"}
Jul 20 09:47:18.101324 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:18.101283    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-bxxd6" event={"ID":"c1428ce9-478b-4367-8e57-00dcda1906c5","Type":"ContainerStarted","Data":"3432df642d1c2b24945c41294837111d007c4a5cd89e7b32b4cc68eef89adb10"}
Jul 20 09:47:18.102993 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:18.102950    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-6kpnm" event={"ID":"e7c22a59-967f-4e99-b2a6-5bf80adfa210","Type":"ContainerStarted","Data":"315d21477105ddecd0d8008288ccb91786a3887a6cfa85f5c30ca391c600ea9c"}
Jul 20 09:47:18.103152 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:18.103032    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"
Jul 20 09:47:18.117351 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:18.117292    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-bxxd6" podStartSLOduration=1.226456614 podStartE2EDuration="5.117275886s" totalImagesPullingTime="3.890819272s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 09:47:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:47:18.115601845 +0000 UTC m=+504.667056302" watchObservedRunningTime="2026-07-20 09:47:18.117275886 +0000 UTC m=+504.668730350"
Jul 20 09:47:18.139612 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:18.139543    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-6kpnm" podStartSLOduration=1.662063734 podStartE2EDuration="6.139519082s" totalImagesPullingTime="4.477455348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 09:47:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:47:18.137859064 +0000 UTC m=+504.689313530" watchObservedRunningTime="2026-07-20 09:47:18.139519082 +0000 UTC m=+504.690973547"
Jul 20 09:47:29.108651 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:29.108619    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"
Jul 20 09:47:45.436934 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:45.436842    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-76gmf"]
Jul 20 09:47:45.443004 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:45.442958    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-76gmf"
Jul 20 09:47:45.449219 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:45.449174    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-76gmf"]
Jul 20 09:47:45.508638 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:45.508599    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nnsc\" (UniqueName: \"kubernetes.io/projected/7588d787-eb6c-429a-b0c1-32880c6a963f-kube-api-access-9nnsc\") pod \"authorino-5d4b67c446-76gmf\" (UID: \"7588d787-eb6c-429a-b0c1-32880c6a963f\") " pod="kuadrant-system/authorino-5d4b67c446-76gmf"
Jul 20 09:47:45.609926 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:45.609882    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nnsc\" (UniqueName: \"kubernetes.io/projected/7588d787-eb6c-429a-b0c1-32880c6a963f-kube-api-access-9nnsc\") pod \"authorino-5d4b67c446-76gmf\" (UID: \"7588d787-eb6c-429a-b0c1-32880c6a963f\") " pod="kuadrant-system/authorino-5d4b67c446-76gmf"
Jul 20 09:47:45.623336 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:45.623303    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nnsc\" (UniqueName: \"kubernetes.io/projected/7588d787-eb6c-429a-b0c1-32880c6a963f-kube-api-access-9nnsc\") pod \"authorino-5d4b67c446-76gmf\" (UID: \"7588d787-eb6c-429a-b0c1-32880c6a963f\") " pod="kuadrant-system/authorino-5d4b67c446-76gmf"
Jul 20 09:47:45.688168 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:45.688074    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-76gmf"]
Jul 20 09:47:45.688347 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:45.688333    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-76gmf"
Jul 20 09:47:45.731933 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:45.731884    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-747499b6d9-b2r2k"]
Jul 20 09:47:45.737732 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:45.737695    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-747499b6d9-b2r2k"
Jul 20 09:47:45.752222 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:45.751760    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-747499b6d9-b2r2k"]
Jul 20 09:47:45.811349 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:45.811296    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsjkr\" (UniqueName: \"kubernetes.io/projected/db4803c7-86ec-46d5-8948-b16314eadfc1-kube-api-access-rsjkr\") pod \"authorino-747499b6d9-b2r2k\" (UID: \"db4803c7-86ec-46d5-8948-b16314eadfc1\") " pod="kuadrant-system/authorino-747499b6d9-b2r2k"
Jul 20 09:47:45.820028 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:45.819994    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-747499b6d9-b2r2k"]
Jul 20 09:47:45.820264 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:47:45.820244    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-rsjkr], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-747499b6d9-b2r2k" podUID="db4803c7-86ec-46d5-8948-b16314eadfc1"
Jul 20 09:47:45.840764 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:45.840738    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-76gmf"]
Jul 20 09:47:45.842686 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:47:45.842657    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7588d787_eb6c_429a_b0c1_32880c6a963f.slice/crio-1e2aa0c33245d9ff3535a505bdaaff376d11d1254aca4d04ca04d3e4010ab1ba WatchSource:0}: Error finding container 1e2aa0c33245d9ff3535a505bdaaff376d11d1254aca4d04ca04d3e4010ab1ba: Status 404 returned error can't find the container with id 1e2aa0c33245d9ff3535a505bdaaff376d11d1254aca4d04ca04d3e4010ab1ba
Jul 20 09:47:45.861514 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:45.859251    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6ddf9f4f7c-8htbz"]
Jul 20 09:47:45.864437 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:45.864412    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6ddf9f4f7c-8htbz"
Jul 20 09:47:45.868087 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:45.868064    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 20 09:47:45.871378 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:45.871351    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6ddf9f4f7c-8htbz"]
Jul 20 09:47:45.912550 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:45.912508    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsjkr\" (UniqueName: \"kubernetes.io/projected/db4803c7-86ec-46d5-8948-b16314eadfc1-kube-api-access-rsjkr\") pod \"authorino-747499b6d9-b2r2k\" (UID: \"db4803c7-86ec-46d5-8948-b16314eadfc1\") " pod="kuadrant-system/authorino-747499b6d9-b2r2k"
Jul 20 09:47:45.923040 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:45.923007    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsjkr\" (UniqueName: \"kubernetes.io/projected/db4803c7-86ec-46d5-8948-b16314eadfc1-kube-api-access-rsjkr\") pod \"authorino-747499b6d9-b2r2k\" (UID: \"db4803c7-86ec-46d5-8948-b16314eadfc1\") " pod="kuadrant-system/authorino-747499b6d9-b2r2k"
Jul 20 09:47:46.013423 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:46.013331    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-686f5\" (UniqueName: \"kubernetes.io/projected/d71c0646-b5af-4b56-870e-68fff4e5f6cc-kube-api-access-686f5\") pod \"authorino-6ddf9f4f7c-8htbz\" (UID: \"d71c0646-b5af-4b56-870e-68fff4e5f6cc\") " pod="kuadrant-system/authorino-6ddf9f4f7c-8htbz"
Jul 20 09:47:46.013570 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:46.013451    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d71c0646-b5af-4b56-870e-68fff4e5f6cc-tls-cert\") pod \"authorino-6ddf9f4f7c-8htbz\" (UID: \"d71c0646-b5af-4b56-870e-68fff4e5f6cc\") " pod="kuadrant-system/authorino-6ddf9f4f7c-8htbz"
Jul 20 09:47:46.114754 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:46.114707    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d71c0646-b5af-4b56-870e-68fff4e5f6cc-tls-cert\") pod \"authorino-6ddf9f4f7c-8htbz\" (UID: \"d71c0646-b5af-4b56-870e-68fff4e5f6cc\") " pod="kuadrant-system/authorino-6ddf9f4f7c-8htbz"
Jul 20 09:47:46.114933 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:46.114786    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-686f5\" (UniqueName: \"kubernetes.io/projected/d71c0646-b5af-4b56-870e-68fff4e5f6cc-kube-api-access-686f5\") pod \"authorino-6ddf9f4f7c-8htbz\" (UID: \"d71c0646-b5af-4b56-870e-68fff4e5f6cc\") " pod="kuadrant-system/authorino-6ddf9f4f7c-8htbz"
Jul 20 09:47:46.117424 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:46.117401    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d71c0646-b5af-4b56-870e-68fff4e5f6cc-tls-cert\") pod \"authorino-6ddf9f4f7c-8htbz\" (UID: \"d71c0646-b5af-4b56-870e-68fff4e5f6cc\") " pod="kuadrant-system/authorino-6ddf9f4f7c-8htbz"
Jul 20 09:47:46.125010 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:46.124961    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-686f5\" (UniqueName: \"kubernetes.io/projected/d71c0646-b5af-4b56-870e-68fff4e5f6cc-kube-api-access-686f5\") pod \"authorino-6ddf9f4f7c-8htbz\" (UID: \"d71c0646-b5af-4b56-870e-68fff4e5f6cc\") " pod="kuadrant-system/authorino-6ddf9f4f7c-8htbz"
Jul 20 09:47:46.174645 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:46.174623    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6ddf9f4f7c-8htbz"
Jul 20 09:47:46.233237 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:46.232919    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-747499b6d9-b2r2k"
Jul 20 09:47:46.233237 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:46.232920    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-76gmf" event={"ID":"7588d787-eb6c-429a-b0c1-32880c6a963f","Type":"ContainerStarted","Data":"1e2aa0c33245d9ff3535a505bdaaff376d11d1254aca4d04ca04d3e4010ab1ba"}
Jul 20 09:47:46.243887 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:46.243856    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-747499b6d9-b2r2k"
Jul 20 09:47:46.316597 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:46.316564    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rsjkr\" (UniqueName: \"kubernetes.io/projected/db4803c7-86ec-46d5-8948-b16314eadfc1-kube-api-access-rsjkr\") pod \"db4803c7-86ec-46d5-8948-b16314eadfc1\" (UID: \"db4803c7-86ec-46d5-8948-b16314eadfc1\") "
Jul 20 09:47:46.319771 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:46.319724    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db4803c7-86ec-46d5-8948-b16314eadfc1-kube-api-access-rsjkr" (OuterVolumeSpecName: "kube-api-access-rsjkr") pod "db4803c7-86ec-46d5-8948-b16314eadfc1" (UID: "db4803c7-86ec-46d5-8948-b16314eadfc1"). InnerVolumeSpecName "kube-api-access-rsjkr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 09:47:46.335825 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:46.335791    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6ddf9f4f7c-8htbz"]
Jul 20 09:47:46.340706 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:47:46.340670    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd71c0646_b5af_4b56_870e_68fff4e5f6cc.slice/crio-f0b4d6d93ac2c7c83068f24f1ddee5f1c908960a657b537cc82b52b113ad2b96 WatchSource:0}: Error finding container f0b4d6d93ac2c7c83068f24f1ddee5f1c908960a657b537cc82b52b113ad2b96: Status 404 returned error can't find the container with id f0b4d6d93ac2c7c83068f24f1ddee5f1c908960a657b537cc82b52b113ad2b96
Jul 20 09:47:46.417571 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:46.417530    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rsjkr\" (UniqueName: \"kubernetes.io/projected/db4803c7-86ec-46d5-8948-b16314eadfc1-kube-api-access-rsjkr\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 09:47:47.238931 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:47.238894    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6ddf9f4f7c-8htbz" event={"ID":"d71c0646-b5af-4b56-870e-68fff4e5f6cc","Type":"ContainerStarted","Data":"99375a565a62a73f4e3985b400dc5ad913001cec8e418c164fabfe8786295bf4"}
Jul 20 09:47:47.239400 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:47.238938    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6ddf9f4f7c-8htbz" event={"ID":"d71c0646-b5af-4b56-870e-68fff4e5f6cc","Type":"ContainerStarted","Data":"f0b4d6d93ac2c7c83068f24f1ddee5f1c908960a657b537cc82b52b113ad2b96"}
Jul 20 09:47:47.240496 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:47.240476    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-747499b6d9-b2r2k"
Jul 20 09:47:47.240629 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:47.240467    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-76gmf" event={"ID":"7588d787-eb6c-429a-b0c1-32880c6a963f","Type":"ContainerStarted","Data":"531cc178013db1ec9e0c6d50429ab3e4d4ab7ddb4fd04e784dd58a965ddd9f12"}
Jul 20 09:47:47.240629 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:47.240546    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-76gmf" podUID="7588d787-eb6c-429a-b0c1-32880c6a963f" containerName="authorino" containerID="cri-o://531cc178013db1ec9e0c6d50429ab3e4d4ab7ddb4fd04e784dd58a965ddd9f12" gracePeriod=30
Jul 20 09:47:47.265211 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:47.265158    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6ddf9f4f7c-8htbz" podStartSLOduration=1.86836264 podStartE2EDuration="2.265140222s" totalImagesPullingTime="396.777582ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 09:47:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:47:47.264297614 +0000 UTC m=+533.815752077" watchObservedRunningTime="2026-07-20 09:47:47.265140222 +0000 UTC m=+533.816594686"
Jul 20 09:47:47.290558 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:47.290495    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-76gmf" podStartSLOduration=1.95934658 podStartE2EDuration="2.29047851s" totalImagesPullingTime="331.13193ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 09:47:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:47:47.289669647 +0000 UTC m=+533.841124125" watchObservedRunningTime="2026-07-20 09:47:47.29047851 +0000 UTC m=+533.841932974"
Jul 20 09:47:47.302061 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:47.302024    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-bxxd6"]
Jul 20 09:47:47.302400 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:47.302259    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-bxxd6" podUID="c1428ce9-478b-4367-8e57-00dcda1906c5" containerName="authorino" containerID="cri-o://3432df642d1c2b24945c41294837111d007c4a5cd89e7b32b4cc68eef89adb10" gracePeriod=30
Jul 20 09:47:47.328716 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:47.328681    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-747499b6d9-b2r2k"]
Jul 20 09:47:47.338577 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:47.338546    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-747499b6d9-b2r2k"]
Jul 20 09:47:47.540038 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:47.540010    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-76gmf"
Jul 20 09:47:47.575226 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:47.575197    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-bxxd6"
Jul 20 09:47:47.628080 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:47.627891    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9nnsc\" (UniqueName: \"kubernetes.io/projected/7588d787-eb6c-429a-b0c1-32880c6a963f-kube-api-access-9nnsc\") pod \"7588d787-eb6c-429a-b0c1-32880c6a963f\" (UID: \"7588d787-eb6c-429a-b0c1-32880c6a963f\") "
Jul 20 09:47:47.630441 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:47.630402    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7588d787-eb6c-429a-b0c1-32880c6a963f-kube-api-access-9nnsc" (OuterVolumeSpecName: "kube-api-access-9nnsc") pod "7588d787-eb6c-429a-b0c1-32880c6a963f" (UID: "7588d787-eb6c-429a-b0c1-32880c6a963f"). InnerVolumeSpecName "kube-api-access-9nnsc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 09:47:47.729131 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:47.729093    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-685vp\" (UniqueName: \"kubernetes.io/projected/c1428ce9-478b-4367-8e57-00dcda1906c5-kube-api-access-685vp\") pod \"c1428ce9-478b-4367-8e57-00dcda1906c5\" (UID: \"c1428ce9-478b-4367-8e57-00dcda1906c5\") "
Jul 20 09:47:47.729333 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:47.729289    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9nnsc\" (UniqueName: \"kubernetes.io/projected/7588d787-eb6c-429a-b0c1-32880c6a963f-kube-api-access-9nnsc\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 09:47:47.731352 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:47.731324    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1428ce9-478b-4367-8e57-00dcda1906c5-kube-api-access-685vp" (OuterVolumeSpecName: "kube-api-access-685vp") pod "c1428ce9-478b-4367-8e57-00dcda1906c5" (UID: "c1428ce9-478b-4367-8e57-00dcda1906c5"). InnerVolumeSpecName "kube-api-access-685vp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 09:47:47.830067 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:47.830029    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-685vp\" (UniqueName: \"kubernetes.io/projected/c1428ce9-478b-4367-8e57-00dcda1906c5-kube-api-access-685vp\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 09:47:47.923099 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:47.923061    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db4803c7-86ec-46d5-8948-b16314eadfc1" path="/var/lib/kubelet/pods/db4803c7-86ec-46d5-8948-b16314eadfc1/volumes"
Jul 20 09:47:48.246926 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:48.246835    2566 generic.go:356] "Generic (PLEG): container finished" podID="c1428ce9-478b-4367-8e57-00dcda1906c5" containerID="3432df642d1c2b24945c41294837111d007c4a5cd89e7b32b4cc68eef89adb10" exitCode=0
Jul 20 09:47:48.246926 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:48.246882    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-bxxd6"
Jul 20 09:47:48.247423 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:48.246921    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-bxxd6" event={"ID":"c1428ce9-478b-4367-8e57-00dcda1906c5","Type":"ContainerDied","Data":"3432df642d1c2b24945c41294837111d007c4a5cd89e7b32b4cc68eef89adb10"}
Jul 20 09:47:48.247423 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:48.246966    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-bxxd6" event={"ID":"c1428ce9-478b-4367-8e57-00dcda1906c5","Type":"ContainerDied","Data":"ef7426131bd3aff95a574833d070ac43e37e04d98702e39b12fc1c523e2ea8e0"}
Jul 20 09:47:48.247423 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:48.246999    2566 scope.go:117] "RemoveContainer" containerID="3432df642d1c2b24945c41294837111d007c4a5cd89e7b32b4cc68eef89adb10"
Jul 20 09:47:48.248378 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:48.248360    2566 generic.go:356] "Generic (PLEG): container finished" podID="7588d787-eb6c-429a-b0c1-32880c6a963f" containerID="531cc178013db1ec9e0c6d50429ab3e4d4ab7ddb4fd04e784dd58a965ddd9f12" exitCode=0
Jul 20 09:47:48.248433 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:48.248411    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-76gmf"
Jul 20 09:47:48.248478 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:48.248425    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-76gmf" event={"ID":"7588d787-eb6c-429a-b0c1-32880c6a963f","Type":"ContainerDied","Data":"531cc178013db1ec9e0c6d50429ab3e4d4ab7ddb4fd04e784dd58a965ddd9f12"}
Jul 20 09:47:48.248478 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:48.248471    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-76gmf" event={"ID":"7588d787-eb6c-429a-b0c1-32880c6a963f","Type":"ContainerDied","Data":"1e2aa0c33245d9ff3535a505bdaaff376d11d1254aca4d04ca04d3e4010ab1ba"}
Jul 20 09:47:48.256009 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:48.255958    2566 scope.go:117] "RemoveContainer" containerID="3432df642d1c2b24945c41294837111d007c4a5cd89e7b32b4cc68eef89adb10"
Jul 20 09:47:48.256319 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:47:48.256299    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3432df642d1c2b24945c41294837111d007c4a5cd89e7b32b4cc68eef89adb10\": container with ID starting with 3432df642d1c2b24945c41294837111d007c4a5cd89e7b32b4cc68eef89adb10 not found: ID does not exist" containerID="3432df642d1c2b24945c41294837111d007c4a5cd89e7b32b4cc68eef89adb10"
Jul 20 09:47:48.256378 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:48.256329    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3432df642d1c2b24945c41294837111d007c4a5cd89e7b32b4cc68eef89adb10"} err="failed to get container status \"3432df642d1c2b24945c41294837111d007c4a5cd89e7b32b4cc68eef89adb10\": rpc error: code = NotFound desc = could not find container \"3432df642d1c2b24945c41294837111d007c4a5cd89e7b32b4cc68eef89adb10\": container with ID starting with 3432df642d1c2b24945c41294837111d007c4a5cd89e7b32b4cc68eef89adb10 not found: ID does not exist"
Jul 20 09:47:48.256378 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:48.256344    2566 scope.go:117] "RemoveContainer" containerID="531cc178013db1ec9e0c6d50429ab3e4d4ab7ddb4fd04e784dd58a965ddd9f12"
Jul 20 09:47:48.264703 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:48.264677    2566 scope.go:117] "RemoveContainer" containerID="531cc178013db1ec9e0c6d50429ab3e4d4ab7ddb4fd04e784dd58a965ddd9f12"
Jul 20 09:47:48.265035 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:47:48.265016    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"531cc178013db1ec9e0c6d50429ab3e4d4ab7ddb4fd04e784dd58a965ddd9f12\": container with ID starting with 531cc178013db1ec9e0c6d50429ab3e4d4ab7ddb4fd04e784dd58a965ddd9f12 not found: ID does not exist" containerID="531cc178013db1ec9e0c6d50429ab3e4d4ab7ddb4fd04e784dd58a965ddd9f12"
Jul 20 09:47:48.265091 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:48.265046    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"531cc178013db1ec9e0c6d50429ab3e4d4ab7ddb4fd04e784dd58a965ddd9f12"} err="failed to get container status \"531cc178013db1ec9e0c6d50429ab3e4d4ab7ddb4fd04e784dd58a965ddd9f12\": rpc error: code = NotFound desc = could not find container \"531cc178013db1ec9e0c6d50429ab3e4d4ab7ddb4fd04e784dd58a965ddd9f12\": container with ID starting with 531cc178013db1ec9e0c6d50429ab3e4d4ab7ddb4fd04e784dd58a965ddd9f12 not found: ID does not exist"
Jul 20 09:47:48.265628 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:48.265608    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-bxxd6"]
Jul 20 09:47:48.270453 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:48.270416    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-bxxd6"]
Jul 20 09:47:48.282857 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:48.282819    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-76gmf"]
Jul 20 09:47:48.290296 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:48.290269    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-76gmf"]
Jul 20 09:47:49.920778 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:49.920740    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7588d787-eb6c-429a-b0c1-32880c6a963f" path="/var/lib/kubelet/pods/7588d787-eb6c-429a-b0c1-32880c6a963f/volumes"
Jul 20 09:47:49.921167 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:47:49.921071    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1428ce9-478b-4367-8e57-00dcda1906c5" path="/var/lib/kubelet/pods/c1428ce9-478b-4367-8e57-00dcda1906c5/volumes"
Jul 20 09:48:28.741091 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:48:28.741050    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 09:48:43.840883 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:48:43.840835    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 09:48:53.909144 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:48:53.909109    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 09:49:07.415788 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:07.415743    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 09:49:14.622099 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:14.622011    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 09:49:19.304018 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:19.303963    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-54xsf"]
Jul 20 09:49:19.304392 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:19.304312    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7588d787-eb6c-429a-b0c1-32880c6a963f" containerName="authorino"
Jul 20 09:49:19.304392 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:19.304323    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7588d787-eb6c-429a-b0c1-32880c6a963f" containerName="authorino"
Jul 20 09:49:19.304392 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:19.304334    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1428ce9-478b-4367-8e57-00dcda1906c5" containerName="authorino"
Jul 20 09:49:19.304392 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:19.304340    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1428ce9-478b-4367-8e57-00dcda1906c5" containerName="authorino"
Jul 20 09:49:19.304520 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:19.304420    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1428ce9-478b-4367-8e57-00dcda1906c5" containerName="authorino"
Jul 20 09:49:19.304520 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:19.304430    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7588d787-eb6c-429a-b0c1-32880c6a963f" containerName="authorino"
Jul 20 09:49:19.307426 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:19.307405    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54xsf"
Jul 20 09:49:19.310382 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:19.310355    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 20 09:49:19.310382 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:19.310373    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 20 09:49:19.310382 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:19.310382    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-hbv28\""
Jul 20 09:49:19.311694 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:19.311670    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 20 09:49:19.318665 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:19.318636    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-54xsf"]
Jul 20 09:49:19.410698 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:19.410658    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36028478-292f-449b-af9b-2679d8cc5d95-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-54xsf\" (UID: \"36028478-292f-449b-af9b-2679d8cc5d95\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54xsf"
Jul 20 09:49:19.410902 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:19.410720    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36028478-292f-449b-af9b-2679d8cc5d95-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-54xsf\" (UID: \"36028478-292f-449b-af9b-2679d8cc5d95\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54xsf"
Jul 20 09:49:19.410902 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:19.410745    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36028478-292f-449b-af9b-2679d8cc5d95-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-54xsf\" (UID: \"36028478-292f-449b-af9b-2679d8cc5d95\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54xsf"
Jul 20 09:49:19.410902 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:19.410762    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36028478-292f-449b-af9b-2679d8cc5d95-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-54xsf\" (UID: \"36028478-292f-449b-af9b-2679d8cc5d95\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54xsf"
Jul 20 09:49:19.410902 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:19.410783    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htrxf\" (UniqueName: \"kubernetes.io/projected/36028478-292f-449b-af9b-2679d8cc5d95-kube-api-access-htrxf\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-54xsf\" (UID: \"36028478-292f-449b-af9b-2679d8cc5d95\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54xsf"
Jul 20 09:49:19.410902 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:19.410819    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36028478-292f-449b-af9b-2679d8cc5d95-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-54xsf\" (UID: \"36028478-292f-449b-af9b-2679d8cc5d95\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54xsf"
Jul 20 09:49:19.511490 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:19.511445    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36028478-292f-449b-af9b-2679d8cc5d95-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-54xsf\" (UID: \"36028478-292f-449b-af9b-2679d8cc5d95\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54xsf"
Jul 20 09:49:19.511490 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:19.511487    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36028478-292f-449b-af9b-2679d8cc5d95-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-54xsf\" (UID: \"36028478-292f-449b-af9b-2679d8cc5d95\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54xsf"
Jul 20 09:49:19.511749 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:19.511513    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htrxf\" (UniqueName: \"kubernetes.io/projected/36028478-292f-449b-af9b-2679d8cc5d95-kube-api-access-htrxf\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-54xsf\" (UID: \"36028478-292f-449b-af9b-2679d8cc5d95\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54xsf"
Jul 20 09:49:19.511749 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:19.511553    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36028478-292f-449b-af9b-2679d8cc5d95-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-54xsf\" (UID: \"36028478-292f-449b-af9b-2679d8cc5d95\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54xsf"
Jul 20 09:49:19.511749 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:19.511575    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36028478-292f-449b-af9b-2679d8cc5d95-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-54xsf\" (UID: \"36028478-292f-449b-af9b-2679d8cc5d95\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54xsf"
Jul 20 09:49:19.511749 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:19.511614    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36028478-292f-449b-af9b-2679d8cc5d95-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-54xsf\" (UID: \"36028478-292f-449b-af9b-2679d8cc5d95\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54xsf"
Jul 20 09:49:19.511967 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:19.511944    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36028478-292f-449b-af9b-2679d8cc5d95-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-54xsf\" (UID: \"36028478-292f-449b-af9b-2679d8cc5d95\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54xsf"
Jul 20 09:49:19.512077 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:19.511950    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36028478-292f-449b-af9b-2679d8cc5d95-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-54xsf\" (UID: \"36028478-292f-449b-af9b-2679d8cc5d95\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54xsf"
Jul 20 09:49:19.512148 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:19.512130    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36028478-292f-449b-af9b-2679d8cc5d95-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-54xsf\" (UID: \"36028478-292f-449b-af9b-2679d8cc5d95\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54xsf"
Jul 20 09:49:19.514214 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:19.514187    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36028478-292f-449b-af9b-2679d8cc5d95-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-54xsf\" (UID: \"36028478-292f-449b-af9b-2679d8cc5d95\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54xsf"
Jul 20 09:49:19.514651 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:19.514628    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36028478-292f-449b-af9b-2679d8cc5d95-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-54xsf\" (UID: \"36028478-292f-449b-af9b-2679d8cc5d95\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54xsf"
Jul 20 09:49:19.521593 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:19.521551    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htrxf\" (UniqueName: \"kubernetes.io/projected/36028478-292f-449b-af9b-2679d8cc5d95-kube-api-access-htrxf\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-54xsf\" (UID: \"36028478-292f-449b-af9b-2679d8cc5d95\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54xsf"
Jul 20 09:49:19.618836 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:19.618742    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54xsf"
Jul 20 09:49:19.784421 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:19.784394    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-54xsf"]
Jul 20 09:49:19.786924 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:49:19.786890    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36028478_292f_449b_af9b_2679d8cc5d95.slice/crio-ef34f68ce8a9b118947903291076badc00ec6db28a433492c869dfb8956702e5 WatchSource:0}: Error finding container ef34f68ce8a9b118947903291076badc00ec6db28a433492c869dfb8956702e5: Status 404 returned error can't find the container with id ef34f68ce8a9b118947903291076badc00ec6db28a433492c869dfb8956702e5
Jul 20 09:49:19.788320 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:19.788298    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 09:49:20.666856 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:20.666808    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54xsf" event={"ID":"36028478-292f-449b-af9b-2679d8cc5d95","Type":"ContainerStarted","Data":"ef34f68ce8a9b118947903291076badc00ec6db28a433492c869dfb8956702e5"}
Jul 20 09:49:21.673880 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:21.673840    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54xsf" event={"ID":"36028478-292f-449b-af9b-2679d8cc5d95","Type":"ContainerStarted","Data":"915878051e1f401cb6a60d1678c641a055ca064a063b20d5a37713ec377690ad"}
Jul 20 09:49:21.694594 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:21.694541    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54xsf" podStartSLOduration=1.661668825 podStartE2EDuration="2.694523489s" totalImagesPullingTime="1.032854664s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 09:49:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:49:21.693528629 +0000 UTC m=+628.244983095" watchObservedRunningTime="2026-07-20 09:49:21.694523489 +0000 UTC m=+628.245977955"
Jul 20 09:49:22.219366 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:22.219322    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 09:49:29.619763 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:29.619717    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54xsf"
Jul 20 09:49:29.620314 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:29.619780    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54xsf"
Jul 20 09:49:29.632526 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:29.632490    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54xsf"
Jul 20 09:49:29.722763 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:29.722726    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-54xsf"
Jul 20 09:49:38.917153 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:49:38.917112    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 09:50:55.352529 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:50:55.352442    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7fdbc5985b-c468h"]
Jul 20 09:50:55.356041 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:50:55.356010    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7fdbc5985b-c468h"
Jul 20 09:50:55.366262 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:50:55.366233    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7fdbc5985b-c468h"]
Jul 20 09:50:55.457242 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:50:55.457200    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a3f446c1-ffcb-4dba-876e-fc267e1f5421-tls-cert\") pod \"authorino-7fdbc5985b-c468h\" (UID: \"a3f446c1-ffcb-4dba-876e-fc267e1f5421\") " pod="kuadrant-system/authorino-7fdbc5985b-c468h"
Jul 20 09:50:55.457438 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:50:55.457258    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtnbq\" (UniqueName: \"kubernetes.io/projected/a3f446c1-ffcb-4dba-876e-fc267e1f5421-kube-api-access-mtnbq\") pod \"authorino-7fdbc5985b-c468h\" (UID: \"a3f446c1-ffcb-4dba-876e-fc267e1f5421\") " pod="kuadrant-system/authorino-7fdbc5985b-c468h"
Jul 20 09:50:55.557906 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:50:55.557867    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a3f446c1-ffcb-4dba-876e-fc267e1f5421-tls-cert\") pod \"authorino-7fdbc5985b-c468h\" (UID: \"a3f446c1-ffcb-4dba-876e-fc267e1f5421\") " pod="kuadrant-system/authorino-7fdbc5985b-c468h"
Jul 20 09:50:55.557906 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:50:55.557915    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtnbq\" (UniqueName: \"kubernetes.io/projected/a3f446c1-ffcb-4dba-876e-fc267e1f5421-kube-api-access-mtnbq\") pod \"authorino-7fdbc5985b-c468h\" (UID: \"a3f446c1-ffcb-4dba-876e-fc267e1f5421\") " pod="kuadrant-system/authorino-7fdbc5985b-c468h"
Jul 20 09:50:55.560613 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:50:55.560581    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a3f446c1-ffcb-4dba-876e-fc267e1f5421-tls-cert\") pod \"authorino-7fdbc5985b-c468h\" (UID: \"a3f446c1-ffcb-4dba-876e-fc267e1f5421\") " pod="kuadrant-system/authorino-7fdbc5985b-c468h"
Jul 20 09:50:55.567100 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:50:55.567070    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtnbq\" (UniqueName: \"kubernetes.io/projected/a3f446c1-ffcb-4dba-876e-fc267e1f5421-kube-api-access-mtnbq\") pod \"authorino-7fdbc5985b-c468h\" (UID: \"a3f446c1-ffcb-4dba-876e-fc267e1f5421\") " pod="kuadrant-system/authorino-7fdbc5985b-c468h"
Jul 20 09:50:55.666900 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:50:55.666803    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7fdbc5985b-c468h"
Jul 20 09:50:55.818888 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:50:55.818859    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7fdbc5985b-c468h"]
Jul 20 09:50:55.822052 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:50:55.822019    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda3f446c1_ffcb_4dba_876e_fc267e1f5421.slice/crio-f8fbb32a00cd4e15f38249aa8771ee85e282574550a6f6469cd4cf30b1a5b365 WatchSource:0}: Error finding container f8fbb32a00cd4e15f38249aa8771ee85e282574550a6f6469cd4cf30b1a5b365: Status 404 returned error can't find the container with id f8fbb32a00cd4e15f38249aa8771ee85e282574550a6f6469cd4cf30b1a5b365
Jul 20 09:50:56.101585 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:50:56.101546    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7fdbc5985b-c468h" event={"ID":"a3f446c1-ffcb-4dba-876e-fc267e1f5421","Type":"ContainerStarted","Data":"f8fbb32a00cd4e15f38249aa8771ee85e282574550a6f6469cd4cf30b1a5b365"}
Jul 20 09:50:57.111255 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:50:57.111214    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7fdbc5985b-c468h" event={"ID":"a3f446c1-ffcb-4dba-876e-fc267e1f5421","Type":"ContainerStarted","Data":"dcfc0a03dbbfa7d7a0d2caa365e8de075720c8013b159ebcb842e820851259a2"}
Jul 20 09:50:57.135755 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:50:57.135689    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7fdbc5985b-c468h" podStartSLOduration=1.356048698 podStartE2EDuration="2.135663197s" totalImagesPullingTime="779.614499ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 09:50:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:50:57.133949335 +0000 UTC m=+723.685403800" watchObservedRunningTime="2026-07-20 09:50:57.135663197 +0000 UTC m=+723.687117659"
Jul 20 09:50:57.192045 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:50:57.183011    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6ddf9f4f7c-8htbz"]
Jul 20 09:50:57.192045 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:50:57.183227    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6ddf9f4f7c-8htbz" podUID="d71c0646-b5af-4b56-870e-68fff4e5f6cc" containerName="authorino" containerID="cri-o://99375a565a62a73f4e3985b400dc5ad913001cec8e418c164fabfe8786295bf4" gracePeriod=30
Jul 20 09:50:57.466720 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:50:57.466683    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6ddf9f4f7c-8htbz"
Jul 20 09:50:57.573662 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:50:57.573626    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d71c0646-b5af-4b56-870e-68fff4e5f6cc-tls-cert\") pod \"d71c0646-b5af-4b56-870e-68fff4e5f6cc\" (UID: \"d71c0646-b5af-4b56-870e-68fff4e5f6cc\") "
Jul 20 09:50:57.573825 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:50:57.573704    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-686f5\" (UniqueName: \"kubernetes.io/projected/d71c0646-b5af-4b56-870e-68fff4e5f6cc-kube-api-access-686f5\") pod \"d71c0646-b5af-4b56-870e-68fff4e5f6cc\" (UID: \"d71c0646-b5af-4b56-870e-68fff4e5f6cc\") "
Jul 20 09:50:57.576050 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:50:57.576007    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d71c0646-b5af-4b56-870e-68fff4e5f6cc-kube-api-access-686f5" (OuterVolumeSpecName: "kube-api-access-686f5") pod "d71c0646-b5af-4b56-870e-68fff4e5f6cc" (UID: "d71c0646-b5af-4b56-870e-68fff4e5f6cc"). InnerVolumeSpecName "kube-api-access-686f5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 09:50:57.585475 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:50:57.585435    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d71c0646-b5af-4b56-870e-68fff4e5f6cc-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "d71c0646-b5af-4b56-870e-68fff4e5f6cc" (UID: "d71c0646-b5af-4b56-870e-68fff4e5f6cc"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 09:50:57.674805 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:50:57.674712    2566 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d71c0646-b5af-4b56-870e-68fff4e5f6cc-tls-cert\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 09:50:57.674805 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:50:57.674749    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-686f5\" (UniqueName: \"kubernetes.io/projected/d71c0646-b5af-4b56-870e-68fff4e5f6cc-kube-api-access-686f5\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 09:50:58.124327 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:50:58.124289    2566 generic.go:356] "Generic (PLEG): container finished" podID="d71c0646-b5af-4b56-870e-68fff4e5f6cc" containerID="99375a565a62a73f4e3985b400dc5ad913001cec8e418c164fabfe8786295bf4" exitCode=0
Jul 20 09:50:58.124791 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:50:58.124353    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6ddf9f4f7c-8htbz"
Jul 20 09:50:58.124791 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:50:58.124381    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6ddf9f4f7c-8htbz" event={"ID":"d71c0646-b5af-4b56-870e-68fff4e5f6cc","Type":"ContainerDied","Data":"99375a565a62a73f4e3985b400dc5ad913001cec8e418c164fabfe8786295bf4"}
Jul 20 09:50:58.124791 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:50:58.124430    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6ddf9f4f7c-8htbz" event={"ID":"d71c0646-b5af-4b56-870e-68fff4e5f6cc","Type":"ContainerDied","Data":"f0b4d6d93ac2c7c83068f24f1ddee5f1c908960a657b537cc82b52b113ad2b96"}
Jul 20 09:50:58.124791 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:50:58.124451    2566 scope.go:117] "RemoveContainer" containerID="99375a565a62a73f4e3985b400dc5ad913001cec8e418c164fabfe8786295bf4"
Jul 20 09:50:58.133849 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:50:58.133824    2566 scope.go:117] "RemoveContainer" containerID="99375a565a62a73f4e3985b400dc5ad913001cec8e418c164fabfe8786295bf4"
Jul 20 09:50:58.134199 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:50:58.134181    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99375a565a62a73f4e3985b400dc5ad913001cec8e418c164fabfe8786295bf4\": container with ID starting with 99375a565a62a73f4e3985b400dc5ad913001cec8e418c164fabfe8786295bf4 not found: ID does not exist" containerID="99375a565a62a73f4e3985b400dc5ad913001cec8e418c164fabfe8786295bf4"
Jul 20 09:50:58.134280 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:50:58.134214    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99375a565a62a73f4e3985b400dc5ad913001cec8e418c164fabfe8786295bf4"} err="failed to get container status \"99375a565a62a73f4e3985b400dc5ad913001cec8e418c164fabfe8786295bf4\": rpc error: code = NotFound desc = could not find container \"99375a565a62a73f4e3985b400dc5ad913001cec8e418c164fabfe8786295bf4\": container with ID starting with 99375a565a62a73f4e3985b400dc5ad913001cec8e418c164fabfe8786295bf4 not found: ID does not exist"
Jul 20 09:50:58.141665 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:50:58.141632    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6ddf9f4f7c-8htbz"]
Jul 20 09:50:58.147390 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:50:58.147357    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6ddf9f4f7c-8htbz"]
Jul 20 09:50:59.920413 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:50:59.920381    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d71c0646-b5af-4b56-870e-68fff4e5f6cc" path="/var/lib/kubelet/pods/d71c0646-b5af-4b56-870e-68fff4e5f6cc/volumes"
Jul 20 09:51:27.921657 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:51:27.921625    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 09:51:37.520612 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:51:37.520558    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 09:52:11.623816 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:52:11.623781    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 09:53:55.123669 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:53:55.123628    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 09:54:04.516852 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:54:04.516801    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 09:54:18.020425 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:54:18.020386    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 09:54:35.216073 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:54:35.216021    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 09:54:44.421857 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:54:44.421818    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 09:54:55.409548 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:54:55.409506    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 09:55:03.416726 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:55:03.416688    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 09:55:13.029062 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:55:13.029018    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 09:55:23.719370 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:55:23.719268    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 09:55:36.630533 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:55:36.630493    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 09:55:44.421166 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:55:44.421123    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 09:56:07.619335 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:56:07.619285    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 09:56:20.110531 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:56:20.110488    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 09:56:41.920583 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:56:41.920548    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 09:56:53.518380 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:56:53.518272    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 09:57:09.816310 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:57:09.816263    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 09:57:19.820961 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:57:19.820914    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 09:57:30.028697 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:57:30.028661    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 09:57:42.322486 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:57:42.322436    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 09:58:13.385387 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:58:13.385347    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7bdl"]
Jul 20 09:58:13.386035 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:58:13.385574    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7bdl" podUID="a1c0d635-044b-46db-a732-57eda1799859" containerName="manager" containerID="cri-o://33eef42ec4bb4648cf2c8a118540bb781ddd6fc1210531cadaf26355c6ef6581" gracePeriod=10
Jul 20 09:58:16.550422 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:58:16.550394    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7bdl"
Jul 20 09:58:16.608191 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:58:16.608151    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a1c0d635-044b-46db-a732-57eda1799859-extensions-socket-volume\") pod \"a1c0d635-044b-46db-a732-57eda1799859\" (UID: \"a1c0d635-044b-46db-a732-57eda1799859\") "
Jul 20 09:58:16.608391 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:58:16.608210    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8wpmw\" (UniqueName: \"kubernetes.io/projected/a1c0d635-044b-46db-a732-57eda1799859-kube-api-access-8wpmw\") pod \"a1c0d635-044b-46db-a732-57eda1799859\" (UID: \"a1c0d635-044b-46db-a732-57eda1799859\") "
Jul 20 09:58:16.608526 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:58:16.608502    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1c0d635-044b-46db-a732-57eda1799859-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a1c0d635-044b-46db-a732-57eda1799859" (UID: "a1c0d635-044b-46db-a732-57eda1799859"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 09:58:16.610583 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:58:16.610508    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1c0d635-044b-46db-a732-57eda1799859-kube-api-access-8wpmw" (OuterVolumeSpecName: "kube-api-access-8wpmw") pod "a1c0d635-044b-46db-a732-57eda1799859" (UID: "a1c0d635-044b-46db-a732-57eda1799859"). InnerVolumeSpecName "kube-api-access-8wpmw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 09:58:16.708932 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:58:16.708873    2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a1c0d635-044b-46db-a732-57eda1799859-extensions-socket-volume\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 09:58:16.708932 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:58:16.708917    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wpmw\" (UniqueName: \"kubernetes.io/projected/a1c0d635-044b-46db-a732-57eda1799859-kube-api-access-8wpmw\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 09:58:17.185624 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:58:17.185579    2566 generic.go:356] "Generic (PLEG): container finished" podID="a1c0d635-044b-46db-a732-57eda1799859" containerID="33eef42ec4bb4648cf2c8a118540bb781ddd6fc1210531cadaf26355c6ef6581" exitCode=0
Jul 20 09:58:17.185790 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:58:17.185650    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7bdl"
Jul 20 09:58:17.185790 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:58:17.185665    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7bdl" event={"ID":"a1c0d635-044b-46db-a732-57eda1799859","Type":"ContainerDied","Data":"33eef42ec4bb4648cf2c8a118540bb781ddd6fc1210531cadaf26355c6ef6581"}
Jul 20 09:58:17.185790 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:58:17.185724    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7bdl" event={"ID":"a1c0d635-044b-46db-a732-57eda1799859","Type":"ContainerDied","Data":"1def6fe671b9ac9ceb2aac72fb13b1d67a7ba09a403e059f1cdfcb09f0113b6b"}
Jul 20 09:58:17.185790 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:58:17.185746    2566 scope.go:117] "RemoveContainer" containerID="33eef42ec4bb4648cf2c8a118540bb781ddd6fc1210531cadaf26355c6ef6581"
Jul 20 09:58:17.195871 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:58:17.195851    2566 scope.go:117] "RemoveContainer" containerID="33eef42ec4bb4648cf2c8a118540bb781ddd6fc1210531cadaf26355c6ef6581"
Jul 20 09:58:17.196243 ip-10-0-140-134 kubenswrapper[2566]: E0720 09:58:17.196219    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33eef42ec4bb4648cf2c8a118540bb781ddd6fc1210531cadaf26355c6ef6581\": container with ID starting with 33eef42ec4bb4648cf2c8a118540bb781ddd6fc1210531cadaf26355c6ef6581 not found: ID does not exist" containerID="33eef42ec4bb4648cf2c8a118540bb781ddd6fc1210531cadaf26355c6ef6581"
Jul 20 09:58:17.196298 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:58:17.196254    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33eef42ec4bb4648cf2c8a118540bb781ddd6fc1210531cadaf26355c6ef6581"} err="failed to get container status \"33eef42ec4bb4648cf2c8a118540bb781ddd6fc1210531cadaf26355c6ef6581\": rpc error: code = NotFound desc = could not find container \"33eef42ec4bb4648cf2c8a118540bb781ddd6fc1210531cadaf26355c6ef6581\": container with ID starting with 33eef42ec4bb4648cf2c8a118540bb781ddd6fc1210531cadaf26355c6ef6581 not found: ID does not exist"
Jul 20 09:58:17.211548 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:58:17.211510    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7bdl"]
Jul 20 09:58:17.222619 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:58:17.222588    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-c7bdl"]
Jul 20 09:58:17.920122 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:58:17.920086    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1c0d635-044b-46db-a732-57eda1799859" path="/var/lib/kubelet/pods/a1c0d635-044b-46db-a732-57eda1799859/volumes"
Jul 20 09:59:14.651601 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:59:14.651564    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ss27q"]
Jul 20 09:59:14.652102 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:59:14.651905    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1c0d635-044b-46db-a732-57eda1799859" containerName="manager"
Jul 20 09:59:14.652102 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:59:14.651917    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1c0d635-044b-46db-a732-57eda1799859" containerName="manager"
Jul 20 09:59:14.652102 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:59:14.651927    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d71c0646-b5af-4b56-870e-68fff4e5f6cc" containerName="authorino"
Jul 20 09:59:14.652102 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:59:14.651933    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d71c0646-b5af-4b56-870e-68fff4e5f6cc" containerName="authorino"
Jul 20 09:59:14.652102 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:59:14.652017    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1c0d635-044b-46db-a732-57eda1799859" containerName="manager"
Jul 20 09:59:14.652102 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:59:14.652027    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d71c0646-b5af-4b56-870e-68fff4e5f6cc" containerName="authorino"
Jul 20 09:59:14.656299 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:59:14.656274    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ss27q"
Jul 20 09:59:14.659085 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:59:14.659063    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-xctf2\""
Jul 20 09:59:14.672244 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:59:14.672205    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ss27q"]
Jul 20 09:59:14.688622 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:59:14.688588    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nh7qs\" (UniqueName: \"kubernetes.io/projected/0859aa77-3466-49d5-ac4f-bf326c67b529-kube-api-access-nh7qs\") pod \"kuadrant-operator-controller-manager-86878c769f-ss27q\" (UID: \"0859aa77-3466-49d5-ac4f-bf326c67b529\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ss27q"
Jul 20 09:59:14.688799 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:59:14.688660    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0859aa77-3466-49d5-ac4f-bf326c67b529-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ss27q\" (UID: \"0859aa77-3466-49d5-ac4f-bf326c67b529\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ss27q"
Jul 20 09:59:14.789794 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:59:14.789754    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nh7qs\" (UniqueName: \"kubernetes.io/projected/0859aa77-3466-49d5-ac4f-bf326c67b529-kube-api-access-nh7qs\") pod \"kuadrant-operator-controller-manager-86878c769f-ss27q\" (UID: \"0859aa77-3466-49d5-ac4f-bf326c67b529\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ss27q"
Jul 20 09:59:14.789990 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:59:14.789857    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0859aa77-3466-49d5-ac4f-bf326c67b529-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ss27q\" (UID: \"0859aa77-3466-49d5-ac4f-bf326c67b529\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ss27q"
Jul 20 09:59:14.790280 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:59:14.790257    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0859aa77-3466-49d5-ac4f-bf326c67b529-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ss27q\" (UID: \"0859aa77-3466-49d5-ac4f-bf326c67b529\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ss27q"
Jul 20 09:59:14.802297 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:59:14.802253    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nh7qs\" (UniqueName: \"kubernetes.io/projected/0859aa77-3466-49d5-ac4f-bf326c67b529-kube-api-access-nh7qs\") pod \"kuadrant-operator-controller-manager-86878c769f-ss27q\" (UID: \"0859aa77-3466-49d5-ac4f-bf326c67b529\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ss27q"
Jul 20 09:59:14.967053 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:59:14.966947    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ss27q"
Jul 20 09:59:15.153936 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:59:15.153909    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ss27q"]
Jul 20 09:59:15.156135 ip-10-0-140-134 kubenswrapper[2566]: W0720 09:59:15.156103    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0859aa77_3466_49d5_ac4f_bf326c67b529.slice/crio-9e1ba1c93d323c6eac146cc9ab3205b5ed0d27a3a9487556dc5ff567c40f5ea7 WatchSource:0}: Error finding container 9e1ba1c93d323c6eac146cc9ab3205b5ed0d27a3a9487556dc5ff567c40f5ea7: Status 404 returned error can't find the container with id 9e1ba1c93d323c6eac146cc9ab3205b5ed0d27a3a9487556dc5ff567c40f5ea7
Jul 20 09:59:15.446521 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:59:15.446476    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ss27q" event={"ID":"0859aa77-3466-49d5-ac4f-bf326c67b529","Type":"ContainerStarted","Data":"f65645e72b2b6d9339dc8b04eda70efded2c32cc54b4732171504002d5c055e5"}
Jul 20 09:59:15.446717 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:59:15.446601    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ss27q" event={"ID":"0859aa77-3466-49d5-ac4f-bf326c67b529","Type":"ContainerStarted","Data":"9e1ba1c93d323c6eac146cc9ab3205b5ed0d27a3a9487556dc5ff567c40f5ea7"}
Jul 20 09:59:15.446717 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:59:15.446634    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ss27q"
Jul 20 09:59:15.469407 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:59:15.469335    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ss27q" podStartSLOduration=1.46931119 podStartE2EDuration="1.46931119s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 09:59:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 09:59:15.468265365 +0000 UTC m=+1222.019719829" watchObservedRunningTime="2026-07-20 09:59:15.46931119 +0000 UTC m=+1222.020765656"
Jul 20 09:59:26.452875 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:59:26.452841    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ss27q"
Jul 20 09:59:36.524923 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:59:36.524883    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 09:59:47.715732 ip-10-0-140-134 kubenswrapper[2566]: I0720 09:59:47.715641    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:00:10.424769 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:00:10.424722    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:00:23.517215 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:00:23.517174    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:00:33.711792 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:00:33.711748    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:00:45.110128 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:00:45.110084    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:00:52.913665 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:00:52.913620    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:01:05.023272 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:01:05.023228    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:01:15.413390 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:01:15.413306    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:01:25.012687 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:01:25.012647    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:01:43.012676 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:01:43.012628    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:01:53.313853 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:01:53.313812    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:01:55.629125 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:01:55.629082    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl"]
Jul 20 10:01:55.632413 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:01:55.632388    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl"
Jul 20 10:01:55.635246 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:01:55.635215    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-db33ff6f-kserve-self-signed-certs\""
Jul 20 10:01:55.641459 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:01:55.641427    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl"]
Jul 20 10:01:55.672658 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:01:55.672608    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8fc84ba7-5834-4cc5-9d67-22a74cea2f5d-home\") pod \"e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl\" (UID: \"8fc84ba7-5834-4cc5-9d67-22a74cea2f5d\") " pod="llm/e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl"
Jul 20 10:01:55.672868 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:01:55.672679    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnv9x\" (UniqueName: \"kubernetes.io/projected/8fc84ba7-5834-4cc5-9d67-22a74cea2f5d-kube-api-access-gnv9x\") pod \"e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl\" (UID: \"8fc84ba7-5834-4cc5-9d67-22a74cea2f5d\") " pod="llm/e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl"
Jul 20 10:01:55.672868 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:01:55.672774    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8fc84ba7-5834-4cc5-9d67-22a74cea2f5d-dshm\") pod \"e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl\" (UID: \"8fc84ba7-5834-4cc5-9d67-22a74cea2f5d\") " pod="llm/e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl"
Jul 20 10:01:55.672868 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:01:55.672808    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8fc84ba7-5834-4cc5-9d67-22a74cea2f5d-model-cache\") pod \"e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl\" (UID: \"8fc84ba7-5834-4cc5-9d67-22a74cea2f5d\") " pod="llm/e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl"
Jul 20 10:01:55.673068 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:01:55.672900    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8fc84ba7-5834-4cc5-9d67-22a74cea2f5d-tmp-dir\") pod \"e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl\" (UID: \"8fc84ba7-5834-4cc5-9d67-22a74cea2f5d\") " pod="llm/e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl"
Jul 20 10:01:55.673068 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:01:55.672950    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fc84ba7-5834-4cc5-9d67-22a74cea2f5d-tls-certs\") pod \"e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl\" (UID: \"8fc84ba7-5834-4cc5-9d67-22a74cea2f5d\") " pod="llm/e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl"
Jul 20 10:01:55.774158 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:01:55.774113    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fc84ba7-5834-4cc5-9d67-22a74cea2f5d-tls-certs\") pod \"e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl\" (UID: \"8fc84ba7-5834-4cc5-9d67-22a74cea2f5d\") " pod="llm/e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl"
Jul 20 10:01:55.774363 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:01:55.774176    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8fc84ba7-5834-4cc5-9d67-22a74cea2f5d-home\") pod \"e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl\" (UID: \"8fc84ba7-5834-4cc5-9d67-22a74cea2f5d\") " pod="llm/e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl"
Jul 20 10:01:55.774363 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:01:55.774200    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnv9x\" (UniqueName: \"kubernetes.io/projected/8fc84ba7-5834-4cc5-9d67-22a74cea2f5d-kube-api-access-gnv9x\") pod \"e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl\" (UID: \"8fc84ba7-5834-4cc5-9d67-22a74cea2f5d\") " pod="llm/e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl"
Jul 20 10:01:55.774363 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:01:55.774221    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8fc84ba7-5834-4cc5-9d67-22a74cea2f5d-dshm\") pod \"e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl\" (UID: \"8fc84ba7-5834-4cc5-9d67-22a74cea2f5d\") " pod="llm/e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl"
Jul 20 10:01:55.774363 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:01:55.774237    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8fc84ba7-5834-4cc5-9d67-22a74cea2f5d-model-cache\") pod \"e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl\" (UID: \"8fc84ba7-5834-4cc5-9d67-22a74cea2f5d\") " pod="llm/e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl"
Jul 20 10:01:55.774363 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:01:55.774288    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8fc84ba7-5834-4cc5-9d67-22a74cea2f5d-tmp-dir\") pod \"e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl\" (UID: \"8fc84ba7-5834-4cc5-9d67-22a74cea2f5d\") " pod="llm/e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl"
Jul 20 10:01:55.774705 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:01:55.774572    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8fc84ba7-5834-4cc5-9d67-22a74cea2f5d-home\") pod \"e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl\" (UID: \"8fc84ba7-5834-4cc5-9d67-22a74cea2f5d\") " pod="llm/e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl"
Jul 20 10:01:55.774705 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:01:55.774664    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8fc84ba7-5834-4cc5-9d67-22a74cea2f5d-model-cache\") pod \"e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl\" (UID: \"8fc84ba7-5834-4cc5-9d67-22a74cea2f5d\") " pod="llm/e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl"
Jul 20 10:01:55.774773 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:01:55.774721    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8fc84ba7-5834-4cc5-9d67-22a74cea2f5d-tmp-dir\") pod \"e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl\" (UID: \"8fc84ba7-5834-4cc5-9d67-22a74cea2f5d\") " pod="llm/e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl"
Jul 20 10:01:55.776696 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:01:55.776673    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8fc84ba7-5834-4cc5-9d67-22a74cea2f5d-dshm\") pod \"e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl\" (UID: \"8fc84ba7-5834-4cc5-9d67-22a74cea2f5d\") " pod="llm/e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl"
Jul 20 10:01:55.776953 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:01:55.776932    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fc84ba7-5834-4cc5-9d67-22a74cea2f5d-tls-certs\") pod \"e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl\" (UID: \"8fc84ba7-5834-4cc5-9d67-22a74cea2f5d\") " pod="llm/e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl"
Jul 20 10:01:55.782842 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:01:55.782812    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnv9x\" (UniqueName: \"kubernetes.io/projected/8fc84ba7-5834-4cc5-9d67-22a74cea2f5d-kube-api-access-gnv9x\") pod \"e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl\" (UID: \"8fc84ba7-5834-4cc5-9d67-22a74cea2f5d\") " pod="llm/e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl"
Jul 20 10:01:55.943179 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:01:55.943089    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl"
Jul 20 10:01:56.093375 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:01:56.093347    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl"]
Jul 20 10:01:56.095619 ip-10-0-140-134 kubenswrapper[2566]: W0720 10:01:56.095589    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8fc84ba7_5834_4cc5_9d67_22a74cea2f5d.slice/crio-d34f4b3a46d99a8201e810f5fa1c9d98b36acda8fc28dd72858ddc02cbc8f605 WatchSource:0}: Error finding container d34f4b3a46d99a8201e810f5fa1c9d98b36acda8fc28dd72858ddc02cbc8f605: Status 404 returned error can't find the container with id d34f4b3a46d99a8201e810f5fa1c9d98b36acda8fc28dd72858ddc02cbc8f605
Jul 20 10:01:56.213888 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:01:56.213796    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl" event={"ID":"8fc84ba7-5834-4cc5-9d67-22a74cea2f5d","Type":"ContainerStarted","Data":"7f7adcb748f682508d7e7a60e491e0a64ab4bd94350b7d39d7c525c133147fe4"}
Jul 20 10:01:56.213888 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:01:56.213837    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl" event={"ID":"8fc84ba7-5834-4cc5-9d67-22a74cea2f5d","Type":"ContainerStarted","Data":"d34f4b3a46d99a8201e810f5fa1c9d98b36acda8fc28dd72858ddc02cbc8f605"}
Jul 20 10:01:56.233303 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:01:56.233246    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl" podStartSLOduration=1.233228701 podStartE2EDuration="1.233228701s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 10:01:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 10:01:56.231727696 +0000 UTC m=+1382.783182160" watchObservedRunningTime="2026-07-20 10:01:56.233228701 +0000 UTC m=+1382.784683165"
Jul 20 10:02:05.944160 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:05.944060    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl"
Jul 20 10:02:05.944160 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:05.944117    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl"
Jul 20 10:02:05.957038 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:05.956995    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl"
Jul 20 10:02:06.274174 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:06.274087    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl"
Jul 20 10:02:07.216527 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:07.216482    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:02:12.222052 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:12.222006    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr"]
Jul 20 10:02:12.226223 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:12.226195    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr"
Jul 20 10:02:12.230155 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:12.230128    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-db33ff6f-kserve-self-signed-certs\""
Jul 20 10:02:12.237524 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:12.237496    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr"]
Jul 20 10:02:12.320618 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:12.320576    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd3acf6a-a489-436e-87d7-97488bc297f6-tmp-dir\") pod \"e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr\" (UID: \"bd3acf6a-a489-436e-87d7-97488bc297f6\") " pod="llm/e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr"
Jul 20 10:02:12.320833 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:12.320629    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd3acf6a-a489-436e-87d7-97488bc297f6-home\") pod \"e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr\" (UID: \"bd3acf6a-a489-436e-87d7-97488bc297f6\") " pod="llm/e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr"
Jul 20 10:02:12.320833 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:12.320733    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd3acf6a-a489-436e-87d7-97488bc297f6-dshm\") pod \"e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr\" (UID: \"bd3acf6a-a489-436e-87d7-97488bc297f6\") " pod="llm/e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr"
Jul 20 10:02:12.320833 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:12.320793    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd3acf6a-a489-436e-87d7-97488bc297f6-tls-certs\") pod \"e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr\" (UID: \"bd3acf6a-a489-436e-87d7-97488bc297f6\") " pod="llm/e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr"
Jul 20 10:02:12.320833 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:12.320821    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd3acf6a-a489-436e-87d7-97488bc297f6-model-cache\") pod \"e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr\" (UID: \"bd3acf6a-a489-436e-87d7-97488bc297f6\") " pod="llm/e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr"
Jul 20 10:02:12.321061 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:12.320871    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htmj6\" (UniqueName: \"kubernetes.io/projected/bd3acf6a-a489-436e-87d7-97488bc297f6-kube-api-access-htmj6\") pod \"e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr\" (UID: \"bd3acf6a-a489-436e-87d7-97488bc297f6\") " pod="llm/e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr"
Jul 20 10:02:12.422045 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:12.421968    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd3acf6a-a489-436e-87d7-97488bc297f6-tmp-dir\") pod \"e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr\" (UID: \"bd3acf6a-a489-436e-87d7-97488bc297f6\") " pod="llm/e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr"
Jul 20 10:02:12.422242 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:12.422060    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd3acf6a-a489-436e-87d7-97488bc297f6-home\") pod \"e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr\" (UID: \"bd3acf6a-a489-436e-87d7-97488bc297f6\") " pod="llm/e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr"
Jul 20 10:02:12.422242 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:12.422112    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd3acf6a-a489-436e-87d7-97488bc297f6-dshm\") pod \"e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr\" (UID: \"bd3acf6a-a489-436e-87d7-97488bc297f6\") " pod="llm/e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr"
Jul 20 10:02:12.422242 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:12.422154    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd3acf6a-a489-436e-87d7-97488bc297f6-tls-certs\") pod \"e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr\" (UID: \"bd3acf6a-a489-436e-87d7-97488bc297f6\") " pod="llm/e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr"
Jul 20 10:02:12.422414 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:12.422271    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd3acf6a-a489-436e-87d7-97488bc297f6-model-cache\") pod \"e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr\" (UID: \"bd3acf6a-a489-436e-87d7-97488bc297f6\") " pod="llm/e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr"
Jul 20 10:02:12.422414 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:12.422340    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htmj6\" (UniqueName: \"kubernetes.io/projected/bd3acf6a-a489-436e-87d7-97488bc297f6-kube-api-access-htmj6\") pod \"e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr\" (UID: \"bd3acf6a-a489-436e-87d7-97488bc297f6\") " pod="llm/e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr"
Jul 20 10:02:12.422532 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:12.422460    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd3acf6a-a489-436e-87d7-97488bc297f6-tmp-dir\") pod \"e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr\" (UID: \"bd3acf6a-a489-436e-87d7-97488bc297f6\") " pod="llm/e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr"
Jul 20 10:02:12.422532 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:12.422498    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd3acf6a-a489-436e-87d7-97488bc297f6-home\") pod \"e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr\" (UID: \"bd3acf6a-a489-436e-87d7-97488bc297f6\") " pod="llm/e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr"
Jul 20 10:02:12.422617 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:12.422590    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd3acf6a-a489-436e-87d7-97488bc297f6-model-cache\") pod \"e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr\" (UID: \"bd3acf6a-a489-436e-87d7-97488bc297f6\") " pod="llm/e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr"
Jul 20 10:02:12.424570 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:12.424542    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd3acf6a-a489-436e-87d7-97488bc297f6-dshm\") pod \"e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr\" (UID: \"bd3acf6a-a489-436e-87d7-97488bc297f6\") " pod="llm/e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr"
Jul 20 10:02:12.424728 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:12.424711    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd3acf6a-a489-436e-87d7-97488bc297f6-tls-certs\") pod \"e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr\" (UID: \"bd3acf6a-a489-436e-87d7-97488bc297f6\") " pod="llm/e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr"
Jul 20 10:02:12.438048 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:12.438014    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htmj6\" (UniqueName: \"kubernetes.io/projected/bd3acf6a-a489-436e-87d7-97488bc297f6-kube-api-access-htmj6\") pod \"e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr\" (UID: \"bd3acf6a-a489-436e-87d7-97488bc297f6\") " pod="llm/e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr"
Jul 20 10:02:12.538201 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:12.538111    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr"
Jul 20 10:02:12.708200 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:12.708171    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr"]
Jul 20 10:02:12.710959 ip-10-0-140-134 kubenswrapper[2566]: W0720 10:02:12.710924    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd3acf6a_a489_436e_87d7_97488bc297f6.slice/crio-76324b8d5a169cd02a134787222909193d69648e8dcf25c2f9ac2c343133a389 WatchSource:0}: Error finding container 76324b8d5a169cd02a134787222909193d69648e8dcf25c2f9ac2c343133a389: Status 404 returned error can't find the container with id 76324b8d5a169cd02a134787222909193d69648e8dcf25c2f9ac2c343133a389
Jul 20 10:02:13.305749 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:13.305702    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr" event={"ID":"bd3acf6a-a489-436e-87d7-97488bc297f6","Type":"ContainerStarted","Data":"5e0113f06aa8b4bef5efea4eb3d2a52f705a1e8c148131f465eb1aa081acdab0"}
Jul 20 10:02:13.305749 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:13.305748    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr" event={"ID":"bd3acf6a-a489-436e-87d7-97488bc297f6","Type":"ContainerStarted","Data":"76324b8d5a169cd02a134787222909193d69648e8dcf25c2f9ac2c343133a389"}
Jul 20 10:02:13.332358 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:13.332301    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr" podStartSLOduration=1.332278837 podStartE2EDuration="1.332278837s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 10:02:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 10:02:13.331085903 +0000 UTC m=+1399.882540367" watchObservedRunningTime="2026-07-20 10:02:13.332278837 +0000 UTC m=+1399.883733300"
Jul 20 10:02:15.731857 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:15.731813    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:02:22.539154 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:22.539111    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr"
Jul 20 10:02:22.539610 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:22.539169    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr"
Jul 20 10:02:22.551857 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:22.551826    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr"
Jul 20 10:02:23.367053 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:23.367021    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr"
Jul 20 10:02:44.401745 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:44.401706    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl"]
Jul 20 10:02:46.250077 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:46.250033    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr"]
Jul 20 10:02:47.223045 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:47.223012    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:02:59.423306 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:59.423259    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl" podUID="8fc84ba7-5834-4cc5-9d67-22a74cea2f5d" containerName="main" containerID="cri-o://7f7adcb748f682508d7e7a60e491e0a64ab4bd94350b7d39d7c525c133147fe4" gracePeriod=45
Jul 20 10:02:59.685300 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:59.685220    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl"
Jul 20 10:02:59.749727 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:59.749682    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fc84ba7-5834-4cc5-9d67-22a74cea2f5d-tls-certs\") pod \"8fc84ba7-5834-4cc5-9d67-22a74cea2f5d\" (UID: \"8fc84ba7-5834-4cc5-9d67-22a74cea2f5d\") "
Jul 20 10:02:59.749929 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:59.749790    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8fc84ba7-5834-4cc5-9d67-22a74cea2f5d-model-cache\") pod \"8fc84ba7-5834-4cc5-9d67-22a74cea2f5d\" (UID: \"8fc84ba7-5834-4cc5-9d67-22a74cea2f5d\") "
Jul 20 10:02:59.749929 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:59.749824    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8fc84ba7-5834-4cc5-9d67-22a74cea2f5d-home\") pod \"8fc84ba7-5834-4cc5-9d67-22a74cea2f5d\" (UID: \"8fc84ba7-5834-4cc5-9d67-22a74cea2f5d\") "
Jul 20 10:02:59.749929 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:59.749840    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8fc84ba7-5834-4cc5-9d67-22a74cea2f5d-tmp-dir\") pod \"8fc84ba7-5834-4cc5-9d67-22a74cea2f5d\" (UID: \"8fc84ba7-5834-4cc5-9d67-22a74cea2f5d\") "
Jul 20 10:02:59.749929 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:59.749871    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gnv9x\" (UniqueName: \"kubernetes.io/projected/8fc84ba7-5834-4cc5-9d67-22a74cea2f5d-kube-api-access-gnv9x\") pod \"8fc84ba7-5834-4cc5-9d67-22a74cea2f5d\" (UID: \"8fc84ba7-5834-4cc5-9d67-22a74cea2f5d\") "
Jul 20 10:02:59.749929 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:59.749904    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8fc84ba7-5834-4cc5-9d67-22a74cea2f5d-dshm\") pod \"8fc84ba7-5834-4cc5-9d67-22a74cea2f5d\" (UID: \"8fc84ba7-5834-4cc5-9d67-22a74cea2f5d\") "
Jul 20 10:02:59.750229 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:59.750158    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fc84ba7-5834-4cc5-9d67-22a74cea2f5d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8fc84ba7-5834-4cc5-9d67-22a74cea2f5d" (UID: "8fc84ba7-5834-4cc5-9d67-22a74cea2f5d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 10:02:59.750229 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:59.750173    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fc84ba7-5834-4cc5-9d67-22a74cea2f5d-home" (OuterVolumeSpecName: "home") pod "8fc84ba7-5834-4cc5-9d67-22a74cea2f5d" (UID: "8fc84ba7-5834-4cc5-9d67-22a74cea2f5d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 10:02:59.750229 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:59.750185    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fc84ba7-5834-4cc5-9d67-22a74cea2f5d-model-cache" (OuterVolumeSpecName: "model-cache") pod "8fc84ba7-5834-4cc5-9d67-22a74cea2f5d" (UID: "8fc84ba7-5834-4cc5-9d67-22a74cea2f5d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 10:02:59.752339 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:59.752299    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fc84ba7-5834-4cc5-9d67-22a74cea2f5d-dshm" (OuterVolumeSpecName: "dshm") pod "8fc84ba7-5834-4cc5-9d67-22a74cea2f5d" (UID: "8fc84ba7-5834-4cc5-9d67-22a74cea2f5d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 10:02:59.752339 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:59.752304    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fc84ba7-5834-4cc5-9d67-22a74cea2f5d-kube-api-access-gnv9x" (OuterVolumeSpecName: "kube-api-access-gnv9x") pod "8fc84ba7-5834-4cc5-9d67-22a74cea2f5d" (UID: "8fc84ba7-5834-4cc5-9d67-22a74cea2f5d"). InnerVolumeSpecName "kube-api-access-gnv9x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 10:02:59.752487 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:59.752366    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fc84ba7-5834-4cc5-9d67-22a74cea2f5d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8fc84ba7-5834-4cc5-9d67-22a74cea2f5d" (UID: "8fc84ba7-5834-4cc5-9d67-22a74cea2f5d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 10:02:59.851192 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:59.851150    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8fc84ba7-5834-4cc5-9d67-22a74cea2f5d-model-cache\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:02:59.851192 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:59.851186    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8fc84ba7-5834-4cc5-9d67-22a74cea2f5d-home\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:02:59.851399 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:59.851218    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8fc84ba7-5834-4cc5-9d67-22a74cea2f5d-tmp-dir\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:02:59.851399 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:59.851227    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gnv9x\" (UniqueName: \"kubernetes.io/projected/8fc84ba7-5834-4cc5-9d67-22a74cea2f5d-kube-api-access-gnv9x\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:02:59.851399 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:59.851236    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8fc84ba7-5834-4cc5-9d67-22a74cea2f5d-dshm\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:02:59.851399 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:02:59.851245    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8fc84ba7-5834-4cc5-9d67-22a74cea2f5d-tls-certs\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:03:00.532944 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:00.532905    2566 generic.go:356] "Generic (PLEG): container finished" podID="8fc84ba7-5834-4cc5-9d67-22a74cea2f5d" containerID="7f7adcb748f682508d7e7a60e491e0a64ab4bd94350b7d39d7c525c133147fe4" exitCode=0
Jul 20 10:03:00.533404 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:00.532950    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl"
Jul 20 10:03:00.533404 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:00.533006    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl" event={"ID":"8fc84ba7-5834-4cc5-9d67-22a74cea2f5d","Type":"ContainerDied","Data":"7f7adcb748f682508d7e7a60e491e0a64ab4bd94350b7d39d7c525c133147fe4"}
Jul 20 10:03:00.533404 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:00.533048    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl" event={"ID":"8fc84ba7-5834-4cc5-9d67-22a74cea2f5d","Type":"ContainerDied","Data":"d34f4b3a46d99a8201e810f5fa1c9d98b36acda8fc28dd72858ddc02cbc8f605"}
Jul 20 10:03:00.533404 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:00.533066    2566 scope.go:117] "RemoveContainer" containerID="7f7adcb748f682508d7e7a60e491e0a64ab4bd94350b7d39d7c525c133147fe4"
Jul 20 10:03:00.542231 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:00.542208    2566 scope.go:117] "RemoveContainer" containerID="7f7adcb748f682508d7e7a60e491e0a64ab4bd94350b7d39d7c525c133147fe4"
Jul 20 10:03:00.542593 ip-10-0-140-134 kubenswrapper[2566]: E0720 10:03:00.542571    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f7adcb748f682508d7e7a60e491e0a64ab4bd94350b7d39d7c525c133147fe4\": container with ID starting with 7f7adcb748f682508d7e7a60e491e0a64ab4bd94350b7d39d7c525c133147fe4 not found: ID does not exist" containerID="7f7adcb748f682508d7e7a60e491e0a64ab4bd94350b7d39d7c525c133147fe4"
Jul 20 10:03:00.542675 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:00.542601    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f7adcb748f682508d7e7a60e491e0a64ab4bd94350b7d39d7c525c133147fe4"} err="failed to get container status \"7f7adcb748f682508d7e7a60e491e0a64ab4bd94350b7d39d7c525c133147fe4\": rpc error: code = NotFound desc = could not find container \"7f7adcb748f682508d7e7a60e491e0a64ab4bd94350b7d39d7c525c133147fe4\": container with ID starting with 7f7adcb748f682508d7e7a60e491e0a64ab4bd94350b7d39d7c525c133147fe4 not found: ID does not exist"
Jul 20 10:03:00.552783 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:00.552738    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl"]
Jul 20 10:03:00.557147 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:00.557112    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-db33ff6f-kserve-5778978987-cttpl"]
Jul 20 10:03:01.273507 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:01.273456    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr" podUID="bd3acf6a-a489-436e-87d7-97488bc297f6" containerName="main" containerID="cri-o://5e0113f06aa8b4bef5efea4eb3d2a52f705a1e8c148131f465eb1aa081acdab0" gracePeriod=45
Jul 20 10:03:01.528679 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:01.528616    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr"
Jul 20 10:03:01.538532 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:01.538497    2566 generic.go:356] "Generic (PLEG): container finished" podID="bd3acf6a-a489-436e-87d7-97488bc297f6" containerID="5e0113f06aa8b4bef5efea4eb3d2a52f705a1e8c148131f465eb1aa081acdab0" exitCode=0
Jul 20 10:03:01.538957 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:01.538549    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr"
Jul 20 10:03:01.538957 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:01.538584    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr" event={"ID":"bd3acf6a-a489-436e-87d7-97488bc297f6","Type":"ContainerDied","Data":"5e0113f06aa8b4bef5efea4eb3d2a52f705a1e8c148131f465eb1aa081acdab0"}
Jul 20 10:03:01.538957 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:01.538634    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr" event={"ID":"bd3acf6a-a489-436e-87d7-97488bc297f6","Type":"ContainerDied","Data":"76324b8d5a169cd02a134787222909193d69648e8dcf25c2f9ac2c343133a389"}
Jul 20 10:03:01.538957 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:01.538663    2566 scope.go:117] "RemoveContainer" containerID="5e0113f06aa8b4bef5efea4eb3d2a52f705a1e8c148131f465eb1aa081acdab0"
Jul 20 10:03:01.548728 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:01.548691    2566 scope.go:117] "RemoveContainer" containerID="5e0113f06aa8b4bef5efea4eb3d2a52f705a1e8c148131f465eb1aa081acdab0"
Jul 20 10:03:01.549097 ip-10-0-140-134 kubenswrapper[2566]: E0720 10:03:01.549073    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e0113f06aa8b4bef5efea4eb3d2a52f705a1e8c148131f465eb1aa081acdab0\": container with ID starting with 5e0113f06aa8b4bef5efea4eb3d2a52f705a1e8c148131f465eb1aa081acdab0 not found: ID does not exist" containerID="5e0113f06aa8b4bef5efea4eb3d2a52f705a1e8c148131f465eb1aa081acdab0"
Jul 20 10:03:01.549177 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:01.549106    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e0113f06aa8b4bef5efea4eb3d2a52f705a1e8c148131f465eb1aa081acdab0"} err="failed to get container status \"5e0113f06aa8b4bef5efea4eb3d2a52f705a1e8c148131f465eb1aa081acdab0\": rpc error: code = NotFound desc = could not find container \"5e0113f06aa8b4bef5efea4eb3d2a52f705a1e8c148131f465eb1aa081acdab0\": container with ID starting with 5e0113f06aa8b4bef5efea4eb3d2a52f705a1e8c148131f465eb1aa081acdab0 not found: ID does not exist"
Jul 20 10:03:01.667655 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:01.667612    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd3acf6a-a489-436e-87d7-97488bc297f6-model-cache\") pod \"bd3acf6a-a489-436e-87d7-97488bc297f6\" (UID: \"bd3acf6a-a489-436e-87d7-97488bc297f6\") "
Jul 20 10:03:01.667655 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:01.667652    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd3acf6a-a489-436e-87d7-97488bc297f6-tls-certs\") pod \"bd3acf6a-a489-436e-87d7-97488bc297f6\" (UID: \"bd3acf6a-a489-436e-87d7-97488bc297f6\") "
Jul 20 10:03:01.667895 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:01.667711    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd3acf6a-a489-436e-87d7-97488bc297f6-dshm\") pod \"bd3acf6a-a489-436e-87d7-97488bc297f6\" (UID: \"bd3acf6a-a489-436e-87d7-97488bc297f6\") "
Jul 20 10:03:01.667895 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:01.667752    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd3acf6a-a489-436e-87d7-97488bc297f6-home\") pod \"bd3acf6a-a489-436e-87d7-97488bc297f6\" (UID: \"bd3acf6a-a489-436e-87d7-97488bc297f6\") "
Jul 20 10:03:01.667895 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:01.667779    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd3acf6a-a489-436e-87d7-97488bc297f6-tmp-dir\") pod \"bd3acf6a-a489-436e-87d7-97488bc297f6\" (UID: \"bd3acf6a-a489-436e-87d7-97488bc297f6\") "
Jul 20 10:03:01.667895 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:01.667795    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-htmj6\" (UniqueName: \"kubernetes.io/projected/bd3acf6a-a489-436e-87d7-97488bc297f6-kube-api-access-htmj6\") pod \"bd3acf6a-a489-436e-87d7-97488bc297f6\" (UID: \"bd3acf6a-a489-436e-87d7-97488bc297f6\") "
Jul 20 10:03:01.668119 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:01.668025    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd3acf6a-a489-436e-87d7-97488bc297f6-model-cache" (OuterVolumeSpecName: "model-cache") pod "bd3acf6a-a489-436e-87d7-97488bc297f6" (UID: "bd3acf6a-a489-436e-87d7-97488bc297f6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 10:03:01.668119 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:01.668057    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd3acf6a-a489-436e-87d7-97488bc297f6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bd3acf6a-a489-436e-87d7-97488bc297f6" (UID: "bd3acf6a-a489-436e-87d7-97488bc297f6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 10:03:01.668119 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:01.668073    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd3acf6a-a489-436e-87d7-97488bc297f6-home" (OuterVolumeSpecName: "home") pod "bd3acf6a-a489-436e-87d7-97488bc297f6" (UID: "bd3acf6a-a489-436e-87d7-97488bc297f6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 10:03:01.670069 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:01.670046    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd3acf6a-a489-436e-87d7-97488bc297f6-dshm" (OuterVolumeSpecName: "dshm") pod "bd3acf6a-a489-436e-87d7-97488bc297f6" (UID: "bd3acf6a-a489-436e-87d7-97488bc297f6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 10:03:01.670175 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:01.670104    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd3acf6a-a489-436e-87d7-97488bc297f6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bd3acf6a-a489-436e-87d7-97488bc297f6" (UID: "bd3acf6a-a489-436e-87d7-97488bc297f6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 10:03:01.670175 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:01.670104    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd3acf6a-a489-436e-87d7-97488bc297f6-kube-api-access-htmj6" (OuterVolumeSpecName: "kube-api-access-htmj6") pod "bd3acf6a-a489-436e-87d7-97488bc297f6" (UID: "bd3acf6a-a489-436e-87d7-97488bc297f6"). InnerVolumeSpecName "kube-api-access-htmj6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 10:03:01.769070 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:01.769025    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd3acf6a-a489-436e-87d7-97488bc297f6-model-cache\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:03:01.769070 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:01.769058    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd3acf6a-a489-436e-87d7-97488bc297f6-tls-certs\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:03:01.769070 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:01.769066    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd3acf6a-a489-436e-87d7-97488bc297f6-dshm\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:03:01.769070 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:01.769073    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd3acf6a-a489-436e-87d7-97488bc297f6-home\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:03:01.769070 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:01.769081    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd3acf6a-a489-436e-87d7-97488bc297f6-tmp-dir\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:03:01.769353 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:01.769089    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-htmj6\" (UniqueName: \"kubernetes.io/projected/bd3acf6a-a489-436e-87d7-97488bc297f6-kube-api-access-htmj6\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:03:01.863070 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:01.863025    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr"]
Jul 20 10:03:01.869755 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:01.869715    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-db33ff6f-kserve-6859fb89bc-t56jr"]
Jul 20 10:03:01.920731 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:01.920692    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fc84ba7-5834-4cc5-9d67-22a74cea2f5d" path="/var/lib/kubelet/pods/8fc84ba7-5834-4cc5-9d67-22a74cea2f5d/volumes"
Jul 20 10:03:01.921167 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:01.921151    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd3acf6a-a489-436e-87d7-97488bc297f6" path="/var/lib/kubelet/pods/bd3acf6a-a489-436e-87d7-97488bc297f6/volumes"
Jul 20 10:03:02.113103 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:02.113008    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:03:11.519576 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:11.519529    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:03:24.021264 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:24.021221    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:03:38.820422 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:38.820382    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:03:48.418353 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:03:48.418309    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:04:02.439751 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:04:02.439710    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:04:12.324337 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:04:12.324297    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:04:23.221786 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:04:23.221743    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:04:40.913654 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:04:40.913605    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:04:51.251854 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:04:51.251816    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:05:02.022174 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:05:02.022128    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:05:12.012288 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:05:12.012250    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:05:31.122483 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:05:31.122426    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:05:43.539355 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:05:43.539312    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:05:55.719335 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:05:55.719250    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:06:18.217043 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:06:18.217003    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:06:36.517770 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:06:36.517728    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d4aaf29e-9wt49"]
Jul 20 10:06:36.518224 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:06:36.518087    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fc84ba7-5834-4cc5-9d67-22a74cea2f5d" containerName="main"
Jul 20 10:06:36.518224 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:06:36.518099    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fc84ba7-5834-4cc5-9d67-22a74cea2f5d" containerName="main"
Jul 20 10:06:36.518224 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:06:36.518111    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd3acf6a-a489-436e-87d7-97488bc297f6" containerName="main"
Jul 20 10:06:36.518224 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:06:36.518117    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd3acf6a-a489-436e-87d7-97488bc297f6" containerName="main"
Jul 20 10:06:36.518224 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:06:36.518169    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd3acf6a-a489-436e-87d7-97488bc297f6" containerName="main"
Jul 20 10:06:36.518224 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:06:36.518179    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fc84ba7-5834-4cc5-9d67-22a74cea2f5d" containerName="main"
Jul 20 10:06:36.521224 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:06:36.521203    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d4aaf29e-9wt49"
Jul 20 10:06:36.524101 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:06:36.524069    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 10:06:36.524250 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:06:36.524083    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6jlzx\""
Jul 20 10:06:36.528327 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:06:36.528210    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d4aaf29e-9wt49"]
Jul 20 10:06:36.674346 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:06:36.674302    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/94d54f09-eec5-4d1c-9369-224f793a9593-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d4aaf29e-9wt49\" (UID: \"94d54f09-eec5-4d1c-9369-224f793a9593\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d4aaf29e-9wt49"
Jul 20 10:06:36.775242 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:06:36.775148    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/94d54f09-eec5-4d1c-9369-224f793a9593-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d4aaf29e-9wt49\" (UID: \"94d54f09-eec5-4d1c-9369-224f793a9593\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d4aaf29e-9wt49"
Jul 20 10:06:36.775759 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:06:36.775736    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/94d54f09-eec5-4d1c-9369-224f793a9593-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-d4aaf29e-9wt49\" (UID: \"94d54f09-eec5-4d1c-9369-224f793a9593\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d4aaf29e-9wt49"
Jul 20 10:06:36.834666 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:06:36.834636    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d4aaf29e-9wt49"
Jul 20 10:06:36.978163 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:06:36.978059    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d4aaf29e-9wt49"]
Jul 20 10:06:36.980552 ip-10-0-140-134 kubenswrapper[2566]: W0720 10:06:36.980519    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94d54f09_eec5_4d1c_9369_224f793a9593.slice/crio-3e6623378355984731bde476d169915cd415600e3adbed6627c0cbf7282677aa WatchSource:0}: Error finding container 3e6623378355984731bde476d169915cd415600e3adbed6627c0cbf7282677aa: Status 404 returned error can't find the container with id 3e6623378355984731bde476d169915cd415600e3adbed6627c0cbf7282677aa
Jul 20 10:06:36.982439 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:06:36.982410    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 10:06:37.557276 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:06:37.557240    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d4aaf29e-9wt49" event={"ID":"94d54f09-eec5-4d1c-9369-224f793a9593","Type":"ContainerStarted","Data":"3e6623378355984731bde476d169915cd415600e3adbed6627c0cbf7282677aa"}
Jul 20 10:06:41.578272 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:06:41.578233    2566 generic.go:356] "Generic (PLEG): container finished" podID="94d54f09-eec5-4d1c-9369-224f793a9593" containerID="1eae987c5d9a0b56098f2daa5a6c62101dc87e202dae7968b20f79e277ecdb5f" exitCode=0
Jul 20 10:06:41.578716 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:06:41.578324    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d4aaf29e-9wt49" event={"ID":"94d54f09-eec5-4d1c-9369-224f793a9593","Type":"ContainerDied","Data":"1eae987c5d9a0b56098f2daa5a6c62101dc87e202dae7968b20f79e277ecdb5f"}
Jul 20 10:06:42.722244 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:06:42.722214    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d4aaf29e-9wt49"
Jul 20 10:06:42.827421 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:06:42.827383    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/94d54f09-eec5-4d1c-9369-224f793a9593-maas-api-service-ca\") pod \"94d54f09-eec5-4d1c-9369-224f793a9593\" (UID: \"94d54f09-eec5-4d1c-9369-224f793a9593\") "
Jul 20 10:06:42.827771 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:06:42.827736    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94d54f09-eec5-4d1c-9369-224f793a9593-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "94d54f09-eec5-4d1c-9369-224f793a9593" (UID: "94d54f09-eec5-4d1c-9369-224f793a9593"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 10:06:42.928376 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:06:42.928279    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/94d54f09-eec5-4d1c-9369-224f793a9593-maas-api-service-ca\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:06:43.588263 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:06:43.588226    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d4aaf29e-9wt49" event={"ID":"94d54f09-eec5-4d1c-9369-224f793a9593","Type":"ContainerDied","Data":"3e6623378355984731bde476d169915cd415600e3adbed6627c0cbf7282677aa"}
Jul 20 10:06:43.588263 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:06:43.588260    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d4aaf29e-9wt49"
Jul 20 10:06:43.588468 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:06:43.588277    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3e6623378355984731bde476d169915cd415600e3adbed6627c0cbf7282677aa"
Jul 20 10:06:49.920358 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:06:49.920317    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:06:51.592895 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:06:51.592854    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d4aaf29e-9wt49"]
Jul 20 10:06:51.595885 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:06:51.595856    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-d4aaf29e-9wt49"]
Jul 20 10:06:51.920797 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:06:51.920708    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94d54f09-eec5-4d1c-9369-224f793a9593" path="/var/lib/kubelet/pods/94d54f09-eec5-4d1c-9369-224f793a9593/volumes"
Jul 20 10:07:01.413529 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:07:01.413483    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:07:20.332550 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:07:20.332510    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d95536fc-nmpct"]
Jul 20 10:07:20.332957 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:07:20.332857    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94d54f09-eec5-4d1c-9369-224f793a9593" containerName="revoke-keys"
Jul 20 10:07:20.332957 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:07:20.332869    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="94d54f09-eec5-4d1c-9369-224f793a9593" containerName="revoke-keys"
Jul 20 10:07:20.332957 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:07:20.332933    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="94d54f09-eec5-4d1c-9369-224f793a9593" containerName="revoke-keys"
Jul 20 10:07:20.338531 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:07:20.338503    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d95536fc-nmpct"
Jul 20 10:07:20.342056 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:07:20.341911    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6jlzx\""
Jul 20 10:07:20.342056 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:07:20.341994    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 10:07:20.342525 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:07:20.342494    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d95536fc-nmpct"]
Jul 20 10:07:20.455735 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:07:20.455695    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e22030f4-b5a5-4f66-8557-07b79de01eb0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-d95536fc-nmpct\" (UID: \"e22030f4-b5a5-4f66-8557-07b79de01eb0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d95536fc-nmpct"
Jul 20 10:07:20.557668 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:07:20.557609    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e22030f4-b5a5-4f66-8557-07b79de01eb0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-d95536fc-nmpct\" (UID: \"e22030f4-b5a5-4f66-8557-07b79de01eb0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d95536fc-nmpct"
Jul 20 10:07:20.558722 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:07:20.558688    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e22030f4-b5a5-4f66-8557-07b79de01eb0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-d95536fc-nmpct\" (UID: \"e22030f4-b5a5-4f66-8557-07b79de01eb0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d95536fc-nmpct"
Jul 20 10:07:20.651497 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:07:20.651395    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d95536fc-nmpct"
Jul 20 10:07:20.798514 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:07:20.798484    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d95536fc-nmpct"]
Jul 20 10:07:20.801294 ip-10-0-140-134 kubenswrapper[2566]: W0720 10:07:20.801263    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode22030f4_b5a5_4f66_8557_07b79de01eb0.slice/crio-37d6301759775d8394d6e29f6b1a92dce94900e4f7e18c9c77dc5ff966fa82b9 WatchSource:0}: Error finding container 37d6301759775d8394d6e29f6b1a92dce94900e4f7e18c9c77dc5ff966fa82b9: Status 404 returned error can't find the container with id 37d6301759775d8394d6e29f6b1a92dce94900e4f7e18c9c77dc5ff966fa82b9
Jul 20 10:07:21.765571 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:07:21.765466    2566 generic.go:356] "Generic (PLEG): container finished" podID="e22030f4-b5a5-4f66-8557-07b79de01eb0" containerID="5b6fc82c8ec6885e13fe43ee1648fa273f849726115a73f622e1468641218bc1" exitCode=0
Jul 20 10:07:21.765571 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:07:21.765551    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d95536fc-nmpct" event={"ID":"e22030f4-b5a5-4f66-8557-07b79de01eb0","Type":"ContainerDied","Data":"5b6fc82c8ec6885e13fe43ee1648fa273f849726115a73f622e1468641218bc1"}
Jul 20 10:07:21.766076 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:07:21.765606    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d95536fc-nmpct" event={"ID":"e22030f4-b5a5-4f66-8557-07b79de01eb0","Type":"ContainerStarted","Data":"37d6301759775d8394d6e29f6b1a92dce94900e4f7e18c9c77dc5ff966fa82b9"}
Jul 20 10:07:22.905001 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:07:22.904949    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d95536fc-nmpct"
Jul 20 10:07:22.983525 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:07:22.983482    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e22030f4-b5a5-4f66-8557-07b79de01eb0-maas-api-service-ca\") pod \"e22030f4-b5a5-4f66-8557-07b79de01eb0\" (UID: \"e22030f4-b5a5-4f66-8557-07b79de01eb0\") "
Jul 20 10:07:22.983916 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:07:22.983885    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e22030f4-b5a5-4f66-8557-07b79de01eb0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e22030f4-b5a5-4f66-8557-07b79de01eb0" (UID: "e22030f4-b5a5-4f66-8557-07b79de01eb0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 10:07:23.084462 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:07:23.084425    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e22030f4-b5a5-4f66-8557-07b79de01eb0-maas-api-service-ca\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:07:23.776256 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:07:23.776225    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d95536fc-nmpct"
Jul 20 10:07:23.776437 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:07:23.776227    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d95536fc-nmpct" event={"ID":"e22030f4-b5a5-4f66-8557-07b79de01eb0","Type":"ContainerDied","Data":"37d6301759775d8394d6e29f6b1a92dce94900e4f7e18c9c77dc5ff966fa82b9"}
Jul 20 10:07:23.776437 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:07:23.776358    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="37d6301759775d8394d6e29f6b1a92dce94900e4f7e18c9c77dc5ff966fa82b9"
Jul 20 10:07:33.906930 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:07:33.906885    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:07:35.403572 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:07:35.403532    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d95536fc-nmpct"]
Jul 20 10:07:35.405375 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:07:35.405347    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d95536fc-nmpct"]
Jul 20 10:07:35.920899 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:07:35.920861    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e22030f4-b5a5-4f66-8557-07b79de01eb0" path="/var/lib/kubelet/pods/e22030f4-b5a5-4f66-8557-07b79de01eb0/volumes"
Jul 20 10:07:45.213529 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:07:45.213489    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:08:04.269198 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:04.269158    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c5c080c1-z962f"]
Jul 20 10:08:04.269796 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:04.269657    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e22030f4-b5a5-4f66-8557-07b79de01eb0" containerName="revoke-keys"
Jul 20 10:08:04.269796 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:04.269677    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e22030f4-b5a5-4f66-8557-07b79de01eb0" containerName="revoke-keys"
Jul 20 10:08:04.269796 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:04.269778    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e22030f4-b5a5-4f66-8557-07b79de01eb0" containerName="revoke-keys"
Jul 20 10:08:04.272777 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:04.272746    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c5c080c1-z962f"
Jul 20 10:08:04.275621 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:04.275596    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 10:08:04.275772 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:04.275692    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6jlzx\""
Jul 20 10:08:04.279721 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:04.279687    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c5c080c1-z962f"]
Jul 20 10:08:04.342663 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:04.342619    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/598190a6-4763-4214-9db8-e1cd91f54ee7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c5c080c1-z962f\" (UID: \"598190a6-4763-4214-9db8-e1cd91f54ee7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c5c080c1-z962f"
Jul 20 10:08:04.443235 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:04.443177    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/598190a6-4763-4214-9db8-e1cd91f54ee7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c5c080c1-z962f\" (UID: \"598190a6-4763-4214-9db8-e1cd91f54ee7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c5c080c1-z962f"
Jul 20 10:08:04.443824 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:04.443804    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/598190a6-4763-4214-9db8-e1cd91f54ee7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c5c080c1-z962f\" (UID: \"598190a6-4763-4214-9db8-e1cd91f54ee7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c5c080c1-z962f"
Jul 20 10:08:04.586084 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:04.586043    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c5c080c1-z962f"
Jul 20 10:08:04.727464 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:04.727394    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c5c080c1-z962f"]
Jul 20 10:08:04.729803 ip-10-0-140-134 kubenswrapper[2566]: W0720 10:08:04.729768    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod598190a6_4763_4214_9db8_e1cd91f54ee7.slice/crio-db0a1d406cb7e324060cc82778fdbd1ccd49b4fdc5f0b0af5c9b8163984f98ed WatchSource:0}: Error finding container db0a1d406cb7e324060cc82778fdbd1ccd49b4fdc5f0b0af5c9b8163984f98ed: Status 404 returned error can't find the container with id db0a1d406cb7e324060cc82778fdbd1ccd49b4fdc5f0b0af5c9b8163984f98ed
Jul 20 10:08:04.978047 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:04.978013    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c5c080c1-z962f" event={"ID":"598190a6-4763-4214-9db8-e1cd91f54ee7","Type":"ContainerStarted","Data":"64cecf28eaee29394c73fa5f86c6f851eec2b9567c8bc01580b90aaa5c846ad2"}
Jul 20 10:08:04.978212 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:04.978070    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c5c080c1-z962f" event={"ID":"598190a6-4763-4214-9db8-e1cd91f54ee7","Type":"ContainerStarted","Data":"db0a1d406cb7e324060cc82778fdbd1ccd49b4fdc5f0b0af5c9b8163984f98ed"}
Jul 20 10:08:04.993184 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:04.993125    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c5c080c1-z962f" podStartSLOduration=0.993107699 podStartE2EDuration="993.107699ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 10:08:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 10:08:04.991722175 +0000 UTC m=+1751.543176641" watchObservedRunningTime="2026-07-20 10:08:04.993107699 +0000 UTC m=+1751.544562163"
Jul 20 10:08:05.984110 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:05.984073    2566 generic.go:356] "Generic (PLEG): container finished" podID="598190a6-4763-4214-9db8-e1cd91f54ee7" containerID="64cecf28eaee29394c73fa5f86c6f851eec2b9567c8bc01580b90aaa5c846ad2" exitCode=0
Jul 20 10:08:05.984493 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:05.984175    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c5c080c1-z962f" event={"ID":"598190a6-4763-4214-9db8-e1cd91f54ee7","Type":"ContainerDied","Data":"64cecf28eaee29394c73fa5f86c6f851eec2b9567c8bc01580b90aaa5c846ad2"}
Jul 20 10:08:07.126903 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:07.126871    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c5c080c1-z962f"
Jul 20 10:08:07.267533 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:07.267443    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/598190a6-4763-4214-9db8-e1cd91f54ee7-maas-api-service-ca\") pod \"598190a6-4763-4214-9db8-e1cd91f54ee7\" (UID: \"598190a6-4763-4214-9db8-e1cd91f54ee7\") "
Jul 20 10:08:07.267847 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:07.267822    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/598190a6-4763-4214-9db8-e1cd91f54ee7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "598190a6-4763-4214-9db8-e1cd91f54ee7" (UID: "598190a6-4763-4214-9db8-e1cd91f54ee7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 10:08:07.368820 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:07.368781    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/598190a6-4763-4214-9db8-e1cd91f54ee7-maas-api-service-ca\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:08:07.994701 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:07.994670    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c5c080c1-z962f"
Jul 20 10:08:07.994701 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:07.994683    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c5c080c1-z962f" event={"ID":"598190a6-4763-4214-9db8-e1cd91f54ee7","Type":"ContainerDied","Data":"db0a1d406cb7e324060cc82778fdbd1ccd49b4fdc5f0b0af5c9b8163984f98ed"}
Jul 20 10:08:07.994895 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:07.994726    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="db0a1d406cb7e324060cc82778fdbd1ccd49b4fdc5f0b0af5c9b8163984f98ed"
Jul 20 10:08:17.810519 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:17.810474    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:08:19.345334 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:19.345291    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c5c080c1-z962f"]
Jul 20 10:08:19.347102 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:19.347077    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c5c080c1-z962f"]
Jul 20 10:08:19.920646 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:19.920612    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="598190a6-4763-4214-9db8-e1cd91f54ee7" path="/var/lib/kubelet/pods/598190a6-4763-4214-9db8-e1cd91f54ee7/volumes"
Jul 20 10:08:29.501734 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:29.501692    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:08:48.588822 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:48.588733    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-26b8b528-xp884"]
Jul 20 10:08:48.589293 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:48.589098    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="598190a6-4763-4214-9db8-e1cd91f54ee7" containerName="revoke-keys"
Jul 20 10:08:48.589293 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:48.589109    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="598190a6-4763-4214-9db8-e1cd91f54ee7" containerName="revoke-keys"
Jul 20 10:08:48.589293 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:48.589180    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="598190a6-4763-4214-9db8-e1cd91f54ee7" containerName="revoke-keys"
Jul 20 10:08:48.592434 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:48.592412    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-26b8b528-xp884"
Jul 20 10:08:48.595196 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:48.595169    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6jlzx\""
Jul 20 10:08:48.595295 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:48.595182    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 10:08:48.598950 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:48.598914    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-26b8b528-xp884"]
Jul 20 10:08:48.619355 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:48.619307    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf39eea1-37af-49d0-8e5d-d0ba2de5eb8a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-26b8b528-xp884\" (UID: \"bf39eea1-37af-49d0-8e5d-d0ba2de5eb8a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-26b8b528-xp884"
Jul 20 10:08:48.719810 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:48.719768    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf39eea1-37af-49d0-8e5d-d0ba2de5eb8a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-26b8b528-xp884\" (UID: \"bf39eea1-37af-49d0-8e5d-d0ba2de5eb8a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-26b8b528-xp884"
Jul 20 10:08:48.720435 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:48.720414    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf39eea1-37af-49d0-8e5d-d0ba2de5eb8a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-26b8b528-xp884\" (UID: \"bf39eea1-37af-49d0-8e5d-d0ba2de5eb8a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-26b8b528-xp884"
Jul 20 10:08:48.905859 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:48.905758    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-26b8b528-xp884"
Jul 20 10:08:49.047819 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:49.047789    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-26b8b528-xp884"]
Jul 20 10:08:49.049638 ip-10-0-140-134 kubenswrapper[2566]: W0720 10:08:49.049604    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf39eea1_37af_49d0_8e5d_d0ba2de5eb8a.slice/crio-6a4dce5bffeb852bf98c426557eef7074e5172f264d1f5a52094df04a2906dc4 WatchSource:0}: Error finding container 6a4dce5bffeb852bf98c426557eef7074e5172f264d1f5a52094df04a2906dc4: Status 404 returned error can't find the container with id 6a4dce5bffeb852bf98c426557eef7074e5172f264d1f5a52094df04a2906dc4
Jul 20 10:08:49.191705 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:49.191614    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-26b8b528-xp884" event={"ID":"bf39eea1-37af-49d0-8e5d-d0ba2de5eb8a","Type":"ContainerStarted","Data":"6a4dce5bffeb852bf98c426557eef7074e5172f264d1f5a52094df04a2906dc4"}
Jul 20 10:08:50.198086 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:50.198043    2566 generic.go:356] "Generic (PLEG): container finished" podID="bf39eea1-37af-49d0-8e5d-d0ba2de5eb8a" containerID="d9bba246cc428550952b850543bbef6dc15686be4e87b87579c67563288b5928" exitCode=0
Jul 20 10:08:50.198478 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:50.198125    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-26b8b528-xp884" event={"ID":"bf39eea1-37af-49d0-8e5d-d0ba2de5eb8a","Type":"ContainerDied","Data":"d9bba246cc428550952b850543bbef6dc15686be4e87b87579c67563288b5928"}
Jul 20 10:08:51.342569 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:51.342541    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-26b8b528-xp884"
Jul 20 10:08:51.442528 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:51.442487    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf39eea1-37af-49d0-8e5d-d0ba2de5eb8a-maas-api-service-ca\") pod \"bf39eea1-37af-49d0-8e5d-d0ba2de5eb8a\" (UID: \"bf39eea1-37af-49d0-8e5d-d0ba2de5eb8a\") "
Jul 20 10:08:51.442897 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:51.442872    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf39eea1-37af-49d0-8e5d-d0ba2de5eb8a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bf39eea1-37af-49d0-8e5d-d0ba2de5eb8a" (UID: "bf39eea1-37af-49d0-8e5d-d0ba2de5eb8a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 10:08:51.543496 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:51.543406    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf39eea1-37af-49d0-8e5d-d0ba2de5eb8a-maas-api-service-ca\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:08:52.209606 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:52.209519    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-26b8b528-xp884"
Jul 20 10:08:52.209606 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:52.209526    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-26b8b528-xp884" event={"ID":"bf39eea1-37af-49d0-8e5d-d0ba2de5eb8a","Type":"ContainerDied","Data":"6a4dce5bffeb852bf98c426557eef7074e5172f264d1f5a52094df04a2906dc4"}
Jul 20 10:08:52.209783 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:08:52.209641    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6a4dce5bffeb852bf98c426557eef7074e5172f264d1f5a52094df04a2906dc4"
Jul 20 10:09:02.209111 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:09:02.209065    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:09:03.657594 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:09:03.657540    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-26b8b528-xp884"]
Jul 20 10:09:03.659836 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:09:03.659802    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-26b8b528-xp884"]
Jul 20 10:09:03.921486 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:09:03.921401    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf39eea1-37af-49d0-8e5d-d0ba2de5eb8a" path="/var/lib/kubelet/pods/bf39eea1-37af-49d0-8e5d-d0ba2de5eb8a/volumes"
Jul 20 10:10:30.307369 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:10:30.307278    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:10:36.809989 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:10:36.809928    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:10:45.611740 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:10:45.611696    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:10:55.790035 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:10:55.789984    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bc2fab5a-b7rdd"]
Jul 20 10:10:55.790541 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:10:55.790513    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf39eea1-37af-49d0-8e5d-d0ba2de5eb8a" containerName="revoke-keys"
Jul 20 10:10:55.790541 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:10:55.790534    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf39eea1-37af-49d0-8e5d-d0ba2de5eb8a" containerName="revoke-keys"
Jul 20 10:10:55.790664 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:10:55.790651    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf39eea1-37af-49d0-8e5d-d0ba2de5eb8a" containerName="revoke-keys"
Jul 20 10:10:55.793707 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:10:55.793679    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bc2fab5a-b7rdd"
Jul 20 10:10:55.796590 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:10:55.796562    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6jlzx\""
Jul 20 10:10:55.796743 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:10:55.796600    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 10:10:55.800490 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:10:55.800448    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bc2fab5a-b7rdd"]
Jul 20 10:10:55.914565 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:10:55.914531    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/652c56eb-6d5e-4196-ada7-bd15f8e93a41-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-bc2fab5a-b7rdd\" (UID: \"652c56eb-6d5e-4196-ada7-bd15f8e93a41\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bc2fab5a-b7rdd"
Jul 20 10:10:56.015537 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:10:56.015496    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/652c56eb-6d5e-4196-ada7-bd15f8e93a41-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-bc2fab5a-b7rdd\" (UID: \"652c56eb-6d5e-4196-ada7-bd15f8e93a41\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bc2fab5a-b7rdd"
Jul 20 10:10:56.016168 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:10:56.016149    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/652c56eb-6d5e-4196-ada7-bd15f8e93a41-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-bc2fab5a-b7rdd\" (UID: \"652c56eb-6d5e-4196-ada7-bd15f8e93a41\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bc2fab5a-b7rdd"
Jul 20 10:10:56.106603 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:10:56.106566    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bc2fab5a-b7rdd"
Jul 20 10:10:56.256366 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:10:56.256335    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bc2fab5a-b7rdd"]
Jul 20 10:10:56.258908 ip-10-0-140-134 kubenswrapper[2566]: W0720 10:10:56.258878    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod652c56eb_6d5e_4196_ada7_bd15f8e93a41.slice/crio-7244ad3904c2cbad312a179fa71f7f50c096b24a510658c8b4f2b7fd861789b9 WatchSource:0}: Error finding container 7244ad3904c2cbad312a179fa71f7f50c096b24a510658c8b4f2b7fd861789b9: Status 404 returned error can't find the container with id 7244ad3904c2cbad312a179fa71f7f50c096b24a510658c8b4f2b7fd861789b9
Jul 20 10:10:56.831788 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:10:56.831735    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bc2fab5a-b7rdd" event={"ID":"652c56eb-6d5e-4196-ada7-bd15f8e93a41","Type":"ContainerStarted","Data":"a7018ec729e8cb8a46e66467a19013366d49d5bc558b55f76259bf09be1bb137"}
Jul 20 10:10:56.832213 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:10:56.831842    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bc2fab5a-b7rdd" event={"ID":"652c56eb-6d5e-4196-ada7-bd15f8e93a41","Type":"ContainerStarted","Data":"7244ad3904c2cbad312a179fa71f7f50c096b24a510658c8b4f2b7fd861789b9"}
Jul 20 10:10:56.847746 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:10:56.847690    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bc2fab5a-b7rdd" podStartSLOduration=1.847675511 podStartE2EDuration="1.847675511s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 10:10:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 10:10:56.846674885 +0000 UTC m=+1923.398129353" watchObservedRunningTime="2026-07-20 10:10:56.847675511 +0000 UTC m=+1923.399129974"
Jul 20 10:10:57.838549 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:10:57.838513    2566 generic.go:356] "Generic (PLEG): container finished" podID="652c56eb-6d5e-4196-ada7-bd15f8e93a41" containerID="a7018ec729e8cb8a46e66467a19013366d49d5bc558b55f76259bf09be1bb137" exitCode=0
Jul 20 10:10:57.838947 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:10:57.838612    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bc2fab5a-b7rdd" event={"ID":"652c56eb-6d5e-4196-ada7-bd15f8e93a41","Type":"ContainerDied","Data":"a7018ec729e8cb8a46e66467a19013366d49d5bc558b55f76259bf09be1bb137"}
Jul 20 10:10:58.983497 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:10:58.983456    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bc2fab5a-b7rdd"
Jul 20 10:10:59.039057 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:10:59.039017    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/652c56eb-6d5e-4196-ada7-bd15f8e93a41-maas-api-service-ca\") pod \"652c56eb-6d5e-4196-ada7-bd15f8e93a41\" (UID: \"652c56eb-6d5e-4196-ada7-bd15f8e93a41\") "
Jul 20 10:10:59.039391 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:10:59.039369    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/652c56eb-6d5e-4196-ada7-bd15f8e93a41-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "652c56eb-6d5e-4196-ada7-bd15f8e93a41" (UID: "652c56eb-6d5e-4196-ada7-bd15f8e93a41"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 10:10:59.139991 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:10:59.139875    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/652c56eb-6d5e-4196-ada7-bd15f8e93a41-maas-api-service-ca\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:10:59.849312 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:10:59.849283    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bc2fab5a-b7rdd"
Jul 20 10:10:59.849312 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:10:59.849300    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bc2fab5a-b7rdd" event={"ID":"652c56eb-6d5e-4196-ada7-bd15f8e93a41","Type":"ContainerDied","Data":"7244ad3904c2cbad312a179fa71f7f50c096b24a510658c8b4f2b7fd861789b9"}
Jul 20 10:10:59.849530 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:10:59.849344    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7244ad3904c2cbad312a179fa71f7f50c096b24a510658c8b4f2b7fd861789b9"
Jul 20 10:11:07.515490 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:11:07.515439    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:11:10.909941 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:11:10.909888    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bc2fab5a-b7rdd"]
Jul 20 10:11:10.912601 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:11:10.912559    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bc2fab5a-b7rdd"]
Jul 20 10:11:11.921262 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:11:11.921221    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="652c56eb-6d5e-4196-ada7-bd15f8e93a41" path="/var/lib/kubelet/pods/652c56eb-6d5e-4196-ada7-bd15f8e93a41/volumes"
Jul 20 10:11:45.907379 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:11:45.907277    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:11:54.226713 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:11:54.226678    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-3a0aa251/e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm"]
Jul 20 10:11:54.227193 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:11:54.227053    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="652c56eb-6d5e-4196-ada7-bd15f8e93a41" containerName="revoke-keys"
Jul 20 10:11:54.227193 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:11:54.227065    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="652c56eb-6d5e-4196-ada7-bd15f8e93a41" containerName="revoke-keys"
Jul 20 10:11:54.227193 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:11:54.227129    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="652c56eb-6d5e-4196-ada7-bd15f8e93a41" containerName="revoke-keys"
Jul 20 10:11:54.230245 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:11:54.230223    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3a0aa251/e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm"
Jul 20 10:11:54.233067 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:11:54.233039    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-3a0aa251\"/\"default-dockercfg-sn52b\""
Jul 20 10:11:54.233223 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:11:54.233039    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-3a0aa251\"/\"kube-root-ca.crt\""
Jul 20 10:11:54.234188 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:11:54.234166    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-3a0aa251\"/\"openshift-service-ca.crt\""
Jul 20 10:11:54.234278 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:11:54.234241    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-3a0aa251\"/\"e2e-lifecycle-model-3a0aa251-kserve-self-signed-certs\""
Jul 20 10:11:54.237007 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:11:54.236959    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-3a0aa251/e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm"]
Jul 20 10:11:54.324875 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:11:54.324833    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466-tmp-dir\") pod \"e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm\" (UID: \"1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466\") " pod="ai-tenant-e2e-mt-3a0aa251/e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm"
Jul 20 10:11:54.324875 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:11:54.324880    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466-model-cache\") pod \"e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm\" (UID: \"1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466\") " pod="ai-tenant-e2e-mt-3a0aa251/e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm"
Jul 20 10:11:54.325134 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:11:54.324907    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466-dshm\") pod \"e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm\" (UID: \"1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466\") " pod="ai-tenant-e2e-mt-3a0aa251/e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm"
Jul 20 10:11:54.325134 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:11:54.324966    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jfgb\" (UniqueName: \"kubernetes.io/projected/1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466-kube-api-access-9jfgb\") pod \"e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm\" (UID: \"1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466\") " pod="ai-tenant-e2e-mt-3a0aa251/e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm"
Jul 20 10:11:54.325134 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:11:54.325048    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466-home\") pod \"e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm\" (UID: \"1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466\") " pod="ai-tenant-e2e-mt-3a0aa251/e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm"
Jul 20 10:11:54.325134 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:11:54.325070    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466-tls-certs\") pod \"e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm\" (UID: \"1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466\") " pod="ai-tenant-e2e-mt-3a0aa251/e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm"
Jul 20 10:11:54.425653 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:11:54.425613    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466-tmp-dir\") pod \"e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm\" (UID: \"1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466\") " pod="ai-tenant-e2e-mt-3a0aa251/e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm"
Jul 20 10:11:54.425653 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:11:54.425659    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466-model-cache\") pod \"e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm\" (UID: \"1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466\") " pod="ai-tenant-e2e-mt-3a0aa251/e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm"
Jul 20 10:11:54.425907 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:11:54.425806    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466-dshm\") pod \"e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm\" (UID: \"1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466\") " pod="ai-tenant-e2e-mt-3a0aa251/e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm"
Jul 20 10:11:54.425907 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:11:54.425894    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jfgb\" (UniqueName: \"kubernetes.io/projected/1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466-kube-api-access-9jfgb\") pod \"e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm\" (UID: \"1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466\") " pod="ai-tenant-e2e-mt-3a0aa251/e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm"
Jul 20 10:11:54.426049 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:11:54.425939    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466-home\") pod \"e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm\" (UID: \"1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466\") " pod="ai-tenant-e2e-mt-3a0aa251/e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm"
Jul 20 10:11:54.426049 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:11:54.426003    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466-tls-certs\") pod \"e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm\" (UID: \"1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466\") " pod="ai-tenant-e2e-mt-3a0aa251/e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm"
Jul 20 10:11:54.426148 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:11:54.426130    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466-model-cache\") pod \"e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm\" (UID: \"1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466\") " pod="ai-tenant-e2e-mt-3a0aa251/e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm"
Jul 20 10:11:54.426203 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:11:54.426131    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466-tmp-dir\") pod \"e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm\" (UID: \"1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466\") " pod="ai-tenant-e2e-mt-3a0aa251/e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm"
Jul 20 10:11:54.426310 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:11:54.426274    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466-home\") pod \"e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm\" (UID: \"1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466\") " pod="ai-tenant-e2e-mt-3a0aa251/e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm"
Jul 20 10:11:54.428370 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:11:54.428344    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466-dshm\") pod \"e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm\" (UID: \"1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466\") " pod="ai-tenant-e2e-mt-3a0aa251/e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm"
Jul 20 10:11:54.428792 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:11:54.428769    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466-tls-certs\") pod \"e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm\" (UID: \"1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466\") " pod="ai-tenant-e2e-mt-3a0aa251/e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm"
Jul 20 10:11:54.435686 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:11:54.435647    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jfgb\" (UniqueName: \"kubernetes.io/projected/1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466-kube-api-access-9jfgb\") pod \"e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm\" (UID: \"1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466\") " pod="ai-tenant-e2e-mt-3a0aa251/e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm"
Jul 20 10:11:54.541667 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:11:54.541557    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3a0aa251/e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm"
Jul 20 10:11:54.689740 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:11:54.689692    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-3a0aa251/e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm"]
Jul 20 10:11:54.692202 ip-10-0-140-134 kubenswrapper[2566]: W0720 10:11:54.692162    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1bdcf95b_d0f0_4e2b_9ee4_c30ecb0a2466.slice/crio-b6b63b13b2d0fccc5d672350ee79cd096df9fab728e2040d96eba960982f1cc3 WatchSource:0}: Error finding container b6b63b13b2d0fccc5d672350ee79cd096df9fab728e2040d96eba960982f1cc3: Status 404 returned error can't find the container with id b6b63b13b2d0fccc5d672350ee79cd096df9fab728e2040d96eba960982f1cc3
Jul 20 10:11:55.128619 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:11:55.128579    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3a0aa251/e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm" event={"ID":"1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466","Type":"ContainerStarted","Data":"11c98bd3b0a8fe08da84409e7b2b3cf55543e47d50b36febc179a0593c14b94b"}
Jul 20 10:11:55.128619 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:11:55.128624    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3a0aa251/e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm" event={"ID":"1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466","Type":"ContainerStarted","Data":"b6b63b13b2d0fccc5d672350ee79cd096df9fab728e2040d96eba960982f1cc3"}
Jul 20 10:11:55.145928 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:11:55.145873    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-3a0aa251/e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm" podStartSLOduration=1.14585567 podStartE2EDuration="1.14585567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 10:11:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 10:11:55.145165106 +0000 UTC m=+1981.696619571" watchObservedRunningTime="2026-07-20 10:11:55.14585567 +0000 UTC m=+1981.697310140"
Jul 20 10:11:59.309025 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:11:59.308957    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:12:04.542097 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:04.542052    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-3a0aa251/e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm"
Jul 20 10:12:04.542097 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:04.542111    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-3a0aa251/e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm"
Jul 20 10:12:04.555599 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:04.555562    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-3a0aa251/e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm"
Jul 20 10:12:05.190308 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:05.190276    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-3a0aa251/e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm"
Jul 20 10:12:08.604367 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:08.604324    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:12:12.230486 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:12.230443    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a0aa251-75btw"]
Jul 20 10:12:12.233588 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:12.233563    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a0aa251-75btw"
Jul 20 10:12:12.236234 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:12.236199    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 10:12:12.236412 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:12.236242    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6jlzx\""
Jul 20 10:12:12.242016 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:12.241988    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a0aa251-75btw"]
Jul 20 10:12:12.390524 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:12.390475    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a9dabca-6839-4d6a-9bac-d75598939ec0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3a0aa251-75btw\" (UID: \"6a9dabca-6839-4d6a-9bac-d75598939ec0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a0aa251-75btw"
Jul 20 10:12:12.491676 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:12.491575    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a9dabca-6839-4d6a-9bac-d75598939ec0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3a0aa251-75btw\" (UID: \"6a9dabca-6839-4d6a-9bac-d75598939ec0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a0aa251-75btw"
Jul 20 10:12:12.492308 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:12.492285    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a9dabca-6839-4d6a-9bac-d75598939ec0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-3a0aa251-75btw\" (UID: \"6a9dabca-6839-4d6a-9bac-d75598939ec0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a0aa251-75btw"
Jul 20 10:12:12.546697 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:12.546653    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a0aa251-75btw"
Jul 20 10:12:12.696135 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:12.696106    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a0aa251-75btw"]
Jul 20 10:12:12.698296 ip-10-0-140-134 kubenswrapper[2566]: W0720 10:12:12.698255    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a9dabca_6839_4d6a_9bac_d75598939ec0.slice/crio-08e7dd983f0f36b37b2fe8fbfdec1a24dd38e278efa6da755281a62d0c0e1e08 WatchSource:0}: Error finding container 08e7dd983f0f36b37b2fe8fbfdec1a24dd38e278efa6da755281a62d0c0e1e08: Status 404 returned error can't find the container with id 08e7dd983f0f36b37b2fe8fbfdec1a24dd38e278efa6da755281a62d0c0e1e08
Jul 20 10:12:13.221307 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:13.221205    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a0aa251-75btw" event={"ID":"6a9dabca-6839-4d6a-9bac-d75598939ec0","Type":"ContainerStarted","Data":"fc95816609697cdb13438e6ce0df249ec19e52c89f43a20d22bcbdbceb245296"}
Jul 20 10:12:13.221491 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:13.221319    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a0aa251-75btw" event={"ID":"6a9dabca-6839-4d6a-9bac-d75598939ec0","Type":"ContainerStarted","Data":"08e7dd983f0f36b37b2fe8fbfdec1a24dd38e278efa6da755281a62d0c0e1e08"}
Jul 20 10:12:13.236890 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:13.236830    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a0aa251-75btw" podStartSLOduration=1.236813066 podStartE2EDuration="1.236813066s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 10:12:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 10:12:13.235356211 +0000 UTC m=+1999.786810674" watchObservedRunningTime="2026-07-20 10:12:13.236813066 +0000 UTC m=+1999.788267530"
Jul 20 10:12:14.227528 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:14.227490    2566 generic.go:356] "Generic (PLEG): container finished" podID="6a9dabca-6839-4d6a-9bac-d75598939ec0" containerID="fc95816609697cdb13438e6ce0df249ec19e52c89f43a20d22bcbdbceb245296" exitCode=0
Jul 20 10:12:14.227752 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:14.227586    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a0aa251-75btw" event={"ID":"6a9dabca-6839-4d6a-9bac-d75598939ec0","Type":"ContainerDied","Data":"fc95816609697cdb13438e6ce0df249ec19e52c89f43a20d22bcbdbceb245296"}
Jul 20 10:12:15.375589 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:15.375557    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a0aa251-75btw"
Jul 20 10:12:15.518350 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:15.518266    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a9dabca-6839-4d6a-9bac-d75598939ec0-maas-api-service-ca\") pod \"6a9dabca-6839-4d6a-9bac-d75598939ec0\" (UID: \"6a9dabca-6839-4d6a-9bac-d75598939ec0\") "
Jul 20 10:12:15.518615 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:15.518590    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a9dabca-6839-4d6a-9bac-d75598939ec0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6a9dabca-6839-4d6a-9bac-d75598939ec0" (UID: "6a9dabca-6839-4d6a-9bac-d75598939ec0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 10:12:15.619507 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:15.619445    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6a9dabca-6839-4d6a-9bac-d75598939ec0-maas-api-service-ca\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:12:16.239454 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:16.239361    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a0aa251-75btw" event={"ID":"6a9dabca-6839-4d6a-9bac-d75598939ec0","Type":"ContainerDied","Data":"08e7dd983f0f36b37b2fe8fbfdec1a24dd38e278efa6da755281a62d0c0e1e08"}
Jul 20 10:12:16.239454 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:16.239400    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a0aa251-75btw"
Jul 20 10:12:16.239454 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:16.239420    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="08e7dd983f0f36b37b2fe8fbfdec1a24dd38e278efa6da755281a62d0c0e1e08"
Jul 20 10:12:27.619616 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:27.619564    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:12:32.269267 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:32.269220    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a0aa251-75btw"]
Jul 20 10:12:32.271226 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:32.271200    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-3a0aa251-75btw"]
Jul 20 10:12:33.921098 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:33.921054    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a9dabca-6839-4d6a-9bac-d75598939ec0" path="/var/lib/kubelet/pods/6a9dabca-6839-4d6a-9bac-d75598939ec0/volumes"
Jul 20 10:12:36.617125 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:36.617090    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:12:39.447083 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:39.447033    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-3a0aa251/e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm"]
Jul 20 10:12:54.066624 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:54.066589    2566 scope.go:117] "RemoveContainer" containerID="1eae987c5d9a0b56098f2daa5a6c62101dc87e202dae7968b20f79e277ecdb5f"
Jul 20 10:12:54.469842 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:54.469741    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-3a0aa251/e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm" podUID="1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466" containerName="main" containerID="cri-o://11c98bd3b0a8fe08da84409e7b2b3cf55543e47d50b36febc179a0593c14b94b" gracePeriod=45
Jul 20 10:12:54.741301 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:54.741270    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3a0aa251/e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm"
Jul 20 10:12:54.855138 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:54.855093    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9jfgb\" (UniqueName: \"kubernetes.io/projected/1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466-kube-api-access-9jfgb\") pod \"1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466\" (UID: \"1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466\") "
Jul 20 10:12:54.855321 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:54.855152    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466-tls-certs\") pod \"1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466\" (UID: \"1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466\") "
Jul 20 10:12:54.855321 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:54.855191    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466-dshm\") pod \"1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466\" (UID: \"1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466\") "
Jul 20 10:12:54.855321 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:54.855241    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466-tmp-dir\") pod \"1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466\" (UID: \"1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466\") "
Jul 20 10:12:54.855321 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:54.855258    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466-model-cache\") pod \"1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466\" (UID: \"1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466\") "
Jul 20 10:12:54.855321 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:54.855273    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466-home\") pod \"1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466\" (UID: \"1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466\") "
Jul 20 10:12:54.855598 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:54.855543    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466" (UID: "1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 10:12:54.855598 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:54.855560    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466-model-cache" (OuterVolumeSpecName: "model-cache") pod "1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466" (UID: "1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 10:12:54.855685 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:54.855586    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466-home" (OuterVolumeSpecName: "home") pod "1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466" (UID: "1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 10:12:54.857790 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:54.857761    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466-dshm" (OuterVolumeSpecName: "dshm") pod "1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466" (UID: "1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 10:12:54.857905 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:54.857760    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466-kube-api-access-9jfgb" (OuterVolumeSpecName: "kube-api-access-9jfgb") pod "1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466" (UID: "1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466"). InnerVolumeSpecName "kube-api-access-9jfgb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 10:12:54.857905 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:54.857816    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466" (UID: "1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 10:12:54.956453 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:54.956413    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466-dshm\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:12:54.956453 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:54.956448    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466-tmp-dir\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:12:54.956453 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:54.956459    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466-model-cache\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:12:54.956693 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:54.956466    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466-home\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:12:54.956693 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:54.956475    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jfgb\" (UniqueName: \"kubernetes.io/projected/1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466-kube-api-access-9jfgb\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:12:54.956693 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:54.956484    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466-tls-certs\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:12:55.433909 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:55.433868    2566 generic.go:356] "Generic (PLEG): container finished" podID="1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466" containerID="11c98bd3b0a8fe08da84409e7b2b3cf55543e47d50b36febc179a0593c14b94b" exitCode=0
Jul 20 10:12:55.434348 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:55.433920    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3a0aa251/e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm"
Jul 20 10:12:55.434348 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:55.433953    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3a0aa251/e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm" event={"ID":"1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466","Type":"ContainerDied","Data":"11c98bd3b0a8fe08da84409e7b2b3cf55543e47d50b36febc179a0593c14b94b"}
Jul 20 10:12:55.434348 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:55.434014    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3a0aa251/e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm" event={"ID":"1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466","Type":"ContainerDied","Data":"b6b63b13b2d0fccc5d672350ee79cd096df9fab728e2040d96eba960982f1cc3"}
Jul 20 10:12:55.434348 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:55.434031    2566 scope.go:117] "RemoveContainer" containerID="11c98bd3b0a8fe08da84409e7b2b3cf55543e47d50b36febc179a0593c14b94b"
Jul 20 10:12:55.444161 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:55.444134    2566 scope.go:117] "RemoveContainer" containerID="11c98bd3b0a8fe08da84409e7b2b3cf55543e47d50b36febc179a0593c14b94b"
Jul 20 10:12:55.444588 ip-10-0-140-134 kubenswrapper[2566]: E0720 10:12:55.444565    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11c98bd3b0a8fe08da84409e7b2b3cf55543e47d50b36febc179a0593c14b94b\": container with ID starting with 11c98bd3b0a8fe08da84409e7b2b3cf55543e47d50b36febc179a0593c14b94b not found: ID does not exist" containerID="11c98bd3b0a8fe08da84409e7b2b3cf55543e47d50b36febc179a0593c14b94b"
Jul 20 10:12:55.444638 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:55.444598    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11c98bd3b0a8fe08da84409e7b2b3cf55543e47d50b36febc179a0593c14b94b"} err="failed to get container status \"11c98bd3b0a8fe08da84409e7b2b3cf55543e47d50b36febc179a0593c14b94b\": rpc error: code = NotFound desc = could not find container \"11c98bd3b0a8fe08da84409e7b2b3cf55543e47d50b36febc179a0593c14b94b\": container with ID starting with 11c98bd3b0a8fe08da84409e7b2b3cf55543e47d50b36febc179a0593c14b94b not found: ID does not exist"
Jul 20 10:12:55.457118 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:55.457079    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-3a0aa251/e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm"]
Jul 20 10:12:55.463011 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:55.462957    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-3a0aa251/e2e-lifecycle-model-3a0aa251-kserve-7599fd569f-9hqgm"]
Jul 20 10:12:55.921089 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:12:55.921051    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466" path="/var/lib/kubelet/pods/1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466/volumes"
Jul 20 10:13:26.510223 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:13:26.510132    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:13:39.011755 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:13:39.011706    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:13:54.090106 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:13:54.090067    2566 scope.go:117] "RemoveContainer" containerID="5b6fc82c8ec6885e13fe43ee1648fa273f849726115a73f622e1468641218bc1"
Jul 20 10:14:25.014764 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:25.014717    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:14:35.411177 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:35.411123    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:14:45.905258 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:45.905141    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-11ee60/test-model-11ee60-kserve-9495b6df7-wlcr4"]
Jul 20 10:14:45.905716 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:45.905693    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466" containerName="main"
Jul 20 10:14:45.905766 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:45.905722    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466" containerName="main"
Jul 20 10:14:45.905766 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:45.905749    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a9dabca-6839-4d6a-9bac-d75598939ec0" containerName="revoke-keys"
Jul 20 10:14:45.905766 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:45.905758    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a9dabca-6839-4d6a-9bac-d75598939ec0" containerName="revoke-keys"
Jul 20 10:14:45.905877 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:45.905863    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a9dabca-6839-4d6a-9bac-d75598939ec0" containerName="revoke-keys"
Jul 20 10:14:45.905921 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:45.905880    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1bdcf95b-d0f0-4e2b-9ee4-c30ecb0a2466" containerName="main"
Jul 20 10:14:45.909029 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:45.908990    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-11ee60/test-model-11ee60-kserve-9495b6df7-wlcr4"
Jul 20 10:14:45.912090 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:45.912053    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-11ee60\"/\"openshift-service-ca.crt\""
Jul 20 10:14:45.912090 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:45.912083    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-11ee60\"/\"test-model-11ee60-kserve-self-signed-certs\""
Jul 20 10:14:45.913300 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:45.913275    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-11ee60\"/\"kube-root-ca.crt\""
Jul 20 10:14:45.913300 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:45.913277    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-11ee60\"/\"default-dockercfg-wgt98\""
Jul 20 10:14:45.931758 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:45.931717    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-11ee60/test-model-11ee60-kserve-9495b6df7-wlcr4"]
Jul 20 10:14:45.995800 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:45.995749    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd-home\") pod \"test-model-11ee60-kserve-9495b6df7-wlcr4\" (UID: \"3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd\") " pod="ai-tenant-e2e-inf-a-11ee60/test-model-11ee60-kserve-9495b6df7-wlcr4"
Jul 20 10:14:45.995800 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:45.995796    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd-tmp-dir\") pod \"test-model-11ee60-kserve-9495b6df7-wlcr4\" (UID: \"3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd\") " pod="ai-tenant-e2e-inf-a-11ee60/test-model-11ee60-kserve-9495b6df7-wlcr4"
Jul 20 10:14:45.996068 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:45.995862    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfnc9\" (UniqueName: \"kubernetes.io/projected/3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd-kube-api-access-tfnc9\") pod \"test-model-11ee60-kserve-9495b6df7-wlcr4\" (UID: \"3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd\") " pod="ai-tenant-e2e-inf-a-11ee60/test-model-11ee60-kserve-9495b6df7-wlcr4"
Jul 20 10:14:45.996068 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:45.995913    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd-tls-certs\") pod \"test-model-11ee60-kserve-9495b6df7-wlcr4\" (UID: \"3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd\") " pod="ai-tenant-e2e-inf-a-11ee60/test-model-11ee60-kserve-9495b6df7-wlcr4"
Jul 20 10:14:45.996068 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:45.995933    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd-dshm\") pod \"test-model-11ee60-kserve-9495b6df7-wlcr4\" (UID: \"3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd\") " pod="ai-tenant-e2e-inf-a-11ee60/test-model-11ee60-kserve-9495b6df7-wlcr4"
Jul 20 10:14:45.996068 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:45.995950    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd-model-cache\") pod \"test-model-11ee60-kserve-9495b6df7-wlcr4\" (UID: \"3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd\") " pod="ai-tenant-e2e-inf-a-11ee60/test-model-11ee60-kserve-9495b6df7-wlcr4"
Jul 20 10:14:46.097455 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:46.097399    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd-tmp-dir\") pod \"test-model-11ee60-kserve-9495b6df7-wlcr4\" (UID: \"3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd\") " pod="ai-tenant-e2e-inf-a-11ee60/test-model-11ee60-kserve-9495b6df7-wlcr4"
Jul 20 10:14:46.097455 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:46.097456    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfnc9\" (UniqueName: \"kubernetes.io/projected/3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd-kube-api-access-tfnc9\") pod \"test-model-11ee60-kserve-9495b6df7-wlcr4\" (UID: \"3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd\") " pod="ai-tenant-e2e-inf-a-11ee60/test-model-11ee60-kserve-9495b6df7-wlcr4"
Jul 20 10:14:46.097717 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:46.097588    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd-tls-certs\") pod \"test-model-11ee60-kserve-9495b6df7-wlcr4\" (UID: \"3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd\") " pod="ai-tenant-e2e-inf-a-11ee60/test-model-11ee60-kserve-9495b6df7-wlcr4"
Jul 20 10:14:46.097717 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:46.097629    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd-dshm\") pod \"test-model-11ee60-kserve-9495b6df7-wlcr4\" (UID: \"3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd\") " pod="ai-tenant-e2e-inf-a-11ee60/test-model-11ee60-kserve-9495b6df7-wlcr4"
Jul 20 10:14:46.097717 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:46.097655    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd-model-cache\") pod \"test-model-11ee60-kserve-9495b6df7-wlcr4\" (UID: \"3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd\") " pod="ai-tenant-e2e-inf-a-11ee60/test-model-11ee60-kserve-9495b6df7-wlcr4"
Jul 20 10:14:46.097873 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:46.097769    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd-home\") pod \"test-model-11ee60-kserve-9495b6df7-wlcr4\" (UID: \"3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd\") " pod="ai-tenant-e2e-inf-a-11ee60/test-model-11ee60-kserve-9495b6df7-wlcr4"
Jul 20 10:14:46.097873 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:46.097858    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd-tmp-dir\") pod \"test-model-11ee60-kserve-9495b6df7-wlcr4\" (UID: \"3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd\") " pod="ai-tenant-e2e-inf-a-11ee60/test-model-11ee60-kserve-9495b6df7-wlcr4"
Jul 20 10:14:46.098096 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:46.098074    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd-model-cache\") pod \"test-model-11ee60-kserve-9495b6df7-wlcr4\" (UID: \"3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd\") " pod="ai-tenant-e2e-inf-a-11ee60/test-model-11ee60-kserve-9495b6df7-wlcr4"
Jul 20 10:14:46.098161 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:46.098139    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd-home\") pod \"test-model-11ee60-kserve-9495b6df7-wlcr4\" (UID: \"3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd\") " pod="ai-tenant-e2e-inf-a-11ee60/test-model-11ee60-kserve-9495b6df7-wlcr4"
Jul 20 10:14:46.100364 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:46.100329    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd-dshm\") pod \"test-model-11ee60-kserve-9495b6df7-wlcr4\" (UID: \"3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd\") " pod="ai-tenant-e2e-inf-a-11ee60/test-model-11ee60-kserve-9495b6df7-wlcr4"
Jul 20 10:14:46.100590 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:46.100566    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd-tls-certs\") pod \"test-model-11ee60-kserve-9495b6df7-wlcr4\" (UID: \"3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd\") " pod="ai-tenant-e2e-inf-a-11ee60/test-model-11ee60-kserve-9495b6df7-wlcr4"
Jul 20 10:14:46.106316 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:46.106280    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfnc9\" (UniqueName: \"kubernetes.io/projected/3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd-kube-api-access-tfnc9\") pod \"test-model-11ee60-kserve-9495b6df7-wlcr4\" (UID: \"3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd\") " pod="ai-tenant-e2e-inf-a-11ee60/test-model-11ee60-kserve-9495b6df7-wlcr4"
Jul 20 10:14:46.223435 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:46.223325    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-11ee60/test-model-11ee60-kserve-9495b6df7-wlcr4"
Jul 20 10:14:46.379487 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:46.379385    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-11ee60/test-model-11ee60-kserve-9495b6df7-wlcr4"]
Jul 20 10:14:46.382239 ip-10-0-140-134 kubenswrapper[2566]: W0720 10:14:46.382203    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3c05d872_8b01_4e0d_a5a5_dd2b9c95f5dd.slice/crio-ca27051b4e5cff55f523ffc40bb5023d09eb137a037785eb5cadd815d07dd9eb WatchSource:0}: Error finding container ca27051b4e5cff55f523ffc40bb5023d09eb137a037785eb5cadd815d07dd9eb: Status 404 returned error can't find the container with id ca27051b4e5cff55f523ffc40bb5023d09eb137a037785eb5cadd815d07dd9eb
Jul 20 10:14:46.999389 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:46.999342    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-11ee60/test-model-11ee60-kserve-9495b6df7-wlcr4" event={"ID":"3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd","Type":"ContainerStarted","Data":"18f300bc29597dd0f51268c411b0f7ce99ec07afe23046ba2c72d6a8ac11154b"}
Jul 20 10:14:46.999784 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:46.999401    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-11ee60/test-model-11ee60-kserve-9495b6df7-wlcr4" event={"ID":"3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd","Type":"ContainerStarted","Data":"ca27051b4e5cff55f523ffc40bb5023d09eb137a037785eb5cadd815d07dd9eb"}
Jul 20 10:14:47.018154 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:47.018089    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-11ee60/test-model-11ee60-kserve-9495b6df7-wlcr4" podStartSLOduration=2.018069141 podStartE2EDuration="2.018069141s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 10:14:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 10:14:47.01619535 +0000 UTC m=+2153.567649813" watchObservedRunningTime="2026-07-20 10:14:47.018069141 +0000 UTC m=+2153.569523604"
Jul 20 10:14:49.705210 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:49.705167    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:14:54.115253 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:54.115210    2566 scope.go:117] "RemoveContainer" containerID="d9bba246cc428550952b850543bbef6dc15686be4e87b87579c67563288b5928"
Jul 20 10:14:54.124572 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:54.124546    2566 scope.go:117] "RemoveContainer" containerID="64cecf28eaee29394c73fa5f86c6f851eec2b9567c8bc01580b90aaa5c846ad2"
Jul 20 10:14:56.224377 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:56.224332    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-11ee60/test-model-11ee60-kserve-9495b6df7-wlcr4"
Jul 20 10:14:56.224967 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:56.224398    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-11ee60/test-model-11ee60-kserve-9495b6df7-wlcr4"
Jul 20 10:14:56.238533 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:56.238493    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-11ee60/test-model-11ee60-kserve-9495b6df7-wlcr4"
Jul 20 10:14:57.066379 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:14:57.066346    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-11ee60/test-model-11ee60-kserve-9495b6df7-wlcr4"
Jul 20 10:15:00.130860 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:00.130812    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742375-hfc7j"]
Jul 20 10:15:00.133295 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:00.133270    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a2a58-29742375-8fpz5"]
Jul 20 10:15:00.133411 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:00.133386    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742375-hfc7j"
Jul 20 10:15:00.135589 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:00.135565    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a2a58-29742375-8fpz5"
Jul 20 10:15:00.136089 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:00.136071    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6jlzx\""
Jul 20 10:15:00.144334 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:00.144296    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742375-hfc7j"]
Jul 20 10:15:00.146903 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:00.146871    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742375-hfc7j"
Jul 20 10:15:00.159720 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:00.157360    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a2a58-29742375-8fpz5"]
Jul 20 10:15:00.159720 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:00.158573    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a2a58-29742375-8fpz5"
Jul 20 10:15:00.317237 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:00.316677    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29742375-hfc7j"]
Jul 20 10:15:00.320011 ip-10-0-140-134 kubenswrapper[2566]: W0720 10:15:00.319962    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99c5c5e2_2ebe_408c_9ece_3ac5d5bb04a7.slice/crio-89b9e6e5f8f58b8fab9172648341eb1c7eb318058d30e86f25f7d224877c6135 WatchSource:0}: Error finding container 89b9e6e5f8f58b8fab9172648341eb1c7eb318058d30e86f25f7d224877c6135: Status 404 returned error can't find the container with id 89b9e6e5f8f58b8fab9172648341eb1c7eb318058d30e86f25f7d224877c6135
Jul 20 10:15:00.338879 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:00.338843    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a2a58-29742375-8fpz5"]
Jul 20 10:15:00.343718 ip-10-0-140-134 kubenswrapper[2566]: W0720 10:15:00.343602    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27cc30f2_fcee_4b69_9653_a2fa0908d81d.slice/crio-2b235a77d6afc67e871fe074cf20804b366737bbb9c4aef8a1f095b881c293ce WatchSource:0}: Error finding container 2b235a77d6afc67e871fe074cf20804b366737bbb9c4aef8a1f095b881c293ce: Status 404 returned error can't find the container with id 2b235a77d6afc67e871fe074cf20804b366737bbb9c4aef8a1f095b881c293ce
Jul 20 10:15:01.076320 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:01.076283    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a2a58-29742375-8fpz5" event={"ID":"27cc30f2-fcee-4b69-9653-a2fa0908d81d","Type":"ContainerStarted","Data":"3e32e633d889e25a4a589dcbcd0c8f8f7d41884aef6624f9e8b29dbb05bbc776"}
Jul 20 10:15:01.076539 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:01.076347    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a2a58-29742375-8fpz5" event={"ID":"27cc30f2-fcee-4b69-9653-a2fa0908d81d","Type":"ContainerStarted","Data":"2b235a77d6afc67e871fe074cf20804b366737bbb9c4aef8a1f095b881c293ce"}
Jul 20 10:15:01.078067 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:01.078034    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742375-hfc7j" event={"ID":"99c5c5e2-2ebe-408c-9ece-3ac5d5bb04a7","Type":"ContainerStarted","Data":"261226c6f2f8d48749e433eae7c59bb5cd369cf4a33561c0cc6d505a5b557b36"}
Jul 20 10:15:01.078191 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:01.078089    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742375-hfc7j" event={"ID":"99c5c5e2-2ebe-408c-9ece-3ac5d5bb04a7","Type":"ContainerStarted","Data":"89b9e6e5f8f58b8fab9172648341eb1c7eb318058d30e86f25f7d224877c6135"}
Jul 20 10:15:01.092888 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:01.092826    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a2a58-29742375-8fpz5" podStartSLOduration=1.092805958 podStartE2EDuration="1.092805958s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 10:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 10:15:01.090729176 +0000 UTC m=+2167.642183640" watchObservedRunningTime="2026-07-20 10:15:01.092805958 +0000 UTC m=+2167.644260460"
Jul 20 10:15:01.106074 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:01.106012    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742375-hfc7j" podStartSLOduration=1.105992261 podStartE2EDuration="1.105992261s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 10:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 10:15:01.104587056 +0000 UTC m=+2167.656041517" watchObservedRunningTime="2026-07-20 10:15:01.105992261 +0000 UTC m=+2167.657446724"
Jul 20 10:15:01.222951 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:01.222892    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:15:02.085800 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:02.085759    2566 generic.go:356] "Generic (PLEG): container finished" podID="27cc30f2-fcee-4b69-9653-a2fa0908d81d" containerID="3e32e633d889e25a4a589dcbcd0c8f8f7d41884aef6624f9e8b29dbb05bbc776" exitCode=0
Jul 20 10:15:02.086034 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:02.085856    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a2a58-29742375-8fpz5" event={"ID":"27cc30f2-fcee-4b69-9653-a2fa0908d81d","Type":"ContainerDied","Data":"3e32e633d889e25a4a589dcbcd0c8f8f7d41884aef6624f9e8b29dbb05bbc776"}
Jul 20 10:15:02.087597 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:02.087564    2566 generic.go:356] "Generic (PLEG): container finished" podID="99c5c5e2-2ebe-408c-9ece-3ac5d5bb04a7" containerID="261226c6f2f8d48749e433eae7c59bb5cd369cf4a33561c0cc6d505a5b557b36" exitCode=0
Jul 20 10:15:02.087750 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:02.087606    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742375-hfc7j" event={"ID":"99c5c5e2-2ebe-408c-9ece-3ac5d5bb04a7","Type":"ContainerDied","Data":"261226c6f2f8d48749e433eae7c59bb5cd369cf4a33561c0cc6d505a5b557b36"}
Jul 20 10:15:03.254088 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:03.254054    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a2a58-29742375-8fpz5"
Jul 20 10:15:03.276467 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:03.276430    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742375-hfc7j"
Jul 20 10:15:04.099527 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:04.099488    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742375-hfc7j"
Jul 20 10:15:04.099527 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:04.099519    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29742375-hfc7j" event={"ID":"99c5c5e2-2ebe-408c-9ece-3ac5d5bb04a7","Type":"ContainerDied","Data":"89b9e6e5f8f58b8fab9172648341eb1c7eb318058d30e86f25f7d224877c6135"}
Jul 20 10:15:04.099741 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:04.099574    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="89b9e6e5f8f58b8fab9172648341eb1c7eb318058d30e86f25f7d224877c6135"
Jul 20 10:15:04.101061 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:04.101032    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a2a58-29742375-8fpz5" event={"ID":"27cc30f2-fcee-4b69-9653-a2fa0908d81d","Type":"ContainerDied","Data":"2b235a77d6afc67e871fe074cf20804b366737bbb9c4aef8a1f095b881c293ce"}
Jul 20 10:15:04.101217 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:04.101074    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a2a58-29742375-8fpz5"
Jul 20 10:15:04.101217 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:04.101081    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2b235a77d6afc67e871fe074cf20804b366737bbb9c4aef8a1f095b881c293ce"
Jul 20 10:15:14.409559 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:14.409502    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:15:42.101988 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:42.101929    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-11ee60/test-model-11ee60-kserve-9495b6df7-wlcr4"]
Jul 20 10:15:43.290836 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:43.290793    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-11ee60-ft6n8"]
Jul 20 10:15:43.291375 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:43.291318    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99c5c5e2-2ebe-408c-9ece-3ac5d5bb04a7" containerName="cleanup"
Jul 20 10:15:43.291375 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:43.291339    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="99c5c5e2-2ebe-408c-9ece-3ac5d5bb04a7" containerName="cleanup"
Jul 20 10:15:43.291375 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:43.291357    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27cc30f2-fcee-4b69-9653-a2fa0908d81d" containerName="cleanup"
Jul 20 10:15:43.291375 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:43.291365    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="27cc30f2-fcee-4b69-9653-a2fa0908d81d" containerName="cleanup"
Jul 20 10:15:43.291593 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:43.291460    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="99c5c5e2-2ebe-408c-9ece-3ac5d5bb04a7" containerName="cleanup"
Jul 20 10:15:43.291593 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:43.291472    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="27cc30f2-fcee-4b69-9653-a2fa0908d81d" containerName="cleanup"
Jul 20 10:15:43.294545 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:43.294516    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-11ee60-ft6n8"
Jul 20 10:15:43.297259 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:43.297230    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 10:15:43.297436 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:43.297292    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6jlzx\""
Jul 20 10:15:43.301658 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:43.301622    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-11ee60-ft6n8"]
Jul 20 10:15:43.401041 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:43.400967    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2009e19-a4e8-4846-b0e4-443f7c15e38d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-11ee60-ft6n8\" (UID: \"c2009e19-a4e8-4846-b0e4-443f7c15e38d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-11ee60-ft6n8"
Jul 20 10:15:43.502094 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:43.502043    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2009e19-a4e8-4846-b0e4-443f7c15e38d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-11ee60-ft6n8\" (UID: \"c2009e19-a4e8-4846-b0e4-443f7c15e38d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-11ee60-ft6n8"
Jul 20 10:15:43.502765 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:43.502736    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2009e19-a4e8-4846-b0e4-443f7c15e38d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-11ee60-ft6n8\" (UID: \"c2009e19-a4e8-4846-b0e4-443f7c15e38d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-11ee60-ft6n8"
Jul 20 10:15:43.607477 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:43.607436    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-11ee60-ft6n8"
Jul 20 10:15:43.760115 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:43.759891    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-11ee60-ft6n8"]
Jul 20 10:15:43.762764 ip-10-0-140-134 kubenswrapper[2566]: W0720 10:15:43.762730    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2009e19_a4e8_4846_b0e4_443f7c15e38d.slice/crio-d79b7be806d83af2ab8a9b044ee7680af0a02db10dde39d9f7a6e4d0af97f424 WatchSource:0}: Error finding container d79b7be806d83af2ab8a9b044ee7680af0a02db10dde39d9f7a6e4d0af97f424: Status 404 returned error can't find the container with id d79b7be806d83af2ab8a9b044ee7680af0a02db10dde39d9f7a6e4d0af97f424
Jul 20 10:15:44.310039 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:44.309996    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-11ee60-ft6n8" event={"ID":"c2009e19-a4e8-4846-b0e4-443f7c15e38d","Type":"ContainerStarted","Data":"268fbad204471e700892e6a1de3fbdd7a630b97feae499216daeddedcc8deef3"}
Jul 20 10:15:44.310562 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:44.310083    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-11ee60-ft6n8" event={"ID":"c2009e19-a4e8-4846-b0e4-443f7c15e38d","Type":"ContainerStarted","Data":"d79b7be806d83af2ab8a9b044ee7680af0a02db10dde39d9f7a6e4d0af97f424"}
Jul 20 10:15:44.325668 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:44.325613    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-11ee60-ft6n8" podStartSLOduration=1.325597557 podStartE2EDuration="1.325597557s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 10:15:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 10:15:44.324145782 +0000 UTC m=+2210.875600247" watchObservedRunningTime="2026-07-20 10:15:44.325597557 +0000 UTC m=+2210.877052021"
Jul 20 10:15:45.316544 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:45.316506    2566 generic.go:356] "Generic (PLEG): container finished" podID="c2009e19-a4e8-4846-b0e4-443f7c15e38d" containerID="268fbad204471e700892e6a1de3fbdd7a630b97feae499216daeddedcc8deef3" exitCode=0
Jul 20 10:15:45.316952 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:45.316599    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-11ee60-ft6n8" event={"ID":"c2009e19-a4e8-4846-b0e4-443f7c15e38d","Type":"ContainerDied","Data":"268fbad204471e700892e6a1de3fbdd7a630b97feae499216daeddedcc8deef3"}
Jul 20 10:15:46.464239 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:46.464208    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-11ee60-ft6n8"
Jul 20 10:15:46.528337 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:46.528299    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2009e19-a4e8-4846-b0e4-443f7c15e38d-maas-api-service-ca\") pod \"c2009e19-a4e8-4846-b0e4-443f7c15e38d\" (UID: \"c2009e19-a4e8-4846-b0e4-443f7c15e38d\") "
Jul 20 10:15:46.528694 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:46.528670    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2009e19-a4e8-4846-b0e4-443f7c15e38d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c2009e19-a4e8-4846-b0e4-443f7c15e38d" (UID: "c2009e19-a4e8-4846-b0e4-443f7c15e38d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 10:15:46.629003 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:46.628864    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2009e19-a4e8-4846-b0e4-443f7c15e38d-maas-api-service-ca\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:15:46.916714 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:46.916617    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:15:47.328360 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:47.328331    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-11ee60-ft6n8"
Jul 20 10:15:47.328556 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:47.328368    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-11ee60-ft6n8" event={"ID":"c2009e19-a4e8-4846-b0e4-443f7c15e38d","Type":"ContainerDied","Data":"d79b7be806d83af2ab8a9b044ee7680af0a02db10dde39d9f7a6e4d0af97f424"}
Jul 20 10:15:47.328556 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:47.328419    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d79b7be806d83af2ab8a9b044ee7680af0a02db10dde39d9f7a6e4d0af97f424"
Jul 20 10:15:57.124679 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:57.124615    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-11ee60/test-model-11ee60-kserve-9495b6df7-wlcr4" podUID="3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd" containerName="main" containerID="cri-o://18f300bc29597dd0f51268c411b0f7ce99ec07afe23046ba2c72d6a8ac11154b" gracePeriod=45
Jul 20 10:15:57.381897 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:57.381857    2566 generic.go:356] "Generic (PLEG): container finished" podID="3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd" containerID="18f300bc29597dd0f51268c411b0f7ce99ec07afe23046ba2c72d6a8ac11154b" exitCode=0
Jul 20 10:15:57.382100 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:57.381935    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-11ee60/test-model-11ee60-kserve-9495b6df7-wlcr4" event={"ID":"3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd","Type":"ContainerDied","Data":"18f300bc29597dd0f51268c411b0f7ce99ec07afe23046ba2c72d6a8ac11154b"}
Jul 20 10:15:57.382100 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:57.382015    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-11ee60/test-model-11ee60-kserve-9495b6df7-wlcr4" event={"ID":"3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd","Type":"ContainerDied","Data":"ca27051b4e5cff55f523ffc40bb5023d09eb137a037785eb5cadd815d07dd9eb"}
Jul 20 10:15:57.382100 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:57.382030    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ca27051b4e5cff55f523ffc40bb5023d09eb137a037785eb5cadd815d07dd9eb"
Jul 20 10:15:57.393862 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:57.393826    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-11ee60/test-model-11ee60-kserve-9495b6df7-wlcr4"
Jul 20 10:15:57.524213 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:57.524169    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd-dshm\") pod \"3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd\" (UID: \"3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd\") "
Jul 20 10:15:57.524412 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:57.524248    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd-tls-certs\") pod \"3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd\" (UID: \"3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd\") "
Jul 20 10:15:57.524412 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:57.524300    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd-model-cache\") pod \"3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd\" (UID: \"3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd\") "
Jul 20 10:15:57.524412 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:57.524341    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd-home\") pod \"3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd\" (UID: \"3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd\") "
Jul 20 10:15:57.524412 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:57.524372    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tfnc9\" (UniqueName: \"kubernetes.io/projected/3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd-kube-api-access-tfnc9\") pod \"3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd\" (UID: \"3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd\") "
Jul 20 10:15:57.524412 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:57.524412    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd-tmp-dir\") pod \"3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd\" (UID: \"3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd\") "
Jul 20 10:15:57.524682 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:57.524643    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd-model-cache" (OuterVolumeSpecName: "model-cache") pod "3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd" (UID: "3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 10:15:57.524682 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:57.524663    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd-home" (OuterVolumeSpecName: "home") pod "3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd" (UID: "3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 10:15:57.524784 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:57.524767    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd" (UID: "3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 10:15:57.526800 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:57.526769    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd-dshm" (OuterVolumeSpecName: "dshm") pod "3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd" (UID: "3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 10:15:57.526800 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:57.526767    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd" (UID: "3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 10:15:57.526965 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:57.526862    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd-kube-api-access-tfnc9" (OuterVolumeSpecName: "kube-api-access-tfnc9") pod "3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd" (UID: "3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd"). InnerVolumeSpecName "kube-api-access-tfnc9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 10:15:57.625238 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:57.625189    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd-dshm\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:15:57.625238 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:57.625231    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd-tls-certs\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:15:57.625238 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:57.625247    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd-model-cache\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:15:57.625495 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:57.625259    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd-home\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:15:57.625495 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:57.625271    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tfnc9\" (UniqueName: \"kubernetes.io/projected/3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd-kube-api-access-tfnc9\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:15:57.625495 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:57.625285    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd-tmp-dir\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:15:58.370211 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:58.370163    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-11ee60-ft6n8"]
Jul 20 10:15:58.375025 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:58.374985    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-11ee60-ft6n8"]
Jul 20 10:15:58.386385 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:58.386344    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-11ee60/test-model-11ee60-kserve-9495b6df7-wlcr4"
Jul 20 10:15:58.402365 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:58.402306    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-11ee60/test-model-11ee60-kserve-9495b6df7-wlcr4"]
Jul 20 10:15:58.405985 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:58.405940    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-11ee60/test-model-11ee60-kserve-9495b6df7-wlcr4"]
Jul 20 10:15:59.920400 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:59.920360    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd" path="/var/lib/kubelet/pods/3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd/volumes"
Jul 20 10:15:59.920780 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:15:59.920691    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2009e19-a4e8-4846-b0e4-443f7c15e38d" path="/var/lib/kubelet/pods/c2009e19-a4e8-4846-b0e4-443f7c15e38d/volumes"
Jul 20 10:16:00.313729 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:00.313687    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:16:05.410770 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:05.410722    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a2a58-cs97k"]
Jul 20 10:16:05.411292 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:05.411084    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2009e19-a4e8-4846-b0e4-443f7c15e38d" containerName="revoke-keys"
Jul 20 10:16:05.411292 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:05.411096    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2009e19-a4e8-4846-b0e4-443f7c15e38d" containerName="revoke-keys"
Jul 20 10:16:05.411292 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:05.411113    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd" containerName="main"
Jul 20 10:16:05.411292 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:05.411118    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd" containerName="main"
Jul 20 10:16:05.411292 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:05.411182    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c05d872-8b01-4e0d-a5a5-dd2b9c95f5dd" containerName="main"
Jul 20 10:16:05.411292 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:05.411193    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2009e19-a4e8-4846-b0e4-443f7c15e38d" containerName="revoke-keys"
Jul 20 10:16:05.414475 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:05.414447    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a2a58-cs97k"
Jul 20 10:16:05.417279 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:05.417251    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 10:16:05.417465 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:05.417250    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6jlzx\""
Jul 20 10:16:05.422518 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:05.422485    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a2a58-cs97k"]
Jul 20 10:16:05.498591 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:05.498540    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e821473-26ee-4237-b4eb-8288c64d3622-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-8a2a58-cs97k\" (UID: \"6e821473-26ee-4237-b4eb-8288c64d3622\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a2a58-cs97k"
Jul 20 10:16:05.599939 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:05.599898    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e821473-26ee-4237-b4eb-8288c64d3622-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-8a2a58-cs97k\" (UID: \"6e821473-26ee-4237-b4eb-8288c64d3622\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a2a58-cs97k"
Jul 20 10:16:05.600556 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:05.600535    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e821473-26ee-4237-b4eb-8288c64d3622-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-8a2a58-cs97k\" (UID: \"6e821473-26ee-4237-b4eb-8288c64d3622\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a2a58-cs97k"
Jul 20 10:16:05.726901 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:05.726802    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a2a58-cs97k"
Jul 20 10:16:05.880543 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:05.880515    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a2a58-cs97k"]
Jul 20 10:16:05.886138 ip-10-0-140-134 kubenswrapper[2566]: W0720 10:16:05.886100    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e821473_26ee_4237_b4eb_8288c64d3622.slice/crio-bc7cf30b6d50e8a84a91b9683a9f76ed89e705ec99aaea24c0d0b2f0c304f3ce WatchSource:0}: Error finding container bc7cf30b6d50e8a84a91b9683a9f76ed89e705ec99aaea24c0d0b2f0c304f3ce: Status 404 returned error can't find the container with id bc7cf30b6d50e8a84a91b9683a9f76ed89e705ec99aaea24c0d0b2f0c304f3ce
Jul 20 10:16:06.428856 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:06.428755    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a2a58-cs97k" event={"ID":"6e821473-26ee-4237-b4eb-8288c64d3622","Type":"ContainerStarted","Data":"495049b81399dcad4cfdcda769b7e99db0b5e86401eb66b40e5a2b2b60a00fa5"}
Jul 20 10:16:06.428856 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:06.428818    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a2a58-cs97k" event={"ID":"6e821473-26ee-4237-b4eb-8288c64d3622","Type":"ContainerStarted","Data":"bc7cf30b6d50e8a84a91b9683a9f76ed89e705ec99aaea24c0d0b2f0c304f3ce"}
Jul 20 10:16:06.444557 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:06.444501    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a2a58-cs97k" podStartSLOduration=1.444482801 podStartE2EDuration="1.444482801s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 10:16:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 10:16:06.442942678 +0000 UTC m=+2232.994397143" watchObservedRunningTime="2026-07-20 10:16:06.444482801 +0000 UTC m=+2232.995937299"
Jul 20 10:16:07.437231 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:07.437188    2566 generic.go:356] "Generic (PLEG): container finished" podID="6e821473-26ee-4237-b4eb-8288c64d3622" containerID="495049b81399dcad4cfdcda769b7e99db0b5e86401eb66b40e5a2b2b60a00fa5" exitCode=0
Jul 20 10:16:07.437629 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:07.437286    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a2a58-cs97k" event={"ID":"6e821473-26ee-4237-b4eb-8288c64d3622","Type":"ContainerDied","Data":"495049b81399dcad4cfdcda769b7e99db0b5e86401eb66b40e5a2b2b60a00fa5"}
Jul 20 10:16:08.601331 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:08.601292    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a2a58-cs97k"
Jul 20 10:16:08.728964 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:08.728855    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e821473-26ee-4237-b4eb-8288c64d3622-maas-api-service-ca\") pod \"6e821473-26ee-4237-b4eb-8288c64d3622\" (UID: \"6e821473-26ee-4237-b4eb-8288c64d3622\") "
Jul 20 10:16:08.729309 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:08.729283    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e821473-26ee-4237-b4eb-8288c64d3622-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6e821473-26ee-4237-b4eb-8288c64d3622" (UID: "6e821473-26ee-4237-b4eb-8288c64d3622"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 10:16:08.830309 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:08.830262    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e821473-26ee-4237-b4eb-8288c64d3622-maas-api-service-ca\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:16:09.448924 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:09.448885    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a2a58-cs97k"
Jul 20 10:16:09.449140 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:09.448915    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a2a58-cs97k" event={"ID":"6e821473-26ee-4237-b4eb-8288c64d3622","Type":"ContainerDied","Data":"bc7cf30b6d50e8a84a91b9683a9f76ed89e705ec99aaea24c0d0b2f0c304f3ce"}
Jul 20 10:16:09.449140 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:09.449000    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bc7cf30b6d50e8a84a91b9683a9f76ed89e705ec99aaea24c0d0b2f0c304f3ce"
Jul 20 10:16:15.827507 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:15.827420    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a2a58-29742375-8fpz5"]
Jul 20 10:16:15.830486 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:15.830454    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a2a58-29742375-8fpz5"]
Jul 20 10:16:15.920703 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:15.920663    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27cc30f2-fcee-4b69-9653-a2fa0908d81d" path="/var/lib/kubelet/pods/27cc30f2-fcee-4b69-9653-a2fa0908d81d/volumes"
Jul 20 10:16:18.912574 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:18.912531    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:16:20.481049 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:20.481003    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a2a58-cs97k"]
Jul 20 10:16:20.482795 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:20.482762    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-8a2a58-cs97k"]
Jul 20 10:16:21.921061 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:21.921021    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e821473-26ee-4237-b4eb-8288c64d3622" path="/var/lib/kubelet/pods/6e821473-26ee-4237-b4eb-8288c64d3622/volumes"
Jul 20 10:16:36.420464 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:36.420414    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:16:47.310582 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:47.310537    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:16:52.027659 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:52.027614    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-817a64-gj4d4"]
Jul 20 10:16:52.028080 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:52.027962    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e821473-26ee-4237-b4eb-8288c64d3622" containerName="revoke-keys"
Jul 20 10:16:52.028080 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:52.027988    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e821473-26ee-4237-b4eb-8288c64d3622" containerName="revoke-keys"
Jul 20 10:16:52.028080 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:52.028049    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e821473-26ee-4237-b4eb-8288c64d3622" containerName="revoke-keys"
Jul 20 10:16:52.031278 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:52.031252    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-817a64-gj4d4"
Jul 20 10:16:52.033903 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:52.033872    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6jlzx\""
Jul 20 10:16:52.034101 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:52.033872    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 10:16:52.039788 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:52.039756    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-817a64-gj4d4"]
Jul 20 10:16:52.126477 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:52.126421    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eba97bf0-5446-49d8-9e8f-6ece64d4ed43-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-817a64-gj4d4\" (UID: \"eba97bf0-5446-49d8-9e8f-6ece64d4ed43\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-817a64-gj4d4"
Jul 20 10:16:52.227433 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:52.227384    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eba97bf0-5446-49d8-9e8f-6ece64d4ed43-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-817a64-gj4d4\" (UID: \"eba97bf0-5446-49d8-9e8f-6ece64d4ed43\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-817a64-gj4d4"
Jul 20 10:16:52.228096 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:52.228074    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eba97bf0-5446-49d8-9e8f-6ece64d4ed43-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-817a64-gj4d4\" (UID: \"eba97bf0-5446-49d8-9e8f-6ece64d4ed43\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-817a64-gj4d4"
Jul 20 10:16:52.344126 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:52.344083    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-817a64-gj4d4"
Jul 20 10:16:52.496223 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:52.496189    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-817a64-gj4d4"]
Jul 20 10:16:52.499219 ip-10-0-140-134 kubenswrapper[2566]: W0720 10:16:52.499175    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeba97bf0_5446_49d8_9e8f_6ece64d4ed43.slice/crio-142bfd65f1641fddb1ed8cea6ee1d3c08af4a9ee388462c01826e8ec8f4d5d79 WatchSource:0}: Error finding container 142bfd65f1641fddb1ed8cea6ee1d3c08af4a9ee388462c01826e8ec8f4d5d79: Status 404 returned error can't find the container with id 142bfd65f1641fddb1ed8cea6ee1d3c08af4a9ee388462c01826e8ec8f4d5d79
Jul 20 10:16:52.677327 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:52.677290    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-817a64-gj4d4" event={"ID":"eba97bf0-5446-49d8-9e8f-6ece64d4ed43","Type":"ContainerStarted","Data":"142bfd65f1641fddb1ed8cea6ee1d3c08af4a9ee388462c01826e8ec8f4d5d79"}
Jul 20 10:16:53.684094 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:53.684059    2566 generic.go:356] "Generic (PLEG): container finished" podID="eba97bf0-5446-49d8-9e8f-6ece64d4ed43" containerID="5f04634ce1420995e2244848f0106de6603d16c846df23f2cca1c996478efce2" exitCode=0
Jul 20 10:16:53.684506 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:53.684157    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-817a64-gj4d4" event={"ID":"eba97bf0-5446-49d8-9e8f-6ece64d4ed43","Type":"ContainerDied","Data":"5f04634ce1420995e2244848f0106de6603d16c846df23f2cca1c996478efce2"}
Jul 20 10:16:54.842678 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:54.842642    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-817a64-gj4d4"
Jul 20 10:16:54.951448 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:54.951333    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eba97bf0-5446-49d8-9e8f-6ece64d4ed43-maas-api-service-ca\") pod \"eba97bf0-5446-49d8-9e8f-6ece64d4ed43\" (UID: \"eba97bf0-5446-49d8-9e8f-6ece64d4ed43\") "
Jul 20 10:16:54.951701 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:54.951676    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eba97bf0-5446-49d8-9e8f-6ece64d4ed43-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "eba97bf0-5446-49d8-9e8f-6ece64d4ed43" (UID: "eba97bf0-5446-49d8-9e8f-6ece64d4ed43"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 10:16:55.052382 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:55.052334    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eba97bf0-5446-49d8-9e8f-6ece64d4ed43-maas-api-service-ca\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:16:55.696365 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:55.696334    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-817a64-gj4d4"
Jul 20 10:16:55.696550 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:55.696369    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-817a64-gj4d4" event={"ID":"eba97bf0-5446-49d8-9e8f-6ece64d4ed43","Type":"ContainerDied","Data":"142bfd65f1641fddb1ed8cea6ee1d3c08af4a9ee388462c01826e8ec8f4d5d79"}
Jul 20 10:16:55.696550 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:16:55.696420    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="142bfd65f1641fddb1ed8cea6ee1d3c08af4a9ee388462c01826e8ec8f4d5d79"
Jul 20 10:17:10.419217 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:17:10.419177    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:17:12.107057 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:17:12.107012    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-817a64-gj4d4"]
Jul 20 10:17:12.109826 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:17:12.109782    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-817a64-gj4d4"]
Jul 20 10:17:13.920237 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:17:13.920196    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eba97bf0-5446-49d8-9e8f-6ece64d4ed43" path="/var/lib/kubelet/pods/eba97bf0-5446-49d8-9e8f-6ece64d4ed43/volumes"
Jul 20 10:17:18.830065 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:17:18.830020    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-39467e-c7r8j"]
Jul 20 10:17:18.830595 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:17:18.830366    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eba97bf0-5446-49d8-9e8f-6ece64d4ed43" containerName="revoke-keys"
Jul 20 10:17:18.830595 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:17:18.830378    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="eba97bf0-5446-49d8-9e8f-6ece64d4ed43" containerName="revoke-keys"
Jul 20 10:17:18.830595 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:17:18.830454    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="eba97bf0-5446-49d8-9e8f-6ece64d4ed43" containerName="revoke-keys"
Jul 20 10:17:18.833544 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:17:18.833522    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-39467e-c7r8j"
Jul 20 10:17:18.836087 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:17:18.836061    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6jlzx\""
Jul 20 10:17:18.836226 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:17:18.836061    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 10:17:18.840017 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:17:18.839955    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-39467e-c7r8j"]
Jul 20 10:17:18.841811 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:17:18.841775    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4a63403-e5e3-49c7-accb-a6c0ebb14ea7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-39467e-c7r8j\" (UID: \"d4a63403-e5e3-49c7-accb-a6c0ebb14ea7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-39467e-c7r8j"
Jul 20 10:17:18.942498 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:17:18.942446    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4a63403-e5e3-49c7-accb-a6c0ebb14ea7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-39467e-c7r8j\" (UID: \"d4a63403-e5e3-49c7-accb-a6c0ebb14ea7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-39467e-c7r8j"
Jul 20 10:17:18.943128 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:17:18.943106    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4a63403-e5e3-49c7-accb-a6c0ebb14ea7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-39467e-c7r8j\" (UID: \"d4a63403-e5e3-49c7-accb-a6c0ebb14ea7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-39467e-c7r8j"
Jul 20 10:17:19.146361 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:17:19.146249    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-39467e-c7r8j"
Jul 20 10:17:19.293333 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:17:19.293302    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-39467e-c7r8j"]
Jul 20 10:17:19.295669 ip-10-0-140-134 kubenswrapper[2566]: W0720 10:17:19.295633    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4a63403_e5e3_49c7_accb_a6c0ebb14ea7.slice/crio-0d2b72744ca2c54f856bce0e06d05eb61bd80bbfb07222041cbb7814785b850b WatchSource:0}: Error finding container 0d2b72744ca2c54f856bce0e06d05eb61bd80bbfb07222041cbb7814785b850b: Status 404 returned error can't find the container with id 0d2b72744ca2c54f856bce0e06d05eb61bd80bbfb07222041cbb7814785b850b
Jul 20 10:17:19.821196 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:17:19.821156    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-39467e-c7r8j" event={"ID":"d4a63403-e5e3-49c7-accb-a6c0ebb14ea7","Type":"ContainerStarted","Data":"78a7859d8775b86a10121e6afe72d31d6afb8860de8e3acfc26145fc576a90d0"}
Jul 20 10:17:19.821408 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:17:19.821220    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-39467e-c7r8j" event={"ID":"d4a63403-e5e3-49c7-accb-a6c0ebb14ea7","Type":"ContainerStarted","Data":"0d2b72744ca2c54f856bce0e06d05eb61bd80bbfb07222041cbb7814785b850b"}
Jul 20 10:17:19.837568 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:17:19.837507    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-39467e-c7r8j" podStartSLOduration=1.8374876869999999 podStartE2EDuration="1.837487687s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 10:17:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 10:17:19.835925791 +0000 UTC m=+2306.387380260" watchObservedRunningTime="2026-07-20 10:17:19.837487687 +0000 UTC m=+2306.388942150"
Jul 20 10:17:20.827881 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:17:20.827839    2566 generic.go:356] "Generic (PLEG): container finished" podID="d4a63403-e5e3-49c7-accb-a6c0ebb14ea7" containerID="78a7859d8775b86a10121e6afe72d31d6afb8860de8e3acfc26145fc576a90d0" exitCode=0
Jul 20 10:17:20.828100 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:17:20.827931    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-39467e-c7r8j" event={"ID":"d4a63403-e5e3-49c7-accb-a6c0ebb14ea7","Type":"ContainerDied","Data":"78a7859d8775b86a10121e6afe72d31d6afb8860de8e3acfc26145fc576a90d0"}
Jul 20 10:17:21.978946 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:17:21.978909    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-39467e-c7r8j"
Jul 20 10:17:22.067582 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:17:22.067539    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4a63403-e5e3-49c7-accb-a6c0ebb14ea7-maas-api-service-ca\") pod \"d4a63403-e5e3-49c7-accb-a6c0ebb14ea7\" (UID: \"d4a63403-e5e3-49c7-accb-a6c0ebb14ea7\") "
Jul 20 10:17:22.067932 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:17:22.067905    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4a63403-e5e3-49c7-accb-a6c0ebb14ea7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d4a63403-e5e3-49c7-accb-a6c0ebb14ea7" (UID: "d4a63403-e5e3-49c7-accb-a6c0ebb14ea7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 10:17:22.168581 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:17:22.168476    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4a63403-e5e3-49c7-accb-a6c0ebb14ea7-maas-api-service-ca\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:17:22.839471 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:17:22.839431    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-39467e-c7r8j" event={"ID":"d4a63403-e5e3-49c7-accb-a6c0ebb14ea7","Type":"ContainerDied","Data":"0d2b72744ca2c54f856bce0e06d05eb61bd80bbfb07222041cbb7814785b850b"}
Jul 20 10:17:22.839651 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:17:22.839484    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0d2b72744ca2c54f856bce0e06d05eb61bd80bbfb07222041cbb7814785b850b"
Jul 20 10:17:22.839651 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:17:22.839450    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-39467e-c7r8j"
Jul 20 10:17:38.896027 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:17:38.895962    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-39467e-c7r8j"]
Jul 20 10:17:38.898879 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:17:38.898841    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-39467e-c7r8j"]
Jul 20 10:17:39.765766 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:17:39.765722    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:17:39.921827 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:17:39.921787    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4a63403-e5e3-49c7-accb-a6c0ebb14ea7" path="/var/lib/kubelet/pods/d4a63403-e5e3-49c7-accb-a6c0ebb14ea7/volumes"
Jul 20 10:17:54.164539 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:17:54.164500    2566 scope.go:117] "RemoveContainer" containerID="a7018ec729e8cb8a46e66467a19013366d49d5bc558b55f76259bf09be1bb137"
Jul 20 10:17:55.212827 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:17:55.212774    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:18:07.813657 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:18:07.813617    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:18:33.922533 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:18:33.922494    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-b7ed4c/auth-test-model-b7ed4c-kserve-6f479fd879-xm25v"]
Jul 20 10:18:33.923125 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:18:33.922802    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4a63403-e5e3-49c7-accb-a6c0ebb14ea7" containerName="revoke-keys"
Jul 20 10:18:33.923125 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:18:33.922813    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4a63403-e5e3-49c7-accb-a6c0ebb14ea7" containerName="revoke-keys"
Jul 20 10:18:33.923125 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:18:33.922883    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4a63403-e5e3-49c7-accb-a6c0ebb14ea7" containerName="revoke-keys"
Jul 20 10:18:33.925858 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:18:33.925835    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b7ed4c/auth-test-model-b7ed4c-kserve-6f479fd879-xm25v"
Jul 20 10:18:33.929118 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:18:33.929092    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b7ed4c\"/\"default-dockercfg-7ddw5\""
Jul 20 10:18:33.930349 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:18:33.930323    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b7ed4c\"/\"openshift-service-ca.crt\""
Jul 20 10:18:33.930449 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:18:33.930363    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b7ed4c\"/\"auth-test-model-b7ed4c-kserve-self-signed-certs\""
Jul 20 10:18:33.930514 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:18:33.930492    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b7ed4c\"/\"kube-root-ca.crt\""
Jul 20 10:18:33.936742 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:18:33.936711    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b7ed4c/auth-test-model-b7ed4c-kserve-6f479fd879-xm25v"]
Jul 20 10:18:33.966604 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:18:33.966552    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9842c630-e1f2-4075-9401-e700052bf6a8-home\") pod \"auth-test-model-b7ed4c-kserve-6f479fd879-xm25v\" (UID: \"9842c630-e1f2-4075-9401-e700052bf6a8\") " pod="ai-tenant-e2e-shared-a-b7ed4c/auth-test-model-b7ed4c-kserve-6f479fd879-xm25v"
Jul 20 10:18:33.966604 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:18:33.966600    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9842c630-e1f2-4075-9401-e700052bf6a8-tls-certs\") pod \"auth-test-model-b7ed4c-kserve-6f479fd879-xm25v\" (UID: \"9842c630-e1f2-4075-9401-e700052bf6a8\") " pod="ai-tenant-e2e-shared-a-b7ed4c/auth-test-model-b7ed4c-kserve-6f479fd879-xm25v"
Jul 20 10:18:33.966865 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:18:33.966704    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9842c630-e1f2-4075-9401-e700052bf6a8-model-cache\") pod \"auth-test-model-b7ed4c-kserve-6f479fd879-xm25v\" (UID: \"9842c630-e1f2-4075-9401-e700052bf6a8\") " pod="ai-tenant-e2e-shared-a-b7ed4c/auth-test-model-b7ed4c-kserve-6f479fd879-xm25v"
Jul 20 10:18:33.966865 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:18:33.966742    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfnct\" (UniqueName: \"kubernetes.io/projected/9842c630-e1f2-4075-9401-e700052bf6a8-kube-api-access-qfnct\") pod \"auth-test-model-b7ed4c-kserve-6f479fd879-xm25v\" (UID: \"9842c630-e1f2-4075-9401-e700052bf6a8\") " pod="ai-tenant-e2e-shared-a-b7ed4c/auth-test-model-b7ed4c-kserve-6f479fd879-xm25v"
Jul 20 10:18:33.966865 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:18:33.966809    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9842c630-e1f2-4075-9401-e700052bf6a8-dshm\") pod \"auth-test-model-b7ed4c-kserve-6f479fd879-xm25v\" (UID: \"9842c630-e1f2-4075-9401-e700052bf6a8\") " pod="ai-tenant-e2e-shared-a-b7ed4c/auth-test-model-b7ed4c-kserve-6f479fd879-xm25v"
Jul 20 10:18:33.967049 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:18:33.967027    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9842c630-e1f2-4075-9401-e700052bf6a8-tmp-dir\") pod \"auth-test-model-b7ed4c-kserve-6f479fd879-xm25v\" (UID: \"9842c630-e1f2-4075-9401-e700052bf6a8\") " pod="ai-tenant-e2e-shared-a-b7ed4c/auth-test-model-b7ed4c-kserve-6f479fd879-xm25v"
Jul 20 10:18:34.067769 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:18:34.067714    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9842c630-e1f2-4075-9401-e700052bf6a8-home\") pod \"auth-test-model-b7ed4c-kserve-6f479fd879-xm25v\" (UID: \"9842c630-e1f2-4075-9401-e700052bf6a8\") " pod="ai-tenant-e2e-shared-a-b7ed4c/auth-test-model-b7ed4c-kserve-6f479fd879-xm25v"
Jul 20 10:18:34.067769 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:18:34.067765    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9842c630-e1f2-4075-9401-e700052bf6a8-tls-certs\") pod \"auth-test-model-b7ed4c-kserve-6f479fd879-xm25v\" (UID: \"9842c630-e1f2-4075-9401-e700052bf6a8\") " pod="ai-tenant-e2e-shared-a-b7ed4c/auth-test-model-b7ed4c-kserve-6f479fd879-xm25v"
Jul 20 10:18:34.068111 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:18:34.067804    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9842c630-e1f2-4075-9401-e700052bf6a8-model-cache\") pod \"auth-test-model-b7ed4c-kserve-6f479fd879-xm25v\" (UID: \"9842c630-e1f2-4075-9401-e700052bf6a8\") " pod="ai-tenant-e2e-shared-a-b7ed4c/auth-test-model-b7ed4c-kserve-6f479fd879-xm25v"
Jul 20 10:18:34.068111 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:18:34.067823    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfnct\" (UniqueName: \"kubernetes.io/projected/9842c630-e1f2-4075-9401-e700052bf6a8-kube-api-access-qfnct\") pod \"auth-test-model-b7ed4c-kserve-6f479fd879-xm25v\" (UID: \"9842c630-e1f2-4075-9401-e700052bf6a8\") " pod="ai-tenant-e2e-shared-a-b7ed4c/auth-test-model-b7ed4c-kserve-6f479fd879-xm25v"
Jul 20 10:18:34.068111 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:18:34.067856    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9842c630-e1f2-4075-9401-e700052bf6a8-dshm\") pod \"auth-test-model-b7ed4c-kserve-6f479fd879-xm25v\" (UID: \"9842c630-e1f2-4075-9401-e700052bf6a8\") " pod="ai-tenant-e2e-shared-a-b7ed4c/auth-test-model-b7ed4c-kserve-6f479fd879-xm25v"
Jul 20 10:18:34.068111 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:18:34.068012    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9842c630-e1f2-4075-9401-e700052bf6a8-tmp-dir\") pod \"auth-test-model-b7ed4c-kserve-6f479fd879-xm25v\" (UID: \"9842c630-e1f2-4075-9401-e700052bf6a8\") " pod="ai-tenant-e2e-shared-a-b7ed4c/auth-test-model-b7ed4c-kserve-6f479fd879-xm25v"
Jul 20 10:18:34.068332 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:18:34.068227    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9842c630-e1f2-4075-9401-e700052bf6a8-home\") pod \"auth-test-model-b7ed4c-kserve-6f479fd879-xm25v\" (UID: \"9842c630-e1f2-4075-9401-e700052bf6a8\") " pod="ai-tenant-e2e-shared-a-b7ed4c/auth-test-model-b7ed4c-kserve-6f479fd879-xm25v"
Jul 20 10:18:34.068332 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:18:34.068294    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9842c630-e1f2-4075-9401-e700052bf6a8-model-cache\") pod \"auth-test-model-b7ed4c-kserve-6f479fd879-xm25v\" (UID: \"9842c630-e1f2-4075-9401-e700052bf6a8\") " pod="ai-tenant-e2e-shared-a-b7ed4c/auth-test-model-b7ed4c-kserve-6f479fd879-xm25v"
Jul 20 10:18:34.068447 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:18:34.068411    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9842c630-e1f2-4075-9401-e700052bf6a8-tmp-dir\") pod \"auth-test-model-b7ed4c-kserve-6f479fd879-xm25v\" (UID: \"9842c630-e1f2-4075-9401-e700052bf6a8\") " pod="ai-tenant-e2e-shared-a-b7ed4c/auth-test-model-b7ed4c-kserve-6f479fd879-xm25v"
Jul 20 10:18:34.070438 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:18:34.070386    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9842c630-e1f2-4075-9401-e700052bf6a8-dshm\") pod \"auth-test-model-b7ed4c-kserve-6f479fd879-xm25v\" (UID: \"9842c630-e1f2-4075-9401-e700052bf6a8\") " pod="ai-tenant-e2e-shared-a-b7ed4c/auth-test-model-b7ed4c-kserve-6f479fd879-xm25v"
Jul 20 10:18:34.070654 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:18:34.070636    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9842c630-e1f2-4075-9401-e700052bf6a8-tls-certs\") pod \"auth-test-model-b7ed4c-kserve-6f479fd879-xm25v\" (UID: \"9842c630-e1f2-4075-9401-e700052bf6a8\") " pod="ai-tenant-e2e-shared-a-b7ed4c/auth-test-model-b7ed4c-kserve-6f479fd879-xm25v"
Jul 20 10:18:34.076875 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:18:34.076847    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfnct\" (UniqueName: \"kubernetes.io/projected/9842c630-e1f2-4075-9401-e700052bf6a8-kube-api-access-qfnct\") pod \"auth-test-model-b7ed4c-kserve-6f479fd879-xm25v\" (UID: \"9842c630-e1f2-4075-9401-e700052bf6a8\") " pod="ai-tenant-e2e-shared-a-b7ed4c/auth-test-model-b7ed4c-kserve-6f479fd879-xm25v"
Jul 20 10:18:34.237109 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:18:34.237010    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b7ed4c/auth-test-model-b7ed4c-kserve-6f479fd879-xm25v"
Jul 20 10:18:34.388017 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:18:34.387165    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b7ed4c/auth-test-model-b7ed4c-kserve-6f479fd879-xm25v"]
Jul 20 10:18:34.395128 ip-10-0-140-134 kubenswrapper[2566]: W0720 10:18:34.395073    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9842c630_e1f2_4075_9401_e700052bf6a8.slice/crio-4b7973c561acd816bfc835a2c4f13614f06e63a26916b7791809f48a61ebaae3 WatchSource:0}: Error finding container 4b7973c561acd816bfc835a2c4f13614f06e63a26916b7791809f48a61ebaae3: Status 404 returned error can't find the container with id 4b7973c561acd816bfc835a2c4f13614f06e63a26916b7791809f48a61ebaae3
Jul 20 10:18:35.200510 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:18:35.200467    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b7ed4c/auth-test-model-b7ed4c-kserve-6f479fd879-xm25v" event={"ID":"9842c630-e1f2-4075-9401-e700052bf6a8","Type":"ContainerStarted","Data":"c0d091f0f224d5dadfbef2f5d4878d2fe50c107d02358e125483380430811fcc"}
Jul 20 10:18:35.200510 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:18:35.200513    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b7ed4c/auth-test-model-b7ed4c-kserve-6f479fd879-xm25v" event={"ID":"9842c630-e1f2-4075-9401-e700052bf6a8","Type":"ContainerStarted","Data":"4b7973c561acd816bfc835a2c4f13614f06e63a26916b7791809f48a61ebaae3"}
Jul 20 10:18:35.219902 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:18:35.219838    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-b7ed4c/auth-test-model-b7ed4c-kserve-6f479fd879-xm25v" podStartSLOduration=2.219817057 podStartE2EDuration="2.219817057s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 10:18:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 10:18:35.21756205 +0000 UTC m=+2381.769016514" watchObservedRunningTime="2026-07-20 10:18:35.219817057 +0000 UTC m=+2381.771271551"
Jul 20 10:18:37.517051 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:18:37.517014    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:18:44.237459 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:18:44.237407    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-b7ed4c/auth-test-model-b7ed4c-kserve-6f479fd879-xm25v"
Jul 20 10:18:44.237961 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:18:44.237482    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-b7ed4c/auth-test-model-b7ed4c-kserve-6f479fd879-xm25v"
Jul 20 10:18:44.250851 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:18:44.250823    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-b7ed4c/auth-test-model-b7ed4c-kserve-6f479fd879-xm25v"
Jul 20 10:18:44.262936 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:18:44.262894    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-b7ed4c/auth-test-model-b7ed4c-kserve-6f479fd879-xm25v"
Jul 20 10:18:54.186621 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:18:54.186581    2566 scope.go:117] "RemoveContainer" containerID="fc95816609697cdb13438e6ce0df249ec19e52c89f43a20d22bcbdbceb245296"
Jul 20 10:19:30.307034 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:19:30.306940    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:19:49.712462 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:19:49.712413    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:20:09.313727 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:20:09.313684    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:20:27.908559 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:20:27.908514    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:20:44.213851 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:20:44.213806    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:20:54.213307 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:20:54.213271    2566 scope.go:117] "RemoveContainer" containerID="18f300bc29597dd0f51268c411b0f7ce99ec07afe23046ba2c72d6a8ac11154b"
Jul 20 10:21:03.708542 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:21:03.708496    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:21:22.514177 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:21:22.514127    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:21:38.469321 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:21:38.469283    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b7ed4c/auth-test-model-b7ed4c-kserve-6f479fd879-xm25v"]
Jul 20 10:21:42.612803 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:21:42.612759    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:21:53.491078 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:21:53.491033    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-b7ed4c/auth-test-model-b7ed4c-kserve-6f479fd879-xm25v" podUID="9842c630-e1f2-4075-9401-e700052bf6a8" containerName="main" containerID="cri-o://c0d091f0f224d5dadfbef2f5d4878d2fe50c107d02358e125483380430811fcc" gracePeriod=45
Jul 20 10:21:53.751710 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:21:53.751643    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b7ed4c/auth-test-model-b7ed4c-kserve-6f479fd879-xm25v"
Jul 20 10:21:53.804710 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:21:53.804659    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9842c630-e1f2-4075-9401-e700052bf6a8-tmp-dir\") pod \"9842c630-e1f2-4075-9401-e700052bf6a8\" (UID: \"9842c630-e1f2-4075-9401-e700052bf6a8\") "
Jul 20 10:21:53.804710 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:21:53.804701    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9842c630-e1f2-4075-9401-e700052bf6a8-dshm\") pod \"9842c630-e1f2-4075-9401-e700052bf6a8\" (UID: \"9842c630-e1f2-4075-9401-e700052bf6a8\") "
Jul 20 10:21:53.805007 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:21:53.804726    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9842c630-e1f2-4075-9401-e700052bf6a8-model-cache\") pod \"9842c630-e1f2-4075-9401-e700052bf6a8\" (UID: \"9842c630-e1f2-4075-9401-e700052bf6a8\") "
Jul 20 10:21:53.805007 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:21:53.804762    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9842c630-e1f2-4075-9401-e700052bf6a8-tls-certs\") pod \"9842c630-e1f2-4075-9401-e700052bf6a8\" (UID: \"9842c630-e1f2-4075-9401-e700052bf6a8\") "
Jul 20 10:21:53.805007 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:21:53.804793    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9842c630-e1f2-4075-9401-e700052bf6a8-home\") pod \"9842c630-e1f2-4075-9401-e700052bf6a8\" (UID: \"9842c630-e1f2-4075-9401-e700052bf6a8\") "
Jul 20 10:21:53.805007 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:21:53.804830    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qfnct\" (UniqueName: \"kubernetes.io/projected/9842c630-e1f2-4075-9401-e700052bf6a8-kube-api-access-qfnct\") pod \"9842c630-e1f2-4075-9401-e700052bf6a8\" (UID: \"9842c630-e1f2-4075-9401-e700052bf6a8\") "
Jul 20 10:21:53.805232 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:21:53.805059    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9842c630-e1f2-4075-9401-e700052bf6a8-model-cache" (OuterVolumeSpecName: "model-cache") pod "9842c630-e1f2-4075-9401-e700052bf6a8" (UID: "9842c630-e1f2-4075-9401-e700052bf6a8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 10:21:53.805232 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:21:53.805077    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9842c630-e1f2-4075-9401-e700052bf6a8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9842c630-e1f2-4075-9401-e700052bf6a8" (UID: "9842c630-e1f2-4075-9401-e700052bf6a8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 10:21:53.805232 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:21:53.805140    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9842c630-e1f2-4075-9401-e700052bf6a8-home" (OuterVolumeSpecName: "home") pod "9842c630-e1f2-4075-9401-e700052bf6a8" (UID: "9842c630-e1f2-4075-9401-e700052bf6a8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 10:21:53.807323 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:21:53.807281    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9842c630-e1f2-4075-9401-e700052bf6a8-dshm" (OuterVolumeSpecName: "dshm") pod "9842c630-e1f2-4075-9401-e700052bf6a8" (UID: "9842c630-e1f2-4075-9401-e700052bf6a8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 10:21:53.807461 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:21:53.807358    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9842c630-e1f2-4075-9401-e700052bf6a8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9842c630-e1f2-4075-9401-e700052bf6a8" (UID: "9842c630-e1f2-4075-9401-e700052bf6a8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 10:21:53.807461 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:21:53.807361    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9842c630-e1f2-4075-9401-e700052bf6a8-kube-api-access-qfnct" (OuterVolumeSpecName: "kube-api-access-qfnct") pod "9842c630-e1f2-4075-9401-e700052bf6a8" (UID: "9842c630-e1f2-4075-9401-e700052bf6a8"). InnerVolumeSpecName "kube-api-access-qfnct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 10:21:53.905783 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:21:53.905736    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9842c630-e1f2-4075-9401-e700052bf6a8-home\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:21:53.905783 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:21:53.905774    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qfnct\" (UniqueName: \"kubernetes.io/projected/9842c630-e1f2-4075-9401-e700052bf6a8-kube-api-access-qfnct\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:21:53.905783 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:21:53.905786    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9842c630-e1f2-4075-9401-e700052bf6a8-tmp-dir\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:21:53.905783 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:21:53.905794    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9842c630-e1f2-4075-9401-e700052bf6a8-dshm\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:21:53.906165 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:21:53.905802    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9842c630-e1f2-4075-9401-e700052bf6a8-model-cache\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:21:53.906165 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:21:53.905810    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9842c630-e1f2-4075-9401-e700052bf6a8-tls-certs\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:21:54.191451 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:21:54.191411    2566 generic.go:356] "Generic (PLEG): container finished" podID="9842c630-e1f2-4075-9401-e700052bf6a8" containerID="c0d091f0f224d5dadfbef2f5d4878d2fe50c107d02358e125483380430811fcc" exitCode=0
Jul 20 10:21:54.191451 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:21:54.191454    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b7ed4c/auth-test-model-b7ed4c-kserve-6f479fd879-xm25v" event={"ID":"9842c630-e1f2-4075-9401-e700052bf6a8","Type":"ContainerDied","Data":"c0d091f0f224d5dadfbef2f5d4878d2fe50c107d02358e125483380430811fcc"}
Jul 20 10:21:54.191667 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:21:54.191486    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b7ed4c/auth-test-model-b7ed4c-kserve-6f479fd879-xm25v" event={"ID":"9842c630-e1f2-4075-9401-e700052bf6a8","Type":"ContainerDied","Data":"4b7973c561acd816bfc835a2c4f13614f06e63a26916b7791809f48a61ebaae3"}
Jul 20 10:21:54.191667 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:21:54.191502    2566 scope.go:117] "RemoveContainer" containerID="c0d091f0f224d5dadfbef2f5d4878d2fe50c107d02358e125483380430811fcc"
Jul 20 10:21:54.191667 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:21:54.191457    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b7ed4c/auth-test-model-b7ed4c-kserve-6f479fd879-xm25v"
Jul 20 10:21:54.200732 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:21:54.200710    2566 scope.go:117] "RemoveContainer" containerID="c0d091f0f224d5dadfbef2f5d4878d2fe50c107d02358e125483380430811fcc"
Jul 20 10:21:54.201036 ip-10-0-140-134 kubenswrapper[2566]: E0720 10:21:54.201013    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0d091f0f224d5dadfbef2f5d4878d2fe50c107d02358e125483380430811fcc\": container with ID starting with c0d091f0f224d5dadfbef2f5d4878d2fe50c107d02358e125483380430811fcc not found: ID does not exist" containerID="c0d091f0f224d5dadfbef2f5d4878d2fe50c107d02358e125483380430811fcc"
Jul 20 10:21:54.201103 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:21:54.201045    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0d091f0f224d5dadfbef2f5d4878d2fe50c107d02358e125483380430811fcc"} err="failed to get container status \"c0d091f0f224d5dadfbef2f5d4878d2fe50c107d02358e125483380430811fcc\": rpc error: code = NotFound desc = could not find container \"c0d091f0f224d5dadfbef2f5d4878d2fe50c107d02358e125483380430811fcc\": container with ID starting with c0d091f0f224d5dadfbef2f5d4878d2fe50c107d02358e125483380430811fcc not found: ID does not exist"
Jul 20 10:21:54.207323 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:21:54.207287    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b7ed4c/auth-test-model-b7ed4c-kserve-6f479fd879-xm25v"]
Jul 20 10:21:54.212531 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:21:54.212506    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-b7ed4c/auth-test-model-b7ed4c-kserve-6f479fd879-xm25v"]
Jul 20 10:21:54.233066 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:21:54.233043    2566 scope.go:117] "RemoveContainer" containerID="3e32e633d889e25a4a589dcbcd0c8f8f7d41884aef6624f9e8b29dbb05bbc776"
Jul 20 10:21:54.241235 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:21:54.241214    2566 scope.go:117] "RemoveContainer" containerID="268fbad204471e700892e6a1de3fbdd7a630b97feae499216daeddedcc8deef3"
Jul 20 10:21:55.925812 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:21:55.925775    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9842c630-e1f2-4075-9401-e700052bf6a8" path="/var/lib/kubelet/pods/9842c630-e1f2-4075-9401-e700052bf6a8/volumes"
Jul 20 10:22:06.205180 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:22:06.205127    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-b7ed4c/sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b"]
Jul 20 10:22:06.205649 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:22:06.205514    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9842c630-e1f2-4075-9401-e700052bf6a8" containerName="main"
Jul 20 10:22:06.205649 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:22:06.205527    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9842c630-e1f2-4075-9401-e700052bf6a8" containerName="main"
Jul 20 10:22:06.205649 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:22:06.205602    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9842c630-e1f2-4075-9401-e700052bf6a8" containerName="main"
Jul 20 10:22:06.208360 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:22:06.208329    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b7ed4c/sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b"
Jul 20 10:22:06.211226 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:22:06.211203    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b7ed4c\"/\"kube-root-ca.crt\""
Jul 20 10:22:06.212502 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:22:06.212474    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b7ed4c\"/\"openshift-service-ca.crt\""
Jul 20 10:22:06.212645 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:22:06.212474    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b7ed4c\"/\"sub-test-model-b7ed4c-kserve-self-signed-certs\""
Jul 20 10:22:06.212645 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:22:06.212520    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b7ed4c\"/\"default-dockercfg-7ddw5\""
Jul 20 10:22:06.218124 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:22:06.218092    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b7ed4c/sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b"]
Jul 20 10:22:06.309775 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:22:06.309735    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f71c0c3-f2dd-4f50-ab63-b39881de6227-home\") pod \"sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b\" (UID: \"7f71c0c3-f2dd-4f50-ab63-b39881de6227\") " pod="ai-tenant-e2e-shared-a-b7ed4c/sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b"
Jul 20 10:22:06.309963 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:22:06.309798    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f71c0c3-f2dd-4f50-ab63-b39881de6227-dshm\") pod \"sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b\" (UID: \"7f71c0c3-f2dd-4f50-ab63-b39881de6227\") " pod="ai-tenant-e2e-shared-a-b7ed4c/sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b"
Jul 20 10:22:06.309963 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:22:06.309845    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9s4h\" (UniqueName: \"kubernetes.io/projected/7f71c0c3-f2dd-4f50-ab63-b39881de6227-kube-api-access-z9s4h\") pod \"sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b\" (UID: \"7f71c0c3-f2dd-4f50-ab63-b39881de6227\") " pod="ai-tenant-e2e-shared-a-b7ed4c/sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b"
Jul 20 10:22:06.309963 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:22:06.309876    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f71c0c3-f2dd-4f50-ab63-b39881de6227-tmp-dir\") pod \"sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b\" (UID: \"7f71c0c3-f2dd-4f50-ab63-b39881de6227\") " pod="ai-tenant-e2e-shared-a-b7ed4c/sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b"
Jul 20 10:22:06.309963 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:22:06.309912    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f71c0c3-f2dd-4f50-ab63-b39881de6227-model-cache\") pod \"sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b\" (UID: \"7f71c0c3-f2dd-4f50-ab63-b39881de6227\") " pod="ai-tenant-e2e-shared-a-b7ed4c/sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b"
Jul 20 10:22:06.309963 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:22:06.309939    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f71c0c3-f2dd-4f50-ab63-b39881de6227-tls-certs\") pod \"sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b\" (UID: \"7f71c0c3-f2dd-4f50-ab63-b39881de6227\") " pod="ai-tenant-e2e-shared-a-b7ed4c/sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b"
Jul 20 10:22:06.410360 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:22:06.410320    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f71c0c3-f2dd-4f50-ab63-b39881de6227-tls-certs\") pod \"sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b\" (UID: \"7f71c0c3-f2dd-4f50-ab63-b39881de6227\") " pod="ai-tenant-e2e-shared-a-b7ed4c/sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b"
Jul 20 10:22:06.410360 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:22:06.410365    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f71c0c3-f2dd-4f50-ab63-b39881de6227-home\") pod \"sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b\" (UID: \"7f71c0c3-f2dd-4f50-ab63-b39881de6227\") " pod="ai-tenant-e2e-shared-a-b7ed4c/sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b"
Jul 20 10:22:06.410652 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:22:06.410400    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f71c0c3-f2dd-4f50-ab63-b39881de6227-dshm\") pod \"sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b\" (UID: \"7f71c0c3-f2dd-4f50-ab63-b39881de6227\") " pod="ai-tenant-e2e-shared-a-b7ed4c/sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b"
Jul 20 10:22:06.410652 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:22:06.410437    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9s4h\" (UniqueName: \"kubernetes.io/projected/7f71c0c3-f2dd-4f50-ab63-b39881de6227-kube-api-access-z9s4h\") pod \"sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b\" (UID: \"7f71c0c3-f2dd-4f50-ab63-b39881de6227\") " pod="ai-tenant-e2e-shared-a-b7ed4c/sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b"
Jul 20 10:22:06.410652 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:22:06.410494    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f71c0c3-f2dd-4f50-ab63-b39881de6227-tmp-dir\") pod \"sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b\" (UID: \"7f71c0c3-f2dd-4f50-ab63-b39881de6227\") " pod="ai-tenant-e2e-shared-a-b7ed4c/sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b"
Jul 20 10:22:06.410652 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:22:06.410524    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f71c0c3-f2dd-4f50-ab63-b39881de6227-model-cache\") pod \"sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b\" (UID: \"7f71c0c3-f2dd-4f50-ab63-b39881de6227\") " pod="ai-tenant-e2e-shared-a-b7ed4c/sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b"
Jul 20 10:22:06.410872 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:22:06.410847    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f71c0c3-f2dd-4f50-ab63-b39881de6227-home\") pod \"sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b\" (UID: \"7f71c0c3-f2dd-4f50-ab63-b39881de6227\") " pod="ai-tenant-e2e-shared-a-b7ed4c/sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b"
Jul 20 10:22:06.410872 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:22:06.410867    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f71c0c3-f2dd-4f50-ab63-b39881de6227-model-cache\") pod \"sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b\" (UID: \"7f71c0c3-f2dd-4f50-ab63-b39881de6227\") " pod="ai-tenant-e2e-shared-a-b7ed4c/sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b"
Jul 20 10:22:06.411029 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:22:06.410928    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f71c0c3-f2dd-4f50-ab63-b39881de6227-tmp-dir\") pod \"sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b\" (UID: \"7f71c0c3-f2dd-4f50-ab63-b39881de6227\") " pod="ai-tenant-e2e-shared-a-b7ed4c/sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b"
Jul 20 10:22:06.413065 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:22:06.413041    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f71c0c3-f2dd-4f50-ab63-b39881de6227-dshm\") pod \"sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b\" (UID: \"7f71c0c3-f2dd-4f50-ab63-b39881de6227\") " pod="ai-tenant-e2e-shared-a-b7ed4c/sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b"
Jul 20 10:22:06.413310 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:22:06.413291    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f71c0c3-f2dd-4f50-ab63-b39881de6227-tls-certs\") pod \"sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b\" (UID: \"7f71c0c3-f2dd-4f50-ab63-b39881de6227\") " pod="ai-tenant-e2e-shared-a-b7ed4c/sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b"
Jul 20 10:22:06.419176 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:22:06.419146    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9s4h\" (UniqueName: \"kubernetes.io/projected/7f71c0c3-f2dd-4f50-ab63-b39881de6227-kube-api-access-z9s4h\") pod \"sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b\" (UID: \"7f71c0c3-f2dd-4f50-ab63-b39881de6227\") " pod="ai-tenant-e2e-shared-a-b7ed4c/sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b"
Jul 20 10:22:06.520801 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:22:06.520694    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b7ed4c/sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b"
Jul 20 10:22:06.674045 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:22:06.674014    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b7ed4c/sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b"]
Jul 20 10:22:06.677233 ip-10-0-140-134 kubenswrapper[2566]: W0720 10:22:06.677202    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7f71c0c3_f2dd_4f50_ab63_b39881de6227.slice/crio-c8bbe37ea4ce30ab708e05d0c26f402fac314e4bc348d9c12a3a2b98ecb6808c WatchSource:0}: Error finding container c8bbe37ea4ce30ab708e05d0c26f402fac314e4bc348d9c12a3a2b98ecb6808c: Status 404 returned error can't find the container with id c8bbe37ea4ce30ab708e05d0c26f402fac314e4bc348d9c12a3a2b98ecb6808c
Jul 20 10:22:07.252895 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:22:07.252860    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b7ed4c/sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b" event={"ID":"7f71c0c3-f2dd-4f50-ab63-b39881de6227","Type":"ContainerStarted","Data":"879ff339eb9a40cd39adc10e5330a36c38ad6ae91f1b9debd2a1558277b45c6a"}
Jul 20 10:22:07.252895 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:22:07.252901    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b7ed4c/sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b" event={"ID":"7f71c0c3-f2dd-4f50-ab63-b39881de6227","Type":"ContainerStarted","Data":"c8bbe37ea4ce30ab708e05d0c26f402fac314e4bc348d9c12a3a2b98ecb6808c"}
Jul 20 10:22:07.273081 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:22:07.273023    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-b7ed4c/sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b" podStartSLOduration=1.273004977 podStartE2EDuration="1.273004977s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 10:22:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 10:22:07.271223668 +0000 UTC m=+2593.822678150" watchObservedRunningTime="2026-07-20 10:22:07.273004977 +0000 UTC m=+2593.824459434"
Jul 20 10:22:10.209507 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:22:10.209462    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:22:16.521125 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:22:16.521006    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-b7ed4c/sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b"
Jul 20 10:22:16.521125 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:22:16.521096    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-b7ed4c/sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b"
Jul 20 10:22:16.534015 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:22:16.533958    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-b7ed4c/sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b"
Jul 20 10:22:17.312117 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:22:17.312081    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-b7ed4c/sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b"
Jul 20 10:22:49.304475 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:22:49.304433    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:22:54.266855 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:22:54.266818    2566 scope.go:117] "RemoveContainer" containerID="495049b81399dcad4cfdcda769b7e99db0b5e86401eb66b40e5a2b2b60a00fa5"
Jul 20 10:22:54.275786 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:22:54.275761    2566 scope.go:117] "RemoveContainer" containerID="5f04634ce1420995e2244848f0106de6603d16c846df23f2cca1c996478efce2"
Jul 20 10:23:04.611878 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:04.611834    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:23:24.013981 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:24.013923    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:23:31.705113 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:31.705069    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b7ed4c/sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b"]
Jul 20 10:23:42.808126 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:42.808088    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-b7ed4c/rate-test-model-b7ed4c-kserve-84bc55d547-m458p"]
Jul 20 10:23:42.811815 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:42.811781    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b7ed4c/rate-test-model-b7ed4c-kserve-84bc55d547-m458p"
Jul 20 10:23:42.814489 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:42.814460    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b7ed4c\"/\"rate-test-model-b7ed4c-kserve-self-signed-certs\""
Jul 20 10:23:42.820827 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:42.820791    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b7ed4c/rate-test-model-b7ed4c-kserve-84bc55d547-m458p"]
Jul 20 10:23:42.946765 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:42.946723    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ece574b-b6f9-4562-93e4-f557bfe0da88-dshm\") pod \"rate-test-model-b7ed4c-kserve-84bc55d547-m458p\" (UID: \"7ece574b-b6f9-4562-93e4-f557bfe0da88\") " pod="ai-tenant-e2e-shared-a-b7ed4c/rate-test-model-b7ed4c-kserve-84bc55d547-m458p"
Jul 20 10:23:42.946765 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:42.946774    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ece574b-b6f9-4562-93e4-f557bfe0da88-home\") pod \"rate-test-model-b7ed4c-kserve-84bc55d547-m458p\" (UID: \"7ece574b-b6f9-4562-93e4-f557bfe0da88\") " pod="ai-tenant-e2e-shared-a-b7ed4c/rate-test-model-b7ed4c-kserve-84bc55d547-m458p"
Jul 20 10:23:42.947030 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:42.946793    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ece574b-b6f9-4562-93e4-f557bfe0da88-tls-certs\") pod \"rate-test-model-b7ed4c-kserve-84bc55d547-m458p\" (UID: \"7ece574b-b6f9-4562-93e4-f557bfe0da88\") " pod="ai-tenant-e2e-shared-a-b7ed4c/rate-test-model-b7ed4c-kserve-84bc55d547-m458p"
Jul 20 10:23:42.947030 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:42.946850    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ece574b-b6f9-4562-93e4-f557bfe0da88-model-cache\") pod \"rate-test-model-b7ed4c-kserve-84bc55d547-m458p\" (UID: \"7ece574b-b6f9-4562-93e4-f557bfe0da88\") " pod="ai-tenant-e2e-shared-a-b7ed4c/rate-test-model-b7ed4c-kserve-84bc55d547-m458p"
Jul 20 10:23:42.947030 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:42.946890    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ece574b-b6f9-4562-93e4-f557bfe0da88-tmp-dir\") pod \"rate-test-model-b7ed4c-kserve-84bc55d547-m458p\" (UID: \"7ece574b-b6f9-4562-93e4-f557bfe0da88\") " pod="ai-tenant-e2e-shared-a-b7ed4c/rate-test-model-b7ed4c-kserve-84bc55d547-m458p"
Jul 20 10:23:42.947030 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:42.946911    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvqnh\" (UniqueName: \"kubernetes.io/projected/7ece574b-b6f9-4562-93e4-f557bfe0da88-kube-api-access-tvqnh\") pod \"rate-test-model-b7ed4c-kserve-84bc55d547-m458p\" (UID: \"7ece574b-b6f9-4562-93e4-f557bfe0da88\") " pod="ai-tenant-e2e-shared-a-b7ed4c/rate-test-model-b7ed4c-kserve-84bc55d547-m458p"
Jul 20 10:23:43.047623 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:43.047576    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ece574b-b6f9-4562-93e4-f557bfe0da88-dshm\") pod \"rate-test-model-b7ed4c-kserve-84bc55d547-m458p\" (UID: \"7ece574b-b6f9-4562-93e4-f557bfe0da88\") " pod="ai-tenant-e2e-shared-a-b7ed4c/rate-test-model-b7ed4c-kserve-84bc55d547-m458p"
Jul 20 10:23:43.047623 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:43.047627    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ece574b-b6f9-4562-93e4-f557bfe0da88-home\") pod \"rate-test-model-b7ed4c-kserve-84bc55d547-m458p\" (UID: \"7ece574b-b6f9-4562-93e4-f557bfe0da88\") " pod="ai-tenant-e2e-shared-a-b7ed4c/rate-test-model-b7ed4c-kserve-84bc55d547-m458p"
Jul 20 10:23:43.047915 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:43.047747    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ece574b-b6f9-4562-93e4-f557bfe0da88-tls-certs\") pod \"rate-test-model-b7ed4c-kserve-84bc55d547-m458p\" (UID: \"7ece574b-b6f9-4562-93e4-f557bfe0da88\") " pod="ai-tenant-e2e-shared-a-b7ed4c/rate-test-model-b7ed4c-kserve-84bc55d547-m458p"
Jul 20 10:23:43.047915 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:43.047794    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ece574b-b6f9-4562-93e4-f557bfe0da88-model-cache\") pod \"rate-test-model-b7ed4c-kserve-84bc55d547-m458p\" (UID: \"7ece574b-b6f9-4562-93e4-f557bfe0da88\") " pod="ai-tenant-e2e-shared-a-b7ed4c/rate-test-model-b7ed4c-kserve-84bc55d547-m458p"
Jul 20 10:23:43.047915 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:43.047834    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ece574b-b6f9-4562-93e4-f557bfe0da88-tmp-dir\") pod \"rate-test-model-b7ed4c-kserve-84bc55d547-m458p\" (UID: \"7ece574b-b6f9-4562-93e4-f557bfe0da88\") " pod="ai-tenant-e2e-shared-a-b7ed4c/rate-test-model-b7ed4c-kserve-84bc55d547-m458p"
Jul 20 10:23:43.047915 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:43.047863    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvqnh\" (UniqueName: \"kubernetes.io/projected/7ece574b-b6f9-4562-93e4-f557bfe0da88-kube-api-access-tvqnh\") pod \"rate-test-model-b7ed4c-kserve-84bc55d547-m458p\" (UID: \"7ece574b-b6f9-4562-93e4-f557bfe0da88\") " pod="ai-tenant-e2e-shared-a-b7ed4c/rate-test-model-b7ed4c-kserve-84bc55d547-m458p"
Jul 20 10:23:43.048218 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:43.048021    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ece574b-b6f9-4562-93e4-f557bfe0da88-home\") pod \"rate-test-model-b7ed4c-kserve-84bc55d547-m458p\" (UID: \"7ece574b-b6f9-4562-93e4-f557bfe0da88\") " pod="ai-tenant-e2e-shared-a-b7ed4c/rate-test-model-b7ed4c-kserve-84bc55d547-m458p"
Jul 20 10:23:43.048218 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:43.048155    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ece574b-b6f9-4562-93e4-f557bfe0da88-model-cache\") pod \"rate-test-model-b7ed4c-kserve-84bc55d547-m458p\" (UID: \"7ece574b-b6f9-4562-93e4-f557bfe0da88\") " pod="ai-tenant-e2e-shared-a-b7ed4c/rate-test-model-b7ed4c-kserve-84bc55d547-m458p"
Jul 20 10:23:43.048218 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:43.048187    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ece574b-b6f9-4562-93e4-f557bfe0da88-tmp-dir\") pod \"rate-test-model-b7ed4c-kserve-84bc55d547-m458p\" (UID: \"7ece574b-b6f9-4562-93e4-f557bfe0da88\") " pod="ai-tenant-e2e-shared-a-b7ed4c/rate-test-model-b7ed4c-kserve-84bc55d547-m458p"
Jul 20 10:23:43.050380 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:43.050343    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ece574b-b6f9-4562-93e4-f557bfe0da88-dshm\") pod \"rate-test-model-b7ed4c-kserve-84bc55d547-m458p\" (UID: \"7ece574b-b6f9-4562-93e4-f557bfe0da88\") " pod="ai-tenant-e2e-shared-a-b7ed4c/rate-test-model-b7ed4c-kserve-84bc55d547-m458p"
Jul 20 10:23:43.050555 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:43.050534    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ece574b-b6f9-4562-93e4-f557bfe0da88-tls-certs\") pod \"rate-test-model-b7ed4c-kserve-84bc55d547-m458p\" (UID: \"7ece574b-b6f9-4562-93e4-f557bfe0da88\") " pod="ai-tenant-e2e-shared-a-b7ed4c/rate-test-model-b7ed4c-kserve-84bc55d547-m458p"
Jul 20 10:23:43.057194 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:43.057159    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvqnh\" (UniqueName: \"kubernetes.io/projected/7ece574b-b6f9-4562-93e4-f557bfe0da88-kube-api-access-tvqnh\") pod \"rate-test-model-b7ed4c-kserve-84bc55d547-m458p\" (UID: \"7ece574b-b6f9-4562-93e4-f557bfe0da88\") " pod="ai-tenant-e2e-shared-a-b7ed4c/rate-test-model-b7ed4c-kserve-84bc55d547-m458p"
Jul 20 10:23:43.122381 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:43.122271    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b7ed4c/rate-test-model-b7ed4c-kserve-84bc55d547-m458p"
Jul 20 10:23:43.275126 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:43.275096    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b7ed4c/rate-test-model-b7ed4c-kserve-84bc55d547-m458p"]
Jul 20 10:23:43.277501 ip-10-0-140-134 kubenswrapper[2566]: W0720 10:23:43.277453    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7ece574b_b6f9_4562_93e4_f557bfe0da88.slice/crio-2092ccecd7a76560d06572046f63f3ed385ba37960618a115a5cb18b4003ff2b WatchSource:0}: Error finding container 2092ccecd7a76560d06572046f63f3ed385ba37960618a115a5cb18b4003ff2b: Status 404 returned error can't find the container with id 2092ccecd7a76560d06572046f63f3ed385ba37960618a115a5cb18b4003ff2b
Jul 20 10:23:43.723932 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:43.723890    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b7ed4c/rate-test-model-b7ed4c-kserve-84bc55d547-m458p" event={"ID":"7ece574b-b6f9-4562-93e4-f557bfe0da88","Type":"ContainerStarted","Data":"32693ccf92ff6a172232ff91429cbcb2ee39cd6c46cd3f6c35407f49192bc657"}
Jul 20 10:23:43.724146 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:43.723948    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b7ed4c/rate-test-model-b7ed4c-kserve-84bc55d547-m458p" event={"ID":"7ece574b-b6f9-4562-93e4-f557bfe0da88","Type":"ContainerStarted","Data":"2092ccecd7a76560d06572046f63f3ed385ba37960618a115a5cb18b4003ff2b"}
Jul 20 10:23:43.741358 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:43.741299    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-b7ed4c/rate-test-model-b7ed4c-kserve-84bc55d547-m458p" podStartSLOduration=1.741282792 podStartE2EDuration="1.741282792s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 10:23:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 10:23:43.740253049 +0000 UTC m=+2690.291707533" watchObservedRunningTime="2026-07-20 10:23:43.741282792 +0000 UTC m=+2690.292737257"
Jul 20 10:23:44.225462 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:44.225419    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:23:46.726688 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:46.726637    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-b7ed4c/sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b" podUID="7f71c0c3-f2dd-4f50-ab63-b39881de6227" containerName="main" containerID="cri-o://879ff339eb9a40cd39adc10e5330a36c38ad6ae91f1b9debd2a1558277b45c6a" gracePeriod=45
Jul 20 10:23:46.995002 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:46.994955    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b7ed4c/sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b"
Jul 20 10:23:47.088660 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:47.088619    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f71c0c3-f2dd-4f50-ab63-b39881de6227-dshm\") pod \"7f71c0c3-f2dd-4f50-ab63-b39881de6227\" (UID: \"7f71c0c3-f2dd-4f50-ab63-b39881de6227\") "
Jul 20 10:23:47.088660 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:47.088670    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f71c0c3-f2dd-4f50-ab63-b39881de6227-tls-certs\") pod \"7f71c0c3-f2dd-4f50-ab63-b39881de6227\" (UID: \"7f71c0c3-f2dd-4f50-ab63-b39881de6227\") "
Jul 20 10:23:47.088921 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:47.088713    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f71c0c3-f2dd-4f50-ab63-b39881de6227-home\") pod \"7f71c0c3-f2dd-4f50-ab63-b39881de6227\" (UID: \"7f71c0c3-f2dd-4f50-ab63-b39881de6227\") "
Jul 20 10:23:47.088921 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:47.088735    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9s4h\" (UniqueName: \"kubernetes.io/projected/7f71c0c3-f2dd-4f50-ab63-b39881de6227-kube-api-access-z9s4h\") pod \"7f71c0c3-f2dd-4f50-ab63-b39881de6227\" (UID: \"7f71c0c3-f2dd-4f50-ab63-b39881de6227\") "
Jul 20 10:23:47.088921 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:47.088759    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f71c0c3-f2dd-4f50-ab63-b39881de6227-tmp-dir\") pod \"7f71c0c3-f2dd-4f50-ab63-b39881de6227\" (UID: \"7f71c0c3-f2dd-4f50-ab63-b39881de6227\") "
Jul 20 10:23:47.088921 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:47.088813    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f71c0c3-f2dd-4f50-ab63-b39881de6227-model-cache\") pod \"7f71c0c3-f2dd-4f50-ab63-b39881de6227\" (UID: \"7f71c0c3-f2dd-4f50-ab63-b39881de6227\") "
Jul 20 10:23:47.089159 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:47.089061    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f71c0c3-f2dd-4f50-ab63-b39881de6227-home" (OuterVolumeSpecName: "home") pod "7f71c0c3-f2dd-4f50-ab63-b39881de6227" (UID: "7f71c0c3-f2dd-4f50-ab63-b39881de6227"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 10:23:47.089159 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:47.089097    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f71c0c3-f2dd-4f50-ab63-b39881de6227-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7f71c0c3-f2dd-4f50-ab63-b39881de6227" (UID: "7f71c0c3-f2dd-4f50-ab63-b39881de6227"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 10:23:47.089272 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:47.089158    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f71c0c3-f2dd-4f50-ab63-b39881de6227-model-cache" (OuterVolumeSpecName: "model-cache") pod "7f71c0c3-f2dd-4f50-ab63-b39881de6227" (UID: "7f71c0c3-f2dd-4f50-ab63-b39881de6227"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 10:23:47.091260 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:47.091225    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f71c0c3-f2dd-4f50-ab63-b39881de6227-dshm" (OuterVolumeSpecName: "dshm") pod "7f71c0c3-f2dd-4f50-ab63-b39881de6227" (UID: "7f71c0c3-f2dd-4f50-ab63-b39881de6227"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 10:23:47.091260 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:47.091245    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f71c0c3-f2dd-4f50-ab63-b39881de6227-kube-api-access-z9s4h" (OuterVolumeSpecName: "kube-api-access-z9s4h") pod "7f71c0c3-f2dd-4f50-ab63-b39881de6227" (UID: "7f71c0c3-f2dd-4f50-ab63-b39881de6227"). InnerVolumeSpecName "kube-api-access-z9s4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 10:23:47.091437 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:47.091306    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f71c0c3-f2dd-4f50-ab63-b39881de6227-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7f71c0c3-f2dd-4f50-ab63-b39881de6227" (UID: "7f71c0c3-f2dd-4f50-ab63-b39881de6227"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 10:23:47.189958 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:47.189916    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f71c0c3-f2dd-4f50-ab63-b39881de6227-home\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:23:47.189958 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:47.189950    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9s4h\" (UniqueName: \"kubernetes.io/projected/7f71c0c3-f2dd-4f50-ab63-b39881de6227-kube-api-access-z9s4h\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:23:47.189958 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:47.189961    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f71c0c3-f2dd-4f50-ab63-b39881de6227-tmp-dir\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:23:47.190228 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:47.190003    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f71c0c3-f2dd-4f50-ab63-b39881de6227-model-cache\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:23:47.190228 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:47.190019    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f71c0c3-f2dd-4f50-ab63-b39881de6227-dshm\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:23:47.190228 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:47.190028    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f71c0c3-f2dd-4f50-ab63-b39881de6227-tls-certs\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:23:47.743515 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:47.743473    2566 generic.go:356] "Generic (PLEG): container finished" podID="7f71c0c3-f2dd-4f50-ab63-b39881de6227" containerID="879ff339eb9a40cd39adc10e5330a36c38ad6ae91f1b9debd2a1558277b45c6a" exitCode=0
Jul 20 10:23:47.743515 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:47.743514    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b7ed4c/sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b"
Jul 20 10:23:47.744022 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:47.743563    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b7ed4c/sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b" event={"ID":"7f71c0c3-f2dd-4f50-ab63-b39881de6227","Type":"ContainerDied","Data":"879ff339eb9a40cd39adc10e5330a36c38ad6ae91f1b9debd2a1558277b45c6a"}
Jul 20 10:23:47.744022 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:47.743605    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b7ed4c/sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b" event={"ID":"7f71c0c3-f2dd-4f50-ab63-b39881de6227","Type":"ContainerDied","Data":"c8bbe37ea4ce30ab708e05d0c26f402fac314e4bc348d9c12a3a2b98ecb6808c"}
Jul 20 10:23:47.744022 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:47.743627    2566 scope.go:117] "RemoveContainer" containerID="879ff339eb9a40cd39adc10e5330a36c38ad6ae91f1b9debd2a1558277b45c6a"
Jul 20 10:23:47.754145 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:47.754117    2566 scope.go:117] "RemoveContainer" containerID="879ff339eb9a40cd39adc10e5330a36c38ad6ae91f1b9debd2a1558277b45c6a"
Jul 20 10:23:47.754560 ip-10-0-140-134 kubenswrapper[2566]: E0720 10:23:47.754536    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"879ff339eb9a40cd39adc10e5330a36c38ad6ae91f1b9debd2a1558277b45c6a\": container with ID starting with 879ff339eb9a40cd39adc10e5330a36c38ad6ae91f1b9debd2a1558277b45c6a not found: ID does not exist" containerID="879ff339eb9a40cd39adc10e5330a36c38ad6ae91f1b9debd2a1558277b45c6a"
Jul 20 10:23:47.754646 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:47.754574    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"879ff339eb9a40cd39adc10e5330a36c38ad6ae91f1b9debd2a1558277b45c6a"} err="failed to get container status \"879ff339eb9a40cd39adc10e5330a36c38ad6ae91f1b9debd2a1558277b45c6a\": rpc error: code = NotFound desc = could not find container \"879ff339eb9a40cd39adc10e5330a36c38ad6ae91f1b9debd2a1558277b45c6a\": container with ID starting with 879ff339eb9a40cd39adc10e5330a36c38ad6ae91f1b9debd2a1558277b45c6a not found: ID does not exist"
Jul 20 10:23:47.765052 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:47.765002    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b7ed4c/sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b"]
Jul 20 10:23:47.768633 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:47.768604    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-b7ed4c/sub-test-model-b7ed4c-kserve-7d94b8fbf-klk8b"]
Jul 20 10:23:47.921633 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:47.921592    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f71c0c3-f2dd-4f50-ab63-b39881de6227" path="/var/lib/kubelet/pods/7f71c0c3-f2dd-4f50-ab63-b39881de6227/volumes"
Jul 20 10:23:53.122578 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:53.122538    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-b7ed4c/rate-test-model-b7ed4c-kserve-84bc55d547-m458p"
Jul 20 10:23:53.123014 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:53.122597    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-b7ed4c/rate-test-model-b7ed4c-kserve-84bc55d547-m458p"
Jul 20 10:23:53.135324 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:53.135295    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-b7ed4c/rate-test-model-b7ed4c-kserve-84bc55d547-m458p"
Jul 20 10:23:53.785455 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:53.785415    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-b7ed4c/rate-test-model-b7ed4c-kserve-84bc55d547-m458p"
Jul 20 10:23:54.301350 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:54.301311    2566 scope.go:117] "RemoveContainer" containerID="78a7859d8775b86a10121e6afe72d31d6afb8860de8e3acfc26145fc576a90d0"
Jul 20 10:23:54.912120 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:23:54.912080    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:24:16.103901 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:24:16.103860    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:24:30.408022 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:24:30.407953    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:24:51.708339 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:24:51.708292    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:24:58.645100 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:24:58.645047    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b7ed4c/rate-test-model-b7ed4c-kserve-84bc55d547-m458p"]
Jul 20 10:25:00.635186 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:00.635154    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7659-140489860367536"]
Jul 20 10:25:00.635589 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:00.635473    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f71c0c3-f2dd-4f50-ab63-b39881de6227" containerName="main"
Jul 20 10:25:00.635589 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:00.635483    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f71c0c3-f2dd-4f50-ab63-b39881de6227" containerName="main"
Jul 20 10:25:00.635589 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:00.635547    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f71c0c3-f2dd-4f50-ab63-b39881de6227" containerName="main"
Jul 20 10:25:00.638725 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:00.638703    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-140489860367536"
Jul 20 10:25:00.641421 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:00.641395    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-hggv6\""
Jul 20 10:25:00.643638 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:00.643610    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7659-140489860367536"]
Jul 20 10:25:00.732571 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:00.732523    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmz6f\" (UniqueName: \"kubernetes.io/projected/11e788bf-c50a-4d1a-a27c-0a8144086e1f-kube-api-access-lmz6f\") pod \"test-curl-7659-140489860367536\" (UID: \"11e788bf-c50a-4d1a-a27c-0a8144086e1f\") " pod="opendatahub/test-curl-7659-140489860367536"
Jul 20 10:25:00.833833 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:00.833795    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmz6f\" (UniqueName: \"kubernetes.io/projected/11e788bf-c50a-4d1a-a27c-0a8144086e1f-kube-api-access-lmz6f\") pod \"test-curl-7659-140489860367536\" (UID: \"11e788bf-c50a-4d1a-a27c-0a8144086e1f\") " pod="opendatahub/test-curl-7659-140489860367536"
Jul 20 10:25:00.843479 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:00.843447    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmz6f\" (UniqueName: \"kubernetes.io/projected/11e788bf-c50a-4d1a-a27c-0a8144086e1f-kube-api-access-lmz6f\") pod \"test-curl-7659-140489860367536\" (UID: \"11e788bf-c50a-4d1a-a27c-0a8144086e1f\") " pod="opendatahub/test-curl-7659-140489860367536"
Jul 20 10:25:00.950877 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:00.950769    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-140489860367536"
Jul 20 10:25:01.087431 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:01.087404    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7659-140489860367536"]
Jul 20 10:25:01.089393 ip-10-0-140-134 kubenswrapper[2566]: W0720 10:25:01.089360    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod11e788bf_c50a_4d1a_a27c_0a8144086e1f.slice/crio-83353fb415483a48333ade674667c8f4dbc6dece2b807dcad090a5904d663d8d WatchSource:0}: Error finding container 83353fb415483a48333ade674667c8f4dbc6dece2b807dcad090a5904d663d8d: Status 404 returned error can't find the container with id 83353fb415483a48333ade674667c8f4dbc6dece2b807dcad090a5904d663d8d
Jul 20 10:25:01.090697 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:01.090678    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 20 10:25:01.097894 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:01.097865    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7659-140489860367536" event={"ID":"11e788bf-c50a-4d1a-a27c-0a8144086e1f","Type":"ContainerStarted","Data":"83353fb415483a48333ade674667c8f4dbc6dece2b807dcad090a5904d663d8d"}
Jul 20 10:25:02.103875 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:02.103838    2566 generic.go:356] "Generic (PLEG): container finished" podID="11e788bf-c50a-4d1a-a27c-0a8144086e1f" containerID="67632bc390bca36f4e058e626d2d2cd2a0658e4ec9e1fe530a8ba6624130b960" exitCode=0
Jul 20 10:25:02.104279 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:02.103929    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7659-140489860367536" event={"ID":"11e788bf-c50a-4d1a-a27c-0a8144086e1f","Type":"ContainerDied","Data":"67632bc390bca36f4e058e626d2d2cd2a0658e4ec9e1fe530a8ba6624130b960"}
Jul 20 10:25:03.250833 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:03.250807    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-140489860367536"
Jul 20 10:25:03.286742 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:03.286710    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7659-140489860367536_11e788bf-c50a-4d1a-a27c-0a8144086e1f/test-curl-7659-140489860367536/0.log"
Jul 20 10:25:03.310267 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:03.310221    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7659-140489860367536"]
Jul 20 10:25:03.313686 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:03.313652    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7659-140489860367536"]
Jul 20 10:25:03.356543 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:03.356510    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lmz6f\" (UniqueName: \"kubernetes.io/projected/11e788bf-c50a-4d1a-a27c-0a8144086e1f-kube-api-access-lmz6f\") pod \"11e788bf-c50a-4d1a-a27c-0a8144086e1f\" (UID: \"11e788bf-c50a-4d1a-a27c-0a8144086e1f\") "
Jul 20 10:25:03.359065 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:03.358996    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11e788bf-c50a-4d1a-a27c-0a8144086e1f-kube-api-access-lmz6f" (OuterVolumeSpecName: "kube-api-access-lmz6f") pod "11e788bf-c50a-4d1a-a27c-0a8144086e1f" (UID: "11e788bf-c50a-4d1a-a27c-0a8144086e1f"). InnerVolumeSpecName "kube-api-access-lmz6f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 10:25:03.445199 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:03.445167    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7659-140489860364208"]
Jul 20 10:25:03.445514 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:03.445500    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11e788bf-c50a-4d1a-a27c-0a8144086e1f" containerName="test-curl-7659-140489860367536"
Jul 20 10:25:03.445557 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:03.445515    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="11e788bf-c50a-4d1a-a27c-0a8144086e1f" containerName="test-curl-7659-140489860367536"
Jul 20 10:25:03.445592 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:03.445576    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="11e788bf-c50a-4d1a-a27c-0a8144086e1f" containerName="test-curl-7659-140489860367536"
Jul 20 10:25:03.448473 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:03.448455    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-140489860364208"
Jul 20 10:25:03.454333 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:03.454304    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7659-140489860364208"]
Jul 20 10:25:03.457383 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:03.457361    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmz6f\" (UniqueName: \"kubernetes.io/projected/11e788bf-c50a-4d1a-a27c-0a8144086e1f-kube-api-access-lmz6f\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:25:03.558275 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:03.558233    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxvx6\" (UniqueName: \"kubernetes.io/projected/7ec6210c-168b-43fb-b0ed-22d4991f30b6-kube-api-access-pxvx6\") pod \"test-curl-7659-140489860364208\" (UID: \"7ec6210c-168b-43fb-b0ed-22d4991f30b6\") " pod="opendatahub/test-curl-7659-140489860364208"
Jul 20 10:25:03.659378 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:03.659271    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxvx6\" (UniqueName: \"kubernetes.io/projected/7ec6210c-168b-43fb-b0ed-22d4991f30b6-kube-api-access-pxvx6\") pod \"test-curl-7659-140489860364208\" (UID: \"7ec6210c-168b-43fb-b0ed-22d4991f30b6\") " pod="opendatahub/test-curl-7659-140489860364208"
Jul 20 10:25:03.668038 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:03.668001    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxvx6\" (UniqueName: \"kubernetes.io/projected/7ec6210c-168b-43fb-b0ed-22d4991f30b6-kube-api-access-pxvx6\") pod \"test-curl-7659-140489860364208\" (UID: \"7ec6210c-168b-43fb-b0ed-22d4991f30b6\") " pod="opendatahub/test-curl-7659-140489860364208"
Jul 20 10:25:03.760517 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:03.760475    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-140489860364208"
Jul 20 10:25:03.900086 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:03.900034    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7659-140489860364208"]
Jul 20 10:25:03.906443 ip-10-0-140-134 kubenswrapper[2566]: W0720 10:25:03.906410    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7ec6210c_168b_43fb_b0ed_22d4991f30b6.slice/crio-73c08e152b1a69d22103d1ef70792e8aea52d4352622c327e75a42d0a05bd596 WatchSource:0}: Error finding container 73c08e152b1a69d22103d1ef70792e8aea52d4352622c327e75a42d0a05bd596: Status 404 returned error can't find the container with id 73c08e152b1a69d22103d1ef70792e8aea52d4352622c327e75a42d0a05bd596
Jul 20 10:25:03.920348 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:03.920265    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11e788bf-c50a-4d1a-a27c-0a8144086e1f" path="/var/lib/kubelet/pods/11e788bf-c50a-4d1a-a27c-0a8144086e1f/volumes"
Jul 20 10:25:04.114641 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:04.114607    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-140489860367536"
Jul 20 10:25:04.114641 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:04.114626    2566 scope.go:117] "RemoveContainer" containerID="67632bc390bca36f4e058e626d2d2cd2a0658e4ec9e1fe530a8ba6624130b960"
Jul 20 10:25:04.116032 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:04.116005    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7659-140489860364208" event={"ID":"7ec6210c-168b-43fb-b0ed-22d4991f30b6","Type":"ContainerStarted","Data":"73c08e152b1a69d22103d1ef70792e8aea52d4352622c327e75a42d0a05bd596"}
Jul 20 10:25:05.123286 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:05.123247    2566 generic.go:356] "Generic (PLEG): container finished" podID="7ec6210c-168b-43fb-b0ed-22d4991f30b6" containerID="ae6ba988c171f420db7fdd09538642c29cafc50a3f3d8b71138d2cfc2667ef0f" exitCode=0
Jul 20 10:25:05.123674 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:05.123301    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7659-140489860364208" event={"ID":"7ec6210c-168b-43fb-b0ed-22d4991f30b6","Type":"ContainerDied","Data":"ae6ba988c171f420db7fdd09538642c29cafc50a3f3d8b71138d2cfc2667ef0f"}
Jul 20 10:25:06.266129 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:06.266103    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-140489860364208"
Jul 20 10:25:06.289233 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:06.289196    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7659-140489860364208_7ec6210c-168b-43fb-b0ed-22d4991f30b6/test-curl-7659-140489860364208/0.log"
Jul 20 10:25:06.318051 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:06.318017    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7659-140489860364208"]
Jul 20 10:25:06.323916 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:06.323885    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7659-140489860364208"]
Jul 20 10:25:06.383429 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:06.383344    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pxvx6\" (UniqueName: \"kubernetes.io/projected/7ec6210c-168b-43fb-b0ed-22d4991f30b6-kube-api-access-pxvx6\") pod \"7ec6210c-168b-43fb-b0ed-22d4991f30b6\" (UID: \"7ec6210c-168b-43fb-b0ed-22d4991f30b6\") "
Jul 20 10:25:06.385688 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:06.385658    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ec6210c-168b-43fb-b0ed-22d4991f30b6-kube-api-access-pxvx6" (OuterVolumeSpecName: "kube-api-access-pxvx6") pod "7ec6210c-168b-43fb-b0ed-22d4991f30b6" (UID: "7ec6210c-168b-43fb-b0ed-22d4991f30b6"). InnerVolumeSpecName "kube-api-access-pxvx6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 10:25:06.484285 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:06.484244    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxvx6\" (UniqueName: \"kubernetes.io/projected/7ec6210c-168b-43fb-b0ed-22d4991f30b6-kube-api-access-pxvx6\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:25:06.542022 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:06.541989    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7659-140489860389024"]
Jul 20 10:25:06.542339 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:06.542325    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ec6210c-168b-43fb-b0ed-22d4991f30b6" containerName="test-curl-7659-140489860364208"
Jul 20 10:25:06.542395 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:06.542340    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ec6210c-168b-43fb-b0ed-22d4991f30b6" containerName="test-curl-7659-140489860364208"
Jul 20 10:25:06.542431 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:06.542405    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ec6210c-168b-43fb-b0ed-22d4991f30b6" containerName="test-curl-7659-140489860364208"
Jul 20 10:25:06.545313 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:06.545294    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-140489860389024"
Jul 20 10:25:06.550124 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:06.550095    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7659-140489860389024"]
Jul 20 10:25:06.685434 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:06.685341    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbpvc\" (UniqueName: \"kubernetes.io/projected/10f8147d-9676-46ea-82da-31dda3dbea3e-kube-api-access-cbpvc\") pod \"test-curl-7659-140489860389024\" (UID: \"10f8147d-9676-46ea-82da-31dda3dbea3e\") " pod="opendatahub/test-curl-7659-140489860389024"
Jul 20 10:25:06.786152 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:06.786115    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbpvc\" (UniqueName: \"kubernetes.io/projected/10f8147d-9676-46ea-82da-31dda3dbea3e-kube-api-access-cbpvc\") pod \"test-curl-7659-140489860389024\" (UID: \"10f8147d-9676-46ea-82da-31dda3dbea3e\") " pod="opendatahub/test-curl-7659-140489860389024"
Jul 20 10:25:06.794504 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:06.794475    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbpvc\" (UniqueName: \"kubernetes.io/projected/10f8147d-9676-46ea-82da-31dda3dbea3e-kube-api-access-cbpvc\") pod \"test-curl-7659-140489860389024\" (UID: \"10f8147d-9676-46ea-82da-31dda3dbea3e\") " pod="opendatahub/test-curl-7659-140489860389024"
Jul 20 10:25:06.820427 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:06.820391    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:25:06.857100 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:06.857047    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-140489860389024"
Jul 20 10:25:06.997225 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:06.997198    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7659-140489860389024"]
Jul 20 10:25:06.999434 ip-10-0-140-134 kubenswrapper[2566]: W0720 10:25:06.999409    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod10f8147d_9676_46ea_82da_31dda3dbea3e.slice/crio-b10445e15d5e94c55cf231dbe426723b6006a8648a2cdf04f4ed4738ad21bfba WatchSource:0}: Error finding container b10445e15d5e94c55cf231dbe426723b6006a8648a2cdf04f4ed4738ad21bfba: Status 404 returned error can't find the container with id b10445e15d5e94c55cf231dbe426723b6006a8648a2cdf04f4ed4738ad21bfba
Jul 20 10:25:07.133385 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:07.133339    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7659-140489860389024" event={"ID":"10f8147d-9676-46ea-82da-31dda3dbea3e","Type":"ContainerStarted","Data":"b10445e15d5e94c55cf231dbe426723b6006a8648a2cdf04f4ed4738ad21bfba"}
Jul 20 10:25:07.134708 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:07.134688    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="73c08e152b1a69d22103d1ef70792e8aea52d4352622c327e75a42d0a05bd596"
Jul 20 10:25:07.134828 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:07.134743    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-140489860364208"
Jul 20 10:25:07.145456 ip-10-0-140-134 kubenswrapper[2566]: E0720 10:25:07.145424    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7659-140489860364208\" is forbidden: User \"system:node:ip-10-0-140-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-134.ec2.internal' and this object" podUID="7ec6210c-168b-43fb-b0ed-22d4991f30b6" pod="opendatahub/test-curl-7659-140489860364208"
Jul 20 10:25:07.920448 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:07.920410    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ec6210c-168b-43fb-b0ed-22d4991f30b6" path="/var/lib/kubelet/pods/7ec6210c-168b-43fb-b0ed-22d4991f30b6/volumes"
Jul 20 10:25:08.140219 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:08.140182    2566 generic.go:356] "Generic (PLEG): container finished" podID="10f8147d-9676-46ea-82da-31dda3dbea3e" containerID="d3a37747e9e3902cfad90166edb835dda6aed766d21c99aa5abf439e96a59dee" exitCode=0
Jul 20 10:25:08.140389 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:08.140265    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7659-140489860389024" event={"ID":"10f8147d-9676-46ea-82da-31dda3dbea3e","Type":"ContainerDied","Data":"d3a37747e9e3902cfad90166edb835dda6aed766d21c99aa5abf439e96a59dee"}
Jul 20 10:25:09.284151 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:09.284125    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-140489860389024"
Jul 20 10:25:09.304792 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:09.304764    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7659-140489860389024_10f8147d-9676-46ea-82da-31dda3dbea3e/test-curl-7659-140489860389024/0.log"
Jul 20 10:25:09.334213 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:09.334173    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7659-140489860389024"]
Jul 20 10:25:09.338040 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:09.338011    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7659-140489860389024"]
Jul 20 10:25:09.409338 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:09.409248    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbpvc\" (UniqueName: \"kubernetes.io/projected/10f8147d-9676-46ea-82da-31dda3dbea3e-kube-api-access-cbpvc\") pod \"10f8147d-9676-46ea-82da-31dda3dbea3e\" (UID: \"10f8147d-9676-46ea-82da-31dda3dbea3e\") "
Jul 20 10:25:09.411528 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:09.411492    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10f8147d-9676-46ea-82da-31dda3dbea3e-kube-api-access-cbpvc" (OuterVolumeSpecName: "kube-api-access-cbpvc") pod "10f8147d-9676-46ea-82da-31dda3dbea3e" (UID: "10f8147d-9676-46ea-82da-31dda3dbea3e"). InnerVolumeSpecName "kube-api-access-cbpvc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 10:25:09.470574 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:09.470533    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7659-140489860389024"]
Jul 20 10:25:09.470875 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:09.470862    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10f8147d-9676-46ea-82da-31dda3dbea3e" containerName="test-curl-7659-140489860389024"
Jul 20 10:25:09.470930 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:09.470877    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="10f8147d-9676-46ea-82da-31dda3dbea3e" containerName="test-curl-7659-140489860389024"
Jul 20 10:25:09.470993 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:09.470939    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="10f8147d-9676-46ea-82da-31dda3dbea3e" containerName="test-curl-7659-140489860389024"
Jul 20 10:25:09.473802 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:09.473782    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-140489860389024"
Jul 20 10:25:09.479405 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:09.479372    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7659-140489860389024"]
Jul 20 10:25:09.510096 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:09.510054    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbpvc\" (UniqueName: \"kubernetes.io/projected/10f8147d-9676-46ea-82da-31dda3dbea3e-kube-api-access-cbpvc\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:25:09.610528 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:09.610484    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkzb5\" (UniqueName: \"kubernetes.io/projected/7c4c3c79-ba45-42b7-b649-44dbaf7db4ba-kube-api-access-lkzb5\") pod \"test-curl-7659-140489860389024\" (UID: \"7c4c3c79-ba45-42b7-b649-44dbaf7db4ba\") " pod="opendatahub/test-curl-7659-140489860389024"
Jul 20 10:25:09.711250 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:09.711147    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkzb5\" (UniqueName: \"kubernetes.io/projected/7c4c3c79-ba45-42b7-b649-44dbaf7db4ba-kube-api-access-lkzb5\") pod \"test-curl-7659-140489860389024\" (UID: \"7c4c3c79-ba45-42b7-b649-44dbaf7db4ba\") " pod="opendatahub/test-curl-7659-140489860389024"
Jul 20 10:25:09.720118 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:09.720085    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkzb5\" (UniqueName: \"kubernetes.io/projected/7c4c3c79-ba45-42b7-b649-44dbaf7db4ba-kube-api-access-lkzb5\") pod \"test-curl-7659-140489860389024\" (UID: \"7c4c3c79-ba45-42b7-b649-44dbaf7db4ba\") " pod="opendatahub/test-curl-7659-140489860389024"
Jul 20 10:25:09.784117 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:09.784080    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-140489860389024"
Jul 20 10:25:09.920690 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:09.920659    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10f8147d-9676-46ea-82da-31dda3dbea3e" path="/var/lib/kubelet/pods/10f8147d-9676-46ea-82da-31dda3dbea3e/volumes"
Jul 20 10:25:09.930788 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:09.930768    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7659-140489860389024"]
Jul 20 10:25:09.932674 ip-10-0-140-134 kubenswrapper[2566]: W0720 10:25:09.932649    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7c4c3c79_ba45_42b7_b649_44dbaf7db4ba.slice/crio-9e48df1988a60477b677f766ad5ee38696990c4b0c2539e399eace028ac8f923 WatchSource:0}: Error finding container 9e48df1988a60477b677f766ad5ee38696990c4b0c2539e399eace028ac8f923: Status 404 returned error can't find the container with id 9e48df1988a60477b677f766ad5ee38696990c4b0c2539e399eace028ac8f923
Jul 20 10:25:10.151749 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:10.151718    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-140489860389024"
Jul 20 10:25:10.151948 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:10.151724    2566 scope.go:117] "RemoveContainer" containerID="d3a37747e9e3902cfad90166edb835dda6aed766d21c99aa5abf439e96a59dee"
Jul 20 10:25:10.153128 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:10.153007    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7659-140489860389024" event={"ID":"7c4c3c79-ba45-42b7-b649-44dbaf7db4ba","Type":"ContainerStarted","Data":"9e48df1988a60477b677f766ad5ee38696990c4b0c2539e399eace028ac8f923"}
Jul 20 10:25:11.161275 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:11.161231    2566 generic.go:356] "Generic (PLEG): container finished" podID="7c4c3c79-ba45-42b7-b649-44dbaf7db4ba" containerID="b809dd94110452a16f82b664c557e554ae278dca58ae571bbff6945d94289118" exitCode=0
Jul 20 10:25:11.161660 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:11.161310    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7659-140489860389024" event={"ID":"7c4c3c79-ba45-42b7-b649-44dbaf7db4ba","Type":"ContainerDied","Data":"b809dd94110452a16f82b664c557e554ae278dca58ae571bbff6945d94289118"}
Jul 20 10:25:12.307827 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:12.307783    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-140489860389024"
Jul 20 10:25:12.332561 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:12.332519    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7659-140489860389024_7c4c3c79-ba45-42b7-b649-44dbaf7db4ba/test-curl-7659-140489860389024/0.log"
Jul 20 10:25:12.359614 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:12.359572    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7659-140489860389024"]
Jul 20 10:25:12.364048 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:12.364017    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7659-140489860389024"]
Jul 20 10:25:12.435932 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:12.435827    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lkzb5\" (UniqueName: \"kubernetes.io/projected/7c4c3c79-ba45-42b7-b649-44dbaf7db4ba-kube-api-access-lkzb5\") pod \"7c4c3c79-ba45-42b7-b649-44dbaf7db4ba\" (UID: \"7c4c3c79-ba45-42b7-b649-44dbaf7db4ba\") "
Jul 20 10:25:12.438166 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:12.438136    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c4c3c79-ba45-42b7-b649-44dbaf7db4ba-kube-api-access-lkzb5" (OuterVolumeSpecName: "kube-api-access-lkzb5") pod "7c4c3c79-ba45-42b7-b649-44dbaf7db4ba" (UID: "7c4c3c79-ba45-42b7-b649-44dbaf7db4ba"). InnerVolumeSpecName "kube-api-access-lkzb5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 10:25:12.485063 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:12.485020    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7659-140489860390032"]
Jul 20 10:25:12.485392 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:12.485378    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c4c3c79-ba45-42b7-b649-44dbaf7db4ba" containerName="test-curl-7659-140489860389024"
Jul 20 10:25:12.485442 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:12.485394    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c4c3c79-ba45-42b7-b649-44dbaf7db4ba" containerName="test-curl-7659-140489860389024"
Jul 20 10:25:12.485482 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:12.485463    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c4c3c79-ba45-42b7-b649-44dbaf7db4ba" containerName="test-curl-7659-140489860389024"
Jul 20 10:25:12.493559 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:12.493522    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-140489860390032"
Jul 20 10:25:12.495244 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:12.495212    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7659-140489860390032"]
Jul 20 10:25:12.536742 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:12.536702    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lkzb5\" (UniqueName: \"kubernetes.io/projected/7c4c3c79-ba45-42b7-b649-44dbaf7db4ba-kube-api-access-lkzb5\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:25:12.637524 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:12.637483    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsbhz\" (UniqueName: \"kubernetes.io/projected/f50bc131-8db4-4366-8cf1-34f56deafcf3-kube-api-access-nsbhz\") pod \"test-curl-7659-140489860390032\" (UID: \"f50bc131-8db4-4366-8cf1-34f56deafcf3\") " pod="opendatahub/test-curl-7659-140489860390032"
Jul 20 10:25:12.739079 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:12.738953    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsbhz\" (UniqueName: \"kubernetes.io/projected/f50bc131-8db4-4366-8cf1-34f56deafcf3-kube-api-access-nsbhz\") pod \"test-curl-7659-140489860390032\" (UID: \"f50bc131-8db4-4366-8cf1-34f56deafcf3\") " pod="opendatahub/test-curl-7659-140489860390032"
Jul 20 10:25:12.748196 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:12.748164    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsbhz\" (UniqueName: \"kubernetes.io/projected/f50bc131-8db4-4366-8cf1-34f56deafcf3-kube-api-access-nsbhz\") pod \"test-curl-7659-140489860390032\" (UID: \"f50bc131-8db4-4366-8cf1-34f56deafcf3\") " pod="opendatahub/test-curl-7659-140489860390032"
Jul 20 10:25:12.804079 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:12.804027    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-140489860390032"
Jul 20 10:25:12.942707 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:12.942669    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7659-140489860390032"]
Jul 20 10:25:12.945014 ip-10-0-140-134 kubenswrapper[2566]: W0720 10:25:12.944960    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf50bc131_8db4_4366_8cf1_34f56deafcf3.slice/crio-8f2fa1110379e86f129b0ba33eb63ff9da73f365593fe5a625ce01aecd1c4fe5 WatchSource:0}: Error finding container 8f2fa1110379e86f129b0ba33eb63ff9da73f365593fe5a625ce01aecd1c4fe5: Status 404 returned error can't find the container with id 8f2fa1110379e86f129b0ba33eb63ff9da73f365593fe5a625ce01aecd1c4fe5
Jul 20 10:25:13.174308 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:13.174273    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9e48df1988a60477b677f766ad5ee38696990c4b0c2539e399eace028ac8f923"
Jul 20 10:25:13.174308 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:13.174280    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-140489860389024"
Jul 20 10:25:13.175748 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:13.175717    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7659-140489860390032" event={"ID":"f50bc131-8db4-4366-8cf1-34f56deafcf3","Type":"ContainerStarted","Data":"8f2fa1110379e86f129b0ba33eb63ff9da73f365593fe5a625ce01aecd1c4fe5"}
Jul 20 10:25:13.185475 ip-10-0-140-134 kubenswrapper[2566]: E0720 10:25:13.185438    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7659-140489860389024\" is forbidden: User \"system:node:ip-10-0-140-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-134.ec2.internal' and this object" podUID="7c4c3c79-ba45-42b7-b649-44dbaf7db4ba" pod="opendatahub/test-curl-7659-140489860389024"
Jul 20 10:25:13.671615 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:13.671567    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-b7ed4c/rate-test-model-b7ed4c-kserve-84bc55d547-m458p" podUID="7ece574b-b6f9-4562-93e4-f557bfe0da88" containerName="main" containerID="cri-o://32693ccf92ff6a172232ff91429cbcb2ee39cd6c46cd3f6c35407f49192bc657" gracePeriod=45
Jul 20 10:25:13.774823 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:13.774773    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-a-b7ed4c/rate-test-model-b7ed4c-kserve-84bc55d547-m458p" podUID="7ece574b-b6f9-4562-93e4-f557bfe0da88" containerName="main" probeResult="failure" output="Get \"https://10.133.0.58:8000/ready\": dial tcp 10.133.0.58:8000: connect: connection refused"
Jul 20 10:25:13.919660 ip-10-0-140-134 kubenswrapper[2566]: E0720 10:25:13.919615    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7659-140489860389024\" is forbidden: User \"system:node:ip-10-0-140-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-134.ec2.internal' and this object" podUID="7c4c3c79-ba45-42b7-b649-44dbaf7db4ba" pod="opendatahub/test-curl-7659-140489860389024"
Jul 20 10:25:13.920875 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:13.920848    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c4c3c79-ba45-42b7-b649-44dbaf7db4ba" path="/var/lib/kubelet/pods/7c4c3c79-ba45-42b7-b649-44dbaf7db4ba/volumes"
Jul 20 10:25:13.927322 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:13.927234    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b7ed4c/rate-test-model-b7ed4c-kserve-84bc55d547-m458p"
Jul 20 10:25:13.948261 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:13.948226    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ece574b-b6f9-4562-93e4-f557bfe0da88-tls-certs\") pod \"7ece574b-b6f9-4562-93e4-f557bfe0da88\" (UID: \"7ece574b-b6f9-4562-93e4-f557bfe0da88\") "
Jul 20 10:25:13.948423 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:13.948286    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ece574b-b6f9-4562-93e4-f557bfe0da88-tmp-dir\") pod \"7ece574b-b6f9-4562-93e4-f557bfe0da88\" (UID: \"7ece574b-b6f9-4562-93e4-f557bfe0da88\") "
Jul 20 10:25:13.948423 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:13.948346    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tvqnh\" (UniqueName: \"kubernetes.io/projected/7ece574b-b6f9-4562-93e4-f557bfe0da88-kube-api-access-tvqnh\") pod \"7ece574b-b6f9-4562-93e4-f557bfe0da88\" (UID: \"7ece574b-b6f9-4562-93e4-f557bfe0da88\") "
Jul 20 10:25:13.948423 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:13.948379    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ece574b-b6f9-4562-93e4-f557bfe0da88-model-cache\") pod \"7ece574b-b6f9-4562-93e4-f557bfe0da88\" (UID: \"7ece574b-b6f9-4562-93e4-f557bfe0da88\") "
Jul 20 10:25:13.948423 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:13.948408    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ece574b-b6f9-4562-93e4-f557bfe0da88-home\") pod \"7ece574b-b6f9-4562-93e4-f557bfe0da88\" (UID: \"7ece574b-b6f9-4562-93e4-f557bfe0da88\") "
Jul 20 10:25:13.948656 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:13.948439    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ece574b-b6f9-4562-93e4-f557bfe0da88-dshm\") pod \"7ece574b-b6f9-4562-93e4-f557bfe0da88\" (UID: \"7ece574b-b6f9-4562-93e4-f557bfe0da88\") "
Jul 20 10:25:13.948656 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:13.948580    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ece574b-b6f9-4562-93e4-f557bfe0da88-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7ece574b-b6f9-4562-93e4-f557bfe0da88" (UID: "7ece574b-b6f9-4562-93e4-f557bfe0da88"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 10:25:13.948750 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:13.948728    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ece574b-b6f9-4562-93e4-f557bfe0da88-tmp-dir\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:25:13.949021 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:13.948845    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ece574b-b6f9-4562-93e4-f557bfe0da88-model-cache" (OuterVolumeSpecName: "model-cache") pod "7ece574b-b6f9-4562-93e4-f557bfe0da88" (UID: "7ece574b-b6f9-4562-93e4-f557bfe0da88"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 10:25:13.949124 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:13.949095    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ece574b-b6f9-4562-93e4-f557bfe0da88-home" (OuterVolumeSpecName: "home") pod "7ece574b-b6f9-4562-93e4-f557bfe0da88" (UID: "7ece574b-b6f9-4562-93e4-f557bfe0da88"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 10:25:13.951055 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:13.951026    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ece574b-b6f9-4562-93e4-f557bfe0da88-dshm" (OuterVolumeSpecName: "dshm") pod "7ece574b-b6f9-4562-93e4-f557bfe0da88" (UID: "7ece574b-b6f9-4562-93e4-f557bfe0da88"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 20 10:25:13.951312 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:13.951285    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ece574b-b6f9-4562-93e4-f557bfe0da88-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7ece574b-b6f9-4562-93e4-f557bfe0da88" (UID: "7ece574b-b6f9-4562-93e4-f557bfe0da88"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 20 10:25:13.951454 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:13.951427    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ece574b-b6f9-4562-93e4-f557bfe0da88-kube-api-access-tvqnh" (OuterVolumeSpecName: "kube-api-access-tvqnh") pod "7ece574b-b6f9-4562-93e4-f557bfe0da88" (UID: "7ece574b-b6f9-4562-93e4-f557bfe0da88"). InnerVolumeSpecName "kube-api-access-tvqnh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 10:25:14.050034 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:14.049942    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tvqnh\" (UniqueName: \"kubernetes.io/projected/7ece574b-b6f9-4562-93e4-f557bfe0da88-kube-api-access-tvqnh\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:25:14.050034 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:14.050030    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ece574b-b6f9-4562-93e4-f557bfe0da88-model-cache\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:25:14.050034 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:14.050046    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ece574b-b6f9-4562-93e4-f557bfe0da88-home\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:25:14.050292 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:14.050058    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ece574b-b6f9-4562-93e4-f557bfe0da88-dshm\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:25:14.050292 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:14.050069    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ece574b-b6f9-4562-93e4-f557bfe0da88-tls-certs\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:25:14.183444 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:14.183344    2566 generic.go:356] "Generic (PLEG): container finished" podID="f50bc131-8db4-4366-8cf1-34f56deafcf3" containerID="ca8e65fe737a9a4b3b6519b1d1d3552dee8e92b8f27d4d8b2dce60d529a89470" exitCode=0
Jul 20 10:25:14.183444 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:14.183433    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7659-140489860390032" event={"ID":"f50bc131-8db4-4366-8cf1-34f56deafcf3","Type":"ContainerDied","Data":"ca8e65fe737a9a4b3b6519b1d1d3552dee8e92b8f27d4d8b2dce60d529a89470"}
Jul 20 10:25:14.184735 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:14.184703    2566 generic.go:356] "Generic (PLEG): container finished" podID="7ece574b-b6f9-4562-93e4-f557bfe0da88" containerID="32693ccf92ff6a172232ff91429cbcb2ee39cd6c46cd3f6c35407f49192bc657" exitCode=0
Jul 20 10:25:14.184867 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:14.184739    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b7ed4c/rate-test-model-b7ed4c-kserve-84bc55d547-m458p"
Jul 20 10:25:14.184867 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:14.184752    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b7ed4c/rate-test-model-b7ed4c-kserve-84bc55d547-m458p" event={"ID":"7ece574b-b6f9-4562-93e4-f557bfe0da88","Type":"ContainerDied","Data":"32693ccf92ff6a172232ff91429cbcb2ee39cd6c46cd3f6c35407f49192bc657"}
Jul 20 10:25:14.184867 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:14.184792    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b7ed4c/rate-test-model-b7ed4c-kserve-84bc55d547-m458p" event={"ID":"7ece574b-b6f9-4562-93e4-f557bfe0da88","Type":"ContainerDied","Data":"2092ccecd7a76560d06572046f63f3ed385ba37960618a115a5cb18b4003ff2b"}
Jul 20 10:25:14.184867 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:14.184811    2566 scope.go:117] "RemoveContainer" containerID="32693ccf92ff6a172232ff91429cbcb2ee39cd6c46cd3f6c35407f49192bc657"
Jul 20 10:25:14.193695 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:14.193669    2566 scope.go:117] "RemoveContainer" containerID="32693ccf92ff6a172232ff91429cbcb2ee39cd6c46cd3f6c35407f49192bc657"
Jul 20 10:25:14.194056 ip-10-0-140-134 kubenswrapper[2566]: E0720 10:25:14.194036    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32693ccf92ff6a172232ff91429cbcb2ee39cd6c46cd3f6c35407f49192bc657\": container with ID starting with 32693ccf92ff6a172232ff91429cbcb2ee39cd6c46cd3f6c35407f49192bc657 not found: ID does not exist" containerID="32693ccf92ff6a172232ff91429cbcb2ee39cd6c46cd3f6c35407f49192bc657"
Jul 20 10:25:14.194126 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:14.194067    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32693ccf92ff6a172232ff91429cbcb2ee39cd6c46cd3f6c35407f49192bc657"} err="failed to get container status \"32693ccf92ff6a172232ff91429cbcb2ee39cd6c46cd3f6c35407f49192bc657\": rpc error: code = NotFound desc = could not find container \"32693ccf92ff6a172232ff91429cbcb2ee39cd6c46cd3f6c35407f49192bc657\": container with ID starting with 32693ccf92ff6a172232ff91429cbcb2ee39cd6c46cd3f6c35407f49192bc657 not found: ID does not exist"
Jul 20 10:25:14.211624 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:14.211584    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b7ed4c/rate-test-model-b7ed4c-kserve-84bc55d547-m458p"]
Jul 20 10:25:14.215306 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:14.215273    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-b7ed4c/rate-test-model-b7ed4c-kserve-84bc55d547-m458p"]
Jul 20 10:25:15.333947 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:15.333910    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-140489860390032"
Jul 20 10:25:15.353252 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:15.353217    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7659-140489860390032_f50bc131-8db4-4366-8cf1-34f56deafcf3/test-curl-7659-140489860390032/0.log"
Jul 20 10:25:15.361692 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:15.361652    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nsbhz\" (UniqueName: \"kubernetes.io/projected/f50bc131-8db4-4366-8cf1-34f56deafcf3-kube-api-access-nsbhz\") pod \"f50bc131-8db4-4366-8cf1-34f56deafcf3\" (UID: \"f50bc131-8db4-4366-8cf1-34f56deafcf3\") "
Jul 20 10:25:15.364038 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:15.364004    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f50bc131-8db4-4366-8cf1-34f56deafcf3-kube-api-access-nsbhz" (OuterVolumeSpecName: "kube-api-access-nsbhz") pod "f50bc131-8db4-4366-8cf1-34f56deafcf3" (UID: "f50bc131-8db4-4366-8cf1-34f56deafcf3"). InnerVolumeSpecName "kube-api-access-nsbhz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 10:25:15.372500 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:15.372457    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7659-140489860390032"]
Jul 20 10:25:15.378634 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:15.378602    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7659-140489860390032"]
Jul 20 10:25:15.462346 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:15.462249    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nsbhz\" (UniqueName: \"kubernetes.io/projected/f50bc131-8db4-4366-8cf1-34f56deafcf3-kube-api-access-nsbhz\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:25:15.503889 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:15.503832    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7659-140489860390032"]
Jul 20 10:25:15.504210 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:15.504197    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ece574b-b6f9-4562-93e4-f557bfe0da88" containerName="main"
Jul 20 10:25:15.504252 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:15.504213    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ece574b-b6f9-4562-93e4-f557bfe0da88" containerName="main"
Jul 20 10:25:15.504252 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:15.504226    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f50bc131-8db4-4366-8cf1-34f56deafcf3" containerName="test-curl-7659-140489860390032"
Jul 20 10:25:15.504252 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:15.504232    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f50bc131-8db4-4366-8cf1-34f56deafcf3" containerName="test-curl-7659-140489860390032"
Jul 20 10:25:15.504366 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:15.504294    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f50bc131-8db4-4366-8cf1-34f56deafcf3" containerName="test-curl-7659-140489860390032"
Jul 20 10:25:15.504366 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:15.504301    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ece574b-b6f9-4562-93e4-f557bfe0da88" containerName="main"
Jul 20 10:25:15.507209 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:15.507188    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-140489860390032"
Jul 20 10:25:15.521784 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:15.519770    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7659-140489860390032"]
Jul 20 10:25:15.563634 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:15.563589    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72xpw\" (UniqueName: \"kubernetes.io/projected/257e5318-967a-4c4a-a0a3-b3b674216cea-kube-api-access-72xpw\") pod \"test-curl-7659-140489860390032\" (UID: \"257e5318-967a-4c4a-a0a3-b3b674216cea\") " pod="opendatahub/test-curl-7659-140489860390032"
Jul 20 10:25:15.664601 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:15.664551    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72xpw\" (UniqueName: \"kubernetes.io/projected/257e5318-967a-4c4a-a0a3-b3b674216cea-kube-api-access-72xpw\") pod \"test-curl-7659-140489860390032\" (UID: \"257e5318-967a-4c4a-a0a3-b3b674216cea\") " pod="opendatahub/test-curl-7659-140489860390032"
Jul 20 10:25:15.672849 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:15.672821    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72xpw\" (UniqueName: \"kubernetes.io/projected/257e5318-967a-4c4a-a0a3-b3b674216cea-kube-api-access-72xpw\") pod \"test-curl-7659-140489860390032\" (UID: \"257e5318-967a-4c4a-a0a3-b3b674216cea\") " pod="opendatahub/test-curl-7659-140489860390032"
Jul 20 10:25:15.819947 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:15.819905    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-140489860390032"
Jul 20 10:25:15.923065 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:15.923033    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ece574b-b6f9-4562-93e4-f557bfe0da88" path="/var/lib/kubelet/pods/7ece574b-b6f9-4562-93e4-f557bfe0da88/volumes"
Jul 20 10:25:15.923395 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:15.923378    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f50bc131-8db4-4366-8cf1-34f56deafcf3" path="/var/lib/kubelet/pods/f50bc131-8db4-4366-8cf1-34f56deafcf3/volumes"
Jul 20 10:25:15.957838 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:15.957696    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7659-140489860390032"]
Jul 20 10:25:15.960113 ip-10-0-140-134 kubenswrapper[2566]: W0720 10:25:15.960086    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod257e5318_967a_4c4a_a0a3_b3b674216cea.slice/crio-035831a55a7f5641724c83a73bad579f30d664e7a4a2660b453d7af47e6fbecc WatchSource:0}: Error finding container 035831a55a7f5641724c83a73bad579f30d664e7a4a2660b453d7af47e6fbecc: Status 404 returned error can't find the container with id 035831a55a7f5641724c83a73bad579f30d664e7a4a2660b453d7af47e6fbecc
Jul 20 10:25:16.196380 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:16.196283    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7659-140489860390032" event={"ID":"257e5318-967a-4c4a-a0a3-b3b674216cea","Type":"ContainerStarted","Data":"035831a55a7f5641724c83a73bad579f30d664e7a4a2660b453d7af47e6fbecc"}
Jul 20 10:25:16.197532 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:16.197511    2566 scope.go:117] "RemoveContainer" containerID="ca8e65fe737a9a4b3b6519b1d1d3552dee8e92b8f27d4d8b2dce60d529a89470"
Jul 20 10:25:16.197681 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:16.197519    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-140489860390032"
Jul 20 10:25:17.205016 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:17.204961    2566 generic.go:356] "Generic (PLEG): container finished" podID="257e5318-967a-4c4a-a0a3-b3b674216cea" containerID="6b348be51b085817c6ed77423d377c753e6229e1b5814015dbd75438b73ad662" exitCode=0
Jul 20 10:25:17.205378 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:17.205018    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7659-140489860390032" event={"ID":"257e5318-967a-4c4a-a0a3-b3b674216cea","Type":"ContainerDied","Data":"6b348be51b085817c6ed77423d377c753e6229e1b5814015dbd75438b73ad662"}
Jul 20 10:25:18.351851 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:18.351819    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-140489860390032"
Jul 20 10:25:18.371188 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:18.371147    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7659-140489860390032_257e5318-967a-4c4a-a0a3-b3b674216cea/test-curl-7659-140489860390032/0.log"
Jul 20 10:25:18.387395 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:18.387362    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-72xpw\" (UniqueName: \"kubernetes.io/projected/257e5318-967a-4c4a-a0a3-b3b674216cea-kube-api-access-72xpw\") pod \"257e5318-967a-4c4a-a0a3-b3b674216cea\" (UID: \"257e5318-967a-4c4a-a0a3-b3b674216cea\") "
Jul 20 10:25:18.389623 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:18.389589    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/257e5318-967a-4c4a-a0a3-b3b674216cea-kube-api-access-72xpw" (OuterVolumeSpecName: "kube-api-access-72xpw") pod "257e5318-967a-4c4a-a0a3-b3b674216cea" (UID: "257e5318-967a-4c4a-a0a3-b3b674216cea"). InnerVolumeSpecName "kube-api-access-72xpw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 20 10:25:18.395289 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:18.395251    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7659-140489860390032"]
Jul 20 10:25:18.404661 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:18.404629    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7659-140489860390032"]
Jul 20 10:25:18.488551 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:18.488455    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72xpw\" (UniqueName: \"kubernetes.io/projected/257e5318-967a-4c4a-a0a3-b3b674216cea-kube-api-access-72xpw\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:25:18.611251 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:18.611208    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b7ed4c-hdnjd"]
Jul 20 10:25:18.611751 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:18.611731    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="257e5318-967a-4c4a-a0a3-b3b674216cea" containerName="test-curl-7659-140489860390032"
Jul 20 10:25:18.611820 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:18.611756    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="257e5318-967a-4c4a-a0a3-b3b674216cea" containerName="test-curl-7659-140489860390032"
Jul 20 10:25:18.611865 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:18.611856    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="257e5318-967a-4c4a-a0a3-b3b674216cea" containerName="test-curl-7659-140489860390032"
Jul 20 10:25:18.614764 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:18.614744    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b7ed4c-hdnjd"
Jul 20 10:25:18.617490 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:18.617467    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6jlzx\""
Jul 20 10:25:18.617627 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:18.617472    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 10:25:18.621947 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:18.621917    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b7ed4c-hdnjd"]
Jul 20 10:25:18.690420 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:18.690372    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3af2d2bc-af15-45be-a148-1ce196db8e5a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-b7ed4c-hdnjd\" (UID: \"3af2d2bc-af15-45be-a148-1ce196db8e5a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b7ed4c-hdnjd"
Jul 20 10:25:18.791390 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:18.791290    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3af2d2bc-af15-45be-a148-1ce196db8e5a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-b7ed4c-hdnjd\" (UID: \"3af2d2bc-af15-45be-a148-1ce196db8e5a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b7ed4c-hdnjd"
Jul 20 10:25:18.791964 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:18.791944    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3af2d2bc-af15-45be-a148-1ce196db8e5a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-b7ed4c-hdnjd\" (UID: \"3af2d2bc-af15-45be-a148-1ce196db8e5a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b7ed4c-hdnjd"
Jul 20 10:25:18.927426 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:18.927383    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b7ed4c-hdnjd"
Jul 20 10:25:19.070919 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:19.070894    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b7ed4c-hdnjd"]
Jul 20 10:25:19.073147 ip-10-0-140-134 kubenswrapper[2566]: W0720 10:25:19.073104    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3af2d2bc_af15_45be_a148_1ce196db8e5a.slice/crio-8a364c1cf7402535920d54e9b10fe0f6180a96cf040b61f5c3f31856d611f5f0 WatchSource:0}: Error finding container 8a364c1cf7402535920d54e9b10fe0f6180a96cf040b61f5c3f31856d611f5f0: Status 404 returned error can't find the container with id 8a364c1cf7402535920d54e9b10fe0f6180a96cf040b61f5c3f31856d611f5f0
Jul 20 10:25:19.217088 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:19.217053    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7659-140489860390032"
Jul 20 10:25:19.217088 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:19.217084    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="035831a55a7f5641724c83a73bad579f30d664e7a4a2660b453d7af47e6fbecc"
Jul 20 10:25:19.218507 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:19.218467    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b7ed4c-hdnjd" event={"ID":"3af2d2bc-af15-45be-a148-1ce196db8e5a","Type":"ContainerStarted","Data":"8a364c1cf7402535920d54e9b10fe0f6180a96cf040b61f5c3f31856d611f5f0"}
Jul 20 10:25:19.227709 ip-10-0-140-134 kubenswrapper[2566]: E0720 10:25:19.227677    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7659-140489860390032\" is forbidden: User \"system:node:ip-10-0-140-134.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-134.ec2.internal' and this object" podUID="257e5318-967a-4c4a-a0a3-b3b674216cea" pod="opendatahub/test-curl-7659-140489860390032"
Jul 20 10:25:19.920489 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:19.920450    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="257e5318-967a-4c4a-a0a3-b3b674216cea" path="/var/lib/kubelet/pods/257e5318-967a-4c4a-a0a3-b3b674216cea/volumes"
Jul 20 10:25:20.225104 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:20.225066    2566 generic.go:356] "Generic (PLEG): container finished" podID="3af2d2bc-af15-45be-a148-1ce196db8e5a" containerID="501d3a6baf8fecd0132cac48edf25e846b0ec5fe54fb86a86b2261d5aefc62dd" exitCode=0
Jul 20 10:25:20.225305 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:20.225160    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b7ed4c-hdnjd" event={"ID":"3af2d2bc-af15-45be-a148-1ce196db8e5a","Type":"ContainerDied","Data":"501d3a6baf8fecd0132cac48edf25e846b0ec5fe54fb86a86b2261d5aefc62dd"}
Jul 20 10:25:21.369121 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:21.369088    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b7ed4c-hdnjd"
Jul 20 10:25:21.412712 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:21.412674    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3af2d2bc-af15-45be-a148-1ce196db8e5a-maas-api-service-ca\") pod \"3af2d2bc-af15-45be-a148-1ce196db8e5a\" (UID: \"3af2d2bc-af15-45be-a148-1ce196db8e5a\") "
Jul 20 10:25:21.413078 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:21.413052    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3af2d2bc-af15-45be-a148-1ce196db8e5a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3af2d2bc-af15-45be-a148-1ce196db8e5a" (UID: "3af2d2bc-af15-45be-a148-1ce196db8e5a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 10:25:21.513294 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:21.513200    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3af2d2bc-af15-45be-a148-1ce196db8e5a-maas-api-service-ca\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:25:22.235180 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:22.235095    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b7ed4c-hdnjd"
Jul 20 10:25:22.235330 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:22.235094    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b7ed4c-hdnjd" event={"ID":"3af2d2bc-af15-45be-a148-1ce196db8e5a","Type":"ContainerDied","Data":"8a364c1cf7402535920d54e9b10fe0f6180a96cf040b61f5c3f31856d611f5f0"}
Jul 20 10:25:22.235330 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:22.235223    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8a364c1cf7402535920d54e9b10fe0f6180a96cf040b61f5c3f31856d611f5f0"
Jul 20 10:25:31.814576 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:31.814535    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:25:33.695940 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:33.695886    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b7ed4c-hdnjd"]
Jul 20 10:25:33.701415 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:33.701374    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-b7ed4c-hdnjd"]
Jul 20 10:25:33.920723 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:33.920685    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3af2d2bc-af15-45be-a148-1ce196db8e5a" path="/var/lib/kubelet/pods/3af2d2bc-af15-45be-a148-1ce196db8e5a/volumes"
Jul 20 10:25:40.531022 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:40.530982    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-186a7e-ggnkn"]
Jul 20 10:25:40.531406 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:40.531327    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3af2d2bc-af15-45be-a148-1ce196db8e5a" containerName="revoke-keys"
Jul 20 10:25:40.531406 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:40.531337    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3af2d2bc-af15-45be-a148-1ce196db8e5a" containerName="revoke-keys"
Jul 20 10:25:40.531406 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:40.531388    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3af2d2bc-af15-45be-a148-1ce196db8e5a" containerName="revoke-keys"
Jul 20 10:25:40.536400 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:40.536375    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-186a7e-ggnkn"
Jul 20 10:25:40.539411 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:40.539380    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 20 10:25:40.539586 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:40.539381    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6jlzx\""
Jul 20 10:25:40.542764 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:40.542738    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-186a7e-ggnkn"]
Jul 20 10:25:40.682898 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:40.682848    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e81a0e09-04db-42dd-bf3e-e61dc6e4b235-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-186a7e-ggnkn\" (UID: \"e81a0e09-04db-42dd-bf3e-e61dc6e4b235\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-186a7e-ggnkn"
Jul 20 10:25:40.783422 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:40.783320    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e81a0e09-04db-42dd-bf3e-e61dc6e4b235-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-186a7e-ggnkn\" (UID: \"e81a0e09-04db-42dd-bf3e-e61dc6e4b235\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-186a7e-ggnkn"
Jul 20 10:25:40.783926 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:40.783902    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e81a0e09-04db-42dd-bf3e-e61dc6e4b235-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-186a7e-ggnkn\" (UID: \"e81a0e09-04db-42dd-bf3e-e61dc6e4b235\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-186a7e-ggnkn"
Jul 20 10:25:40.850375 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:40.850333    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-186a7e-ggnkn"
Jul 20 10:25:40.996849 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:40.996813    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-186a7e-ggnkn"]
Jul 20 10:25:40.999260 ip-10-0-140-134 kubenswrapper[2566]: W0720 10:25:40.999227    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode81a0e09_04db_42dd_bf3e_e61dc6e4b235.slice/crio-29b732c9f1d6a437142779566c4a99dc8bef7e77eb2380baf043f34404134a93 WatchSource:0}: Error finding container 29b732c9f1d6a437142779566c4a99dc8bef7e77eb2380baf043f34404134a93: Status 404 returned error can't find the container with id 29b732c9f1d6a437142779566c4a99dc8bef7e77eb2380baf043f34404134a93
Jul 20 10:25:41.331699 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:41.331663    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-186a7e-ggnkn" event={"ID":"e81a0e09-04db-42dd-bf3e-e61dc6e4b235","Type":"ContainerStarted","Data":"8f6910a0b914d2e8cc60b99d66f8e5693c185936c80b82422d4d1ab696815f34"}
Jul 20 10:25:41.331866 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:41.331725    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-186a7e-ggnkn" event={"ID":"e81a0e09-04db-42dd-bf3e-e61dc6e4b235","Type":"ContainerStarted","Data":"29b732c9f1d6a437142779566c4a99dc8bef7e77eb2380baf043f34404134a93"}
Jul 20 10:25:41.350267 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:41.350211    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-186a7e-ggnkn" podStartSLOduration=1.350193818 podStartE2EDuration="1.350193818s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 10:25:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 10:25:41.34767322 +0000 UTC m=+2807.899127717" watchObservedRunningTime="2026-07-20 10:25:41.350193818 +0000 UTC m=+2807.901648282"
Jul 20 10:25:42.337563 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:42.337530    2566 generic.go:356] "Generic (PLEG): container finished" podID="e81a0e09-04db-42dd-bf3e-e61dc6e4b235" containerID="8f6910a0b914d2e8cc60b99d66f8e5693c185936c80b82422d4d1ab696815f34" exitCode=0
Jul 20 10:25:42.337939 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:42.337616    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-186a7e-ggnkn" event={"ID":"e81a0e09-04db-42dd-bf3e-e61dc6e4b235","Type":"ContainerDied","Data":"8f6910a0b914d2e8cc60b99d66f8e5693c185936c80b82422d4d1ab696815f34"}
Jul 20 10:25:43.487327 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:43.487296    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-186a7e-ggnkn"
Jul 20 10:25:43.609428 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:43.609323    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e81a0e09-04db-42dd-bf3e-e61dc6e4b235-maas-api-service-ca\") pod \"e81a0e09-04db-42dd-bf3e-e61dc6e4b235\" (UID: \"e81a0e09-04db-42dd-bf3e-e61dc6e4b235\") "
Jul 20 10:25:43.609724 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:43.609701    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e81a0e09-04db-42dd-bf3e-e61dc6e4b235-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e81a0e09-04db-42dd-bf3e-e61dc6e4b235" (UID: "e81a0e09-04db-42dd-bf3e-e61dc6e4b235"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 20 10:25:43.710093 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:43.710056    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e81a0e09-04db-42dd-bf3e-e61dc6e4b235-maas-api-service-ca\") on node \"ip-10-0-140-134.ec2.internal\" DevicePath \"\""
Jul 20 10:25:44.349124 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:44.349091    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-186a7e-ggnkn"
Jul 20 10:25:44.349124 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:44.349118    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-186a7e-ggnkn" event={"ID":"e81a0e09-04db-42dd-bf3e-e61dc6e4b235","Type":"ContainerDied","Data":"29b732c9f1d6a437142779566c4a99dc8bef7e77eb2380baf043f34404134a93"}
Jul 20 10:25:44.351438 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:44.349162    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="29b732c9f1d6a437142779566c4a99dc8bef7e77eb2380baf043f34404134a93"
Jul 20 10:25:54.407908 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:54.407868    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6kpnm"]
Jul 20 10:25:55.621868 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:55.621824    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-186a7e-ggnkn"]
Jul 20 10:25:55.623966 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:55.623932    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-186a7e-ggnkn"]
Jul 20 10:25:55.920412 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:25:55.920312    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e81a0e09-04db-42dd-bf3e-e61dc6e4b235" path="/var/lib/kubelet/pods/e81a0e09-04db-42dd-bf3e-e61dc6e4b235/volumes"
Jul 20 10:26:03.089061 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:03.089020    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7fdbc5985b-c468h_a3f446c1-ffcb-4dba-876e-fc267e1f5421/authorino/0.log"
Jul 20 10:26:09.732447 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:09.732410    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-z8w5d_04c45b36-fe41-4276-8b2b-0991fd92123b/manager/0.log"
Jul 20 10:26:09.886625 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:09.886588    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-r5fbn_4f5d4b58-2e51-4be3-b5f3-22a5e75b273a/manager/0.log"
Jul 20 10:26:10.160359 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:10.160329    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-rhthj_ec4dee2b-b4be-4eff-b5a5-d07f9e9d27fc/server/0.log"
Jul 20 10:26:10.298828 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:10.298793    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-rkmvr_c047ca14-af23-48e7-84f8-a2b51edec8bf/manager/0.log"
Jul 20 10:26:10.572694 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:10.572663    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n_06374749-5b32-4b20-8752-34af82ef1d16/copy-manifests/0.log"
Jul 20 10:26:10.580504 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:10.580458    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n_06374749-5b32-4b20-8752-34af82ef1d16/manager/0.log"
Jul 20 10:26:11.336579 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:11.336545    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29742375-hfc7j_99c5c5e2-2ebe-408c-9ece-3ac5d5bb04a7/cleanup/0.log"
Jul 20 10:26:12.811287 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:12.811228    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7fdbc5985b-c468h_a3f446c1-ffcb-4dba-876e-fc267e1f5421/authorino/0.log"
Jul 20 10:26:12.940381 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:12.940337    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-4mff6_ca3131a0-9844-4886-a94d-502f10b00b75/manager/0.log"
Jul 20 10:26:13.059502 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:13.059468    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-fdbb8_f772c922-ff2c-428f-85d6-6cd11bf9f591/manager/0.log"
Jul 20 10:26:13.174862 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:13.174782    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-6bwbt_4659bf4f-c431-41b8-b8c5-1d6a5a1da700/kuadrant-console-plugin/0.log"
Jul 20 10:26:13.303182 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:13.303143    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-n4ljw_93704a5a-df36-40b8-9f69-a7b5342ecc36/registry-server/0.log"
Jul 20 10:26:13.446161 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:13.446040    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-ss27q_0859aa77-3466-49d5-ac4f-bf326c67b529/manager/0.log"
Jul 20 10:26:13.565234 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:13.565202    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-6kpnm_e7c22a59-967f-4e99-b2a6-5bf80adfa210/limitador/0.log"
Jul 20 10:26:13.704633 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:13.704552    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-hv5mg_d3401128-a458-4efc-8f7a-5ffb1bee1597/manager/0.log"
Jul 20 10:26:14.348483 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:14.348451    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b6cfc896b-8t64n_14906f6c-ca75-4f3a-8455-ac29c6cefd38/kube-auth-proxy/0.log"
Jul 20 10:26:15.952130 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:15.952101    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-54xsf_36028478-292f-449b-af9b-2679d8cc5d95/main/0.log"
Jul 20 10:26:24.509308 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:24.509257    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-4lw87_d4a0ef09-53a6-47cb-a39f-15ae72031d8f/global-pull-secret-syncer/0.log"
Jul 20 10:26:24.738166 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:24.738134    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vjr7g_75a6adbe-72f8-4292-a8f6-605d496bffea/konnectivity-agent/0.log"
Jul 20 10:26:24.759918 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:24.759828    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-134.ec2.internal_e04307ae6e708dcc808dbfefa047d3b4/haproxy/0.log"
Jul 20 10:26:30.405406 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:30.405326    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7fdbc5985b-c468h_a3f446c1-ffcb-4dba-876e-fc267e1f5421/authorino/0.log"
Jul 20 10:26:30.440745 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:30.440710    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-4mff6_ca3131a0-9844-4886-a94d-502f10b00b75/manager/0.log"
Jul 20 10:26:30.469647 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:30.469615    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-fdbb8_f772c922-ff2c-428f-85d6-6cd11bf9f591/manager/0.log"
Jul 20 10:26:30.493805 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:30.493772    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-6bwbt_4659bf4f-c431-41b8-b8c5-1d6a5a1da700/kuadrant-console-plugin/0.log"
Jul 20 10:26:30.550118 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:30.550079    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-n4ljw_93704a5a-df36-40b8-9f69-a7b5342ecc36/registry-server/0.log"
Jul 20 10:26:30.785532 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:30.785433    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-ss27q_0859aa77-3466-49d5-ac4f-bf326c67b529/manager/0.log"
Jul 20 10:26:30.806167 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:30.806136    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-6kpnm_e7c22a59-967f-4e99-b2a6-5bf80adfa210/limitador/0.log"
Jul 20 10:26:30.929515 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:30.929476    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-hv5mg_d3401128-a458-4efc-8f7a-5ffb1bee1597/manager/0.log"
Jul 20 10:26:32.968495 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:32.968457    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qkqmv_6b15d8a2-2161-4e9c-b21a-5bc43495db76/node-exporter/0.log"
Jul 20 10:26:32.991852 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:32.991815    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qkqmv_6b15d8a2-2161-4e9c-b21a-5bc43495db76/kube-rbac-proxy/0.log"
Jul 20 10:26:33.011611 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:33.011576    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qkqmv_6b15d8a2-2161-4e9c-b21a-5bc43495db76/init-textfile/0.log"
Jul 20 10:26:36.540278 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:36.540237    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-t6zgv/perf-node-gather-daemonset-7h5pf"]
Jul 20 10:26:36.540743 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:36.540568    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e81a0e09-04db-42dd-bf3e-e61dc6e4b235" containerName="revoke-keys"
Jul 20 10:26:36.540743 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:36.540579    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e81a0e09-04db-42dd-bf3e-e61dc6e4b235" containerName="revoke-keys"
Jul 20 10:26:36.540743 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:36.540643    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e81a0e09-04db-42dd-bf3e-e61dc6e4b235" containerName="revoke-keys"
Jul 20 10:26:36.543643 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:36.543621    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t6zgv/perf-node-gather-daemonset-7h5pf"
Jul 20 10:26:36.546278 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:36.546251    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-t6zgv\"/\"kube-root-ca.crt\""
Jul 20 10:26:36.546459 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:36.546250    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-t6zgv\"/\"default-dockercfg-5qqbj\""
Jul 20 10:26:36.547409 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:36.547387    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-t6zgv\"/\"openshift-service-ca.crt\""
Jul 20 10:26:36.550303 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:36.550276    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t6zgv/perf-node-gather-daemonset-7h5pf"]
Jul 20 10:26:36.667857 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:36.667811    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b719b83e-95aa-4f86-8b04-20152cda2286-sys\") pod \"perf-node-gather-daemonset-7h5pf\" (UID: \"b719b83e-95aa-4f86-8b04-20152cda2286\") " pod="openshift-must-gather-t6zgv/perf-node-gather-daemonset-7h5pf"
Jul 20 10:26:36.667857 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:36.667855    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b719b83e-95aa-4f86-8b04-20152cda2286-lib-modules\") pod \"perf-node-gather-daemonset-7h5pf\" (UID: \"b719b83e-95aa-4f86-8b04-20152cda2286\") " pod="openshift-must-gather-t6zgv/perf-node-gather-daemonset-7h5pf"
Jul 20 10:26:36.668120 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:36.667873    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4nrx\" (UniqueName: \"kubernetes.io/projected/b719b83e-95aa-4f86-8b04-20152cda2286-kube-api-access-f4nrx\") pod \"perf-node-gather-daemonset-7h5pf\" (UID: \"b719b83e-95aa-4f86-8b04-20152cda2286\") " pod="openshift-must-gather-t6zgv/perf-node-gather-daemonset-7h5pf"
Jul 20 10:26:36.668120 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:36.667954    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b719b83e-95aa-4f86-8b04-20152cda2286-proc\") pod \"perf-node-gather-daemonset-7h5pf\" (UID: \"b719b83e-95aa-4f86-8b04-20152cda2286\") " pod="openshift-must-gather-t6zgv/perf-node-gather-daemonset-7h5pf"
Jul 20 10:26:36.668120 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:36.668051    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b719b83e-95aa-4f86-8b04-20152cda2286-podres\") pod \"perf-node-gather-daemonset-7h5pf\" (UID: \"b719b83e-95aa-4f86-8b04-20152cda2286\") " pod="openshift-must-gather-t6zgv/perf-node-gather-daemonset-7h5pf"
Jul 20 10:26:36.768677 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:36.768615    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b719b83e-95aa-4f86-8b04-20152cda2286-sys\") pod \"perf-node-gather-daemonset-7h5pf\" (UID: \"b719b83e-95aa-4f86-8b04-20152cda2286\") " pod="openshift-must-gather-t6zgv/perf-node-gather-daemonset-7h5pf"
Jul 20 10:26:36.768677 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:36.768677    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b719b83e-95aa-4f86-8b04-20152cda2286-lib-modules\") pod \"perf-node-gather-daemonset-7h5pf\" (UID: \"b719b83e-95aa-4f86-8b04-20152cda2286\") " pod="openshift-must-gather-t6zgv/perf-node-gather-daemonset-7h5pf"
Jul 20 10:26:36.768943 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:36.768717    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4nrx\" (UniqueName: \"kubernetes.io/projected/b719b83e-95aa-4f86-8b04-20152cda2286-kube-api-access-f4nrx\") pod \"perf-node-gather-daemonset-7h5pf\" (UID: \"b719b83e-95aa-4f86-8b04-20152cda2286\") " pod="openshift-must-gather-t6zgv/perf-node-gather-daemonset-7h5pf"
Jul 20 10:26:36.768943 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:36.768748    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b719b83e-95aa-4f86-8b04-20152cda2286-proc\") pod \"perf-node-gather-daemonset-7h5pf\" (UID: \"b719b83e-95aa-4f86-8b04-20152cda2286\") " pod="openshift-must-gather-t6zgv/perf-node-gather-daemonset-7h5pf"
Jul 20 10:26:36.768943 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:36.768758    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b719b83e-95aa-4f86-8b04-20152cda2286-sys\") pod \"perf-node-gather-daemonset-7h5pf\" (UID: \"b719b83e-95aa-4f86-8b04-20152cda2286\") " pod="openshift-must-gather-t6zgv/perf-node-gather-daemonset-7h5pf"
Jul 20 10:26:36.768943 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:36.768801    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b719b83e-95aa-4f86-8b04-20152cda2286-podres\") pod \"perf-node-gather-daemonset-7h5pf\" (UID: \"b719b83e-95aa-4f86-8b04-20152cda2286\") " pod="openshift-must-gather-t6zgv/perf-node-gather-daemonset-7h5pf"
Jul 20 10:26:36.768943 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:36.768901    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b719b83e-95aa-4f86-8b04-20152cda2286-lib-modules\") pod \"perf-node-gather-daemonset-7h5pf\" (UID: \"b719b83e-95aa-4f86-8b04-20152cda2286\") " pod="openshift-must-gather-t6zgv/perf-node-gather-daemonset-7h5pf"
Jul 20 10:26:36.768943 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:36.768917    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b719b83e-95aa-4f86-8b04-20152cda2286-podres\") pod \"perf-node-gather-daemonset-7h5pf\" (UID: \"b719b83e-95aa-4f86-8b04-20152cda2286\") " pod="openshift-must-gather-t6zgv/perf-node-gather-daemonset-7h5pf"
Jul 20 10:26:36.768943 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:36.768920    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b719b83e-95aa-4f86-8b04-20152cda2286-proc\") pod \"perf-node-gather-daemonset-7h5pf\" (UID: \"b719b83e-95aa-4f86-8b04-20152cda2286\") " pod="openshift-must-gather-t6zgv/perf-node-gather-daemonset-7h5pf"
Jul 20 10:26:36.776817 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:36.776783    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4nrx\" (UniqueName: \"kubernetes.io/projected/b719b83e-95aa-4f86-8b04-20152cda2286-kube-api-access-f4nrx\") pod \"perf-node-gather-daemonset-7h5pf\" (UID: \"b719b83e-95aa-4f86-8b04-20152cda2286\") " pod="openshift-must-gather-t6zgv/perf-node-gather-daemonset-7h5pf"
Jul 20 10:26:36.855377 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:36.855335    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t6zgv/perf-node-gather-daemonset-7h5pf"
Jul 20 10:26:37.025810 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:37.025777    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t6zgv/perf-node-gather-daemonset-7h5pf"]
Jul 20 10:26:37.028394 ip-10-0-140-134 kubenswrapper[2566]: W0720 10:26:37.028362    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb719b83e_95aa_4f86_8b04_20152cda2286.slice/crio-34cbf68b401973343c9b284cd5291fb4d80896f1323b18cc3131e025099369b7 WatchSource:0}: Error finding container 34cbf68b401973343c9b284cd5291fb4d80896f1323b18cc3131e025099369b7: Status 404 returned error can't find the container with id 34cbf68b401973343c9b284cd5291fb4d80896f1323b18cc3131e025099369b7
Jul 20 10:26:37.535456 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:37.535363    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9fsst_76f390a5-3fce-422a-b2af-10096745a490/dns/0.log"
Jul 20 10:26:37.559554 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:37.559518    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9fsst_76f390a5-3fce-422a-b2af-10096745a490/kube-rbac-proxy/0.log"
Jul 20 10:26:37.607652 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:37.607611    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t6zgv/perf-node-gather-daemonset-7h5pf" event={"ID":"b719b83e-95aa-4f86-8b04-20152cda2286","Type":"ContainerStarted","Data":"3e89e3be5feb828c8b943e2c7b4bb127d27469303c318803ec82fce6b9237982"}
Jul 20 10:26:37.607866 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:37.607663    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t6zgv/perf-node-gather-daemonset-7h5pf" event={"ID":"b719b83e-95aa-4f86-8b04-20152cda2286","Type":"ContainerStarted","Data":"34cbf68b401973343c9b284cd5291fb4d80896f1323b18cc3131e025099369b7"}
Jul 20 10:26:37.607866 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:37.607694    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-t6zgv/perf-node-gather-daemonset-7h5pf"
Jul 20 10:26:37.624455 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:37.624397    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-t6zgv/perf-node-gather-daemonset-7h5pf" podStartSLOduration=1.6243801260000001 podStartE2EDuration="1.624380126s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 10:26:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 10:26:37.62355464 +0000 UTC m=+2864.175009108" watchObservedRunningTime="2026-07-20 10:26:37.624380126 +0000 UTC m=+2864.175834590"
Jul 20 10:26:37.653053 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:37.653021    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mqz47_5d536f0c-af6a-4250-a216-30973ed33030/dns-node-resolver/0.log"
Jul 20 10:26:38.211453 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:38.211394    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-558df75cb7-vz7f9_56a1e57c-315d-463e-9e9a-92a3ce040736/registry/0.log"
Jul 20 10:26:38.235058 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:38.235017    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6p7xh_5fffc7f3-af46-4089-9814-a3932da685a8/node-ca/0.log"
Jul 20 10:26:39.376198 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:39.376165    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b6cfc896b-8t64n_14906f6c-ca75-4f3a-8455-ac29c6cefd38/kube-auth-proxy/0.log"
Jul 20 10:26:40.268348 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:40.268316    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tt5ld_e8908aa8-e9e1-4c71-b064-d230e2dcf434/serve-healthcheck-canary/0.log"
Jul 20 10:26:40.736241 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:40.736207    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kttx7_851041f2-912c-462e-92c1-c2ac3d7022da/kube-rbac-proxy/0.log"
Jul 20 10:26:40.759990 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:40.759947    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kttx7_851041f2-912c-462e-92c1-c2ac3d7022da/exporter/0.log"
Jul 20 10:26:40.783179 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:40.783147    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kttx7_851041f2-912c-462e-92c1-c2ac3d7022da/extractor/0.log"
Jul 20 10:26:43.120422 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:43.120392    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-z8w5d_04c45b36-fe41-4276-8b2b-0991fd92123b/manager/0.log"
Jul 20 10:26:43.461280 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:43.461178    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-r5fbn_4f5d4b58-2e51-4be3-b5f3-22a5e75b273a/manager/0.log"
Jul 20 10:26:43.623722 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:43.623683    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-t6zgv/perf-node-gather-daemonset-7h5pf"
Jul 20 10:26:43.682219 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:43.682187    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-rhthj_ec4dee2b-b4be-4eff-b5a5-d07f9e9d27fc/server/0.log"
Jul 20 10:26:43.803205 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:43.803163    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-rkmvr_c047ca14-af23-48e7-84f8-a2b51edec8bf/manager/0.log"
Jul 20 10:26:43.989670 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:43.989629    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n_06374749-5b32-4b20-8752-34af82ef1d16/manager/0.log"
Jul 20 10:26:44.051252 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:44.051226    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bc4dbfcf4-mbn5n_06374749-5b32-4b20-8752-34af82ef1d16/copy-manifests/0.log"
Jul 20 10:26:44.302066 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:44.302032    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b6cfc896b-8t64n_14906f6c-ca75-4f3a-8455-ac29c6cefd38/kube-auth-proxy/0.log"
Jul 20 10:26:46.007690 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:46.007655    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b6cfc896b-8t64n_14906f6c-ca75-4f3a-8455-ac29c6cefd38/kube-auth-proxy/0.log"
Jul 20 10:26:51.822103 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:51.822074    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gxttc_4727c87a-fe93-4101-adcf-bc4795bfe808/kube-multus-additional-cni-plugins/0.log"
Jul 20 10:26:51.844027 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:51.843990    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gxttc_4727c87a-fe93-4101-adcf-bc4795bfe808/egress-router-binary-copy/0.log"
Jul 20 10:26:51.865342 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:51.865312    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gxttc_4727c87a-fe93-4101-adcf-bc4795bfe808/cni-plugins/0.log"
Jul 20 10:26:51.885425 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:51.885393    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gxttc_4727c87a-fe93-4101-adcf-bc4795bfe808/bond-cni-plugin/0.log"
Jul 20 10:26:51.906667 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:51.906620    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gxttc_4727c87a-fe93-4101-adcf-bc4795bfe808/routeoverride-cni/0.log"
Jul 20 10:26:51.929174 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:51.929139    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gxttc_4727c87a-fe93-4101-adcf-bc4795bfe808/whereabouts-cni-bincopy/0.log"
Jul 20 10:26:51.951226 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:51.951195    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gxttc_4727c87a-fe93-4101-adcf-bc4795bfe808/whereabouts-cni/0.log"
Jul 20 10:26:52.141713 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:52.141618    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-rdqzb_a962283d-f543-4d9f-97d5-87641b30b4c2/kube-multus/0.log"
Jul 20 10:26:52.325380 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:52.325344    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xdlfl_d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de/network-metrics-daemon/0.log"
Jul 20 10:26:52.342155 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:52.342120    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xdlfl_d242fbc4-edf9-46b0-aa2f-4c8f3e8ef9de/kube-rbac-proxy/0.log"
Jul 20 10:26:53.371187 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:53.371157    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-64r4x_5784af38-8a10-4cee-8510-64758ed45746/ovn-controller/0.log"
Jul 20 10:26:53.386750 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:53.386705    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-64r4x_5784af38-8a10-4cee-8510-64758ed45746/ovn-acl-logging/0.log"
Jul 20 10:26:53.408531 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:53.408497    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-64r4x_5784af38-8a10-4cee-8510-64758ed45746/kube-rbac-proxy-node/0.log"
Jul 20 10:26:53.430527 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:53.430490    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-64r4x_5784af38-8a10-4cee-8510-64758ed45746/kube-rbac-proxy-ovn-metrics/0.log"
Jul 20 10:26:53.450907 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:53.450874    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-64r4x_5784af38-8a10-4cee-8510-64758ed45746/northd/0.log"
Jul 20 10:26:53.470477 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:53.470439    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-64r4x_5784af38-8a10-4cee-8510-64758ed45746/nbdb/0.log"
Jul 20 10:26:53.491263 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:53.491226    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-64r4x_5784af38-8a10-4cee-8510-64758ed45746/sbdb/0.log"
Jul 20 10:26:53.582706 ip-10-0-140-134 kubenswrapper[2566]: I0720 10:26:53.582667    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-64r4x_5784af38-8a10-4cee-8510-64758ed45746/ovnkube-controller/0.log"