Jul 22 03:31:31.565234 ip-10-0-132-89 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 22 03:31:31.565246 ip-10-0-132-89 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 22 03:31:31.565256 ip-10-0-132-89 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 22 03:31:31.565747 ip-10-0-132-89 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 22 03:31:41.814585 ip-10-0-132-89 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 22 03:31:41.814609 ip-10-0-132-89 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 312d1177e69d4d41972e8330a21a2789 --
Jul 22 03:33:32.272944 ip-10-0-132-89 systemd[1]: Starting Kubernetes Kubelet...
Jul 22 03:33:32.635999 ip-10-0-132-89 kubenswrapper[2570]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 22 03:33:32.635999 ip-10-0-132-89 kubenswrapper[2570]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 22 03:33:32.635999 ip-10-0-132-89 kubenswrapper[2570]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 22 03:33:32.635999 ip-10-0-132-89 kubenswrapper[2570]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 22 03:33:32.635999 ip-10-0-132-89 kubenswrapper[2570]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 22 03:33:32.637767 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.637682    2570 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 22 03:33:32.641952 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.641934    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 03:33:32.641952 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.641953    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 03:33:32.642029 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.641956    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 03:33:32.642029 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.641959    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 03:33:32.642029 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.641962    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 03:33:32.642029 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.641965    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 03:33:32.642029 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.641968    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 03:33:32.642029 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.641970    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 03:33:32.642029 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.641973    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 03:33:32.642029 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.641975    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 03:33:32.642029 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.641978    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 03:33:32.642029 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.641980    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 03:33:32.642029 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.641983    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 03:33:32.642029 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.641985    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 03:33:32.642029 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.641988    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 03:33:32.642029 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.641990    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 03:33:32.642029 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.641994    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 03:33:32.642029 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.641996    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 03:33:32.642029 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.641999    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 03:33:32.642029 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642001    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 03:33:32.642029 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642011    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 03:33:32.642029 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642014    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 03:33:32.642029 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642017    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 03:33:32.642554 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642020    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 03:33:32.642554 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642022    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 03:33:32.642554 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642025    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 03:33:32.642554 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642027    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 03:33:32.642554 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642030    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 03:33:32.642554 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642033    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 03:33:32.642554 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642035    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 03:33:32.642554 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642038    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 03:33:32.642554 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642041    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 03:33:32.642554 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642044    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 03:33:32.642554 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642046    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 03:33:32.642554 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642049    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 03:33:32.642554 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642051    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 03:33:32.642554 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642054    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 03:33:32.642554 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642056    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 03:33:32.642554 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642059    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 03:33:32.642554 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642061    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 03:33:32.642554 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642064    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 03:33:32.642554 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642066    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 03:33:32.642554 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642069    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 03:33:32.643058 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642071    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 03:33:32.643058 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642073    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 03:33:32.643058 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642075    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 03:33:32.643058 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642078    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 03:33:32.643058 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642081    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 03:33:32.643058 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642083    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 03:33:32.643058 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642085    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 03:33:32.643058 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642088    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 03:33:32.643058 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642094    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 03:33:32.643058 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642098    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 03:33:32.643058 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642101    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 03:33:32.643058 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642104    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 03:33:32.643058 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642107    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 03:33:32.643058 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642110    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 03:33:32.643058 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642113    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 03:33:32.643058 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642116    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 03:33:32.643058 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642119    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 03:33:32.643058 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642121    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 03:33:32.643058 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642124    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 03:33:32.643512 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642126    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 03:33:32.643512 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642130    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 03:33:32.643512 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642132    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 03:33:32.643512 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642134    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 03:33:32.643512 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642137    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 03:33:32.643512 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642139    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 03:33:32.643512 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642143    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 03:33:32.643512 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642146    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 03:33:32.643512 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642148    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 03:33:32.643512 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642151    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 03:33:32.643512 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642153    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 03:33:32.643512 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642156    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 03:33:32.643512 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642158    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 03:33:32.643512 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642163    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 03:33:32.643512 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642166    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 03:33:32.643512 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642169    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 03:33:32.643512 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642172    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 03:33:32.643512 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642174    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 03:33:32.643512 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642177    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 03:33:32.643977 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642179    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 03:33:32.643977 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642182    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 03:33:32.643977 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642184    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 03:33:32.643977 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642189    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 03:33:32.643977 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642192    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 03:33:32.643977 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642196    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 03:33:32.643977 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642198    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 03:33:32.643977 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642201    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 03:33:32.643977 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642204    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 03:33:32.643977 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642206    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 03:33:32.643977 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642209    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 03:33:32.643977 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642211    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 03:33:32.643977 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642214    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 03:33:32.643977 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642216    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 03:33:32.643977 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642219    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 03:33:32.643977 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642221    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 03:33:32.643977 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642224    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 03:33:32.643977 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.642226    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 03:33:32.643977 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643455    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 03:33:32.644421 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643462    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 03:33:32.644421 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643477    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 03:33:32.644421 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643481    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 03:33:32.644421 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643484    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 03:33:32.644421 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643487    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 03:33:32.644421 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643490    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 03:33:32.644421 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643492    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 03:33:32.644421 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643495    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 03:33:32.644421 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643498    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 03:33:32.644421 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643501    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 03:33:32.644421 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643503    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 03:33:32.644421 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643506    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 03:33:32.644421 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643509    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 03:33:32.644421 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643512    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 03:33:32.644421 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643514    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 03:33:32.644421 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643517    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 03:33:32.644421 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643520    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 03:33:32.644421 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643522    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 03:33:32.644421 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643526    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 03:33:32.644421 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643528    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 03:33:32.644925 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643533    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 03:33:32.644925 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643535    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 03:33:32.644925 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643538    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 03:33:32.644925 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643540    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 03:33:32.644925 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643543    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 03:33:32.644925 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643546    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 03:33:32.644925 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643549    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 03:33:32.644925 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643552    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 03:33:32.644925 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643554    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 03:33:32.644925 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643557    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 03:33:32.644925 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643559    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 03:33:32.644925 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643562    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 03:33:32.644925 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643564    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 03:33:32.644925 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643567    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 03:33:32.644925 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643569    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 03:33:32.644925 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643573    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 03:33:32.644925 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643577    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 03:33:32.644925 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643580    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 03:33:32.644925 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643583    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 03:33:32.645385 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643586    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 03:33:32.645385 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643588    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 03:33:32.645385 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643592    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 03:33:32.645385 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643595    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 03:33:32.645385 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643598    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 03:33:32.645385 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643600    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 03:33:32.645385 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643602    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 03:33:32.645385 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643605    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 03:33:32.645385 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643607    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 03:33:32.645385 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643623    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 03:33:32.645385 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643626    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 03:33:32.645385 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643629    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 03:33:32.645385 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643631    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 03:33:32.645385 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643634    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 03:33:32.645385 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643637    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 03:33:32.645385 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643640    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 03:33:32.645385 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643643    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 03:33:32.645385 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643646    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 03:33:32.645385 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643649    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 03:33:32.645385 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643651    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 03:33:32.645887 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643654    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 03:33:32.645887 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643656    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 03:33:32.645887 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643659    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 03:33:32.645887 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643661    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 03:33:32.645887 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643664    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 03:33:32.645887 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643666    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 03:33:32.645887 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643668    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 03:33:32.645887 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643672    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 03:33:32.645887 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643675    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 03:33:32.645887 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643678    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 03:33:32.645887 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643681    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 03:33:32.645887 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643684    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 03:33:32.645887 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643687    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 03:33:32.645887 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643689    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 03:33:32.645887 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643692    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 03:33:32.645887 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643695    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 03:33:32.645887 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643698    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 03:33:32.645887 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643701    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 03:33:32.645887 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643703    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 03:33:32.645887 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643706    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 03:33:32.646385 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643708    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 03:33:32.646385 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643710    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 03:33:32.646385 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643713    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 03:33:32.646385 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643715    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 03:33:32.646385 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643718    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 03:33:32.646385 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643721    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 03:33:32.646385 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643723    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 03:33:32.646385 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643726    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 03:33:32.646385 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643728    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 03:33:32.646385 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643732    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 03:33:32.646385 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643735    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 03:33:32.646385 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643738    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 03:33:32.646385 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643741    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 03:33:32.646385 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643744    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 03:33:32.646385 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643746    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 03:33:32.646385 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643749    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 03:33:32.646385 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643751    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 03:33:32.646385 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643754    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 03:33:32.646385 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.643756    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 03:33:32.646864 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644244    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 03:33:32.646864 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644251    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 03:33:32.646864 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644255    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 03:33:32.646864 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644259    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 03:33:32.646864 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644262    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 03:33:32.646864 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644265    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 03:33:32.646864 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644268    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 03:33:32.646864 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644271    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 03:33:32.646864 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644274    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 03:33:32.646864 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644277    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 03:33:32.646864 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644280    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 03:33:32.646864 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644283    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 03:33:32.646864 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644286    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 03:33:32.646864 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644288    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 03:33:32.646864 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644291    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 03:33:32.646864 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644294    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 03:33:32.646864 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644297    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 03:33:32.646864 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644300    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 03:33:32.646864 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644303    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 03:33:32.647318 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644305    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 03:33:32.647318 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644308    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 03:33:32.647318 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644311    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 03:33:32.647318 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644313    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 03:33:32.647318 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644316    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 03:33:32.647318 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644318    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 03:33:32.647318 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644321    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 03:33:32.647318 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644323    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 03:33:32.647318 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644326    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 03:33:32.647318 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644329    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 03:33:32.647318 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644331    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 03:33:32.647318 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644333    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 03:33:32.647318 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644336    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 03:33:32.647318 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644338    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 03:33:32.647318 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644341    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 03:33:32.647318 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644343    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 03:33:32.647318 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644345    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 03:33:32.647318 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644348    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 03:33:32.647318 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644350    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 03:33:32.647318 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644353    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 03:33:32.647849 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644355    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 03:33:32.647849 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644358    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 03:33:32.647849 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644361    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 03:33:32.647849 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644364    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 03:33:32.647849 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644366    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 03:33:32.647849 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644369    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 03:33:32.647849 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644372    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 03:33:32.647849 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644374    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 03:33:32.647849 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644377    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 03:33:32.647849 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644380    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 03:33:32.647849 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644383    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 03:33:32.647849 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644385    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 03:33:32.647849 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644388    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 03:33:32.647849 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644391    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 03:33:32.647849 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644393    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 03:33:32.647849 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644396    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 03:33:32.647849 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644398    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 03:33:32.647849 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644401    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 03:33:32.647849 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644403    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 03:33:32.647849 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644406    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 03:33:32.648336 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644408    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 03:33:32.648336 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644412    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 03:33:32.648336 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644416    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 03:33:32.648336 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644419    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 03:33:32.648336 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644422    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 03:33:32.648336 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644425    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 03:33:32.648336 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644428    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 03:33:32.648336 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644432    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 03:33:32.648336 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644436    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 03:33:32.648336 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644438    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 03:33:32.648336 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644441    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 03:33:32.648336 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644444    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 03:33:32.648336 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644447    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 03:33:32.648336 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644456    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 03:33:32.648336 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644458    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 03:33:32.648336 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644461    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 03:33:32.648336 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644464    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 03:33:32.648336 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644467    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 03:33:32.648336 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644469    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 03:33:32.648813 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644471    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 03:33:32.648813 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644474    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 03:33:32.648813 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644476    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 03:33:32.648813 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644478    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 03:33:32.648813 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644481    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 03:33:32.648813 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644483    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 03:33:32.648813 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644485    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 03:33:32.648813 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644488    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 03:33:32.648813 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644490    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 03:33:32.648813 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644492    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 03:33:32.648813 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644495    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 03:33:32.648813 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644497    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 03:33:32.648813 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644500    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 03:33:32.648813 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644502    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 03:33:32.648813 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644504    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 03:33:32.648813 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644507    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 03:33:32.648813 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644509    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 03:33:32.648813 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644511    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 03:33:32.648813 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644514    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 03:33:32.648813 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644516    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 03:33:32.649323 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.644518    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 03:33:32.649323 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644600    2570 flags.go:64] FLAG: --address="0.0.0.0"
Jul 22 03:33:32.649323 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644608    2570 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 22 03:33:32.649323 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644630    2570 flags.go:64] FLAG: --anonymous-auth="true"
Jul 22 03:33:32.649323 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644634    2570 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 22 03:33:32.649323 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644639    2570 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 22 03:33:32.649323 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644643    2570 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 22 03:33:32.649323 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644648    2570 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 22 03:33:32.649323 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644652    2570 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 22 03:33:32.649323 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644661    2570 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 22 03:33:32.649323 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644665    2570 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 22 03:33:32.649323 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644669    2570 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 22 03:33:32.649323 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644672    2570 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 22 03:33:32.649323 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644675    2570 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 22 03:33:32.649323 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644678    2570 flags.go:64] FLAG: --cgroup-root=""
Jul 22 03:33:32.649323 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644681    2570 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 22 03:33:32.649323 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644684    2570 flags.go:64] FLAG: --client-ca-file=""
Jul 22 03:33:32.649323 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644686    2570 flags.go:64] FLAG: --cloud-provider="external"
Jul 22 03:33:32.649323 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644689    2570 flags.go:64] FLAG: --cluster-dns="[]"
Jul 22 03:33:32.649323 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644696    2570 flags.go:64] FLAG: --cluster-domain=""
Jul 22 03:33:32.649323 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644699    2570 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 22 03:33:32.649323 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644702    2570 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 22 03:33:32.649323 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644705    2570 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 22 03:33:32.649323 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644708    2570 flags.go:64] FLAG: --container-log-max-files="5"
Jul 22 03:33:32.649904 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644712    2570 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 22 03:33:32.649904 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644715    2570 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 22 03:33:32.649904 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644718    2570 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 22 03:33:32.649904 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644721    2570 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 22 03:33:32.649904 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644724    2570 flags.go:64] FLAG: --contention-profiling="false"
Jul 22 03:33:32.649904 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644727    2570 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 22 03:33:32.649904 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644730    2570 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 22 03:33:32.649904 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644733    2570 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 22 03:33:32.649904 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644735    2570 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 22 03:33:32.649904 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644740    2570 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 22 03:33:32.649904 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644742    2570 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 22 03:33:32.649904 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644745    2570 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 22 03:33:32.649904 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644748    2570 flags.go:64] FLAG: --enable-load-reader="false"
Jul 22 03:33:32.649904 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644751    2570 flags.go:64] FLAG: --enable-server="true"
Jul 22 03:33:32.649904 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644755    2570 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 22 03:33:32.649904 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644762    2570 flags.go:64] FLAG: --event-burst="100"
Jul 22 03:33:32.649904 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644765    2570 flags.go:64] FLAG: --event-qps="50"
Jul 22 03:33:32.649904 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644767    2570 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 22 03:33:32.649904 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644777    2570 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 22 03:33:32.649904 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644780    2570 flags.go:64] FLAG: --eviction-hard=""
Jul 22 03:33:32.649904 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644784    2570 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 22 03:33:32.649904 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644787    2570 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 22 03:33:32.649904 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644789    2570 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 22 03:33:32.649904 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644792    2570 flags.go:64] FLAG: --eviction-soft=""
Jul 22 03:33:32.649904 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644795    2570 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 22 03:33:32.650482 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644799    2570 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 22 03:33:32.650482 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644801    2570 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 22 03:33:32.650482 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644804    2570 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 22 03:33:32.650482 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644806    2570 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 22 03:33:32.650482 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644809    2570 flags.go:64] FLAG: --fail-swap-on="true"
Jul 22 03:33:32.650482 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644812    2570 flags.go:64] FLAG: --feature-gates=""
Jul 22 03:33:32.650482 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644816    2570 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 22 03:33:32.650482 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644818    2570 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 22 03:33:32.650482 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644821    2570 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 22 03:33:32.650482 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644824    2570 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 22 03:33:32.650482 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644827    2570 flags.go:64] FLAG: --healthz-port="10248"
Jul 22 03:33:32.650482 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644830    2570 flags.go:64] FLAG: --help="false"
Jul 22 03:33:32.650482 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644832    2570 flags.go:64] FLAG: --hostname-override="ip-10-0-132-89.ec2.internal"
Jul 22 03:33:32.650482 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644835    2570 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 22 03:33:32.650482 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644838    2570 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 22 03:33:32.650482 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644841    2570 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 22 03:33:32.650482 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644844    2570 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 22 03:33:32.650482 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644847    2570 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 22 03:33:32.650482 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644850    2570 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 22 03:33:32.650482 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644853    2570 flags.go:64] FLAG: --image-service-endpoint=""
Jul 22 03:33:32.650482 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644856    2570 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 22 03:33:32.650482 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644860    2570 flags.go:64] FLAG: --kube-api-burst="100"
Jul 22 03:33:32.650482 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644863    2570 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 22 03:33:32.651183 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644866    2570 flags.go:64] FLAG: --kube-api-qps="50"
Jul 22 03:33:32.651183 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644868    2570 flags.go:64] FLAG: --kube-reserved=""
Jul 22 03:33:32.651183 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644872    2570 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 22 03:33:32.651183 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644880    2570 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 22 03:33:32.651183 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644883    2570 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 22 03:33:32.651183 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644886    2570 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 22 03:33:32.651183 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644888    2570 flags.go:64] FLAG: --lock-file=""
Jul 22 03:33:32.651183 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644891    2570 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 22 03:33:32.651183 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644894    2570 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 22 03:33:32.651183 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644897    2570 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 22 03:33:32.651183 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644902    2570 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 22 03:33:32.651183 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644905    2570 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 22 03:33:32.651183 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644907    2570 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 22 03:33:32.651183 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644910    2570 flags.go:64] FLAG: --logging-format="text"
Jul 22 03:33:32.651183 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644913    2570 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 22 03:33:32.651183 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644916    2570 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 22 03:33:32.651183 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644919    2570 flags.go:64] FLAG: --manifest-url=""
Jul 22 03:33:32.651183 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644921    2570 flags.go:64] FLAG: --manifest-url-header=""
Jul 22 03:33:32.651183 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644925    2570 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 22 03:33:32.651183 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644928    2570 flags.go:64] FLAG: --max-open-files="1000000"
Jul 22 03:33:32.651183 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644932    2570 flags.go:64] FLAG: --max-pods="110"
Jul 22 03:33:32.651183 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644935    2570 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 22 03:33:32.651183 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644938    2570 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 22 03:33:32.651183 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644941    2570 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 22 03:33:32.651183 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644944    2570 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 22 03:33:32.651827 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644946    2570 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 22 03:33:32.651827 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644949    2570 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 22 03:33:32.651827 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644953    2570 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 22 03:33:32.651827 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644961    2570 flags.go:64] FLAG: --node-status-max-images="50"
Jul 22 03:33:32.651827 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644963    2570 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 22 03:33:32.651827 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644967    2570 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 22 03:33:32.651827 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644970    2570 flags.go:64] FLAG: --pod-cidr=""
Jul 22 03:33:32.651827 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644973    2570 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 22 03:33:32.651827 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644979    2570 flags.go:64] FLAG: --pod-manifest-path=""
Jul 22 03:33:32.651827 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644982    2570 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 22 03:33:32.651827 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644985    2570 flags.go:64] FLAG: --pods-per-core="0"
Jul 22 03:33:32.651827 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644993    2570 flags.go:64] FLAG: --port="10250"
Jul 22 03:33:32.651827 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644996    2570 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 22 03:33:32.651827 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.644999    2570 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b8daa8ca1a5f6f79"
Jul 22 03:33:32.651827 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.645002    2570 flags.go:64] FLAG: --qos-reserved=""
Jul 22 03:33:32.651827 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.645006    2570 flags.go:64] FLAG: --read-only-port="10255"
Jul 22 03:33:32.651827 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.645008    2570 flags.go:64] FLAG: --register-node="true"
Jul 22 03:33:32.651827 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.645011    2570 flags.go:64] FLAG: --register-with-taints=""
Jul 22 03:33:32.651827 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.645015    2570 flags.go:64] FLAG: --registry-burst="10"
Jul 22 03:33:32.651827 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.645018    2570 flags.go:64] FLAG: --registry-qps="5"
Jul 22 03:33:32.651827 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.645020    2570 flags.go:64] FLAG: --reserved-cpus=""
Jul 22 03:33:32.651827 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.645023    2570 flags.go:64] FLAG: --reserved-memory=""
Jul 22 03:33:32.651827 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.645027    2570 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 22 03:33:32.651827 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.645030    2570 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 22 03:33:32.651827 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.645033    2570 flags.go:64] FLAG: --rotate-certificates="false"
Jul 22 03:33:32.652471 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.645036    2570 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 22 03:33:32.652471 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.645039    2570 flags.go:64] FLAG: --runonce="false"
Jul 22 03:33:32.652471 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.645041    2570 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 22 03:33:32.652471 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.645044    2570 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 22 03:33:32.652471 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.645047    2570 flags.go:64] FLAG: --seccomp-default="false"
Jul 22 03:33:32.652471 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.645050    2570 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 22 03:33:32.652471 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.645053    2570 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 22 03:33:32.652471 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.645056    2570 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 22 03:33:32.652471 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.645059    2570 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 22 03:33:32.652471 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.645062    2570 flags.go:64] FLAG: --storage-driver-password="root"
Jul 22 03:33:32.652471 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.645066    2570 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 22 03:33:32.652471 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.645069    2570 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 22 03:33:32.652471 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.645073    2570 flags.go:64] FLAG: --storage-driver-user="root"
Jul 22 03:33:32.652471 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.645076    2570 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 22 03:33:32.652471 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.645079    2570 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 22 03:33:32.652471 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.645081    2570 flags.go:64] FLAG: --system-cgroups=""
Jul 22 03:33:32.652471 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.645084    2570 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 22 03:33:32.652471 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.645090    2570 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 22 03:33:32.652471 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.645093    2570 flags.go:64] FLAG: --tls-cert-file=""
Jul 22 03:33:32.652471 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.645101    2570 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 22 03:33:32.652471 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.645113    2570 flags.go:64] FLAG: --tls-min-version=""
Jul 22 03:33:32.652471 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.645116    2570 flags.go:64] FLAG: --tls-private-key-file=""
Jul 22 03:33:32.652471 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.645119    2570 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 22 03:33:32.652471 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.645121    2570 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 22 03:33:32.652471 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.645124    2570 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 22 03:33:32.653094 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.645127    2570 flags.go:64] FLAG: --v="2"
Jul 22 03:33:32.653094 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.645131    2570 flags.go:64] FLAG: --version="false"
Jul 22 03:33:32.653094 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.645135    2570 flags.go:64] FLAG: --vmodule=""
Jul 22 03:33:32.653094 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.645139    2570 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 22 03:33:32.653094 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.645142    2570 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 22 03:33:32.653094 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645289    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 03:33:32.653094 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645292    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 03:33:32.653094 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645296    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 03:33:32.653094 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645298    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 03:33:32.653094 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645301    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 03:33:32.653094 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645304    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 03:33:32.653094 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645307    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 03:33:32.653094 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645310    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 03:33:32.653094 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645314    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 03:33:32.653094 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645317    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 03:33:32.653094 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645320    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 03:33:32.653094 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645322    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 03:33:32.653094 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645326    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 03:33:32.653094 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645329    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 03:33:32.653094 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645334    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 03:33:32.653578 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645336    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 03:33:32.653578 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645339    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 03:33:32.653578 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645342    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 03:33:32.653578 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645344    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 03:33:32.653578 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645347    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 03:33:32.653578 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645349    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 03:33:32.653578 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645352    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 03:33:32.653578 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645360    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 03:33:32.653578 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645362    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 03:33:32.653578 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645365    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 03:33:32.653578 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645367    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 03:33:32.653578 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645370    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 03:33:32.653578 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645372    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 03:33:32.653578 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645374    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 03:33:32.653578 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645377    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 03:33:32.653578 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645380    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 03:33:32.653578 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645382    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 03:33:32.653578 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645385    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 03:33:32.653578 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645387    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 03:33:32.653578 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645390    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 03:33:32.654092 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645392    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 03:33:32.654092 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645395    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 03:33:32.654092 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645397    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 03:33:32.654092 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645400    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 03:33:32.654092 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645402    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 03:33:32.654092 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645405    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 03:33:32.654092 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645407    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 03:33:32.654092 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645410    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 03:33:32.654092 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645412    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 03:33:32.654092 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645416    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 03:33:32.654092 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645419    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 03:33:32.654092 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645422    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 03:33:32.654092 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645424    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 03:33:32.654092 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645428    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 03:33:32.654092 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645431    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 03:33:32.654092 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645434    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 03:33:32.654092 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645437    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 03:33:32.654092 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645440    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 03:33:32.654092 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645443    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 03:33:32.654092 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645445    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 03:33:32.654570 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645453    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 03:33:32.654570 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645456    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 03:33:32.654570 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645459    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 03:33:32.654570 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645461    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 03:33:32.654570 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645464    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 03:33:32.654570 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645467    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 03:33:32.654570 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645471    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 03:33:32.654570 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645473    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 03:33:32.654570 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645476    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 03:33:32.654570 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645478    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 03:33:32.654570 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645480    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 03:33:32.654570 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645483    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 03:33:32.654570 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645485    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 03:33:32.654570 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645488    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 03:33:32.654570 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645490    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 03:33:32.654570 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645493    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 03:33:32.654570 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645495    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 03:33:32.654570 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645498    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 03:33:32.654570 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645500    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 03:33:32.655027 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645502    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 03:33:32.655027 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645505    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 03:33:32.655027 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645508    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 03:33:32.655027 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645511    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 03:33:32.655027 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645513    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 03:33:32.655027 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645516    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 03:33:32.655027 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645518    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 03:33:32.655027 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645521    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 03:33:32.655027 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645523    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 03:33:32.655027 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645525    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 03:33:32.655027 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645528    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 03:33:32.655027 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645531    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 03:33:32.655027 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645533    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 03:33:32.655027 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645535    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 03:33:32.655027 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645543    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 03:33:32.655027 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645547    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 03:33:32.655027 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645549    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 03:33:32.655027 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645551    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 03:33:32.655027 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645554    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 03:33:32.655027 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645556    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 03:33:32.655536 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645559    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 03:33:32.655536 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645561    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 03:33:32.655536 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645564    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 03:33:32.655536 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645566    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 03:33:32.655536 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.645568    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 03:33:32.655536 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.646091    2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 22 03:33:32.655536 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.652510    2570 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 22 03:33:32.655536 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.652526    2570 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 22 03:33:32.655536 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.652545    2570 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 22 03:33:32.655536 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.652554    2570 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 22 03:33:32.655536 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652627    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 03:33:32.655536 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652632    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 03:33:32.655536 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652636    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 03:33:32.655536 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652639    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 03:33:32.655536 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652642    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 03:33:32.655957 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652644    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 03:33:32.655957 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652647    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 03:33:32.655957 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652650    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 03:33:32.655957 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652652    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 03:33:32.655957 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652655    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 03:33:32.655957 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652657    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 03:33:32.655957 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652660    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 03:33:32.655957 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652662    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 03:33:32.655957 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652665    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 03:33:32.655957 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652667    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 03:33:32.655957 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652670    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 03:33:32.655957 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652673    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 03:33:32.655957 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652675    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 03:33:32.655957 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652677    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 03:33:32.655957 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652680    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 03:33:32.655957 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652682    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 03:33:32.655957 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652685    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 03:33:32.655957 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652687    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 03:33:32.655957 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652690    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 03:33:32.655957 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652693    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 03:33:32.656436 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652695    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 03:33:32.656436 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652698    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 03:33:32.656436 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652700    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 03:33:32.656436 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652703    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 03:33:32.656436 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652707    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 03:33:32.656436 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652710    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 03:33:32.656436 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652712    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 03:33:32.656436 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652715    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 03:33:32.656436 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652717    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 03:33:32.656436 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652720    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 03:33:32.656436 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652723    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 03:33:32.656436 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652726    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 03:33:32.656436 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652728    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 03:33:32.656436 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652731    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 03:33:32.656436 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652734    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 03:33:32.656436 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652739    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 03:33:32.656436 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652742    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 03:33:32.656436 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652744    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 03:33:32.656436 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652747    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 03:33:32.656436 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652750    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 03:33:32.657113 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652752    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 03:33:32.657113 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652754    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 03:33:32.657113 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652757    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 03:33:32.657113 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652761    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 03:33:32.657113 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652765    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 03:33:32.657113 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652768    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 03:33:32.657113 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652770    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 03:33:32.657113 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652773    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 03:33:32.657113 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652776    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 03:33:32.657113 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652779    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 03:33:32.657113 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652781    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 03:33:32.657113 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652783    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 03:33:32.657113 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652786    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 03:33:32.657113 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652789    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 03:33:32.657113 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652791    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 03:33:32.657113 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652794    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 03:33:32.657113 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652797    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 03:33:32.657113 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652800    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 03:33:32.657113 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652803    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 03:33:32.657587 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652805    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 03:33:32.657587 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652808    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 03:33:32.657587 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652810    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 03:33:32.657587 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652812    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 03:33:32.657587 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652815    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 03:33:32.657587 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652817    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 03:33:32.657587 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652820    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 03:33:32.657587 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652822    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 03:33:32.657587 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652825    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 03:33:32.657587 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652827    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 03:33:32.657587 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652829    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 03:33:32.657587 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652832    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 03:33:32.657587 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652834    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 03:33:32.657587 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652837    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 03:33:32.657587 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652839    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 03:33:32.657587 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652842    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 03:33:32.657587 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652845    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 03:33:32.657587 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652847    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 03:33:32.657587 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652849    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 03:33:32.657587 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652852    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 03:33:32.658345 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652855    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 03:33:32.658345 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652857    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 03:33:32.658345 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652859    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 03:33:32.658345 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652863    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 03:33:32.658345 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652866    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 03:33:32.658345 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652869    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 03:33:32.658345 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652872    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 03:33:32.658345 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652875    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 03:33:32.658345 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652877    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 03:33:32.658345 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652880    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 03:33:32.658345 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652882    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 03:33:32.658345 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652885    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 03:33:32.658345 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652888    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 03:33:32.658345 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652890    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 03:33:32.658345 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.652893    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 03:33:32.658833 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.652898    2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 22 03:33:32.658833 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653010    2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 03:33:32.658833 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653016    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 03:33:32.658833 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653019    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 03:33:32.658833 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653021    2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 03:33:32.658833 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653024    2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 03:33:32.658833 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653027    2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 03:33:32.658833 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653029    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 03:33:32.658833 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653032    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 03:33:32.658833 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653035    2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 03:33:32.658833 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653037    2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 03:33:32.658833 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653040    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 03:33:32.658833 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653043    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 03:33:32.658833 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653046    2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 03:33:32.658833 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653048    2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 03:33:32.659378 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653050    2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 03:33:32.659378 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653053    2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 03:33:32.659378 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653055    2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 03:33:32.659378 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653057    2570 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 03:33:32.659378 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653060    2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 03:33:32.659378 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653062    2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 03:33:32.659378 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653065    2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 03:33:32.659378 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653068    2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 03:33:32.659378 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653072    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 03:33:32.659378 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653075    2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 03:33:32.659378 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653077    2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 03:33:32.659378 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653079    2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 03:33:32.659378 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653082    2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 03:33:32.659378 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653084    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 03:33:32.659378 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653088    2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 03:33:32.659378 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653090    2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 03:33:32.659378 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653093    2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 03:33:32.659378 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653095    2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 03:33:32.659378 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653098    2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 03:33:32.660144 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653101    2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 03:33:32.660144 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653103    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 03:33:32.660144 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653106    2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 03:33:32.660144 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653109    2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 03:33:32.660144 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653111    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 03:33:32.660144 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653114    2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 03:33:32.660144 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653116    2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 03:33:32.660144 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653119    2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 03:33:32.660144 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653121    2570 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 03:33:32.660144 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653123    2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 03:33:32.660144 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653126    2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 03:33:32.660144 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653128    2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 03:33:32.660144 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653130    2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 03:33:32.660144 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653133    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 03:33:32.660144 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653136    2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 03:33:32.660144 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653138    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 03:33:32.660144 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653142    2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 03:33:32.660144 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653145    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 03:33:32.660144 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653148    2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 03:33:32.660806 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653151    2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 03:33:32.660806 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653153    2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 03:33:32.660806 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653156    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 03:33:32.660806 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653158    2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 03:33:32.660806 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653160    2570 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 03:33:32.660806 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653163    2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 03:33:32.660806 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653165    2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 03:33:32.660806 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653168    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 03:33:32.660806 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653170    2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 03:33:32.660806 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653173    2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 03:33:32.660806 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653176    2570 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 03:33:32.660806 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653178    2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 03:33:32.660806 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653181    2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 03:33:32.660806 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653183    2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 03:33:32.660806 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653186    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 03:33:32.660806 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653188    2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 03:33:32.660806 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653191    2570 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 03:33:32.660806 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653193    2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 03:33:32.660806 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653195    2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 03:33:32.660806 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653198    2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 03:33:32.661475 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653200    2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 03:33:32.661475 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653203    2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 03:33:32.661475 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653205    2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 03:33:32.661475 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653207    2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 03:33:32.661475 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653210    2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 03:33:32.661475 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653214    2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 03:33:32.661475 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653217    2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 03:33:32.661475 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653220    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 03:33:32.661475 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653223    2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 03:33:32.661475 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653225    2570 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 03:33:32.661475 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653228    2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 03:33:32.661475 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653230    2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 03:33:32.661475 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653233    2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 03:33:32.661475 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653236    2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 03:33:32.661475 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653238    2570 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 03:33:32.661475 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653241    2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 03:33:32.661475 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653244    2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 03:33:32.661475 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653246    2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 03:33:32.661475 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653249    2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 03:33:32.662171 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653252    2570 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 03:33:32.662171 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653254    2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 03:33:32.662171 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653256    2570 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 03:33:32.662171 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653259    2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 03:33:32.662171 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653262    2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 03:33:32.662171 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653264    2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 03:33:32.662171 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653266    2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 03:33:32.662171 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:32.653269    2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 03:33:32.662171 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.653273    2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 22 03:33:32.662171 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.653380    2570 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 22 03:33:32.662171 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.655505    2570 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 22 03:33:32.662171 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.656638    2570 server.go:1019] "Starting client certificate rotation"
Jul 22 03:33:32.662171 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.656762    2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 22 03:33:32.662171 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.656807    2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 22 03:33:32.662531 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.660861    2570 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 22 03:33:32.662531 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.662074    2570 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 22 03:33:32.672084 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.672069    2570 log.go:25] "Validated CRI v1 runtime API"
Jul 22 03:33:32.674104 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.674087    2570 log.go:25] "Validated CRI v1 image API"
Jul 22 03:33:32.674890 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.674871    2570 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 22 03:33:32.677973 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.677949    2570 fs.go:135] Filesystem UUIDs: map[2c10833c-3a74-4717-8882-f383cf80b3b1:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 8ecd47fc-0424-44ac-b860-d77e199c74f2:/dev/nvme0n1p3]
Jul 22 03:33:32.678041 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.677971    2570 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 22 03:33:32.683766 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.683648    2570 manager.go:216] Machine: {Timestamp:2026-07-22 03:33:32.681504885 +0000 UTC m=+0.299714899 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3140251 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24efead6d1d28d176512036d268c0e SystemUUID:ec24efea-d6d1-d28d-1765-12036d268c0e BootID:312d1177-e69d-4d41-972e-8330a21a2789 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:01:ea:95:42:15 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:01:ea:95:42:15 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:86:1a:4f:30:9a:ba Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 22 03:33:32.683766 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.683758    2570 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 22 03:33:32.683891 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.683835    2570 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 22 03:33:32.684943 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.684926    2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 22 03:33:32.686543 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.686520    2570 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 22 03:33:32.686699 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.686545    2570 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-89.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 22 03:33:32.687258 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.687248    2570 topology_manager.go:138] "Creating topology manager with none policy"
Jul 22 03:33:32.687292 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.687261    2570 container_manager_linux.go:309] "Creating device plugin manager"
Jul 22 03:33:32.687292 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.687268    2570 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 22 03:33:32.687914 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.687903    2570 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 22 03:33:32.688865 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.688855    2570 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 22 03:33:32.692290 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.692278    2570 state_mem.go:36] "Initialized new in-memory state store"
Jul 22 03:33:32.692414 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.692404    2570 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 22 03:33:32.692451 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.692434    2570 kubelet.go:486] "Attempting to sync node with API server"
Jul 22 03:33:32.692451 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.692443    2570 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 22 03:33:32.692505 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.692453    2570 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 22 03:33:32.692505 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.692461    2570 kubelet.go:398] "Adding apiserver pod source"
Jul 22 03:33:32.692505 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.692469    2570 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 22 03:33:32.692840 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.692829    2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 22 03:33:32.692903 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.692847    2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 22 03:33:32.694994 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.694979    2570 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 22 03:33:32.697788 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.697456    2570 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 22 03:33:32.697788 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.697505    2570 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 22 03:33:32.698380 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.698367    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 22 03:33:32.698423 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.698386    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 22 03:33:32.698423 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.698392    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 22 03:33:32.698423 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.698398    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 22 03:33:32.698423 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.698407    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 22 03:33:32.698423 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.698418    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 22 03:33:32.698423 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.698426    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 22 03:33:32.698606 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.698432    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 22 03:33:32.698606 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.698439    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 22 03:33:32.698606 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.698445    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 22 03:33:32.698606 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.698461    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 22 03:33:32.698606 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.698468    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 22 03:33:32.699471 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.699447    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 22 03:33:32.699540 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.699485    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 22 03:33:32.702725 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.702708    2570 server.go:1268] "Started kubelet"
Jul 22 03:33:32.702821 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.702778    2570 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 22 03:33:32.702901 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.702859    2570 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 22 03:33:32.702946 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.702924    2570 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 22 03:33:32.703516 ip-10-0-132-89 systemd[1]: Started Kubernetes Kubelet.
Jul 22 03:33:32.704224 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:32.704065    2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-89.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 22 03:33:32.704224 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.704150    2570 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-89.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 22 03:33:32.704224 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:32.704169    2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 22 03:33:32.704424 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.704372    2570 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 22 03:33:32.704831 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.704818    2570 server.go:310] "Adding debug handlers to kubelet server"
Jul 22 03:33:32.709304 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:32.709283    2570 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 22 03:33:32.711121 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:32.710374    2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-89.ec2.internal.18c47f33c6e6fab7  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-89.ec2.internal,UID:ip-10-0-132-89.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-89.ec2.internal,},FirstTimestamp:2026-07-22 03:33:32.702644919 +0000 UTC m=+0.320854917,LastTimestamp:2026-07-22 03:33:32.702644919 +0000 UTC m=+0.320854917,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-89.ec2.internal,}"
Jul 22 03:33:32.711283 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.711268    2570 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 22 03:33:32.711318 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.711275    2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 22 03:33:32.711520 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:32.711500    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-89.ec2.internal\" not found"
Jul 22 03:33:32.711752 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.711726    2570 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 22 03:33:32.711960 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.711871    2570 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 22 03:33:32.711960 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.711885    2570 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 22 03:33:32.712078 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.711989    2570 reconstruct.go:97] "Volume reconstruction finished"
Jul 22 03:33:32.712078 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.711996    2570 reconciler.go:29] "Reconciler: start to sync state"
Jul 22 03:33:32.712627 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.712586    2570 factory.go:153] Registering CRI-O factory
Jul 22 03:33:32.712719 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.712631    2570 factory.go:223] Registration of the crio container factory successfully
Jul 22 03:33:32.712719 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.712680    2570 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 22 03:33:32.712719 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.712689    2570 factory.go:55] Registering systemd factory
Jul 22 03:33:32.712719 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.712697    2570 factory.go:223] Registration of the systemd container factory successfully
Jul 22 03:33:32.712909 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.712741    2570 factory.go:103] Registering Raw factory
Jul 22 03:33:32.712909 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.712756    2570 manager.go:1196] Started watching for new ooms in manager
Jul 22 03:33:32.713142 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.713129    2570 manager.go:349] Starting recovery of all containers
Jul 22 03:33:32.714544 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:32.714518    2570 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-89.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 22 03:33:32.714656 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:32.714556    2570 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 22 03:33:32.727102 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.727083    2570 manager.go:354] Recovery completed
Jul 22 03:33:32.731277 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.731262    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 03:33:32.733440 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.733418    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-89.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 03:33:32.733508 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.733458    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-89.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 03:33:32.733508 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.733471    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-89.ec2.internal" event="NodeHasSufficientPID"
Jul 22 03:33:32.734044 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.734030    2570 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 22 03:33:32.734044 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.734044    2570 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 22 03:33:32.734185 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.734062    2570 state_mem.go:36] "Initialized new in-memory state store"
Jul 22 03:33:32.734185 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.734045    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 03:33:32.734811 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.734796    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-89.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 03:33:32.734859 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.734823    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-89.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 03:33:32.734859 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.734833    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-89.ec2.internal" event="NodeHasSufficientPID"
Jul 22 03:33:32.735739 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:32.735641    2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-89.ec2.internal.18c47f33c8bce4b3  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-89.ec2.internal,UID:ip-10-0-132-89.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-89.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-89.ec2.internal,},FirstTimestamp:2026-07-22 03:33:32.733441203 +0000 UTC m=+0.351651211,LastTimestamp:2026-07-22 03:33:32.733441203 +0000 UTC m=+0.351651211,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-89.ec2.internal,}"
Jul 22 03:33:32.736283 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.736269    2570 policy_none.go:49] "None policy: Start"
Jul 22 03:33:32.736328 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.736289    2570 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 22 03:33:32.736328 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.736303    2570 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 22 03:33:32.738381 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.738368    2570 policy_none.go:47] "Start"
Jul 22 03:33:32.745270 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:32.745073    2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-89.ec2.internal.18c47f33c8bd3eda  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-89.ec2.internal,UID:ip-10-0-132-89.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-132-89.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-132-89.ec2.internal,},FirstTimestamp:2026-07-22 03:33:32.733464282 +0000 UTC m=+0.351674279,LastTimestamp:2026-07-22 03:33:32.733464282 +0000 UTC m=+0.351674279,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-89.ec2.internal,}"
Jul 22 03:33:32.745440 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.745420    2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zrrqs"
Jul 22 03:33:32.753153 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.753133    2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zrrqs"
Jul 22 03:33:32.753243 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:32.753142    2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-89.ec2.internal.18c47f33c8bd68e4  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-89.ec2.internal,UID:ip-10-0-132-89.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-132-89.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-132-89.ec2.internal,},FirstTimestamp:2026-07-22 03:33:32.733475044 +0000 UTC m=+0.351685043,LastTimestamp:2026-07-22 03:33:32.733475044 +0000 UTC m=+0.351685043,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-89.ec2.internal,}"
Jul 22 03:33:32.779248 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.779224    2570 manager.go:337] "Starting Device Plugin manager"
Jul 22 03:33:32.779364 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:32.779266    2570 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 22 03:33:32.779364 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.779281    2570 server.go:87] "Starting device plugin registration server"
Jul 22 03:33:32.779585 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.779571    2570 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 22 03:33:32.779667 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.779587    2570 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 22 03:33:32.779723 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.779702    2570 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 22 03:33:32.779805 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.779792    2570 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 22 03:33:32.779863 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.779806    2570 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 22 03:33:32.780355 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:32.780334    2570 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 22 03:33:32.780430 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:32.780381    2570 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-89.ec2.internal\" not found"
Jul 22 03:33:32.843828 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.843788    2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 22 03:33:32.846017 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.845279    2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 22 03:33:32.846017 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.845330    2570 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 22 03:33:32.846017 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.845355    2570 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 22 03:33:32.846017 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:32.845393    2570 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 22 03:33:32.847893 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.847872    2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 03:33:32.879740 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.879690    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 03:33:32.880589 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.880571    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-89.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 03:33:32.880711 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.880600    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-89.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 03:33:32.880711 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.880634    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-89.ec2.internal" event="NodeHasSufficientPID"
Jul 22 03:33:32.880711 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.880658    2570 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-89.ec2.internal"
Jul 22 03:33:32.887776 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.887731    2570 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-89.ec2.internal"
Jul 22 03:33:32.887776 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:32.887754    2570 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-89.ec2.internal\": node \"ip-10-0-132-89.ec2.internal\" not found"
Jul 22 03:33:32.913488 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:32.913466    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-89.ec2.internal\" not found"
Jul 22 03:33:32.945529 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.945498    2570 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-89.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-89.ec2.internal"]
Jul 22 03:33:32.945631 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.945572    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 03:33:32.946535 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.946513    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-89.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 03:33:32.946665 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.946553    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-89.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 03:33:32.946665 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.946566    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-89.ec2.internal" event="NodeHasSufficientPID"
Jul 22 03:33:32.948868 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.948853    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 03:33:32.949007 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.948979    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-89.ec2.internal"
Jul 22 03:33:32.949072 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.949029    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 03:33:32.949571 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.949556    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-89.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 03:33:32.949657 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.949580    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-89.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 03:33:32.949657 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.949589    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-89.ec2.internal" event="NodeHasSufficientPID"
Jul 22 03:33:32.949757 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.949662    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-89.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 03:33:32.949757 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.949687    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-89.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 03:33:32.949757 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.949701    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-89.ec2.internal" event="NodeHasSufficientPID"
Jul 22 03:33:32.951719 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.951705    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-89.ec2.internal"
Jul 22 03:33:32.951788 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.951730    2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 03:33:32.952349 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.952333    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-89.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 03:33:32.952413 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.952355    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-89.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 03:33:32.952413 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:32.952369    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-89.ec2.internal" event="NodeHasSufficientPID"
Jul 22 03:33:32.977362 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:32.977337    2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-89.ec2.internal\" not found" node="ip-10-0-132-89.ec2.internal"
Jul 22 03:33:32.981822 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:32.981805    2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-89.ec2.internal\" not found" node="ip-10-0-132-89.ec2.internal"
Jul 22 03:33:33.013814 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:33.013791    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-89.ec2.internal\" not found"
Jul 22 03:33:33.013903 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.013846    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d1ebdc1a61d7d6c7a73ef206e8173b6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-89.ec2.internal\" (UID: \"4d1ebdc1a61d7d6c7a73ef206e8173b6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-89.ec2.internal"
Jul 22 03:33:33.013903 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.013869    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1decd6d269268ae8653188a0a80fb590-config\") pod \"kube-apiserver-proxy-ip-10-0-132-89.ec2.internal\" (UID: \"1decd6d269268ae8653188a0a80fb590\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-89.ec2.internal"
Jul 22 03:33:33.013903 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.013895    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4d1ebdc1a61d7d6c7a73ef206e8173b6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-89.ec2.internal\" (UID: \"4d1ebdc1a61d7d6c7a73ef206e8173b6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-89.ec2.internal"
Jul 22 03:33:33.114081 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:33.114052    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-89.ec2.internal\" not found"
Jul 22 03:33:33.114178 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.114095    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4d1ebdc1a61d7d6c7a73ef206e8173b6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-89.ec2.internal\" (UID: \"4d1ebdc1a61d7d6c7a73ef206e8173b6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-89.ec2.internal"
Jul 22 03:33:33.114178 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.114119    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d1ebdc1a61d7d6c7a73ef206e8173b6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-89.ec2.internal\" (UID: \"4d1ebdc1a61d7d6c7a73ef206e8173b6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-89.ec2.internal"
Jul 22 03:33:33.114178 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.114147    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1decd6d269268ae8653188a0a80fb590-config\") pod \"kube-apiserver-proxy-ip-10-0-132-89.ec2.internal\" (UID: \"1decd6d269268ae8653188a0a80fb590\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-89.ec2.internal"
Jul 22 03:33:33.114267 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.114184    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1decd6d269268ae8653188a0a80fb590-config\") pod \"kube-apiserver-proxy-ip-10-0-132-89.ec2.internal\" (UID: \"1decd6d269268ae8653188a0a80fb590\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-89.ec2.internal"
Jul 22 03:33:33.114267 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.114185    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4d1ebdc1a61d7d6c7a73ef206e8173b6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-89.ec2.internal\" (UID: \"4d1ebdc1a61d7d6c7a73ef206e8173b6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-89.ec2.internal"
Jul 22 03:33:33.114267 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.114226    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d1ebdc1a61d7d6c7a73ef206e8173b6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-89.ec2.internal\" (UID: \"4d1ebdc1a61d7d6c7a73ef206e8173b6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-89.ec2.internal"
Jul 22 03:33:33.214800 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:33.214768    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-89.ec2.internal\" not found"
Jul 22 03:33:33.279981 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.279954    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-89.ec2.internal"
Jul 22 03:33:33.284727 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.284548    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-89.ec2.internal"
Jul 22 03:33:33.315186 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:33.315157    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-89.ec2.internal\" not found"
Jul 22 03:33:33.415554 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:33.415520    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-89.ec2.internal\" not found"
Jul 22 03:33:33.516054 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:33.515957    2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-89.ec2.internal\" not found"
Jul 22 03:33:33.528594 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.528565    2570 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 03:33:33.612117 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.612090    2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-89.ec2.internal"
Jul 22 03:33:33.626172 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.626144    2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 22 03:33:33.627823 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.627807    2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-89.ec2.internal"
Jul 22 03:33:33.637522 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.637502    2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 22 03:33:33.656704 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.656681    2570 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 22 03:33:33.656855 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.656837    2570 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 22 03:33:33.679426 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.679400    2570 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 03:33:33.684687 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.684662    2570 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 03:33:33.692870 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.692854    2570 apiserver.go:52] "Watching apiserver"
Jul 22 03:33:33.701061 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.701042    2570 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 22 03:33:33.701410 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.701391    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-ndfqq","kube-system/konnectivity-agent-thjp8","openshift-cluster-node-tuning-operator/tuned-wt6b5","openshift-dns/node-resolver-tpgsf","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-89.ec2.internal","openshift-multus/multus-xbhc9","openshift-multus/network-metrics-daemon-lctg9","openshift-network-diagnostics/network-check-target-k2tvl","openshift-network-operator/iptables-alerter-sgpqg","kube-system/global-pull-secret-syncer-p867k","kube-system/kube-apiserver-proxy-ip-10-0-132-89.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vt8wv","openshift-image-registry/node-ca-7srsk","openshift-multus/multus-additional-cni-plugins-6wt45"]
Jul 22 03:33:33.703930 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.703913    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lctg9"
Jul 22 03:33:33.704026 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:33.704007    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lctg9" podUID="6adcbd54-1388-4223-8d6d-9423e9ebe09b"
Jul 22 03:33:33.706308 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.706290    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2tvl"
Jul 22 03:33:33.706398 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:33.706363    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2tvl" podUID="6963eadb-e7e5-4f25-a6ea-fc0a57c951b8"
Jul 22 03:33:33.708254 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.708238    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.710245 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.710230    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tpgsf"
Jul 22 03:33:33.710876 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.710859    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-pnz6k\""
Jul 22 03:33:33.711376 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.711360    2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 22 03:33:33.711509 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.711488    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 22 03:33:33.711713 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.711700    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 22 03:33:33.712382 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.712369    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.712955 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.712937    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 22 03:33:33.713028 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.712968    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-pfckn\""
Jul 22 03:33:33.713239 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.713226    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 22 03:33:33.714454 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.714439    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.714959 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.714946    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 22 03:33:33.715589 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.715572    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 22 03:33:33.715905 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.715890    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 22 03:33:33.716122 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.716108    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 22 03:33:33.716209 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.716192    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q79df\" (UniqueName: \"kubernetes.io/projected/6ba9ef79-f774-4c64-b48e-af2461868076-kube-api-access-q79df\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.716267 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.716222    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-run-openvswitch\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.716267 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.716238    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-etc-kubernetes\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.716340 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.716266    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmx9x\" (UniqueName: \"kubernetes.io/projected/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-kube-api-access-jmx9x\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.716340 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.716293    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.716340 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.716320    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6ba9ef79-f774-4c64-b48e-af2461868076-env-overrides\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.716340 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.716339    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-etc-sysctl-d\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.716453 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.716355    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-tmp\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.716453 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.716380    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6ba9ef79-f774-4c64-b48e-af2461868076-ovnkube-script-lib\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.716453 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.716410    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-etc-modprobe-d\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.716453 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.716437    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-run-systemd\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.716565 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.716461    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6ba9ef79-f774-4c64-b48e-af2461868076-ovn-node-metrics-cert\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.716565 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.716481    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctpr8\" (UniqueName: \"kubernetes.io/projected/6adcbd54-1388-4223-8d6d-9423e9ebe09b-kube-api-access-ctpr8\") pod \"network-metrics-daemon-lctg9\" (UID: \"6adcbd54-1388-4223-8d6d-9423e9ebe09b\") " pod="openshift-multus/network-metrics-daemon-lctg9"
Jul 22 03:33:33.716565 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.716497    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-etc-sysconfig\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.716565 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.716530    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-etc-systemd\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.716565 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.716555    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-var-lib-kubelet\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.716817 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.716578    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-host\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.716817 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.716601    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-run-ovn\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.716817 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.716637    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sgpqg"
Jul 22 03:33:33.716817 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.716686    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-host-cni-bin\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.716993 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.716822    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hc9dm\" (UniqueName: \"kubernetes.io/projected/6963eadb-e7e5-4f25-a6ea-fc0a57c951b8-kube-api-access-hc9dm\") pod \"network-check-target-k2tvl\" (UID: \"6963eadb-e7e5-4f25-a6ea-fc0a57c951b8\") " pod="openshift-network-diagnostics/network-check-target-k2tvl"
Jul 22 03:33:33.716993 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.716847    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed43ef9c-bf6c-4f2c-96b5-fc31d24e5fd1-tmp-dir\") pod \"node-resolver-tpgsf\" (UID: \"ed43ef9c-bf6c-4f2c-96b5-fc31d24e5fd1\") " pod="openshift-dns/node-resolver-tpgsf"
Jul 22 03:33:33.716993 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.716870    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-host-kubelet\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.716993 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.716892    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-var-lib-openvswitch\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.716993 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.716916    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-etc-openvswitch\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.716993 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.716939    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-host-run-ovn-kubernetes\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.716993 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.716962    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-host-cni-netd\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.716993 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.716989    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-etc-sysctl-conf\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.717330 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.717005    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-lib-modules\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.717330 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.717018    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-etc-tuned\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.717330 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.717060    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnwlz\" (UniqueName: \"kubernetes.io/projected/ed43ef9c-bf6c-4f2c-96b5-fc31d24e5fd1-kube-api-access-gnwlz\") pod \"node-resolver-tpgsf\" (UID: \"ed43ef9c-bf6c-4f2c-96b5-fc31d24e5fd1\") " pod="openshift-dns/node-resolver-tpgsf"
Jul 22 03:33:33.717330 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.717088    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6ba9ef79-f774-4c64-b48e-af2461868076-ovnkube-config\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.717330 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.717129    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6adcbd54-1388-4223-8d6d-9423e9ebe09b-metrics-certs\") pod \"network-metrics-daemon-lctg9\" (UID: \"6adcbd54-1388-4223-8d6d-9423e9ebe09b\") " pod="openshift-multus/network-metrics-daemon-lctg9"
Jul 22 03:33:33.717330 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.717159    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ed43ef9c-bf6c-4f2c-96b5-fc31d24e5fd1-hosts-file\") pod \"node-resolver-tpgsf\" (UID: \"ed43ef9c-bf6c-4f2c-96b5-fc31d24e5fd1\") " pod="openshift-dns/node-resolver-tpgsf"
Jul 22 03:33:33.717330 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.717184    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-systemd-units\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.717330 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.717207    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-node-log\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.717330 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.717230    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-run\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.717330 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.717253    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-sys\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.717330 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.717276    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-host-slash\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.717330 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.717298    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-host-run-netns\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.717330 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.717320    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-log-socket\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.718098 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.718077    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 22 03:33:33.718208 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.718152    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7wdj7\""
Jul 22 03:33:33.718208 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.718155    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 22 03:33:33.718862 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.718848    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7srsk"
Jul 22 03:33:33.718994 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.718979    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 22 03:33:33.719053 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.718993    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 22 03:33:33.719929 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.719912    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 22 03:33:33.721104 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.721088    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-thjp8"
Jul 22 03:33:33.721854 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.721794    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 22 03:33:33.722287 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.722088    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 22 03:33:33.722287 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.722091    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 22 03:33:33.722287 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.722145    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 22 03:33:33.722287 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.722167    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 22 03:33:33.722287 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.722235    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-xbbs2\""
Jul 22 03:33:33.722287 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.722239    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rg47r\""
Jul 22 03:33:33.722287 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.722257    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-wsn7d\""
Jul 22 03:33:33.722604 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.722541    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 22 03:33:33.722604 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.722559    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 22 03:33:33.723446 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.723432    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p867k"
Jul 22 03:33:33.723509 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:33.723492    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p867k" podUID="16d564e4-f4b5-4293-b5c9-b7bd30239b47"
Jul 22 03:33:33.723668 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.723653    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 22 03:33:33.723821 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.723807    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4b8zd\""
Jul 22 03:33:33.724011 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.723856    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 22 03:33:33.727330 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.727311    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6wt45"
Jul 22 03:33:33.729539 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.729525    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vt8wv"
Jul 22 03:33:33.729699 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.729682    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-4487q\""
Jul 22 03:33:33.729799 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.729783    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 22 03:33:33.729852 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.729826    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 22 03:33:33.730151 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.730136    2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 22 03:33:33.732064 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.732051    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-llztn\""
Jul 22 03:33:33.732541 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.732525    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 22 03:33:33.732625 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.732533    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 22 03:33:33.733009 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.732995    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 22 03:33:33.758044 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.757994    2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-21 03:28:33 +0000 UTC" deadline="2028-04-13 19:37:50.42276425 +0000 UTC"
Jul 22 03:33:33.758044 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.758043    2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15160h4m16.664725407s"
Jul 22 03:33:33.771910 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.771856    2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qfhb2"
Jul 22 03:33:33.788576 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.788551    2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qfhb2"
Jul 22 03:33:33.813235 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.813219    2570 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 22 03:33:33.818299 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.818281    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-multus-socket-dir-parent\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.818365 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.818326    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/56a14a96-971e-4dab-a1bb-8e3edb6b73b2-serviceca\") pod \"node-ca-7srsk\" (UID: \"56a14a96-971e-4dab-a1bb-8e3edb6b73b2\") " pod="openshift-image-registry/node-ca-7srsk"
Jul 22 03:33:33.818365 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.818350    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-etc-sysconfig\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.818436 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.818371    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-etc-systemd\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.818436 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.818389    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc9dm\" (UniqueName: \"kubernetes.io/projected/6963eadb-e7e5-4f25-a6ea-fc0a57c951b8-kube-api-access-hc9dm\") pod \"network-check-target-k2tvl\" (UID: \"6963eadb-e7e5-4f25-a6ea-fc0a57c951b8\") " pod="openshift-network-diagnostics/network-check-target-k2tvl"
Jul 22 03:33:33.818436 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.818404    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-host-run-k8s-cni-cncf-io\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.818436 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.818424    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j78x2\" (UniqueName: \"kubernetes.io/projected/56a14a96-971e-4dab-a1bb-8e3edb6b73b2-kube-api-access-j78x2\") pod \"node-ca-7srsk\" (UID: \"56a14a96-971e-4dab-a1bb-8e3edb6b73b2\") " pod="openshift-image-registry/node-ca-7srsk"
Jul 22 03:33:33.818630 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.818447    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/26250740-d226-4262-bff6-caf04187cf8d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6wt45\" (UID: \"26250740-d226-4262-bff6-caf04187cf8d\") " pod="openshift-multus/multus-additional-cni-plugins-6wt45"
Jul 22 03:33:33.818630 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.818476    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-etc-systemd\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.818630 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.818529    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-etc-sysconfig\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.818630 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.818574    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed43ef9c-bf6c-4f2c-96b5-fc31d24e5fd1-tmp-dir\") pod \"node-resolver-tpgsf\" (UID: \"ed43ef9c-bf6c-4f2c-96b5-fc31d24e5fd1\") " pod="openshift-dns/node-resolver-tpgsf"
Jul 22 03:33:33.818819 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.818627    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-host-kubelet\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.818819 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.818655    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-var-lib-openvswitch\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.818819 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.818680    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-host-cni-netd\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.818819 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.818705    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-multus-daemon-config\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.818819 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.818705    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-host-kubelet\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.818819 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.818729    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/16d564e4-f4b5-4293-b5c9-b7bd30239b47-original-pull-secret\") pod \"global-pull-secret-syncer-p867k\" (UID: \"16d564e4-f4b5-4293-b5c9-b7bd30239b47\") " pod="kube-system/global-pull-secret-syncer-p867k"
Jul 22 03:33:33.818819 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.818739    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-var-lib-openvswitch\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.818819 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.818756    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rfhd\" (UniqueName: \"kubernetes.io/projected/8f72fc0e-f426-4006-8352-c66cc775d6d2-kube-api-access-4rfhd\") pod \"aws-ebs-csi-driver-node-vt8wv\" (UID: \"8f72fc0e-f426-4006-8352-c66cc775d6d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vt8wv"
Jul 22 03:33:33.818819 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.818756    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-host-cni-netd\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.818819 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.818784    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-etc-sysctl-conf\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.819196 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.818833    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-etc-tuned\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.819196 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.818867    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6adcbd54-1388-4223-8d6d-9423e9ebe09b-metrics-certs\") pod \"network-metrics-daemon-lctg9\" (UID: \"6adcbd54-1388-4223-8d6d-9423e9ebe09b\") " pod="openshift-multus/network-metrics-daemon-lctg9"
Jul 22 03:33:33.819196 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.818881    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed43ef9c-bf6c-4f2c-96b5-fc31d24e5fd1-tmp-dir\") pod \"node-resolver-tpgsf\" (UID: \"ed43ef9c-bf6c-4f2c-96b5-fc31d24e5fd1\") " pod="openshift-dns/node-resolver-tpgsf"
Jul 22 03:33:33.819196 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.818890    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-cni-binary-copy\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.819196 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.818910    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-host-var-lib-kubelet\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.819196 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.818932    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8f72fc0e-f426-4006-8352-c66cc775d6d2-etc-selinux\") pod \"aws-ebs-csi-driver-node-vt8wv\" (UID: \"8f72fc0e-f426-4006-8352-c66cc775d6d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vt8wv"
Jul 22 03:33:33.819196 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.818952    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-etc-sysctl-conf\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.819196 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:33.818988    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 03:33:33.819196 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819036    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h666d\" (UniqueName: \"kubernetes.io/projected/d13af00f-708e-4e6d-bd98-7f4d3afd4974-kube-api-access-h666d\") pod \"iptables-alerter-sgpqg\" (UID: \"d13af00f-708e-4e6d-bd98-7f4d3afd4974\") " pod="openshift-network-operator/iptables-alerter-sgpqg"
Jul 22 03:33:33.819196 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:33.819049    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6adcbd54-1388-4223-8d6d-9423e9ebe09b-metrics-certs podName:6adcbd54-1388-4223-8d6d-9423e9ebe09b nodeName:}" failed. No retries permitted until 2026-07-22 03:33:34.319033414 +0000 UTC m=+1.937243413 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6adcbd54-1388-4223-8d6d-9423e9ebe09b-metrics-certs") pod "network-metrics-daemon-lctg9" (UID: "6adcbd54-1388-4223-8d6d-9423e9ebe09b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 03:33:33.819196 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819077    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-systemd-units\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.819196 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819106    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/26250740-d226-4262-bff6-caf04187cf8d-os-release\") pod \"multus-additional-cni-plugins-6wt45\" (UID: \"26250740-d226-4262-bff6-caf04187cf8d\") " pod="openshift-multus/multus-additional-cni-plugins-6wt45"
Jul 22 03:33:33.819196 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819095    2570 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 22 03:33:33.819196 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819128    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-systemd-units\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.819196 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819134    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/26250740-d226-4262-bff6-caf04187cf8d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6wt45\" (UID: \"26250740-d226-4262-bff6-caf04187cf8d\") " pod="openshift-multus/multus-additional-cni-plugins-6wt45"
Jul 22 03:33:33.819196 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819160    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-run\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.819948 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819229    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-host-run-netns\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.819948 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819274    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-log-socket\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.819948 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819281    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-host-run-netns\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.819948 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819275    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-run\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.819948 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819304    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-multus-conf-dir\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.819948 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819325    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/16d564e4-f4b5-4293-b5c9-b7bd30239b47-dbus\") pod \"global-pull-secret-syncer-p867k\" (UID: \"16d564e4-f4b5-4293-b5c9-b7bd30239b47\") " pod="kube-system/global-pull-secret-syncer-p867k"
Jul 22 03:33:33.819948 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819340    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-log-socket\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.819948 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819341    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-run-openvswitch\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.819948 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819407    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-etc-kubernetes\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.819948 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819408    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-run-openvswitch\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.819948 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819436    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.819948 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819459    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-cnibin\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.819948 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819485    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-etc-kubernetes\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.819948 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819487    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8f72fc0e-f426-4006-8352-c66cc775d6d2-socket-dir\") pod \"aws-ebs-csi-driver-node-vt8wv\" (UID: \"8f72fc0e-f426-4006-8352-c66cc775d6d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vt8wv"
Jul 22 03:33:33.819948 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819529    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8f72fc0e-f426-4006-8352-c66cc775d6d2-registration-dir\") pod \"aws-ebs-csi-driver-node-vt8wv\" (UID: \"8f72fc0e-f426-4006-8352-c66cc775d6d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vt8wv"
Jul 22 03:33:33.819948 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819529    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.819948 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819554    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/26250740-d226-4262-bff6-caf04187cf8d-cni-binary-copy\") pod \"multus-additional-cni-plugins-6wt45\" (UID: \"26250740-d226-4262-bff6-caf04187cf8d\") " pod="openshift-multus/multus-additional-cni-plugins-6wt45"
Jul 22 03:33:33.820686 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819585    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/26250740-d226-4262-bff6-caf04187cf8d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6wt45\" (UID: \"26250740-d226-4262-bff6-caf04187cf8d\") " pod="openshift-multus/multus-additional-cni-plugins-6wt45"
Jul 22 03:33:33.820686 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819628    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-etc-sysctl-d\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.820686 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819654    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-tmp\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.820686 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819677    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-host-var-lib-cni-multus\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.820686 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819700    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8f72fc0e-f426-4006-8352-c66cc775d6d2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vt8wv\" (UID: \"8f72fc0e-f426-4006-8352-c66cc775d6d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vt8wv"
Jul 22 03:33:33.820686 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819726    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-run-systemd\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.820686 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819747    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-hostroot\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.820686 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819778    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d13af00f-708e-4e6d-bd98-7f4d3afd4974-host-slash\") pod \"iptables-alerter-sgpqg\" (UID: \"d13af00f-708e-4e6d-bd98-7f4d3afd4974\") " pod="openshift-network-operator/iptables-alerter-sgpqg"
Jul 22 03:33:33.820686 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819790    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-etc-sysctl-d\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.820686 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819800    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-run-systemd\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.820686 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819814    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/26250740-d226-4262-bff6-caf04187cf8d-cnibin\") pod \"multus-additional-cni-plugins-6wt45\" (UID: \"26250740-d226-4262-bff6-caf04187cf8d\") " pod="openshift-multus/multus-additional-cni-plugins-6wt45"
Jul 22 03:33:33.820686 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819836    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-var-lib-kubelet\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.820686 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819853    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-host\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.820686 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819877    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-run-ovn\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.820686 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819901    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-host-cni-bin\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.820686 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819907    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-var-lib-kubelet\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.820686 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819909    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-host\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.821403 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819926    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/56a14a96-971e-4dab-a1bb-8e3edb6b73b2-host\") pod \"node-ca-7srsk\" (UID: \"56a14a96-971e-4dab-a1bb-8e3edb6b73b2\") " pod="openshift-image-registry/node-ca-7srsk"
Jul 22 03:33:33.821403 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819942    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-run-ovn\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.821403 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819953    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-etc-openvswitch\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.821403 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819975    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-host-run-ovn-kubernetes\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.821403 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819981    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-etc-openvswitch\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.821403 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819954    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-host-cni-bin\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.821403 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.819993    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-host-run-netns\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.821403 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.820024    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-etc-kubernetes\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.821403 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.820032    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-host-run-ovn-kubernetes\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.821403 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.820050    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-lib-modules\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.821403 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.820071    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnwlz\" (UniqueName: \"kubernetes.io/projected/ed43ef9c-bf6c-4f2c-96b5-fc31d24e5fd1-kube-api-access-gnwlz\") pod \"node-resolver-tpgsf\" (UID: \"ed43ef9c-bf6c-4f2c-96b5-fc31d24e5fd1\") " pod="openshift-dns/node-resolver-tpgsf"
Jul 22 03:33:33.821403 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.820092    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6ba9ef79-f774-4c64-b48e-af2461868076-ovnkube-config\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.821403 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.820113    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-host-var-lib-cni-bin\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.821403 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.820135    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d13af00f-708e-4e6d-bd98-7f4d3afd4974-iptables-alerter-script\") pod \"iptables-alerter-sgpqg\" (UID: \"d13af00f-708e-4e6d-bd98-7f4d3afd4974\") " pod="openshift-network-operator/iptables-alerter-sgpqg"
Jul 22 03:33:33.821403 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.820159    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/22a4fd03-a5a7-4d6e-97fb-fb63e5be241c-agent-certs\") pod \"konnectivity-agent-thjp8\" (UID: \"22a4fd03-a5a7-4d6e-97fb-fb63e5be241c\") " pod="kube-system/konnectivity-agent-thjp8"
Jul 22 03:33:33.821403 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.820154    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-lib-modules\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.821403 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.820184    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ed43ef9c-bf6c-4f2c-96b5-fc31d24e5fd1-hosts-file\") pod \"node-resolver-tpgsf\" (UID: \"ed43ef9c-bf6c-4f2c-96b5-fc31d24e5fd1\") " pod="openshift-dns/node-resolver-tpgsf"
Jul 22 03:33:33.821921 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.820207    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-node-log\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.821921 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.820223    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65rk4\" (UniqueName: \"kubernetes.io/projected/26250740-d226-4262-bff6-caf04187cf8d-kube-api-access-65rk4\") pod \"multus-additional-cni-plugins-6wt45\" (UID: \"26250740-d226-4262-bff6-caf04187cf8d\") " pod="openshift-multus/multus-additional-cni-plugins-6wt45"
Jul 22 03:33:33.821921 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.820238    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-sys\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.821921 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.820253    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-host-slash\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.821921 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.820265    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ed43ef9c-bf6c-4f2c-96b5-fc31d24e5fd1-hosts-file\") pod \"node-resolver-tpgsf\" (UID: \"ed43ef9c-bf6c-4f2c-96b5-fc31d24e5fd1\") " pod="openshift-dns/node-resolver-tpgsf"
Jul 22 03:33:33.821921 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.820287    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q79df\" (UniqueName: \"kubernetes.io/projected/6ba9ef79-f774-4c64-b48e-af2461868076-kube-api-access-q79df\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.821921 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.820319    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-host-slash\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.821921 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.820318    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-host-run-multus-certs\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.821921 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.820363    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-sys\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.821921 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.820438    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmx9x\" (UniqueName: \"kubernetes.io/projected/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-kube-api-access-jmx9x\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.821921 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.820447    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6ba9ef79-f774-4c64-b48e-af2461868076-node-log\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.821921 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.820480    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6ba9ef79-f774-4c64-b48e-af2461868076-env-overrides\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.821921 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.820597    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-multus-cni-dir\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.821921 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.820599    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6ba9ef79-f774-4c64-b48e-af2461868076-ovnkube-config\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.821921 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.820704    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-os-release\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.821921 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.820757    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tl9lk\" (UniqueName: \"kubernetes.io/projected/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-kube-api-access-tl9lk\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.821921 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.820802    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6ba9ef79-f774-4c64-b48e-af2461868076-ovnkube-script-lib\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.822395 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.820829    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-system-cni-dir\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.822395 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.820853    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/16d564e4-f4b5-4293-b5c9-b7bd30239b47-kubelet-config\") pod \"global-pull-secret-syncer-p867k\" (UID: \"16d564e4-f4b5-4293-b5c9-b7bd30239b47\") " pod="kube-system/global-pull-secret-syncer-p867k"
Jul 22 03:33:33.822395 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.820877    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8f72fc0e-f426-4006-8352-c66cc775d6d2-device-dir\") pod \"aws-ebs-csi-driver-node-vt8wv\" (UID: \"8f72fc0e-f426-4006-8352-c66cc775d6d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vt8wv"
Jul 22 03:33:33.822395 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.820902    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8f72fc0e-f426-4006-8352-c66cc775d6d2-sys-fs\") pod \"aws-ebs-csi-driver-node-vt8wv\" (UID: \"8f72fc0e-f426-4006-8352-c66cc775d6d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vt8wv"
Jul 22 03:33:33.822395 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.820924    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/22a4fd03-a5a7-4d6e-97fb-fb63e5be241c-konnectivity-ca\") pod \"konnectivity-agent-thjp8\" (UID: \"22a4fd03-a5a7-4d6e-97fb-fb63e5be241c\") " pod="kube-system/konnectivity-agent-thjp8"
Jul 22 03:33:33.822395 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.820962    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/26250740-d226-4262-bff6-caf04187cf8d-system-cni-dir\") pod \"multus-additional-cni-plugins-6wt45\" (UID: \"26250740-d226-4262-bff6-caf04187cf8d\") " pod="openshift-multus/multus-additional-cni-plugins-6wt45"
Jul 22 03:33:33.822395 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.820970    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6ba9ef79-f774-4c64-b48e-af2461868076-env-overrides\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.822395 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.820991    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-etc-modprobe-d\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.822395 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.821027    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6ba9ef79-f774-4c64-b48e-af2461868076-ovn-node-metrics-cert\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.822395 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.821057    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctpr8\" (UniqueName: \"kubernetes.io/projected/6adcbd54-1388-4223-8d6d-9423e9ebe09b-kube-api-access-ctpr8\") pod \"network-metrics-daemon-lctg9\" (UID: \"6adcbd54-1388-4223-8d6d-9423e9ebe09b\") " pod="openshift-multus/network-metrics-daemon-lctg9"
Jul 22 03:33:33.822395 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.821090    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-etc-modprobe-d\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.822395 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.821224    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6ba9ef79-f774-4c64-b48e-af2461868076-ovnkube-script-lib\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.822737 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.822446    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-tmp\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.822737 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.822479    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-etc-tuned\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.823052 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.823036    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6ba9ef79-f774-4c64-b48e-af2461868076-ovn-node-metrics-cert\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.835108 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:33.835086    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 03:33:33.835224 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:33.835117    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 03:33:33.835224 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:33.835130    2570 projected.go:196] Error preparing data for projected volume kube-api-access-hc9dm for pod openshift-network-diagnostics/network-check-target-k2tvl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 03:33:33.835224 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:33.835194    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6963eadb-e7e5-4f25-a6ea-fc0a57c951b8-kube-api-access-hc9dm podName:6963eadb-e7e5-4f25-a6ea-fc0a57c951b8 nodeName:}" failed. No retries permitted until 2026-07-22 03:33:34.335174466 +0000 UTC m=+1.953384471 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-hc9dm" (UniqueName: "kubernetes.io/projected/6963eadb-e7e5-4f25-a6ea-fc0a57c951b8-kube-api-access-hc9dm") pod "network-check-target-k2tvl" (UID: "6963eadb-e7e5-4f25-a6ea-fc0a57c951b8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 03:33:33.837027 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.837003    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctpr8\" (UniqueName: \"kubernetes.io/projected/6adcbd54-1388-4223-8d6d-9423e9ebe09b-kube-api-access-ctpr8\") pod \"network-metrics-daemon-lctg9\" (UID: \"6adcbd54-1388-4223-8d6d-9423e9ebe09b\") " pod="openshift-multus/network-metrics-daemon-lctg9"
Jul 22 03:33:33.837027 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.837021    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnwlz\" (UniqueName: \"kubernetes.io/projected/ed43ef9c-bf6c-4f2c-96b5-fc31d24e5fd1-kube-api-access-gnwlz\") pod \"node-resolver-tpgsf\" (UID: \"ed43ef9c-bf6c-4f2c-96b5-fc31d24e5fd1\") " pod="openshift-dns/node-resolver-tpgsf"
Jul 22 03:33:33.837215 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.837197    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q79df\" (UniqueName: \"kubernetes.io/projected/6ba9ef79-f774-4c64-b48e-af2461868076-kube-api-access-q79df\") pod \"ovnkube-node-ndfqq\" (UID: \"6ba9ef79-f774-4c64-b48e-af2461868076\") " pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:33.837549 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.837533    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmx9x\" (UniqueName: \"kubernetes.io/projected/bd93ded5-d41d-4192-967e-ea95b5e4bbe8-kube-api-access-jmx9x\") pod \"tuned-wt6b5\" (UID: \"bd93ded5-d41d-4192-967e-ea95b5e4bbe8\") " pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:33.921311 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.921284    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-host-run-netns\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.921311 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.921313    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-etc-kubernetes\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.921567 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.921330    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-host-var-lib-cni-bin\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.921567 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.921348    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d13af00f-708e-4e6d-bd98-7f4d3afd4974-iptables-alerter-script\") pod \"iptables-alerter-sgpqg\" (UID: \"d13af00f-708e-4e6d-bd98-7f4d3afd4974\") " pod="openshift-network-operator/iptables-alerter-sgpqg"
Jul 22 03:33:33.921567 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.921367    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/22a4fd03-a5a7-4d6e-97fb-fb63e5be241c-agent-certs\") pod \"konnectivity-agent-thjp8\" (UID: \"22a4fd03-a5a7-4d6e-97fb-fb63e5be241c\") " pod="kube-system/konnectivity-agent-thjp8"
Jul 22 03:33:33.921567 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.921392    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65rk4\" (UniqueName: \"kubernetes.io/projected/26250740-d226-4262-bff6-caf04187cf8d-kube-api-access-65rk4\") pod \"multus-additional-cni-plugins-6wt45\" (UID: \"26250740-d226-4262-bff6-caf04187cf8d\") " pod="openshift-multus/multus-additional-cni-plugins-6wt45"
Jul 22 03:33:33.921567 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.921394    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-host-run-netns\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.921567 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.921408    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-etc-kubernetes\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.921567 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.921417    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-host-var-lib-cni-bin\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.921567 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.921441    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-host-run-multus-certs\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.921567 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.921471    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-multus-cni-dir\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.921567 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.921476    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-host-run-multus-certs\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.921567 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.921495    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-os-release\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.921567 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.921518    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tl9lk\" (UniqueName: \"kubernetes.io/projected/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-kube-api-access-tl9lk\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.921567 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.921541    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-system-cni-dir\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.921567 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.921564    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/16d564e4-f4b5-4293-b5c9-b7bd30239b47-kubelet-config\") pod \"global-pull-secret-syncer-p867k\" (UID: \"16d564e4-f4b5-4293-b5c9-b7bd30239b47\") " pod="kube-system/global-pull-secret-syncer-p867k"
Jul 22 03:33:33.922238 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.921587    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8f72fc0e-f426-4006-8352-c66cc775d6d2-device-dir\") pod \"aws-ebs-csi-driver-node-vt8wv\" (UID: \"8f72fc0e-f426-4006-8352-c66cc775d6d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vt8wv"
Jul 22 03:33:33.922238 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.921592    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-multus-cni-dir\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.922238 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.921628    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8f72fc0e-f426-4006-8352-c66cc775d6d2-sys-fs\") pod \"aws-ebs-csi-driver-node-vt8wv\" (UID: \"8f72fc0e-f426-4006-8352-c66cc775d6d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vt8wv"
Jul 22 03:33:33.922238 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.921653    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/22a4fd03-a5a7-4d6e-97fb-fb63e5be241c-konnectivity-ca\") pod \"konnectivity-agent-thjp8\" (UID: \"22a4fd03-a5a7-4d6e-97fb-fb63e5be241c\") " pod="kube-system/konnectivity-agent-thjp8"
Jul 22 03:33:33.922238 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.921676    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-os-release\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.922238 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.921678    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/26250740-d226-4262-bff6-caf04187cf8d-system-cni-dir\") pod \"multus-additional-cni-plugins-6wt45\" (UID: \"26250740-d226-4262-bff6-caf04187cf8d\") " pod="openshift-multus/multus-additional-cni-plugins-6wt45"
Jul 22 03:33:33.922238 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.921684    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/16d564e4-f4b5-4293-b5c9-b7bd30239b47-kubelet-config\") pod \"global-pull-secret-syncer-p867k\" (UID: \"16d564e4-f4b5-4293-b5c9-b7bd30239b47\") " pod="kube-system/global-pull-secret-syncer-p867k"
Jul 22 03:33:33.922238 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.921715    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/26250740-d226-4262-bff6-caf04187cf8d-system-cni-dir\") pod \"multus-additional-cni-plugins-6wt45\" (UID: \"26250740-d226-4262-bff6-caf04187cf8d\") " pod="openshift-multus/multus-additional-cni-plugins-6wt45"
Jul 22 03:33:33.922238 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.921716    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-multus-socket-dir-parent\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.922238 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.921733    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-system-cni-dir\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.922238 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.921747    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/56a14a96-971e-4dab-a1bb-8e3edb6b73b2-serviceca\") pod \"node-ca-7srsk\" (UID: \"56a14a96-971e-4dab-a1bb-8e3edb6b73b2\") " pod="openshift-image-registry/node-ca-7srsk"
Jul 22 03:33:33.922238 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.921754    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-multus-socket-dir-parent\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.922238 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.921777    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8f72fc0e-f426-4006-8352-c66cc775d6d2-sys-fs\") pod \"aws-ebs-csi-driver-node-vt8wv\" (UID: \"8f72fc0e-f426-4006-8352-c66cc775d6d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vt8wv"
Jul 22 03:33:33.922238 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.921794    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-host-run-k8s-cni-cncf-io\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.922238 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.921820    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j78x2\" (UniqueName: \"kubernetes.io/projected/56a14a96-971e-4dab-a1bb-8e3edb6b73b2-kube-api-access-j78x2\") pod \"node-ca-7srsk\" (UID: \"56a14a96-971e-4dab-a1bb-8e3edb6b73b2\") " pod="openshift-image-registry/node-ca-7srsk"
Jul 22 03:33:33.922238 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.921844    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/26250740-d226-4262-bff6-caf04187cf8d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6wt45\" (UID: \"26250740-d226-4262-bff6-caf04187cf8d\") " pod="openshift-multus/multus-additional-cni-plugins-6wt45"
Jul 22 03:33:33.922238 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.921849    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-host-run-k8s-cni-cncf-io\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.922952 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.921820    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8f72fc0e-f426-4006-8352-c66cc775d6d2-device-dir\") pod \"aws-ebs-csi-driver-node-vt8wv\" (UID: \"8f72fc0e-f426-4006-8352-c66cc775d6d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vt8wv"
Jul 22 03:33:33.922952 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.921876    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-multus-daemon-config\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.922952 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.921927    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/16d564e4-f4b5-4293-b5c9-b7bd30239b47-original-pull-secret\") pod \"global-pull-secret-syncer-p867k\" (UID: \"16d564e4-f4b5-4293-b5c9-b7bd30239b47\") " pod="kube-system/global-pull-secret-syncer-p867k"
Jul 22 03:33:33.922952 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.921958    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rfhd\" (UniqueName: \"kubernetes.io/projected/8f72fc0e-f426-4006-8352-c66cc775d6d2-kube-api-access-4rfhd\") pod \"aws-ebs-csi-driver-node-vt8wv\" (UID: \"8f72fc0e-f426-4006-8352-c66cc775d6d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vt8wv"
Jul 22 03:33:33.922952 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.921999    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-cni-binary-copy\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.922952 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.922035    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-host-var-lib-kubelet\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.922952 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.922059    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8f72fc0e-f426-4006-8352-c66cc775d6d2-etc-selinux\") pod \"aws-ebs-csi-driver-node-vt8wv\" (UID: \"8f72fc0e-f426-4006-8352-c66cc775d6d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vt8wv"
Jul 22 03:33:33.922952 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.922057    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d13af00f-708e-4e6d-bd98-7f4d3afd4974-iptables-alerter-script\") pod \"iptables-alerter-sgpqg\" (UID: \"d13af00f-708e-4e6d-bd98-7f4d3afd4974\") " pod="openshift-network-operator/iptables-alerter-sgpqg"
Jul 22 03:33:33.922952 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:33.922087    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 03:33:33.922952 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.922107    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h666d\" (UniqueName: \"kubernetes.io/projected/d13af00f-708e-4e6d-bd98-7f4d3afd4974-kube-api-access-h666d\") pod \"iptables-alerter-sgpqg\" (UID: \"d13af00f-708e-4e6d-bd98-7f4d3afd4974\") " pod="openshift-network-operator/iptables-alerter-sgpqg"
Jul 22 03:33:33.922952 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.922142    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/26250740-d226-4262-bff6-caf04187cf8d-os-release\") pod \"multus-additional-cni-plugins-6wt45\" (UID: \"26250740-d226-4262-bff6-caf04187cf8d\") " pod="openshift-multus/multus-additional-cni-plugins-6wt45"
Jul 22 03:33:33.922952 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.922168    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8f72fc0e-f426-4006-8352-c66cc775d6d2-etc-selinux\") pod \"aws-ebs-csi-driver-node-vt8wv\" (UID: \"8f72fc0e-f426-4006-8352-c66cc775d6d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vt8wv"
Jul 22 03:33:33.922952 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.922169    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/26250740-d226-4262-bff6-caf04187cf8d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6wt45\" (UID: \"26250740-d226-4262-bff6-caf04187cf8d\") " pod="openshift-multus/multus-additional-cni-plugins-6wt45"
Jul 22 03:33:33.922952 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.922201    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/56a14a96-971e-4dab-a1bb-8e3edb6b73b2-serviceca\") pod \"node-ca-7srsk\" (UID: \"56a14a96-971e-4dab-a1bb-8e3edb6b73b2\") " pod="openshift-image-registry/node-ca-7srsk"
Jul 22 03:33:33.922952 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.922203    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-multus-conf-dir\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.922952 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.922238    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-multus-conf-dir\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.922952 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.922251    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/16d564e4-f4b5-4293-b5c9-b7bd30239b47-dbus\") pod \"global-pull-secret-syncer-p867k\" (UID: \"16d564e4-f4b5-4293-b5c9-b7bd30239b47\") " pod="kube-system/global-pull-secret-syncer-p867k"
Jul 22 03:33:33.922952 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.922278    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-cnibin\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.923448 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.922316    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8f72fc0e-f426-4006-8352-c66cc775d6d2-socket-dir\") pod \"aws-ebs-csi-driver-node-vt8wv\" (UID: \"8f72fc0e-f426-4006-8352-c66cc775d6d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vt8wv"
Jul 22 03:33:33.923448 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.922325    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-cnibin\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.923448 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.922331    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/22a4fd03-a5a7-4d6e-97fb-fb63e5be241c-konnectivity-ca\") pod \"konnectivity-agent-thjp8\" (UID: \"22a4fd03-a5a7-4d6e-97fb-fb63e5be241c\") " pod="kube-system/konnectivity-agent-thjp8"
Jul 22 03:33:33.923448 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.922347    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8f72fc0e-f426-4006-8352-c66cc775d6d2-registration-dir\") pod \"aws-ebs-csi-driver-node-vt8wv\" (UID: \"8f72fc0e-f426-4006-8352-c66cc775d6d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vt8wv"
Jul 22 03:33:33.923448 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:33.922373    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16d564e4-f4b5-4293-b5c9-b7bd30239b47-original-pull-secret podName:16d564e4-f4b5-4293-b5c9-b7bd30239b47 nodeName:}" failed. No retries permitted until 2026-07-22 03:33:34.422355914 +0000 UTC m=+2.040565902 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/16d564e4-f4b5-4293-b5c9-b7bd30239b47-original-pull-secret") pod "global-pull-secret-syncer-p867k" (UID: "16d564e4-f4b5-4293-b5c9-b7bd30239b47") : object "kube-system"/"original-pull-secret" not registered
Jul 22 03:33:33.923448 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.922394    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-multus-daemon-config\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.923448 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.922403    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/26250740-d226-4262-bff6-caf04187cf8d-cni-binary-copy\") pod \"multus-additional-cni-plugins-6wt45\" (UID: \"26250740-d226-4262-bff6-caf04187cf8d\") " pod="openshift-multus/multus-additional-cni-plugins-6wt45"
Jul 22 03:33:33.923448 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.922396    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8f72fc0e-f426-4006-8352-c66cc775d6d2-registration-dir\") pod \"aws-ebs-csi-driver-node-vt8wv\" (UID: \"8f72fc0e-f426-4006-8352-c66cc775d6d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vt8wv"
Jul 22 03:33:33.923448 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.922432    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/26250740-d226-4262-bff6-caf04187cf8d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6wt45\" (UID: \"26250740-d226-4262-bff6-caf04187cf8d\") " pod="openshift-multus/multus-additional-cni-plugins-6wt45"
Jul 22 03:33:33.923448 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.922451    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8f72fc0e-f426-4006-8352-c66cc775d6d2-socket-dir\") pod \"aws-ebs-csi-driver-node-vt8wv\" (UID: \"8f72fc0e-f426-4006-8352-c66cc775d6d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vt8wv"
Jul 22 03:33:33.923448 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.922461    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-host-var-lib-cni-multus\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.923448 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.922371    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/16d564e4-f4b5-4293-b5c9-b7bd30239b47-dbus\") pod \"global-pull-secret-syncer-p867k\" (UID: \"16d564e4-f4b5-4293-b5c9-b7bd30239b47\") " pod="kube-system/global-pull-secret-syncer-p867k"
Jul 22 03:33:33.923448 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.922476    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-host-var-lib-kubelet\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.923448 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.922494    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8f72fc0e-f426-4006-8352-c66cc775d6d2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vt8wv\" (UID: \"8f72fc0e-f426-4006-8352-c66cc775d6d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vt8wv"
Jul 22 03:33:33.923448 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.922503    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-host-var-lib-cni-multus\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.923448 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.922520    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-hostroot\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.923938 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.922530    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/26250740-d226-4262-bff6-caf04187cf8d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6wt45\" (UID: \"26250740-d226-4262-bff6-caf04187cf8d\") " pod="openshift-multus/multus-additional-cni-plugins-6wt45"
Jul 22 03:33:33.923938 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.922544    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d13af00f-708e-4e6d-bd98-7f4d3afd4974-host-slash\") pod \"iptables-alerter-sgpqg\" (UID: \"d13af00f-708e-4e6d-bd98-7f4d3afd4974\") " pod="openshift-network-operator/iptables-alerter-sgpqg"
Jul 22 03:33:33.923938 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.922573    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/26250740-d226-4262-bff6-caf04187cf8d-cnibin\") pod \"multus-additional-cni-plugins-6wt45\" (UID: \"26250740-d226-4262-bff6-caf04187cf8d\") " pod="openshift-multus/multus-additional-cni-plugins-6wt45"
Jul 22 03:33:33.923938 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.922580    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8f72fc0e-f426-4006-8352-c66cc775d6d2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vt8wv\" (UID: \"8f72fc0e-f426-4006-8352-c66cc775d6d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vt8wv"
Jul 22 03:33:33.923938 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.922630    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/56a14a96-971e-4dab-a1bb-8e3edb6b73b2-host\") pod \"node-ca-7srsk\" (UID: \"56a14a96-971e-4dab-a1bb-8e3edb6b73b2\") " pod="openshift-image-registry/node-ca-7srsk"
Jul 22 03:33:33.923938 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.922602    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-hostroot\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.923938 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.922646    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d13af00f-708e-4e6d-bd98-7f4d3afd4974-host-slash\") pod \"iptables-alerter-sgpqg\" (UID: \"d13af00f-708e-4e6d-bd98-7f4d3afd4974\") " pod="openshift-network-operator/iptables-alerter-sgpqg"
Jul 22 03:33:33.923938 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.922705    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/56a14a96-971e-4dab-a1bb-8e3edb6b73b2-host\") pod \"node-ca-7srsk\" (UID: \"56a14a96-971e-4dab-a1bb-8e3edb6b73b2\") " pod="openshift-image-registry/node-ca-7srsk"
Jul 22 03:33:33.923938 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.922716    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/26250740-d226-4262-bff6-caf04187cf8d-os-release\") pod \"multus-additional-cni-plugins-6wt45\" (UID: \"26250740-d226-4262-bff6-caf04187cf8d\") " pod="openshift-multus/multus-additional-cni-plugins-6wt45"
Jul 22 03:33:33.923938 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.922744    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/26250740-d226-4262-bff6-caf04187cf8d-cnibin\") pod \"multus-additional-cni-plugins-6wt45\" (UID: \"26250740-d226-4262-bff6-caf04187cf8d\") " pod="openshift-multus/multus-additional-cni-plugins-6wt45"
Jul 22 03:33:33.923938 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.922844    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-cni-binary-copy\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.923938 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.922919    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/26250740-d226-4262-bff6-caf04187cf8d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6wt45\" (UID: \"26250740-d226-4262-bff6-caf04187cf8d\") " pod="openshift-multus/multus-additional-cni-plugins-6wt45"
Jul 22 03:33:33.923938 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.923117    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/26250740-d226-4262-bff6-caf04187cf8d-cni-binary-copy\") pod \"multus-additional-cni-plugins-6wt45\" (UID: \"26250740-d226-4262-bff6-caf04187cf8d\") " pod="openshift-multus/multus-additional-cni-plugins-6wt45"
Jul 22 03:33:33.923938 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.923128    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/26250740-d226-4262-bff6-caf04187cf8d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6wt45\" (UID: \"26250740-d226-4262-bff6-caf04187cf8d\") " pod="openshift-multus/multus-additional-cni-plugins-6wt45"
Jul 22 03:33:33.925105 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.925085    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/22a4fd03-a5a7-4d6e-97fb-fb63e5be241c-agent-certs\") pod \"konnectivity-agent-thjp8\" (UID: \"22a4fd03-a5a7-4d6e-97fb-fb63e5be241c\") " pod="kube-system/konnectivity-agent-thjp8"
Jul 22 03:33:33.931209 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.931184    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tl9lk\" (UniqueName: \"kubernetes.io/projected/00bf5310-aba0-4e3e-9ac8-5d68e7de24bf-kube-api-access-tl9lk\") pod \"multus-xbhc9\" (UID: \"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf\") " pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:33.931308 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.931216    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65rk4\" (UniqueName: \"kubernetes.io/projected/26250740-d226-4262-bff6-caf04187cf8d-kube-api-access-65rk4\") pod \"multus-additional-cni-plugins-6wt45\" (UID: \"26250740-d226-4262-bff6-caf04187cf8d\") " pod="openshift-multus/multus-additional-cni-plugins-6wt45"
Jul 22 03:33:33.931308 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.931186    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h666d\" (UniqueName: \"kubernetes.io/projected/d13af00f-708e-4e6d-bd98-7f4d3afd4974-kube-api-access-h666d\") pod \"iptables-alerter-sgpqg\" (UID: \"d13af00f-708e-4e6d-bd98-7f4d3afd4974\") " pod="openshift-network-operator/iptables-alerter-sgpqg"
Jul 22 03:33:33.931936 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.931921    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rfhd\" (UniqueName: \"kubernetes.io/projected/8f72fc0e-f426-4006-8352-c66cc775d6d2-kube-api-access-4rfhd\") pod \"aws-ebs-csi-driver-node-vt8wv\" (UID: \"8f72fc0e-f426-4006-8352-c66cc775d6d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vt8wv"
Jul 22 03:33:33.932302 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:33.932287    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j78x2\" (UniqueName: \"kubernetes.io/projected/56a14a96-971e-4dab-a1bb-8e3edb6b73b2-kube-api-access-j78x2\") pod \"node-ca-7srsk\" (UID: \"56a14a96-971e-4dab-a1bb-8e3edb6b73b2\") " pod="openshift-image-registry/node-ca-7srsk"
Jul 22 03:33:34.023065 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:34.022996    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d1ebdc1a61d7d6c7a73ef206e8173b6.slice/crio-5d9146a7423244fb796db4e18c22f0932b32e5129851d4001d6e26812c499f23 WatchSource:0}: Error finding container 5d9146a7423244fb796db4e18c22f0932b32e5129851d4001d6e26812c499f23: Status 404 returned error can't find the container with id 5d9146a7423244fb796db4e18c22f0932b32e5129851d4001d6e26812c499f23
Jul 22 03:33:34.029264 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:34.029244    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 03:33:34.037352 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:34.037326    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wt6b5"
Jul 22 03:33:34.037536 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:34.037512    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1decd6d269268ae8653188a0a80fb590.slice/crio-266f48bf39fde3c39844091befdd99d75aefdd6c44b960b3d887b1e7693d7aca WatchSource:0}: Error finding container 266f48bf39fde3c39844091befdd99d75aefdd6c44b960b3d887b1e7693d7aca: Status 404 returned error can't find the container with id 266f48bf39fde3c39844091befdd99d75aefdd6c44b960b3d887b1e7693d7aca
Jul 22 03:33:34.044609 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:34.044553    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tpgsf"
Jul 22 03:33:34.047088 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:34.047060    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd93ded5_d41d_4192_967e_ea95b5e4bbe8.slice/crio-903f0b08b9fd7775d0e77f277bb373dea838dd922f3fc7f7b801879a35828426 WatchSource:0}: Error finding container 903f0b08b9fd7775d0e77f277bb373dea838dd922f3fc7f7b801879a35828426: Status 404 returned error can't find the container with id 903f0b08b9fd7775d0e77f277bb373dea838dd922f3fc7f7b801879a35828426
Jul 22 03:33:34.053587 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:34.053562    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded43ef9c_bf6c_4f2c_96b5_fc31d24e5fd1.slice/crio-579a445cb370061f4bf788c53ac2085ec8c44f19c2f9ea8c502261825557dd24 WatchSource:0}: Error finding container 579a445cb370061f4bf788c53ac2085ec8c44f19c2f9ea8c502261825557dd24: Status 404 returned error can't find the container with id 579a445cb370061f4bf788c53ac2085ec8c44f19c2f9ea8c502261825557dd24
Jul 22 03:33:34.060304 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:34.060283    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:33:34.070070 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:34.070039    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ba9ef79_f774_4c64_b48e_af2461868076.slice/crio-cbc7eca4eaf6800388a4dd4d1c4ae76f36e1caea8a0303195d3ee21b78e7b3d5 WatchSource:0}: Error finding container cbc7eca4eaf6800388a4dd4d1c4ae76f36e1caea8a0303195d3ee21b78e7b3d5: Status 404 returned error can't find the container with id cbc7eca4eaf6800388a4dd4d1c4ae76f36e1caea8a0303195d3ee21b78e7b3d5
Jul 22 03:33:34.076732 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:34.076714    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xbhc9"
Jul 22 03:33:34.085599 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:34.085576    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00bf5310_aba0_4e3e_9ac8_5d68e7de24bf.slice/crio-a9fbf8433d359426dc1698c8541c0beac56b47f397dd8f7c9f24f1f7196c1808 WatchSource:0}: Error finding container a9fbf8433d359426dc1698c8541c0beac56b47f397dd8f7c9f24f1f7196c1808: Status 404 returned error can't find the container with id a9fbf8433d359426dc1698c8541c0beac56b47f397dd8f7c9f24f1f7196c1808
Jul 22 03:33:34.091784 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:34.091754    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-sgpqg"
Jul 22 03:33:34.100607 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:34.100587    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd13af00f_708e_4e6d_bd98_7f4d3afd4974.slice/crio-d22570cd5af98aa399537c991ca8a1c2f3be4f5b0c09bc5b58ab34777b0675ca WatchSource:0}: Error finding container d22570cd5af98aa399537c991ca8a1c2f3be4f5b0c09bc5b58ab34777b0675ca: Status 404 returned error can't find the container with id d22570cd5af98aa399537c991ca8a1c2f3be4f5b0c09bc5b58ab34777b0675ca
Jul 22 03:33:34.105749 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:34.105733    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7srsk"
Jul 22 03:33:34.114688 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:34.114661    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56a14a96_971e_4dab_a1bb_8e3edb6b73b2.slice/crio-80f8f26638b8b97ef150a01d9e9e9ad006f253a4ec259c355bb77dd1b5e20ba7 WatchSource:0}: Error finding container 80f8f26638b8b97ef150a01d9e9e9ad006f253a4ec259c355bb77dd1b5e20ba7: Status 404 returned error can't find the container with id 80f8f26638b8b97ef150a01d9e9e9ad006f253a4ec259c355bb77dd1b5e20ba7
Jul 22 03:33:34.130305 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:34.130279    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-thjp8"
Jul 22 03:33:34.140300 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:34.140278    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22a4fd03_a5a7_4d6e_97fb_fb63e5be241c.slice/crio-68287d38309ec7cace53b2cbbb3c2436d6f704dd84d6ffb8e523d3ca36ea5ec9 WatchSource:0}: Error finding container 68287d38309ec7cace53b2cbbb3c2436d6f704dd84d6ffb8e523d3ca36ea5ec9: Status 404 returned error can't find the container with id 68287d38309ec7cace53b2cbbb3c2436d6f704dd84d6ffb8e523d3ca36ea5ec9
Jul 22 03:33:34.148707 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:34.148684    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6wt45"
Jul 22 03:33:34.155714 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:34.155694    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vt8wv"
Jul 22 03:33:34.158917 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:34.158891    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26250740_d226_4262_bff6_caf04187cf8d.slice/crio-237194d6d685d29733ad0612bd77f4bd311126188927048871eddc1190cfb1e5 WatchSource:0}: Error finding container 237194d6d685d29733ad0612bd77f4bd311126188927048871eddc1190cfb1e5: Status 404 returned error can't find the container with id 237194d6d685d29733ad0612bd77f4bd311126188927048871eddc1190cfb1e5
Jul 22 03:33:34.164542 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:33:34.164520    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f72fc0e_f426_4006_8352_c66cc775d6d2.slice/crio-005205dc72664be4cd90756780837ee8803ee85726ecd8849a161b95a7514e8d WatchSource:0}: Error finding container 005205dc72664be4cd90756780837ee8803ee85726ecd8849a161b95a7514e8d: Status 404 returned error can't find the container with id 005205dc72664be4cd90756780837ee8803ee85726ecd8849a161b95a7514e8d
Jul 22 03:33:34.166291 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:34.166272    2570 kuberuntime_manager.go:1459] "Unhandled Error" err="container csi-driver start failed in pod aws-ebs-csi-driver-node-vt8wv_openshift-cluster-csi-drivers(8f72fc0e-f426-4006-8352-c66cc775d6d2): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Jul 22 03:33:34.167016 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:34.166994    2570 kuberuntime_manager.go:1459] "Unhandled Error" err="container csi-node-driver-registrar start failed in pod aws-ebs-csi-driver-node-vt8wv_openshift-cluster-csi-drivers(8f72fc0e-f426-4006-8352-c66cc775d6d2): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Jul 22 03:33:34.168892 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:34.168839    2570 kuberuntime_manager.go:1459] "Unhandled Error" err="container csi-liveness-probe start failed in pod aws-ebs-csi-driver-node-vt8wv_openshift-cluster-csi-drivers(8f72fc0e-f426-4006-8352-c66cc775d6d2): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Jul 22 03:33:34.168959 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:34.168928    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"csi-driver\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"csi-node-driver-registrar\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"csi-liveness-probe\" with ErrImagePull: \"pull QPS exceeded\"]" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vt8wv" podUID="8f72fc0e-f426-4006-8352-c66cc775d6d2"
Jul 22 03:33:34.325030 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:34.324927    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6adcbd54-1388-4223-8d6d-9423e9ebe09b-metrics-certs\") pod \"network-metrics-daemon-lctg9\" (UID: \"6adcbd54-1388-4223-8d6d-9423e9ebe09b\") " pod="openshift-multus/network-metrics-daemon-lctg9"
Jul 22 03:33:34.325200 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:34.325068    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 03:33:34.325200 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:34.325141    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6adcbd54-1388-4223-8d6d-9423e9ebe09b-metrics-certs podName:6adcbd54-1388-4223-8d6d-9423e9ebe09b nodeName:}" failed. No retries permitted until 2026-07-22 03:33:35.325122668 +0000 UTC m=+2.943332672 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6adcbd54-1388-4223-8d6d-9423e9ebe09b-metrics-certs") pod "network-metrics-daemon-lctg9" (UID: "6adcbd54-1388-4223-8d6d-9423e9ebe09b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 03:33:34.425709 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:34.425656    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/16d564e4-f4b5-4293-b5c9-b7bd30239b47-original-pull-secret\") pod \"global-pull-secret-syncer-p867k\" (UID: \"16d564e4-f4b5-4293-b5c9-b7bd30239b47\") " pod="kube-system/global-pull-secret-syncer-p867k"
Jul 22 03:33:34.425916 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:34.425768    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc9dm\" (UniqueName: \"kubernetes.io/projected/6963eadb-e7e5-4f25-a6ea-fc0a57c951b8-kube-api-access-hc9dm\") pod \"network-check-target-k2tvl\" (UID: \"6963eadb-e7e5-4f25-a6ea-fc0a57c951b8\") " pod="openshift-network-diagnostics/network-check-target-k2tvl"
Jul 22 03:33:34.425916 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:34.425809    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 03:33:34.425916 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:34.425879    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16d564e4-f4b5-4293-b5c9-b7bd30239b47-original-pull-secret podName:16d564e4-f4b5-4293-b5c9-b7bd30239b47 nodeName:}" failed. No retries permitted until 2026-07-22 03:33:35.425862875 +0000 UTC m=+3.044072864 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/16d564e4-f4b5-4293-b5c9-b7bd30239b47-original-pull-secret") pod "global-pull-secret-syncer-p867k" (UID: "16d564e4-f4b5-4293-b5c9-b7bd30239b47") : object "kube-system"/"original-pull-secret" not registered
Jul 22 03:33:34.425916 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:34.425892    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 03:33:34.425916 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:34.425910    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 03:33:34.425916 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:34.425922    2570 projected.go:196] Error preparing data for projected volume kube-api-access-hc9dm for pod openshift-network-diagnostics/network-check-target-k2tvl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 03:33:34.426159 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:34.425978    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6963eadb-e7e5-4f25-a6ea-fc0a57c951b8-kube-api-access-hc9dm podName:6963eadb-e7e5-4f25-a6ea-fc0a57c951b8 nodeName:}" failed. No retries permitted until 2026-07-22 03:33:35.425960415 +0000 UTC m=+3.044170422 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-hc9dm" (UniqueName: "kubernetes.io/projected/6963eadb-e7e5-4f25-a6ea-fc0a57c951b8-kube-api-access-hc9dm") pod "network-check-target-k2tvl" (UID: "6963eadb-e7e5-4f25-a6ea-fc0a57c951b8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 03:33:34.792044 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:34.791888    2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 03:28:34 +0000 UTC" deadline="2028-01-20 21:54:26.004924859 +0000 UTC"
Jul 22 03:33:34.792044 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:34.792042    2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13146h20m51.212887511s"
Jul 22 03:33:34.828985 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:34.828950    2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 03:33:34.849193 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:34.849131    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p867k"
Jul 22 03:33:34.849373 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:34.849313    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p867k" podUID="16d564e4-f4b5-4293-b5c9-b7bd30239b47"
Jul 22 03:33:34.866203 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:34.865997    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-89.ec2.internal" event={"ID":"1decd6d269268ae8653188a0a80fb590","Type":"ContainerStarted","Data":"266f48bf39fde3c39844091befdd99d75aefdd6c44b960b3d887b1e7693d7aca"}
Jul 22 03:33:34.882725 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:34.882589    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-89.ec2.internal" event={"ID":"4d1ebdc1a61d7d6c7a73ef206e8173b6","Type":"ContainerStarted","Data":"5d9146a7423244fb796db4e18c22f0932b32e5129851d4001d6e26812c499f23"}
Jul 22 03:33:34.885795 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:34.885763    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-thjp8" event={"ID":"22a4fd03-a5a7-4d6e-97fb-fb63e5be241c","Type":"ContainerStarted","Data":"68287d38309ec7cace53b2cbbb3c2436d6f704dd84d6ffb8e523d3ca36ea5ec9"}
Jul 22 03:33:34.897155 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:34.897123    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq" event={"ID":"6ba9ef79-f774-4c64-b48e-af2461868076","Type":"ContainerStarted","Data":"cbc7eca4eaf6800388a4dd4d1c4ae76f36e1caea8a0303195d3ee21b78e7b3d5"}
Jul 22 03:33:34.932966 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:34.932848    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wt6b5" event={"ID":"bd93ded5-d41d-4192-967e-ea95b5e4bbe8","Type":"ContainerStarted","Data":"903f0b08b9fd7775d0e77f277bb373dea838dd922f3fc7f7b801879a35828426"}
Jul 22 03:33:34.946873 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:34.946835    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vt8wv" event={"ID":"8f72fc0e-f426-4006-8352-c66cc775d6d2","Type":"ContainerStarted","Data":"005205dc72664be4cd90756780837ee8803ee85726ecd8849a161b95a7514e8d"}
Jul 22 03:33:34.958246 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:34.958157    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"csi-driver\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b13e6eb321323c63d8808dd2edd629ec9e6bb50a1f51642a22e1192625d06647\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"csi-node-driver-registrar\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a241c7873dcd9a33c18e22c17fc2266c2eba1ed07e551f8eb9e60ee375ef576a\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"csi-liveness-probe\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0409c7725f34b67762c9af07735a2803f1edbba13924022918025dff1863c18c\\\": ErrImagePull: pull QPS exceeded\"]" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vt8wv" podUID="8f72fc0e-f426-4006-8352-c66cc775d6d2"
Jul 22 03:33:34.973574 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:34.973505    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6wt45" event={"ID":"26250740-d226-4262-bff6-caf04187cf8d","Type":"ContainerStarted","Data":"237194d6d685d29733ad0612bd77f4bd311126188927048871eddc1190cfb1e5"}
Jul 22 03:33:34.984306 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:34.984192    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7srsk" event={"ID":"56a14a96-971e-4dab-a1bb-8e3edb6b73b2","Type":"ContainerStarted","Data":"80f8f26638b8b97ef150a01d9e9e9ad006f253a4ec259c355bb77dd1b5e20ba7"}
Jul 22 03:33:35.007161 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:35.007120    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sgpqg" event={"ID":"d13af00f-708e-4e6d-bd98-7f4d3afd4974","Type":"ContainerStarted","Data":"d22570cd5af98aa399537c991ca8a1c2f3be4f5b0c09bc5b58ab34777b0675ca"}
Jul 22 03:33:35.014042 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:35.014005    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xbhc9" event={"ID":"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf","Type":"ContainerStarted","Data":"a9fbf8433d359426dc1698c8541c0beac56b47f397dd8f7c9f24f1f7196c1808"}
Jul 22 03:33:35.018720 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:35.018685    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tpgsf" event={"ID":"ed43ef9c-bf6c-4f2c-96b5-fc31d24e5fd1","Type":"ContainerStarted","Data":"579a445cb370061f4bf788c53ac2085ec8c44f19c2f9ea8c502261825557dd24"}
Jul 22 03:33:35.333722 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:35.333678    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6adcbd54-1388-4223-8d6d-9423e9ebe09b-metrics-certs\") pod \"network-metrics-daemon-lctg9\" (UID: \"6adcbd54-1388-4223-8d6d-9423e9ebe09b\") " pod="openshift-multus/network-metrics-daemon-lctg9"
Jul 22 03:33:35.333929 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:35.333852    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 03:33:35.333929 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:35.333922    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6adcbd54-1388-4223-8d6d-9423e9ebe09b-metrics-certs podName:6adcbd54-1388-4223-8d6d-9423e9ebe09b nodeName:}" failed. No retries permitted until 2026-07-22 03:33:37.33390019 +0000 UTC m=+4.952110192 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6adcbd54-1388-4223-8d6d-9423e9ebe09b-metrics-certs") pod "network-metrics-daemon-lctg9" (UID: "6adcbd54-1388-4223-8d6d-9423e9ebe09b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 03:33:35.435654 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:35.434792    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc9dm\" (UniqueName: \"kubernetes.io/projected/6963eadb-e7e5-4f25-a6ea-fc0a57c951b8-kube-api-access-hc9dm\") pod \"network-check-target-k2tvl\" (UID: \"6963eadb-e7e5-4f25-a6ea-fc0a57c951b8\") " pod="openshift-network-diagnostics/network-check-target-k2tvl"
Jul 22 03:33:35.435654 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:35.434853    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/16d564e4-f4b5-4293-b5c9-b7bd30239b47-original-pull-secret\") pod \"global-pull-secret-syncer-p867k\" (UID: \"16d564e4-f4b5-4293-b5c9-b7bd30239b47\") " pod="kube-system/global-pull-secret-syncer-p867k"
Jul 22 03:33:35.435654 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:35.435010    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 03:33:35.435654 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:35.435076    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16d564e4-f4b5-4293-b5c9-b7bd30239b47-original-pull-secret podName:16d564e4-f4b5-4293-b5c9-b7bd30239b47 nodeName:}" failed. No retries permitted until 2026-07-22 03:33:37.435057803 +0000 UTC m=+5.053267807 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/16d564e4-f4b5-4293-b5c9-b7bd30239b47-original-pull-secret") pod "global-pull-secret-syncer-p867k" (UID: "16d564e4-f4b5-4293-b5c9-b7bd30239b47") : object "kube-system"/"original-pull-secret" not registered
Jul 22 03:33:35.435654 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:35.435528    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 03:33:35.435654 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:35.435551    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 03:33:35.435654 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:35.435564    2570 projected.go:196] Error preparing data for projected volume kube-api-access-hc9dm for pod openshift-network-diagnostics/network-check-target-k2tvl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 03:33:35.435654 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:35.435612    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6963eadb-e7e5-4f25-a6ea-fc0a57c951b8-kube-api-access-hc9dm podName:6963eadb-e7e5-4f25-a6ea-fc0a57c951b8 nodeName:}" failed. No retries permitted until 2026-07-22 03:33:37.435596728 +0000 UTC m=+5.053806715 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-hc9dm" (UniqueName: "kubernetes.io/projected/6963eadb-e7e5-4f25-a6ea-fc0a57c951b8-kube-api-access-hc9dm") pod "network-check-target-k2tvl" (UID: "6963eadb-e7e5-4f25-a6ea-fc0a57c951b8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 03:33:35.793398 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:35.793348    2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 03:28:34 +0000 UTC" deadline="2028-01-03 04:13:21.817323558 +0000 UTC"
Jul 22 03:33:35.793398 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:35.793396    2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12720h39m46.023932764s"
Jul 22 03:33:35.846559 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:35.846523    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2tvl"
Jul 22 03:33:35.846770 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:35.846683    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2tvl" podUID="6963eadb-e7e5-4f25-a6ea-fc0a57c951b8"
Jul 22 03:33:35.846846 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:35.846806    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lctg9"
Jul 22 03:33:35.846973 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:35.846911    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lctg9" podUID="6adcbd54-1388-4223-8d6d-9423e9ebe09b"
Jul 22 03:33:36.848374 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:36.848308    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p867k"
Jul 22 03:33:36.848884 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:36.848483    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p867k" podUID="16d564e4-f4b5-4293-b5c9-b7bd30239b47"
Jul 22 03:33:37.352765 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:37.352724    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6adcbd54-1388-4223-8d6d-9423e9ebe09b-metrics-certs\") pod \"network-metrics-daemon-lctg9\" (UID: \"6adcbd54-1388-4223-8d6d-9423e9ebe09b\") " pod="openshift-multus/network-metrics-daemon-lctg9"
Jul 22 03:33:37.352937 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:37.352904    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 03:33:37.352994 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:37.352982    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6adcbd54-1388-4223-8d6d-9423e9ebe09b-metrics-certs podName:6adcbd54-1388-4223-8d6d-9423e9ebe09b nodeName:}" failed. No retries permitted until 2026-07-22 03:33:41.352960693 +0000 UTC m=+8.971170682 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6adcbd54-1388-4223-8d6d-9423e9ebe09b-metrics-certs") pod "network-metrics-daemon-lctg9" (UID: "6adcbd54-1388-4223-8d6d-9423e9ebe09b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 03:33:37.454807 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:37.453917    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/16d564e4-f4b5-4293-b5c9-b7bd30239b47-original-pull-secret\") pod \"global-pull-secret-syncer-p867k\" (UID: \"16d564e4-f4b5-4293-b5c9-b7bd30239b47\") " pod="kube-system/global-pull-secret-syncer-p867k"
Jul 22 03:33:37.454807 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:37.454024    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc9dm\" (UniqueName: \"kubernetes.io/projected/6963eadb-e7e5-4f25-a6ea-fc0a57c951b8-kube-api-access-hc9dm\") pod \"network-check-target-k2tvl\" (UID: \"6963eadb-e7e5-4f25-a6ea-fc0a57c951b8\") " pod="openshift-network-diagnostics/network-check-target-k2tvl"
Jul 22 03:33:37.454807 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:37.454175    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 03:33:37.454807 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:37.454196    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 03:33:37.454807 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:37.454212    2570 projected.go:196] Error preparing data for projected volume kube-api-access-hc9dm for pod openshift-network-diagnostics/network-check-target-k2tvl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 03:33:37.454807 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:37.454277    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6963eadb-e7e5-4f25-a6ea-fc0a57c951b8-kube-api-access-hc9dm podName:6963eadb-e7e5-4f25-a6ea-fc0a57c951b8 nodeName:}" failed. No retries permitted until 2026-07-22 03:33:41.454257747 +0000 UTC m=+9.072467750 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-hc9dm" (UniqueName: "kubernetes.io/projected/6963eadb-e7e5-4f25-a6ea-fc0a57c951b8-kube-api-access-hc9dm") pod "network-check-target-k2tvl" (UID: "6963eadb-e7e5-4f25-a6ea-fc0a57c951b8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 03:33:37.454807 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:37.454667    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 03:33:37.454807 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:37.454736    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16d564e4-f4b5-4293-b5c9-b7bd30239b47-original-pull-secret podName:16d564e4-f4b5-4293-b5c9-b7bd30239b47 nodeName:}" failed. No retries permitted until 2026-07-22 03:33:41.454721213 +0000 UTC m=+9.072931215 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/16d564e4-f4b5-4293-b5c9-b7bd30239b47-original-pull-secret") pod "global-pull-secret-syncer-p867k" (UID: "16d564e4-f4b5-4293-b5c9-b7bd30239b47") : object "kube-system"/"original-pull-secret" not registered
Jul 22 03:33:37.845978 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:37.845936    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lctg9"
Jul 22 03:33:37.846149 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:37.845968    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2tvl"
Jul 22 03:33:37.846231 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:37.846122    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lctg9" podUID="6adcbd54-1388-4223-8d6d-9423e9ebe09b"
Jul 22 03:33:37.846282 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:37.846250    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2tvl" podUID="6963eadb-e7e5-4f25-a6ea-fc0a57c951b8"
Jul 22 03:33:38.847590 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:38.847042    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p867k"
Jul 22 03:33:38.847590 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:38.847171    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p867k" podUID="16d564e4-f4b5-4293-b5c9-b7bd30239b47"
Jul 22 03:33:39.846641 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:39.846541    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2tvl"
Jul 22 03:33:39.846941 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:39.846709    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2tvl" podUID="6963eadb-e7e5-4f25-a6ea-fc0a57c951b8"
Jul 22 03:33:39.847163 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:39.847143    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lctg9"
Jul 22 03:33:39.847289 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:39.847261    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lctg9" podUID="6adcbd54-1388-4223-8d6d-9423e9ebe09b"
Jul 22 03:33:40.846167 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:40.845925    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p867k"
Jul 22 03:33:40.846690 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:40.846650    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p867k" podUID="16d564e4-f4b5-4293-b5c9-b7bd30239b47"
Jul 22 03:33:41.391065 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:41.391029    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6adcbd54-1388-4223-8d6d-9423e9ebe09b-metrics-certs\") pod \"network-metrics-daemon-lctg9\" (UID: \"6adcbd54-1388-4223-8d6d-9423e9ebe09b\") " pod="openshift-multus/network-metrics-daemon-lctg9"
Jul 22 03:33:41.391251 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:41.391183    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 03:33:41.391251 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:41.391250    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6adcbd54-1388-4223-8d6d-9423e9ebe09b-metrics-certs podName:6adcbd54-1388-4223-8d6d-9423e9ebe09b nodeName:}" failed. No retries permitted until 2026-07-22 03:33:49.391229126 +0000 UTC m=+17.009439136 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6adcbd54-1388-4223-8d6d-9423e9ebe09b-metrics-certs") pod "network-metrics-daemon-lctg9" (UID: "6adcbd54-1388-4223-8d6d-9423e9ebe09b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 03:33:41.492192 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:41.492153    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc9dm\" (UniqueName: \"kubernetes.io/projected/6963eadb-e7e5-4f25-a6ea-fc0a57c951b8-kube-api-access-hc9dm\") pod \"network-check-target-k2tvl\" (UID: \"6963eadb-e7e5-4f25-a6ea-fc0a57c951b8\") " pod="openshift-network-diagnostics/network-check-target-k2tvl"
Jul 22 03:33:41.492192 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:41.492205    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/16d564e4-f4b5-4293-b5c9-b7bd30239b47-original-pull-secret\") pod \"global-pull-secret-syncer-p867k\" (UID: \"16d564e4-f4b5-4293-b5c9-b7bd30239b47\") " pod="kube-system/global-pull-secret-syncer-p867k"
Jul 22 03:33:41.492431 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:41.492345    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 03:33:41.492431 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:41.492362    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 03:33:41.492431 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:41.492391    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 03:33:41.492431 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:41.492404    2570 projected.go:196] Error preparing data for projected volume kube-api-access-hc9dm for pod openshift-network-diagnostics/network-check-target-k2tvl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 03:33:41.492608 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:41.492405    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16d564e4-f4b5-4293-b5c9-b7bd30239b47-original-pull-secret podName:16d564e4-f4b5-4293-b5c9-b7bd30239b47 nodeName:}" failed. No retries permitted until 2026-07-22 03:33:49.492386949 +0000 UTC m=+17.110596948 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/16d564e4-f4b5-4293-b5c9-b7bd30239b47-original-pull-secret") pod "global-pull-secret-syncer-p867k" (UID: "16d564e4-f4b5-4293-b5c9-b7bd30239b47") : object "kube-system"/"original-pull-secret" not registered
Jul 22 03:33:41.492608 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:41.492475    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6963eadb-e7e5-4f25-a6ea-fc0a57c951b8-kube-api-access-hc9dm podName:6963eadb-e7e5-4f25-a6ea-fc0a57c951b8 nodeName:}" failed. No retries permitted until 2026-07-22 03:33:49.492459115 +0000 UTC m=+17.110669113 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-hc9dm" (UniqueName: "kubernetes.io/projected/6963eadb-e7e5-4f25-a6ea-fc0a57c951b8-kube-api-access-hc9dm") pod "network-check-target-k2tvl" (UID: "6963eadb-e7e5-4f25-a6ea-fc0a57c951b8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 03:33:41.845895 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:41.845859    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2tvl"
Jul 22 03:33:41.846077 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:41.845873    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lctg9"
Jul 22 03:33:41.846077 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:41.845989    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2tvl" podUID="6963eadb-e7e5-4f25-a6ea-fc0a57c951b8"
Jul 22 03:33:41.846169 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:41.846075    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lctg9" podUID="6adcbd54-1388-4223-8d6d-9423e9ebe09b"
Jul 22 03:33:42.847729 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:42.847701    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p867k"
Jul 22 03:33:42.848120 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:42.847798    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p867k" podUID="16d564e4-f4b5-4293-b5c9-b7bd30239b47"
Jul 22 03:33:43.846223 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:43.846191    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lctg9"
Jul 22 03:33:43.846411 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:43.846328    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lctg9" podUID="6adcbd54-1388-4223-8d6d-9423e9ebe09b"
Jul 22 03:33:43.846811 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:43.846681    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2tvl"
Jul 22 03:33:43.846811 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:43.846769    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2tvl" podUID="6963eadb-e7e5-4f25-a6ea-fc0a57c951b8"
Jul 22 03:33:44.845646 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:44.845600    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p867k"
Jul 22 03:33:44.846051 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:44.845751    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p867k" podUID="16d564e4-f4b5-4293-b5c9-b7bd30239b47"
Jul 22 03:33:45.846327 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:45.846293    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2tvl"
Jul 22 03:33:45.846765 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:45.846293    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lctg9"
Jul 22 03:33:45.846765 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:45.846400    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2tvl" podUID="6963eadb-e7e5-4f25-a6ea-fc0a57c951b8"
Jul 22 03:33:45.846765 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:45.846519    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lctg9" podUID="6adcbd54-1388-4223-8d6d-9423e9ebe09b"
Jul 22 03:33:46.846101 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:46.846062    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p867k"
Jul 22 03:33:46.846275 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:46.846204    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p867k" podUID="16d564e4-f4b5-4293-b5c9-b7bd30239b47"
Jul 22 03:33:47.845888 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:47.845848    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lctg9"
Jul 22 03:33:47.845888 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:47.845880    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2tvl"
Jul 22 03:33:47.846369 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:47.845978    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2tvl" podUID="6963eadb-e7e5-4f25-a6ea-fc0a57c951b8"
Jul 22 03:33:47.846369 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:47.846146    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lctg9" podUID="6adcbd54-1388-4223-8d6d-9423e9ebe09b"
Jul 22 03:33:48.845561 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:48.845527    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p867k"
Jul 22 03:33:48.845735 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:48.845680    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p867k" podUID="16d564e4-f4b5-4293-b5c9-b7bd30239b47"
Jul 22 03:33:49.457374 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:49.457337    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6adcbd54-1388-4223-8d6d-9423e9ebe09b-metrics-certs\") pod \"network-metrics-daemon-lctg9\" (UID: \"6adcbd54-1388-4223-8d6d-9423e9ebe09b\") " pod="openshift-multus/network-metrics-daemon-lctg9"
Jul 22 03:33:49.457856 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:49.457481    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 03:33:49.457856 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:49.457535    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6adcbd54-1388-4223-8d6d-9423e9ebe09b-metrics-certs podName:6adcbd54-1388-4223-8d6d-9423e9ebe09b nodeName:}" failed. No retries permitted until 2026-07-22 03:34:05.457521682 +0000 UTC m=+33.075731669 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6adcbd54-1388-4223-8d6d-9423e9ebe09b-metrics-certs") pod "network-metrics-daemon-lctg9" (UID: "6adcbd54-1388-4223-8d6d-9423e9ebe09b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 03:33:49.558107 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:49.558072    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc9dm\" (UniqueName: \"kubernetes.io/projected/6963eadb-e7e5-4f25-a6ea-fc0a57c951b8-kube-api-access-hc9dm\") pod \"network-check-target-k2tvl\" (UID: \"6963eadb-e7e5-4f25-a6ea-fc0a57c951b8\") " pod="openshift-network-diagnostics/network-check-target-k2tvl"
Jul 22 03:33:49.558107 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:49.558111    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/16d564e4-f4b5-4293-b5c9-b7bd30239b47-original-pull-secret\") pod \"global-pull-secret-syncer-p867k\" (UID: \"16d564e4-f4b5-4293-b5c9-b7bd30239b47\") " pod="kube-system/global-pull-secret-syncer-p867k"
Jul 22 03:33:49.558301 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:49.558224    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 03:33:49.558301 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:49.558261    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 03:33:49.558301 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:49.558285    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 03:33:49.558301 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:49.558290    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16d564e4-f4b5-4293-b5c9-b7bd30239b47-original-pull-secret podName:16d564e4-f4b5-4293-b5c9-b7bd30239b47 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:05.558271434 +0000 UTC m=+33.176481434 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/16d564e4-f4b5-4293-b5c9-b7bd30239b47-original-pull-secret") pod "global-pull-secret-syncer-p867k" (UID: "16d564e4-f4b5-4293-b5c9-b7bd30239b47") : object "kube-system"/"original-pull-secret" not registered
Jul 22 03:33:49.558301 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:49.558294    2570 projected.go:196] Error preparing data for projected volume kube-api-access-hc9dm for pod openshift-network-diagnostics/network-check-target-k2tvl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 03:33:49.558476 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:49.558335    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6963eadb-e7e5-4f25-a6ea-fc0a57c951b8-kube-api-access-hc9dm podName:6963eadb-e7e5-4f25-a6ea-fc0a57c951b8 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:05.558323865 +0000 UTC m=+33.176533869 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-hc9dm" (UniqueName: "kubernetes.io/projected/6963eadb-e7e5-4f25-a6ea-fc0a57c951b8-kube-api-access-hc9dm") pod "network-check-target-k2tvl" (UID: "6963eadb-e7e5-4f25-a6ea-fc0a57c951b8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 03:33:49.845696 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:49.845631    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lctg9"
Jul 22 03:33:49.845696 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:49.845671    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2tvl"
Jul 22 03:33:49.845860 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:49.845760    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lctg9" podUID="6adcbd54-1388-4223-8d6d-9423e9ebe09b"
Jul 22 03:33:49.845895 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:49.845876    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2tvl" podUID="6963eadb-e7e5-4f25-a6ea-fc0a57c951b8"
Jul 22 03:33:50.846458 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:50.846424    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p867k"
Jul 22 03:33:50.846878 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:50.846555    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p867k" podUID="16d564e4-f4b5-4293-b5c9-b7bd30239b47"
Jul 22 03:33:51.846047 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:51.845850    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lctg9"
Jul 22 03:33:51.846236 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:51.845894    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2tvl"
Jul 22 03:33:51.846236 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:51.846137    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lctg9" podUID="6adcbd54-1388-4223-8d6d-9423e9ebe09b"
Jul 22 03:33:51.846236 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:51.846214    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2tvl" podUID="6963eadb-e7e5-4f25-a6ea-fc0a57c951b8"
Jul 22 03:33:52.846475 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:52.846440    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p867k"
Jul 22 03:33:52.846966 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:52.846546    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p867k" podUID="16d564e4-f4b5-4293-b5c9-b7bd30239b47"
Jul 22 03:33:53.846274 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:53.846244    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lctg9"
Jul 22 03:33:53.846444 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:53.846244    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2tvl"
Jul 22 03:33:53.846444 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:53.846372    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lctg9" podUID="6adcbd54-1388-4223-8d6d-9423e9ebe09b"
Jul 22 03:33:53.846528 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:53.846464    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2tvl" podUID="6963eadb-e7e5-4f25-a6ea-fc0a57c951b8"
Jul 22 03:33:54.845961 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:54.845633    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p867k"
Jul 22 03:33:54.846110 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:54.845958    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p867k" podUID="16d564e4-f4b5-4293-b5c9-b7bd30239b47"
Jul 22 03:33:55.069112 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:55.069085    2570 generic.go:356] "Generic (PLEG): container finished" podID="26250740-d226-4262-bff6-caf04187cf8d" containerID="e0002ba316b8b9918de028c837c8997a8af9e8c3030f6d5de67f66d2e3d65c00" exitCode=0
Jul 22 03:33:55.069864 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:55.069176    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6wt45" event={"ID":"26250740-d226-4262-bff6-caf04187cf8d","Type":"ContainerDied","Data":"e0002ba316b8b9918de028c837c8997a8af9e8c3030f6d5de67f66d2e3d65c00"}
Jul 22 03:33:55.070883 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:55.070860    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7srsk" event={"ID":"56a14a96-971e-4dab-a1bb-8e3edb6b73b2","Type":"ContainerStarted","Data":"b3d424cb878a181da030be1dbae503127a0e03ca53694567d8a745ff11b0c9a6"}
Jul 22 03:33:55.072435 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:55.072407    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xbhc9" event={"ID":"00bf5310-aba0-4e3e-9ac8-5d68e7de24bf","Type":"ContainerStarted","Data":"74663dd4642f061b2bf14b3940983cf188fb6f3d782f785b65af3e078a7a2bd1"}
Jul 22 03:33:55.073993 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:55.073965    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tpgsf" event={"ID":"ed43ef9c-bf6c-4f2c-96b5-fc31d24e5fd1","Type":"ContainerStarted","Data":"8f516995aecb78d5113eb31f82b2f68f24888a8692f08ba5045ba9ddb1e1ff0d"}
Jul 22 03:33:55.075515 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:55.075493    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-89.ec2.internal" event={"ID":"1decd6d269268ae8653188a0a80fb590","Type":"ContainerStarted","Data":"f6c6a7d384086c0633698adc4f5c32f68e579c7e1ca43f618a14b6cc6179d5aa"}
Jul 22 03:33:55.077053 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:55.076980    2570 generic.go:356] "Generic (PLEG): container finished" podID="4d1ebdc1a61d7d6c7a73ef206e8173b6" containerID="b2290baa3d1d0219a98c1365fc120a0cc5f41276bcac86e533905b85938b414a" exitCode=0
Jul 22 03:33:55.077053 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:55.077030    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-89.ec2.internal" event={"ID":"4d1ebdc1a61d7d6c7a73ef206e8173b6","Type":"ContainerDied","Data":"b2290baa3d1d0219a98c1365fc120a0cc5f41276bcac86e533905b85938b414a"}
Jul 22 03:33:55.078462 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:55.078440    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-thjp8" event={"ID":"22a4fd03-a5a7-4d6e-97fb-fb63e5be241c","Type":"ContainerStarted","Data":"9a6e9f70145d9e34400deca1ce30302c8b85a0b97cf41c762e45bac880de059e"}
Jul 22 03:33:55.081591 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:55.081558    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq" event={"ID":"6ba9ef79-f774-4c64-b48e-af2461868076","Type":"ContainerStarted","Data":"f6372d7b0c188372bb5377ca917cad0a26aeaf335f89e6fc46ff96f896d69c38"}
Jul 22 03:33:55.081719 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:55.081658    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq" event={"ID":"6ba9ef79-f774-4c64-b48e-af2461868076","Type":"ContainerStarted","Data":"8cbeec74e5e5074843b8ef80563c2565a357ecf6f3fdecdb8e9d5f4f8fda080e"}
Jul 22 03:33:55.081719 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:55.081703    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq" event={"ID":"6ba9ef79-f774-4c64-b48e-af2461868076","Type":"ContainerStarted","Data":"10279826cd2716eb50f1a31c93bd7cb111fb76f3a98504f21f66343d9780dd24"}
Jul 22 03:33:55.081831 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:55.081749    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq" event={"ID":"6ba9ef79-f774-4c64-b48e-af2461868076","Type":"ContainerStarted","Data":"412f20039c7ce3ce59590a6a027bf52c540125dede8640eed3906841c9ad83cb"}
Jul 22 03:33:55.081831 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:55.081795    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq" event={"ID":"6ba9ef79-f774-4c64-b48e-af2461868076","Type":"ContainerStarted","Data":"fde7ef60b3730dfc357e26da21cf85d8cab736a4680c5cc8577a9ae4fd706e1d"}
Jul 22 03:33:55.083339 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:55.083319    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wt6b5" event={"ID":"bd93ded5-d41d-4192-967e-ea95b5e4bbe8","Type":"ContainerStarted","Data":"df341c1bce87d9de24886707a958f5cf0ed3154a1b2401ed1f75ebdccaa69d40"}
Jul 22 03:33:55.131597 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:55.131538    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-wt6b5" podStartSLOduration=1.6784455569999999 podStartE2EDuration="22.131517357s" totalImagesPullingTime="20.4530718s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:33:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:33:55.130862351 +0000 UTC m=+22.749072359" watchObservedRunningTime="2026-07-22 03:33:55.131517357 +0000 UTC m=+22.749727365"
Jul 22 03:33:55.147318 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:55.147274    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-thjp8" podStartSLOduration=1.9570587499999998 podStartE2EDuration="22.147253306s" totalImagesPullingTime="20.190194556s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:33:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:33:55.146594425 +0000 UTC m=+22.764804432" watchObservedRunningTime="2026-07-22 03:33:55.147253306 +0000 UTC m=+22.765463313"
Jul 22 03:33:55.196108 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:55.196050    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-89.ec2.internal" podStartSLOduration=21.196028934 podStartE2EDuration="21.196028934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:33:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:33:55.173967093 +0000 UTC m=+22.792177102" watchObservedRunningTime="2026-07-22 03:33:55.196028934 +0000 UTC m=+22.814238943"
Jul 22 03:33:55.219001 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:55.218934    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-tpgsf" podStartSLOduration=1.9094777330000001 podStartE2EDuration="22.21891399s" totalImagesPullingTime="20.309436257s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:33:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:33:55.19598835 +0000 UTC m=+22.814198378" watchObservedRunningTime="2026-07-22 03:33:55.21891399 +0000 UTC m=+22.837123998"
Jul 22 03:33:55.235808 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:55.235745    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-xbhc9" podStartSLOduration=1.691107833 podStartE2EDuration="22.235721162s" totalImagesPullingTime="20.544613329s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:33:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:33:55.219529492 +0000 UTC m=+22.837739503" watchObservedRunningTime="2026-07-22 03:33:55.235721162 +0000 UTC m=+22.853931171"
Jul 22 03:33:55.236350 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:55.236297    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7srsk" podStartSLOduration=1.847691501 podStartE2EDuration="22.23628387s" totalImagesPullingTime="20.388592369s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:33:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:33:55.23510564 +0000 UTC m=+22.853315645" watchObservedRunningTime="2026-07-22 03:33:55.23628387 +0000 UTC m=+22.854493878"
Jul 22 03:33:55.798881 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:55.798465    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-thjp8"
Jul 22 03:33:55.799172 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:55.799154    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-thjp8"
Jul 22 03:33:55.845736 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:55.845706    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2tvl"
Jul 22 03:33:55.845897 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:55.845823    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2tvl" podUID="6963eadb-e7e5-4f25-a6ea-fc0a57c951b8"
Jul 22 03:33:55.846156 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:55.846140    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lctg9"
Jul 22 03:33:55.846257 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:55.846237    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lctg9" podUID="6adcbd54-1388-4223-8d6d-9423e9ebe09b"
Jul 22 03:33:56.089527 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:56.089429    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq" event={"ID":"6ba9ef79-f774-4c64-b48e-af2461868076","Type":"ContainerStarted","Data":"b5cd7ede8b886c7f29aa568a16b4a2de6d993d25b9bf64f23139b70c66bc01e1"}
Jul 22 03:33:56.091263 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:56.091231    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-sgpqg" event={"ID":"d13af00f-708e-4e6d-bd98-7f4d3afd4974","Type":"ContainerStarted","Data":"4840a39bb2c137c281e654a5d81bd60eef6ebe5c055120853af3e0abb50db467"}
Jul 22 03:33:56.094658 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:56.094635    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-89.ec2.internal" event={"ID":"4d1ebdc1a61d7d6c7a73ef206e8173b6","Type":"ContainerStarted","Data":"f86b437f78e338c0c888bb005bed73c6275a3334c5e8ffe600b9238f54c0bfab"}
Jul 22 03:33:56.096090 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:56.096069    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-thjp8"
Jul 22 03:33:56.096606 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:56.096587    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-thjp8"
Jul 22 03:33:56.107299 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:56.107249    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-sgpqg" podStartSLOduration=2.8769899580000002 podStartE2EDuration="23.107231969s" totalImagesPullingTime="20.230242011s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:33:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:33:56.107088665 +0000 UTC m=+23.725298672" watchObservedRunningTime="2026-07-22 03:33:56.107231969 +0000 UTC m=+23.725441977"
Jul 22 03:33:56.121703 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:56.121643    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-89.ec2.internal" podStartSLOduration=22.121605381 podStartE2EDuration="22.121605381s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:33:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:33:56.121492302 +0000 UTC m=+23.739702328" watchObservedRunningTime="2026-07-22 03:33:56.121605381 +0000 UTC m=+23.739815391"
Jul 22 03:33:56.846233 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:56.846206    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p867k"
Jul 22 03:33:56.846370 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:56.846339    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p867k" podUID="16d564e4-f4b5-4293-b5c9-b7bd30239b47"
Jul 22 03:33:57.097418 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:57.097331    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vt8wv" event={"ID":"8f72fc0e-f426-4006-8352-c66cc775d6d2","Type":"ContainerStarted","Data":"a9383b4a354182978dc78a0dd05b8a8ea9a9ccaa8547dc16e4f718157ef45f79"}
Jul 22 03:33:57.846137 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:57.846099    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lctg9"
Jul 22 03:33:57.846322 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:57.846142    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2tvl"
Jul 22 03:33:57.846322 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:57.846244    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lctg9" podUID="6adcbd54-1388-4223-8d6d-9423e9ebe09b"
Jul 22 03:33:57.846415 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:57.846392    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2tvl" podUID="6963eadb-e7e5-4f25-a6ea-fc0a57c951b8"
Jul 22 03:33:58.103117 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:58.103038    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq" event={"ID":"6ba9ef79-f774-4c64-b48e-af2461868076","Type":"ContainerStarted","Data":"5ea8f443b5855bfe5bdbe5625a747ae80d2ce3288f59cefbba54ebcee770aab6"}
Jul 22 03:33:58.846285 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:58.846246    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p867k"
Jul 22 03:33:58.846506 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:58.846389    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p867k" podUID="16d564e4-f4b5-4293-b5c9-b7bd30239b47"
Jul 22 03:33:59.846422 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:59.846396    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2tvl"
Jul 22 03:33:59.846792 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:59.846403    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lctg9"
Jul 22 03:33:59.846792 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:59.846540    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2tvl" podUID="6963eadb-e7e5-4f25-a6ea-fc0a57c951b8"
Jul 22 03:33:59.846792 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:33:59.846654    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lctg9" podUID="6adcbd54-1388-4223-8d6d-9423e9ebe09b"
Jul 22 03:33:59.890076 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:33:59.889735    2570 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 22 03:34:00.119962 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:00.119725    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vt8wv" event={"ID":"8f72fc0e-f426-4006-8352-c66cc775d6d2","Type":"ContainerStarted","Data":"7193ef30d8b529221b621ee687db05092be0fcbf80f65eec9c73faeabdac9cb1"}
Jul 22 03:34:00.121465 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:00.121438    2570 generic.go:356] "Generic (PLEG): container finished" podID="26250740-d226-4262-bff6-caf04187cf8d" containerID="76bead8c12700976991201b0b0ed27becade1cb7cad6f31bf7a57d87c7df346f" exitCode=0
Jul 22 03:34:00.121568 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:00.121519    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6wt45" event={"ID":"26250740-d226-4262-bff6-caf04187cf8d","Type":"ContainerDied","Data":"76bead8c12700976991201b0b0ed27becade1cb7cad6f31bf7a57d87c7df346f"}
Jul 22 03:34:00.125210 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:00.125191    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq" event={"ID":"6ba9ef79-f774-4c64-b48e-af2461868076","Type":"ContainerStarted","Data":"4ea3d60d32826403c1799b1805d434cb215e60e6d44ff7b5afc615bb212fff15"}
Jul 22 03:34:00.125512 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:00.125493    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:34:00.125636 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:00.125561    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:34:00.125636 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:00.125600    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:34:00.142108 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:00.142088    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:34:00.142509 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:00.142490    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:34:00.165752 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:00.165705    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq" podStartSLOduration=6.69053424 podStartE2EDuration="27.165686755s" totalImagesPullingTime="20.475152515s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:33:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:34:00.164401194 +0000 UTC m=+27.782611202" watchObservedRunningTime="2026-07-22 03:34:00.165686755 +0000 UTC m=+27.783896761"
Jul 22 03:34:00.795916 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:00.795779    2570 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-22T03:33:59.889747909Z","UUID":"f341cc7a-d2ff-45a0-97d7-3ba5f7d308bf","Handler":null,"Name":"","Endpoint":""}
Jul 22 03:34:00.797790 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:00.797463    2570 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 22 03:34:00.797790 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:00.797520    2570 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 22 03:34:00.845856 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:00.845824    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p867k"
Jul 22 03:34:00.846023 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:00.845953    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p867k" podUID="16d564e4-f4b5-4293-b5c9-b7bd30239b47"
Jul 22 03:34:01.846448 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:01.846419    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lctg9"
Jul 22 03:34:01.847229 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:01.846526    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lctg9" podUID="6adcbd54-1388-4223-8d6d-9423e9ebe09b"
Jul 22 03:34:01.847229 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:01.846584    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2tvl"
Jul 22 03:34:01.847229 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:01.846697    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2tvl" podUID="6963eadb-e7e5-4f25-a6ea-fc0a57c951b8"
Jul 22 03:34:01.896244 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:01.896208    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-p867k"]
Jul 22 03:34:01.896393 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:01.896311    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p867k"
Jul 22 03:34:01.896435 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:01.896397    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p867k" podUID="16d564e4-f4b5-4293-b5c9-b7bd30239b47"
Jul 22 03:34:01.899031 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:01.899004    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lctg9"]
Jul 22 03:34:01.901950 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:01.901916    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-k2tvl"]
Jul 22 03:34:02.132406 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:02.132312    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vt8wv" event={"ID":"8f72fc0e-f426-4006-8352-c66cc775d6d2","Type":"ContainerStarted","Data":"c401fa89b6fff32a41e82ea6ccbfc206dc5401b0c9ae87e610701e926a7a03fe"}
Jul 22 03:34:02.137457 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:02.137425    2570 generic.go:356] "Generic (PLEG): container finished" podID="26250740-d226-4262-bff6-caf04187cf8d" containerID="b22c7a4c40956a3dc47d1cb9b62fd8e664532a856e9e0bf0ac19517dc86784a6" exitCode=0
Jul 22 03:34:02.137643 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:02.137490    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lctg9"
Jul 22 03:34:02.137643 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:02.137520    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6wt45" event={"ID":"26250740-d226-4262-bff6-caf04187cf8d","Type":"ContainerDied","Data":"b22c7a4c40956a3dc47d1cb9b62fd8e664532a856e9e0bf0ac19517dc86784a6"}
Jul 22 03:34:02.137643 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:02.137581    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lctg9" podUID="6adcbd54-1388-4223-8d6d-9423e9ebe09b"
Jul 22 03:34:02.137824 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:02.137735    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2tvl"
Jul 22 03:34:02.138126 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:02.138105    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2tvl" podUID="6963eadb-e7e5-4f25-a6ea-fc0a57c951b8"
Jul 22 03:34:02.154389 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:02.154350    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vt8wv" podStartSLOduration=2.088318482 podStartE2EDuration="29.15433595s" totalImagesPullingTime="27.066017468s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:33:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=3 observedRunningTime="2026-07-22 03:34:02.153976998 +0000 UTC m=+29.772187005" watchObservedRunningTime="2026-07-22 03:34:02.15433595 +0000 UTC m=+29.772545956"
Jul 22 03:34:03.846567 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:03.846340    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2tvl"
Jul 22 03:34:03.847026 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:03.846354    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lctg9"
Jul 22 03:34:03.847026 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:03.846590    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2tvl" podUID="6963eadb-e7e5-4f25-a6ea-fc0a57c951b8"
Jul 22 03:34:03.847026 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:03.846384    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p867k"
Jul 22 03:34:03.847026 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:03.846717    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lctg9" podUID="6adcbd54-1388-4223-8d6d-9423e9ebe09b"
Jul 22 03:34:03.847026 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:03.846811    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p867k" podUID="16d564e4-f4b5-4293-b5c9-b7bd30239b47"
Jul 22 03:34:04.143762 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:04.143665    2570 generic.go:356] "Generic (PLEG): container finished" podID="26250740-d226-4262-bff6-caf04187cf8d" containerID="095ea11dd09508b55a184d406ae288bc7c66adc4b225693c811eff32080e1796" exitCode=0
Jul 22 03:34:04.143762 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:04.143732    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6wt45" event={"ID":"26250740-d226-4262-bff6-caf04187cf8d","Type":"ContainerDied","Data":"095ea11dd09508b55a184d406ae288bc7c66adc4b225693c811eff32080e1796"}
Jul 22 03:34:05.467339 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:05.467292    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6adcbd54-1388-4223-8d6d-9423e9ebe09b-metrics-certs\") pod \"network-metrics-daemon-lctg9\" (UID: \"6adcbd54-1388-4223-8d6d-9423e9ebe09b\") " pod="openshift-multus/network-metrics-daemon-lctg9"
Jul 22 03:34:05.467832 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:05.467477    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 03:34:05.467832 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:05.467565    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6adcbd54-1388-4223-8d6d-9423e9ebe09b-metrics-certs podName:6adcbd54-1388-4223-8d6d-9423e9ebe09b nodeName:}" failed. No retries permitted until 2026-07-22 03:34:37.467543726 +0000 UTC m=+65.085753725 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6adcbd54-1388-4223-8d6d-9423e9ebe09b-metrics-certs") pod "network-metrics-daemon-lctg9" (UID: "6adcbd54-1388-4223-8d6d-9423e9ebe09b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 03:34:05.568452 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:05.568420    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc9dm\" (UniqueName: \"kubernetes.io/projected/6963eadb-e7e5-4f25-a6ea-fc0a57c951b8-kube-api-access-hc9dm\") pod \"network-check-target-k2tvl\" (UID: \"6963eadb-e7e5-4f25-a6ea-fc0a57c951b8\") " pod="openshift-network-diagnostics/network-check-target-k2tvl"
Jul 22 03:34:05.568642 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:05.568459    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/16d564e4-f4b5-4293-b5c9-b7bd30239b47-original-pull-secret\") pod \"global-pull-secret-syncer-p867k\" (UID: \"16d564e4-f4b5-4293-b5c9-b7bd30239b47\") " pod="kube-system/global-pull-secret-syncer-p867k"
Jul 22 03:34:05.568642 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:05.568562    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 03:34:05.568642 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:05.568603    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 03:34:05.568642 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:05.568637    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16d564e4-f4b5-4293-b5c9-b7bd30239b47-original-pull-secret podName:16d564e4-f4b5-4293-b5c9-b7bd30239b47 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:37.568603304 +0000 UTC m=+65.186813294 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/16d564e4-f4b5-4293-b5c9-b7bd30239b47-original-pull-secret") pod "global-pull-secret-syncer-p867k" (UID: "16d564e4-f4b5-4293-b5c9-b7bd30239b47") : object "kube-system"/"original-pull-secret" not registered
Jul 22 03:34:05.568817 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:05.568647    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 03:34:05.568817 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:05.568661    2570 projected.go:196] Error preparing data for projected volume kube-api-access-hc9dm for pod openshift-network-diagnostics/network-check-target-k2tvl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 03:34:05.568817 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:05.568720    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6963eadb-e7e5-4f25-a6ea-fc0a57c951b8-kube-api-access-hc9dm podName:6963eadb-e7e5-4f25-a6ea-fc0a57c951b8 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:37.5687028 +0000 UTC m=+65.186912789 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-hc9dm" (UniqueName: "kubernetes.io/projected/6963eadb-e7e5-4f25-a6ea-fc0a57c951b8-kube-api-access-hc9dm") pod "network-check-target-k2tvl" (UID: "6963eadb-e7e5-4f25-a6ea-fc0a57c951b8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 03:34:05.846517 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:05.846433    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p867k"
Jul 22 03:34:05.846517 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:05.846451    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2tvl"
Jul 22 03:34:05.846778 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:05.846432    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lctg9"
Jul 22 03:34:05.846778 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:05.846564    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p867k" podUID="16d564e4-f4b5-4293-b5c9-b7bd30239b47"
Jul 22 03:34:05.846778 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:05.846686    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lctg9" podUID="6adcbd54-1388-4223-8d6d-9423e9ebe09b"
Jul 22 03:34:05.846778 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:05.846765    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k2tvl" podUID="6963eadb-e7e5-4f25-a6ea-fc0a57c951b8"
Jul 22 03:34:07.213521 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.213485    2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-89.ec2.internal" event="NodeReady"
Jul 22 03:34:07.213998 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.213649    2570 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 22 03:34:07.249414 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.249370    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jgckh"]
Jul 22 03:34:07.270303 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.270263    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qtwrs"]
Jul 22 03:34:07.272421 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.272351    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jgckh"
Jul 22 03:34:07.275482 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.275454    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 22 03:34:07.279984 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.279406    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 22 03:34:07.280854 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.280831    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 22 03:34:07.281716 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.281483    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-tq62s\""
Jul 22 03:34:07.281716 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.281544    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 22 03:34:07.284549 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.284106    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4hpsp"]
Jul 22 03:34:07.285179 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.285163    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qtwrs"
Jul 22 03:34:07.288860 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.288800    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 22 03:34:07.288972 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.288891    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 22 03:34:07.288972 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.288931    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 22 03:34:07.289067 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.289026    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 22 03:34:07.289172 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.289150    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-2jd9g\""
Jul 22 03:34:07.308173 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.308141    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-ptsm9"]
Jul 22 03:34:07.308339 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.308274    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4hpsp"
Jul 22 03:34:07.311086 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.311057    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 22 03:34:07.311086 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.311065    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 22 03:34:07.311471 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.311451    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-ktzh2\""
Jul 22 03:34:07.326446 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.326419    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7749b46548-cf9gh"]
Jul 22 03:34:07.326578 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.326537    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-ptsm9"
Jul 22 03:34:07.329382 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.329360    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 22 03:34:07.329506 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.329413    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 22 03:34:07.329506 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.329427    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 22 03:34:07.330470 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.330415    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 22 03:34:07.330470 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.330428    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-9frgm\""
Jul 22 03:34:07.338939 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.338918    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 22 03:34:07.342807 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.342789    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-lqg9w"]
Jul 22 03:34:07.342940 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.342922    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7749b46548-cf9gh"
Jul 22 03:34:07.345286 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.345268    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-dfp92\""
Jul 22 03:34:07.345390 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.345337    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 22 03:34:07.345390 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.345356    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 22 03:34:07.345390 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.345372    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 22 03:34:07.354347 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.354314    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 22 03:34:07.356581 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.356560    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jds8z"]
Jul 22 03:34:07.356737 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.356718    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-lqg9w"
Jul 22 03:34:07.359054 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.359035    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 22 03:34:07.359233 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.359041    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 22 03:34:07.359325 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.359075    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 22 03:34:07.359325 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.359092    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 22 03:34:07.359454 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.359108    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-7q5c6\""
Jul 22 03:34:07.374702 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.374672    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-2p4qp"]
Jul 22 03:34:07.374843 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.374709    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jds8z"
Jul 22 03:34:07.377165 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.377144    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 22 03:34:07.377320 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.377301    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 22 03:34:07.377398 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.377355    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 22 03:34:07.377731 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.377562    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-jj4kt\""
Jul 22 03:34:07.381529 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.381496    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/367c6e07-f8b3-494b-a12c-3c11e916e61b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-jgckh\" (UID: \"367c6e07-f8b3-494b-a12c-3c11e916e61b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jgckh"
Jul 22 03:34:07.381628 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.381542    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/367c6e07-f8b3-494b-a12c-3c11e916e61b-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-jgckh\" (UID: \"367c6e07-f8b3-494b-a12c-3c11e916e61b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jgckh"
Jul 22 03:34:07.381628 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.381564    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ff77d659-f9dd-4d0a-8f75-2dbadc25d260-config\") pod \"service-ca-operator-5bcbc94c78-qtwrs\" (UID: \"ff77d659-f9dd-4d0a-8f75-2dbadc25d260\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qtwrs"
Jul 22 03:34:07.381628 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.381585    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ff77d659-f9dd-4d0a-8f75-2dbadc25d260-serving-cert\") pod \"service-ca-operator-5bcbc94c78-qtwrs\" (UID: \"ff77d659-f9dd-4d0a-8f75-2dbadc25d260\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qtwrs"
Jul 22 03:34:07.381749 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.381608    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7wsm\" (UniqueName: \"kubernetes.io/projected/ff77d659-f9dd-4d0a-8f75-2dbadc25d260-kube-api-access-n7wsm\") pod \"service-ca-operator-5bcbc94c78-qtwrs\" (UID: \"ff77d659-f9dd-4d0a-8f75-2dbadc25d260\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qtwrs"
Jul 22 03:34:07.381749 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.381656    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htdz8\" (UniqueName: \"kubernetes.io/projected/7c29ba98-5a7e-4603-8731-661eeb2b7bf5-kube-api-access-htdz8\") pod \"volume-data-source-validator-78cf4c6f6-4hpsp\" (UID: \"7c29ba98-5a7e-4603-8731-661eeb2b7bf5\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4hpsp"
Jul 22 03:34:07.381749 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.381701    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvw6g\" (UniqueName: \"kubernetes.io/projected/367c6e07-f8b3-494b-a12c-3c11e916e61b-kube-api-access-kvw6g\") pod \"cluster-monitoring-operator-76cf8966f8-jgckh\" (UID: \"367c6e07-f8b3-494b-a12c-3c11e916e61b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jgckh"
Jul 22 03:34:07.394564 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.394535    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7455857b66-nlzjg"]
Jul 22 03:34:07.394716 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.394677    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-2p4qp"
Jul 22 03:34:07.397235 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.397211    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 22 03:34:07.397366 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.397239    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 22 03:34:07.397366 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.397248    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 22 03:34:07.397366 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.397240    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-2chww\""
Jul 22 03:34:07.397366 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.397211    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 22 03:34:07.403350 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.403324    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 22 03:34:07.408704 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.408679    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jgckh"]
Jul 22 03:34:07.408813 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.408722    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4hpsp"]
Jul 22 03:34:07.408813 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.408739    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qtwrs"]
Jul 22 03:34:07.408813 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.408765    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-fldh9"]
Jul 22 03:34:07.408924 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.408821    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7455857b66-nlzjg"
Jul 22 03:34:07.411203 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.411185    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 22 03:34:07.411422 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.411402    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 22 03:34:07.411521 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.411491    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 22 03:34:07.411675 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.411654    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 22 03:34:07.412084 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.411823    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 22 03:34:07.412084 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.411897    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-f5f4v\""
Jul 22 03:34:07.412084 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.411922    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 22 03:34:07.424954 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.424929    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7749b46548-cf9gh"]
Jul 22 03:34:07.425071 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.424970    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jds8z"]
Jul 22 03:34:07.425071 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.424999    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-2p4qp"]
Jul 22 03:34:07.425071 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.425017    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-lqg9w"]
Jul 22 03:34:07.425071 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.425029    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fldh9"
Jul 22 03:34:07.425071 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.425039    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-g2cfz"]
Jul 22 03:34:07.427471 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.427447    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 22 03:34:07.427471 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.427468    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 22 03:34:07.427712 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.427468    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-8dq25\""
Jul 22 03:34:07.427712 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.427491    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 22 03:34:07.439014 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.438989    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7455857b66-nlzjg"]
Jul 22 03:34:07.439126 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.439020    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-ptsm9"]
Jul 22 03:34:07.439126 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.439033    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-g2cfz"]
Jul 22 03:34:07.439126 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.439054    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fldh9"]
Jul 22 03:34:07.439126 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.439098    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-g2cfz"
Jul 22 03:34:07.441372 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.441351    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jn6xf\""
Jul 22 03:34:07.441454 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.441351    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 22 03:34:07.442236 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.442218    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 22 03:34:07.482405 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.482354    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f23b727c-e88a-42b1-80e8-02384f07c8f6-stats-auth\") pod \"router-default-7455857b66-nlzjg\" (UID: \"f23b727c-e88a-42b1-80e8-02384f07c8f6\") " pod="openshift-ingress/router-default-7455857b66-nlzjg"
Jul 22 03:34:07.482405 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.482400    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ff77d659-f9dd-4d0a-8f75-2dbadc25d260-serving-cert\") pod \"service-ca-operator-5bcbc94c78-qtwrs\" (UID: \"ff77d659-f9dd-4d0a-8f75-2dbadc25d260\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qtwrs"
Jul 22 03:34:07.482649 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.482417    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9f0defcc-ccff-43be-8206-54c772885080-serving-cert\") pod \"console-operator-565b7b569d-2p4qp\" (UID: \"9f0defcc-ccff-43be-8206-54c772885080\") " pod="openshift-console-operator/console-operator-565b7b569d-2p4qp"
Jul 22 03:34:07.482649 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.482481    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ee83bd5e-e31f-4049-947f-8f609f93908f-snapshots\") pod \"insights-operator-555d9d7885-ptsm9\" (UID: \"ee83bd5e-e31f-4049-947f-8f609f93908f\") " pod="openshift-insights/insights-operator-555d9d7885-ptsm9"
Jul 22 03:34:07.482649 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.482521    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9f0defcc-ccff-43be-8206-54c772885080-config\") pod \"console-operator-565b7b569d-2p4qp\" (UID: \"9f0defcc-ccff-43be-8206-54c772885080\") " pod="openshift-console-operator/console-operator-565b7b569d-2p4qp"
Jul 22 03:34:07.482649 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.482556    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-registry-certificates\") pod \"image-registry-7749b46548-cf9gh\" (UID: \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\") " pod="openshift-image-registry/image-registry-7749b46548-cf9gh"
Jul 22 03:34:07.482805 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.482648    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvw6g\" (UniqueName: \"kubernetes.io/projected/367c6e07-f8b3-494b-a12c-3c11e916e61b-kube-api-access-kvw6g\") pod \"cluster-monitoring-operator-76cf8966f8-jgckh\" (UID: \"367c6e07-f8b3-494b-a12c-3c11e916e61b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jgckh"
Jul 22 03:34:07.482805 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.482684    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee83bd5e-e31f-4049-947f-8f609f93908f-tmp\") pod \"insights-operator-555d9d7885-ptsm9\" (UID: \"ee83bd5e-e31f-4049-947f-8f609f93908f\") " pod="openshift-insights/insights-operator-555d9d7885-ptsm9"
Jul 22 03:34:07.482805 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.482713    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-ca-trust-extracted\") pod \"image-registry-7749b46548-cf9gh\" (UID: \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\") " pod="openshift-image-registry/image-registry-7749b46548-cf9gh"
Jul 22 03:34:07.482805 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.482734    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-trusted-ca\") pod \"image-registry-7749b46548-cf9gh\" (UID: \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\") " pod="openshift-image-registry/image-registry-7749b46548-cf9gh"
Jul 22 03:34:07.482805 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.482785    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-installation-pull-secrets\") pod \"image-registry-7749b46548-cf9gh\" (UID: \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\") " pod="openshift-image-registry/image-registry-7749b46548-cf9gh"
Jul 22 03:34:07.482973 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.482829    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee83bd5e-e31f-4049-947f-8f609f93908f-service-ca-bundle\") pod \"insights-operator-555d9d7885-ptsm9\" (UID: \"ee83bd5e-e31f-4049-947f-8f609f93908f\") " pod="openshift-insights/insights-operator-555d9d7885-ptsm9"
Jul 22 03:34:07.482973 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.482861    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-image-registry-private-configuration\") pod \"image-registry-7749b46548-cf9gh\" (UID: \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\") " pod="openshift-image-registry/image-registry-7749b46548-cf9gh"
Jul 22 03:34:07.482973 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.482895    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ee83bd5e-e31f-4049-947f-8f609f93908f-serving-cert\") pod \"insights-operator-555d9d7885-ptsm9\" (UID: \"ee83bd5e-e31f-4049-947f-8f609f93908f\") " pod="openshift-insights/insights-operator-555d9d7885-ptsm9"
Jul 22 03:34:07.482973 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.482918    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8cdfda8a-dc86-4da1-b192-4d0ab5cd37f4-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-lqg9w\" (UID: \"8cdfda8a-dc86-4da1-b192-4d0ab5cd37f4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-lqg9w"
Jul 22 03:34:07.482973 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.482949    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/367c6e07-f8b3-494b-a12c-3c11e916e61b-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-jgckh\" (UID: \"367c6e07-f8b3-494b-a12c-3c11e916e61b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jgckh"
Jul 22 03:34:07.482973 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.482970    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f23b727c-e88a-42b1-80e8-02384f07c8f6-default-certificate\") pod \"router-default-7455857b66-nlzjg\" (UID: \"f23b727c-e88a-42b1-80e8-02384f07c8f6\") " pod="openshift-ingress/router-default-7455857b66-nlzjg"
Jul 22 03:34:07.483196 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.482991    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2zd8\" (UniqueName: \"kubernetes.io/projected/ee83bd5e-e31f-4049-947f-8f609f93908f-kube-api-access-z2zd8\") pod \"insights-operator-555d9d7885-ptsm9\" (UID: \"ee83bd5e-e31f-4049-947f-8f609f93908f\") " pod="openshift-insights/insights-operator-555d9d7885-ptsm9"
Jul 22 03:34:07.483196 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.483014    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f23b727c-e88a-42b1-80e8-02384f07c8f6-service-ca-bundle\") pod \"router-default-7455857b66-nlzjg\" (UID: \"f23b727c-e88a-42b1-80e8-02384f07c8f6\") " pod="openshift-ingress/router-default-7455857b66-nlzjg"
Jul 22 03:34:07.483196 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.483035    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-registry-tls\") pod \"image-registry-7749b46548-cf9gh\" (UID: \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\") " pod="openshift-image-registry/image-registry-7749b46548-cf9gh"
Jul 22 03:34:07.483196 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.483057    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gnfk\" (UniqueName: \"kubernetes.io/projected/f23b727c-e88a-42b1-80e8-02384f07c8f6-kube-api-access-2gnfk\") pod \"router-default-7455857b66-nlzjg\" (UID: \"f23b727c-e88a-42b1-80e8-02384f07c8f6\") " pod="openshift-ingress/router-default-7455857b66-nlzjg"
Jul 22 03:34:07.483196 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.483082    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7wsm\" (UniqueName: \"kubernetes.io/projected/ff77d659-f9dd-4d0a-8f75-2dbadc25d260-kube-api-access-n7wsm\") pod \"service-ca-operator-5bcbc94c78-qtwrs\" (UID: \"ff77d659-f9dd-4d0a-8f75-2dbadc25d260\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qtwrs"
Jul 22 03:34:07.483196 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.483108    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htdz8\" (UniqueName: \"kubernetes.io/projected/7c29ba98-5a7e-4603-8731-661eeb2b7bf5-kube-api-access-htdz8\") pod \"volume-data-source-validator-78cf4c6f6-4hpsp\" (UID: \"7c29ba98-5a7e-4603-8731-661eeb2b7bf5\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4hpsp"
Jul 22 03:34:07.483196 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.483130    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9f0defcc-ccff-43be-8206-54c772885080-trusted-ca\") pod \"console-operator-565b7b569d-2p4qp\" (UID: \"9f0defcc-ccff-43be-8206-54c772885080\") " pod="openshift-console-operator/console-operator-565b7b569d-2p4qp"
Jul 22 03:34:07.483517 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.483262    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-bound-sa-token\") pod \"image-registry-7749b46548-cf9gh\" (UID: \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\") " pod="openshift-image-registry/image-registry-7749b46548-cf9gh"
Jul 22 03:34:07.483517 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.483315    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9z6j\" (UniqueName: \"kubernetes.io/projected/7bc03f15-dc0d-4a86-adc6-7ec711363d87-kube-api-access-j9z6j\") pod \"cluster-samples-operator-79b7c55f87-jds8z\" (UID: \"7bc03f15-dc0d-4a86-adc6-7ec711363d87\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jds8z"
Jul 22 03:34:07.483517 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.483355    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8cdfda8a-dc86-4da1-b192-4d0ab5cd37f4-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-lqg9w\" (UID: \"8cdfda8a-dc86-4da1-b192-4d0ab5cd37f4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-lqg9w"
Jul 22 03:34:07.483517 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.483391    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9b6bv\" (UniqueName: \"kubernetes.io/projected/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-kube-api-access-9b6bv\") pod \"image-registry-7749b46548-cf9gh\" (UID: \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\") " pod="openshift-image-registry/image-registry-7749b46548-cf9gh"
Jul 22 03:34:07.483517 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.483423    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7bc03f15-dc0d-4a86-adc6-7ec711363d87-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-jds8z\" (UID: \"7bc03f15-dc0d-4a86-adc6-7ec711363d87\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jds8z"
Jul 22 03:34:07.483517 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.483457    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnq2p\" (UniqueName: \"kubernetes.io/projected/9f0defcc-ccff-43be-8206-54c772885080-kube-api-access-vnq2p\") pod \"console-operator-565b7b569d-2p4qp\" (UID: \"9f0defcc-ccff-43be-8206-54c772885080\") " pod="openshift-console-operator/console-operator-565b7b569d-2p4qp"
Jul 22 03:34:07.483517 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.483493    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/367c6e07-f8b3-494b-a12c-3c11e916e61b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-jgckh\" (UID: \"367c6e07-f8b3-494b-a12c-3c11e916e61b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jgckh"
Jul 22 03:34:07.483809 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.483525    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f23b727c-e88a-42b1-80e8-02384f07c8f6-metrics-certs\") pod \"router-default-7455857b66-nlzjg\" (UID: \"f23b727c-e88a-42b1-80e8-02384f07c8f6\") " pod="openshift-ingress/router-default-7455857b66-nlzjg"
Jul 22 03:34:07.483809 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.483552    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee83bd5e-e31f-4049-947f-8f609f93908f-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-ptsm9\" (UID: \"ee83bd5e-e31f-4049-947f-8f609f93908f\") " pod="openshift-insights/insights-operator-555d9d7885-ptsm9"
Jul 22 03:34:07.483809 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:07.483560    2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 22 03:34:07.483809 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.483605    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ff77d659-f9dd-4d0a-8f75-2dbadc25d260-config\") pod \"service-ca-operator-5bcbc94c78-qtwrs\" (UID: \"ff77d659-f9dd-4d0a-8f75-2dbadc25d260\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qtwrs"
Jul 22 03:34:07.483809 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.483637    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/367c6e07-f8b3-494b-a12c-3c11e916e61b-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-jgckh\" (UID: \"367c6e07-f8b3-494b-a12c-3c11e916e61b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jgckh"
Jul 22 03:34:07.483809 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:07.483656    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/367c6e07-f8b3-494b-a12c-3c11e916e61b-cluster-monitoring-operator-tls podName:367c6e07-f8b3-494b-a12c-3c11e916e61b nodeName:}" failed. No retries permitted until 2026-07-22 03:34:07.983636734 +0000 UTC m=+35.601846724 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/367c6e07-f8b3-494b-a12c-3c11e916e61b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-jgckh" (UID: "367c6e07-f8b3-494b-a12c-3c11e916e61b") : secret "cluster-monitoring-operator-tls" not found
Jul 22 03:34:07.483809 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.483691    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w298n\" (UniqueName: \"kubernetes.io/projected/8cdfda8a-dc86-4da1-b192-4d0ab5cd37f4-kube-api-access-w298n\") pod \"kube-storage-version-migrator-operator-7658d9b749-lqg9w\" (UID: \"8cdfda8a-dc86-4da1-b192-4d0ab5cd37f4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-lqg9w"
Jul 22 03:34:07.484063 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.484047    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ff77d659-f9dd-4d0a-8f75-2dbadc25d260-config\") pod \"service-ca-operator-5bcbc94c78-qtwrs\" (UID: \"ff77d659-f9dd-4d0a-8f75-2dbadc25d260\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qtwrs"
Jul 22 03:34:07.487018 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.486995    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ff77d659-f9dd-4d0a-8f75-2dbadc25d260-serving-cert\") pod \"service-ca-operator-5bcbc94c78-qtwrs\" (UID: \"ff77d659-f9dd-4d0a-8f75-2dbadc25d260\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qtwrs"
Jul 22 03:34:07.492762 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.492737    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7wsm\" (UniqueName: \"kubernetes.io/projected/ff77d659-f9dd-4d0a-8f75-2dbadc25d260-kube-api-access-n7wsm\") pod \"service-ca-operator-5bcbc94c78-qtwrs\" (UID: \"ff77d659-f9dd-4d0a-8f75-2dbadc25d260\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qtwrs"
Jul 22 03:34:07.493431 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.493407    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvw6g\" (UniqueName: \"kubernetes.io/projected/367c6e07-f8b3-494b-a12c-3c11e916e61b-kube-api-access-kvw6g\") pod \"cluster-monitoring-operator-76cf8966f8-jgckh\" (UID: \"367c6e07-f8b3-494b-a12c-3c11e916e61b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jgckh"
Jul 22 03:34:07.503174 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.503142    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htdz8\" (UniqueName: \"kubernetes.io/projected/7c29ba98-5a7e-4603-8731-661eeb2b7bf5-kube-api-access-htdz8\") pod \"volume-data-source-validator-78cf4c6f6-4hpsp\" (UID: \"7c29ba98-5a7e-4603-8731-661eeb2b7bf5\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4hpsp"
Jul 22 03:34:07.584734 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.584650    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6ced761e-e20b-4b10-b8b1-fe77cd501818-metrics-tls\") pod \"dns-default-g2cfz\" (UID: \"6ced761e-e20b-4b10-b8b1-fe77cd501818\") " pod="openshift-dns/dns-default-g2cfz"
Jul 22 03:34:07.584869 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.584721    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9f0defcc-ccff-43be-8206-54c772885080-trusted-ca\") pod \"console-operator-565b7b569d-2p4qp\" (UID: \"9f0defcc-ccff-43be-8206-54c772885080\") " pod="openshift-console-operator/console-operator-565b7b569d-2p4qp"
Jul 22 03:34:07.584869 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.584770    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-bound-sa-token\") pod \"image-registry-7749b46548-cf9gh\" (UID: \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\") " pod="openshift-image-registry/image-registry-7749b46548-cf9gh"
Jul 22 03:34:07.584869 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.584823    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9z6j\" (UniqueName: \"kubernetes.io/projected/7bc03f15-dc0d-4a86-adc6-7ec711363d87-kube-api-access-j9z6j\") pod \"cluster-samples-operator-79b7c55f87-jds8z\" (UID: \"7bc03f15-dc0d-4a86-adc6-7ec711363d87\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jds8z"
Jul 22 03:34:07.584869 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.584854    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8cdfda8a-dc86-4da1-b192-4d0ab5cd37f4-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-lqg9w\" (UID: \"8cdfda8a-dc86-4da1-b192-4d0ab5cd37f4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-lqg9w"
Jul 22 03:34:07.584992 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.584892    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9b6bv\" (UniqueName: \"kubernetes.io/projected/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-kube-api-access-9b6bv\") pod \"image-registry-7749b46548-cf9gh\" (UID: \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\") " pod="openshift-image-registry/image-registry-7749b46548-cf9gh"
Jul 22 03:34:07.584992 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.584923    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7bc03f15-dc0d-4a86-adc6-7ec711363d87-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-jds8z\" (UID: \"7bc03f15-dc0d-4a86-adc6-7ec711363d87\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jds8z"
Jul 22 03:34:07.584992 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.584959    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnq2p\" (UniqueName: \"kubernetes.io/projected/9f0defcc-ccff-43be-8206-54c772885080-kube-api-access-vnq2p\") pod \"console-operator-565b7b569d-2p4qp\" (UID: \"9f0defcc-ccff-43be-8206-54c772885080\") " pod="openshift-console-operator/console-operator-565b7b569d-2p4qp"
Jul 22 03:34:07.585079 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.585007    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ced761e-e20b-4b10-b8b1-fe77cd501818-tmp-dir\") pod \"dns-default-g2cfz\" (UID: \"6ced761e-e20b-4b10-b8b1-fe77cd501818\") " pod="openshift-dns/dns-default-g2cfz"
Jul 22 03:34:07.585079 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.585040    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f23b727c-e88a-42b1-80e8-02384f07c8f6-metrics-certs\") pod \"router-default-7455857b66-nlzjg\" (UID: \"f23b727c-e88a-42b1-80e8-02384f07c8f6\") " pod="openshift-ingress/router-default-7455857b66-nlzjg"
Jul 22 03:34:07.585079 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.585072    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee83bd5e-e31f-4049-947f-8f609f93908f-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-ptsm9\" (UID: \"ee83bd5e-e31f-4049-947f-8f609f93908f\") " pod="openshift-insights/insights-operator-555d9d7885-ptsm9"
Jul 22 03:34:07.585165 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.585105    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w298n\" (UniqueName: \"kubernetes.io/projected/8cdfda8a-dc86-4da1-b192-4d0ab5cd37f4-kube-api-access-w298n\") pod \"kube-storage-version-migrator-operator-7658d9b749-lqg9w\" (UID: \"8cdfda8a-dc86-4da1-b192-4d0ab5cd37f4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-lqg9w"
Jul 22 03:34:07.585165 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.585138    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f23b727c-e88a-42b1-80e8-02384f07c8f6-stats-auth\") pod \"router-default-7455857b66-nlzjg\" (UID: \"f23b727c-e88a-42b1-80e8-02384f07c8f6\") " pod="openshift-ingress/router-default-7455857b66-nlzjg"
Jul 22 03:34:07.585222 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.585174    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9f0defcc-ccff-43be-8206-54c772885080-serving-cert\") pod \"console-operator-565b7b569d-2p4qp\" (UID: \"9f0defcc-ccff-43be-8206-54c772885080\") " pod="openshift-console-operator/console-operator-565b7b569d-2p4qp"
Jul 22 03:34:07.585222 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.585202    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ee83bd5e-e31f-4049-947f-8f609f93908f-snapshots\") pod \"insights-operator-555d9d7885-ptsm9\" (UID: \"ee83bd5e-e31f-4049-947f-8f609f93908f\") " pod="openshift-insights/insights-operator-555d9d7885-ptsm9"
Jul 22 03:34:07.585302 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.585241    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9f0defcc-ccff-43be-8206-54c772885080-config\") pod \"console-operator-565b7b569d-2p4qp\" (UID: \"9f0defcc-ccff-43be-8206-54c772885080\") " pod="openshift-console-operator/console-operator-565b7b569d-2p4qp"
Jul 22 03:34:07.585302 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.585271    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-registry-certificates\") pod \"image-registry-7749b46548-cf9gh\" (UID: \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\") " pod="openshift-image-registry/image-registry-7749b46548-cf9gh"
Jul 22 03:34:07.585446 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:07.585428    2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 22 03:34:07.585515 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:07.585504    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bc03f15-dc0d-4a86-adc6-7ec711363d87-samples-operator-tls podName:7bc03f15-dc0d-4a86-adc6-7ec711363d87 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:08.085483529 +0000 UTC m=+35.703693516 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7bc03f15-dc0d-4a86-adc6-7ec711363d87-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-jds8z" (UID: "7bc03f15-dc0d-4a86-adc6-7ec711363d87") : secret "samples-operator-tls" not found
Jul 22 03:34:07.585515 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:07.585502    2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 22 03:34:07.586035 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:07.586003    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f23b727c-e88a-42b1-80e8-02384f07c8f6-metrics-certs podName:f23b727c-e88a-42b1-80e8-02384f07c8f6 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:08.085983472 +0000 UTC m=+35.704193457 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f23b727c-e88a-42b1-80e8-02384f07c8f6-metrics-certs") pod "router-default-7455857b66-nlzjg" (UID: "f23b727c-e88a-42b1-80e8-02384f07c8f6") : secret "router-metrics-certs-default" not found
Jul 22 03:34:07.586322 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.586161    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ee83bd5e-e31f-4049-947f-8f609f93908f-snapshots\") pod \"insights-operator-555d9d7885-ptsm9\" (UID: \"ee83bd5e-e31f-4049-947f-8f609f93908f\") " pod="openshift-insights/insights-operator-555d9d7885-ptsm9"
Jul 22 03:34:07.587641 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.586686    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-registry-certificates\") pod \"image-registry-7749b46548-cf9gh\" (UID: \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\") " pod="openshift-image-registry/image-registry-7749b46548-cf9gh"
Jul 22 03:34:07.587641 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.587637    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9f0defcc-ccff-43be-8206-54c772885080-config\") pod \"console-operator-565b7b569d-2p4qp\" (UID: \"9f0defcc-ccff-43be-8206-54c772885080\") " pod="openshift-console-operator/console-operator-565b7b569d-2p4qp"
Jul 22 03:34:07.587931 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.587909    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee83bd5e-e31f-4049-947f-8f609f93908f-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-ptsm9\" (UID: \"ee83bd5e-e31f-4049-947f-8f609f93908f\") " pod="openshift-insights/insights-operator-555d9d7885-ptsm9"
Jul 22 03:34:07.589903 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.588444    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee83bd5e-e31f-4049-947f-8f609f93908f-tmp\") pod \"insights-operator-555d9d7885-ptsm9\" (UID: \"ee83bd5e-e31f-4049-947f-8f609f93908f\") " pod="openshift-insights/insights-operator-555d9d7885-ptsm9"
Jul 22 03:34:07.589903 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.588540    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-ca-trust-extracted\") pod \"image-registry-7749b46548-cf9gh\" (UID: \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\") " pod="openshift-image-registry/image-registry-7749b46548-cf9gh"
Jul 22 03:34:07.589903 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.588573    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9f0defcc-ccff-43be-8206-54c772885080-serving-cert\") pod \"console-operator-565b7b569d-2p4qp\" (UID: \"9f0defcc-ccff-43be-8206-54c772885080\") " pod="openshift-console-operator/console-operator-565b7b569d-2p4qp"
Jul 22 03:34:07.589903 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.588586    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fptn6\" (UniqueName: \"kubernetes.io/projected/f3317954-0d7e-49df-ae2a-4a4b48382557-kube-api-access-fptn6\") pod \"ingress-canary-fldh9\" (UID: \"f3317954-0d7e-49df-ae2a-4a4b48382557\") " pod="openshift-ingress-canary/ingress-canary-fldh9"
Jul 22 03:34:07.589903 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.588541    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9f0defcc-ccff-43be-8206-54c772885080-trusted-ca\") pod \"console-operator-565b7b569d-2p4qp\" (UID: \"9f0defcc-ccff-43be-8206-54c772885080\") " pod="openshift-console-operator/console-operator-565b7b569d-2p4qp"
Jul 22 03:34:07.589903 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.588638    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kfxj\" (UniqueName: \"kubernetes.io/projected/6ced761e-e20b-4b10-b8b1-fe77cd501818-kube-api-access-2kfxj\") pod \"dns-default-g2cfz\" (UID: \"6ced761e-e20b-4b10-b8b1-fe77cd501818\") " pod="openshift-dns/dns-default-g2cfz"
Jul 22 03:34:07.589903 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.588645    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f23b727c-e88a-42b1-80e8-02384f07c8f6-stats-auth\") pod \"router-default-7455857b66-nlzjg\" (UID: \"f23b727c-e88a-42b1-80e8-02384f07c8f6\") " pod="openshift-ingress/router-default-7455857b66-nlzjg"
Jul 22 03:34:07.589903 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.588698    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-trusted-ca\") pod \"image-registry-7749b46548-cf9gh\" (UID: \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\") " pod="openshift-image-registry/image-registry-7749b46548-cf9gh"
Jul 22 03:34:07.589903 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.588735    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6ced761e-e20b-4b10-b8b1-fe77cd501818-config-volume\") pod \"dns-default-g2cfz\" (UID: \"6ced761e-e20b-4b10-b8b1-fe77cd501818\") " pod="openshift-dns/dns-default-g2cfz"
Jul 22 03:34:07.589903 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.588767    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-installation-pull-secrets\") pod \"image-registry-7749b46548-cf9gh\" (UID: \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\") " pod="openshift-image-registry/image-registry-7749b46548-cf9gh"
Jul 22 03:34:07.589903 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.588802    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee83bd5e-e31f-4049-947f-8f609f93908f-service-ca-bundle\") pod \"insights-operator-555d9d7885-ptsm9\" (UID: \"ee83bd5e-e31f-4049-947f-8f609f93908f\") " pod="openshift-insights/insights-operator-555d9d7885-ptsm9"
Jul 22 03:34:07.589903 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.588804    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee83bd5e-e31f-4049-947f-8f609f93908f-tmp\") pod \"insights-operator-555d9d7885-ptsm9\" (UID: \"ee83bd5e-e31f-4049-947f-8f609f93908f\") " pod="openshift-insights/insights-operator-555d9d7885-ptsm9"
Jul 22 03:34:07.589903 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.588832    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-image-registry-private-configuration\") pod \"image-registry-7749b46548-cf9gh\" (UID: \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\") " pod="openshift-image-registry/image-registry-7749b46548-cf9gh"
Jul 22 03:34:07.589903 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.588864    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ee83bd5e-e31f-4049-947f-8f609f93908f-serving-cert\") pod \"insights-operator-555d9d7885-ptsm9\" (UID: \"ee83bd5e-e31f-4049-947f-8f609f93908f\") " pod="openshift-insights/insights-operator-555d9d7885-ptsm9"
Jul 22 03:34:07.589903 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.588890    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-ca-trust-extracted\") pod \"image-registry-7749b46548-cf9gh\" (UID: \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\") " pod="openshift-image-registry/image-registry-7749b46548-cf9gh"
Jul 22 03:34:07.589903 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.589365    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee83bd5e-e31f-4049-947f-8f609f93908f-service-ca-bundle\") pod \"insights-operator-555d9d7885-ptsm9\" (UID: \"ee83bd5e-e31f-4049-947f-8f609f93908f\") " pod="openshift-insights/insights-operator-555d9d7885-ptsm9"
Jul 22 03:34:07.590682 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.589550    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8cdfda8a-dc86-4da1-b192-4d0ab5cd37f4-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-lqg9w\" (UID: \"8cdfda8a-dc86-4da1-b192-4d0ab5cd37f4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-lqg9w"
Jul 22 03:34:07.590682 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.589607    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8cdfda8a-dc86-4da1-b192-4d0ab5cd37f4-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-lqg9w\" (UID: \"8cdfda8a-dc86-4da1-b192-4d0ab5cd37f4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-lqg9w"
Jul 22 03:34:07.590682 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.589672    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f3317954-0d7e-49df-ae2a-4a4b48382557-cert\") pod \"ingress-canary-fldh9\" (UID: \"f3317954-0d7e-49df-ae2a-4a4b48382557\") " pod="openshift-ingress-canary/ingress-canary-fldh9"
Jul 22 03:34:07.590682 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.589709    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f23b727c-e88a-42b1-80e8-02384f07c8f6-default-certificate\") pod \"router-default-7455857b66-nlzjg\" (UID: \"f23b727c-e88a-42b1-80e8-02384f07c8f6\") " pod="openshift-ingress/router-default-7455857b66-nlzjg"
Jul 22 03:34:07.590682 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.589737    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2zd8\" (UniqueName: \"kubernetes.io/projected/ee83bd5e-e31f-4049-947f-8f609f93908f-kube-api-access-z2zd8\") pod \"insights-operator-555d9d7885-ptsm9\" (UID: \"ee83bd5e-e31f-4049-947f-8f609f93908f\") " pod="openshift-insights/insights-operator-555d9d7885-ptsm9"
Jul 22 03:34:07.590682 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.589771    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f23b727c-e88a-42b1-80e8-02384f07c8f6-service-ca-bundle\") pod \"router-default-7455857b66-nlzjg\" (UID: \"f23b727c-e88a-42b1-80e8-02384f07c8f6\") " pod="openshift-ingress/router-default-7455857b66-nlzjg"
Jul 22 03:34:07.590682 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.589800    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-registry-tls\") pod \"image-registry-7749b46548-cf9gh\" (UID: \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\") " pod="openshift-image-registry/image-registry-7749b46548-cf9gh"
Jul 22 03:34:07.590682 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.589833    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gnfk\" (UniqueName: \"kubernetes.io/projected/f23b727c-e88a-42b1-80e8-02384f07c8f6-kube-api-access-2gnfk\") pod \"router-default-7455857b66-nlzjg\" (UID: \"f23b727c-e88a-42b1-80e8-02384f07c8f6\") " pod="openshift-ingress/router-default-7455857b66-nlzjg"
Jul 22 03:34:07.590682 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.590547    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-trusted-ca\") pod \"image-registry-7749b46548-cf9gh\" (UID: \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\") " pod="openshift-image-registry/image-registry-7749b46548-cf9gh"
Jul 22 03:34:07.591945 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.591481    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-installation-pull-secrets\") pod \"image-registry-7749b46548-cf9gh\" (UID: \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\") " pod="openshift-image-registry/image-registry-7749b46548-cf9gh"
Jul 22 03:34:07.591945 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:07.591554    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 22 03:34:07.591945 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:07.591681    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7749b46548-cf9gh: secret "image-registry-tls" not found
Jul 22 03:34:07.591945 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:07.591641    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f23b727c-e88a-42b1-80e8-02384f07c8f6-service-ca-bundle podName:f23b727c-e88a-42b1-80e8-02384f07c8f6 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:08.091593462 +0000 UTC m=+35.709803464 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f23b727c-e88a-42b1-80e8-02384f07c8f6-service-ca-bundle") pod "router-default-7455857b66-nlzjg" (UID: "f23b727c-e88a-42b1-80e8-02384f07c8f6") : configmap references non-existent config key: service-ca.crt
Jul 22 03:34:07.591945 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:07.591731    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-registry-tls podName:7492beee-6a0b-4b5a-85c6-ec0064b61bbf nodeName:}" failed. No retries permitted until 2026-07-22 03:34:08.091717451 +0000 UTC m=+35.709927440 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-registry-tls") pod "image-registry-7749b46548-cf9gh" (UID: "7492beee-6a0b-4b5a-85c6-ec0064b61bbf") : secret "image-registry-tls" not found
Jul 22 03:34:07.591945 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.591677    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8cdfda8a-dc86-4da1-b192-4d0ab5cd37f4-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-lqg9w\" (UID: \"8cdfda8a-dc86-4da1-b192-4d0ab5cd37f4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-lqg9w"
Jul 22 03:34:07.592256 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.591947    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-image-registry-private-configuration\") pod \"image-registry-7749b46548-cf9gh\" (UID: \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\") " pod="openshift-image-registry/image-registry-7749b46548-cf9gh"
Jul 22 03:34:07.592256 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.592058    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ee83bd5e-e31f-4049-947f-8f609f93908f-serving-cert\") pod \"insights-operator-555d9d7885-ptsm9\" (UID: \"ee83bd5e-e31f-4049-947f-8f609f93908f\") " pod="openshift-insights/insights-operator-555d9d7885-ptsm9"
Jul 22 03:34:07.595043 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.594909    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f23b727c-e88a-42b1-80e8-02384f07c8f6-default-certificate\") pod \"router-default-7455857b66-nlzjg\" (UID: \"f23b727c-e88a-42b1-80e8-02384f07c8f6\") " pod="openshift-ingress/router-default-7455857b66-nlzjg"
Jul 22 03:34:07.596774 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.596751    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9b6bv\" (UniqueName: \"kubernetes.io/projected/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-kube-api-access-9b6bv\") pod \"image-registry-7749b46548-cf9gh\" (UID: \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\") " pod="openshift-image-registry/image-registry-7749b46548-cf9gh"
Jul 22 03:34:07.597173 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.597150    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnq2p\" (UniqueName: \"kubernetes.io/projected/9f0defcc-ccff-43be-8206-54c772885080-kube-api-access-vnq2p\") pod \"console-operator-565b7b569d-2p4qp\" (UID: \"9f0defcc-ccff-43be-8206-54c772885080\") " pod="openshift-console-operator/console-operator-565b7b569d-2p4qp"
Jul 22 03:34:07.597838 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.597814    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9z6j\" (UniqueName: \"kubernetes.io/projected/7bc03f15-dc0d-4a86-adc6-7ec711363d87-kube-api-access-j9z6j\") pod \"cluster-samples-operator-79b7c55f87-jds8z\" (UID: \"7bc03f15-dc0d-4a86-adc6-7ec711363d87\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jds8z"
Jul 22 03:34:07.598410 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.598388    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-bound-sa-token\") pod \"image-registry-7749b46548-cf9gh\" (UID: \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\") " pod="openshift-image-registry/image-registry-7749b46548-cf9gh"
Jul 22 03:34:07.598654 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.598605    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w298n\" (UniqueName: \"kubernetes.io/projected/8cdfda8a-dc86-4da1-b192-4d0ab5cd37f4-kube-api-access-w298n\") pod \"kube-storage-version-migrator-operator-7658d9b749-lqg9w\" (UID: \"8cdfda8a-dc86-4da1-b192-4d0ab5cd37f4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-lqg9w"
Jul 22 03:34:07.600157 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.600135    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gnfk\" (UniqueName: \"kubernetes.io/projected/f23b727c-e88a-42b1-80e8-02384f07c8f6-kube-api-access-2gnfk\") pod \"router-default-7455857b66-nlzjg\" (UID: \"f23b727c-e88a-42b1-80e8-02384f07c8f6\") " pod="openshift-ingress/router-default-7455857b66-nlzjg"
Jul 22 03:34:07.600258 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.600244    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qtwrs"
Jul 22 03:34:07.601568 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.601542    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2zd8\" (UniqueName: \"kubernetes.io/projected/ee83bd5e-e31f-4049-947f-8f609f93908f-kube-api-access-z2zd8\") pod \"insights-operator-555d9d7885-ptsm9\" (UID: \"ee83bd5e-e31f-4049-947f-8f609f93908f\") " pod="openshift-insights/insights-operator-555d9d7885-ptsm9"
Jul 22 03:34:07.618730 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.618702    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4hpsp"
Jul 22 03:34:07.638165 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.637944    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-ptsm9"
Jul 22 03:34:07.665988 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.665959    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-lqg9w"
Jul 22 03:34:07.692215 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.692188    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6ced761e-e20b-4b10-b8b1-fe77cd501818-metrics-tls\") pod \"dns-default-g2cfz\" (UID: \"6ced761e-e20b-4b10-b8b1-fe77cd501818\") " pod="openshift-dns/dns-default-g2cfz"
Jul 22 03:34:07.692356 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.692269    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ced761e-e20b-4b10-b8b1-fe77cd501818-tmp-dir\") pod \"dns-default-g2cfz\" (UID: \"6ced761e-e20b-4b10-b8b1-fe77cd501818\") " pod="openshift-dns/dns-default-g2cfz"
Jul 22 03:34:07.692356 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:07.692322    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 03:34:07.692356 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.692337    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fptn6\" (UniqueName: \"kubernetes.io/projected/f3317954-0d7e-49df-ae2a-4a4b48382557-kube-api-access-fptn6\") pod \"ingress-canary-fldh9\" (UID: \"f3317954-0d7e-49df-ae2a-4a4b48382557\") " pod="openshift-ingress-canary/ingress-canary-fldh9"
Jul 22 03:34:07.692499 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.692362    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kfxj\" (UniqueName: \"kubernetes.io/projected/6ced761e-e20b-4b10-b8b1-fe77cd501818-kube-api-access-2kfxj\") pod \"dns-default-g2cfz\" (UID: \"6ced761e-e20b-4b10-b8b1-fe77cd501818\") " pod="openshift-dns/dns-default-g2cfz"
Jul 22 03:34:07.692499 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:07.692396    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ced761e-e20b-4b10-b8b1-fe77cd501818-metrics-tls podName:6ced761e-e20b-4b10-b8b1-fe77cd501818 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:08.192373806 +0000 UTC m=+35.810583799 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6ced761e-e20b-4b10-b8b1-fe77cd501818-metrics-tls") pod "dns-default-g2cfz" (UID: "6ced761e-e20b-4b10-b8b1-fe77cd501818") : secret "dns-default-metrics-tls" not found
Jul 22 03:34:07.692499 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.692421    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6ced761e-e20b-4b10-b8b1-fe77cd501818-config-volume\") pod \"dns-default-g2cfz\" (UID: \"6ced761e-e20b-4b10-b8b1-fe77cd501818\") " pod="openshift-dns/dns-default-g2cfz"
Jul 22 03:34:07.692499 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.692470    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f3317954-0d7e-49df-ae2a-4a4b48382557-cert\") pod \"ingress-canary-fldh9\" (UID: \"f3317954-0d7e-49df-ae2a-4a4b48382557\") " pod="openshift-ingress-canary/ingress-canary-fldh9"
Jul 22 03:34:07.692721 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.692654    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ced761e-e20b-4b10-b8b1-fe77cd501818-tmp-dir\") pod \"dns-default-g2cfz\" (UID: \"6ced761e-e20b-4b10-b8b1-fe77cd501818\") " pod="openshift-dns/dns-default-g2cfz"
Jul 22 03:34:07.693088 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.693067    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6ced761e-e20b-4b10-b8b1-fe77cd501818-config-volume\") pod \"dns-default-g2cfz\" (UID: \"6ced761e-e20b-4b10-b8b1-fe77cd501818\") " pod="openshift-dns/dns-default-g2cfz"
Jul 22 03:34:07.695264 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.695241    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f3317954-0d7e-49df-ae2a-4a4b48382557-cert\") pod \"ingress-canary-fldh9\" (UID: \"f3317954-0d7e-49df-ae2a-4a4b48382557\") " pod="openshift-ingress-canary/ingress-canary-fldh9"
Jul 22 03:34:07.701432 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.701409    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kfxj\" (UniqueName: \"kubernetes.io/projected/6ced761e-e20b-4b10-b8b1-fe77cd501818-kube-api-access-2kfxj\") pod \"dns-default-g2cfz\" (UID: \"6ced761e-e20b-4b10-b8b1-fe77cd501818\") " pod="openshift-dns/dns-default-g2cfz"
Jul 22 03:34:07.701608 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.701590    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fptn6\" (UniqueName: \"kubernetes.io/projected/f3317954-0d7e-49df-ae2a-4a4b48382557-kube-api-access-fptn6\") pod \"ingress-canary-fldh9\" (UID: \"f3317954-0d7e-49df-ae2a-4a4b48382557\") " pod="openshift-ingress-canary/ingress-canary-fldh9"
Jul 22 03:34:07.706393 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.706372    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-2p4qp"
Jul 22 03:34:07.735859 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.735831    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fldh9"
Jul 22 03:34:07.846038 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.845950    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2tvl"
Jul 22 03:34:07.846038 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.845980    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p867k"
Jul 22 03:34:07.846038 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.845980    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lctg9"
Jul 22 03:34:07.849034 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.848852    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 22 03:34:07.849034 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.848885    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 22 03:34:07.849034 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.848936    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-h4vks\""
Jul 22 03:34:07.849530 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.849512    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 22 03:34:07.849644 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.849592    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 22 03:34:07.849644 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.849596    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-dzrs8\""
Jul 22 03:34:07.995304 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:07.995260    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/367c6e07-f8b3-494b-a12c-3c11e916e61b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-jgckh\" (UID: \"367c6e07-f8b3-494b-a12c-3c11e916e61b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jgckh"
Jul 22 03:34:07.995467 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:07.995443    2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 22 03:34:07.995555 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:07.995531    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/367c6e07-f8b3-494b-a12c-3c11e916e61b-cluster-monitoring-operator-tls podName:367c6e07-f8b3-494b-a12c-3c11e916e61b nodeName:}" failed. No retries permitted until 2026-07-22 03:34:08.995508206 +0000 UTC m=+36.613718210 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/367c6e07-f8b3-494b-a12c-3c11e916e61b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-jgckh" (UID: "367c6e07-f8b3-494b-a12c-3c11e916e61b") : secret "cluster-monitoring-operator-tls" not found
Jul 22 03:34:08.096433 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:08.096342    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f23b727c-e88a-42b1-80e8-02384f07c8f6-service-ca-bundle\") pod \"router-default-7455857b66-nlzjg\" (UID: \"f23b727c-e88a-42b1-80e8-02384f07c8f6\") " pod="openshift-ingress/router-default-7455857b66-nlzjg"
Jul 22 03:34:08.096433 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:08.096389    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-registry-tls\") pod \"image-registry-7749b46548-cf9gh\" (UID: \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\") " pod="openshift-image-registry/image-registry-7749b46548-cf9gh"
Jul 22 03:34:08.096433 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:08.096430    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7bc03f15-dc0d-4a86-adc6-7ec711363d87-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-jds8z\" (UID: \"7bc03f15-dc0d-4a86-adc6-7ec711363d87\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jds8z"
Jul 22 03:34:08.096723 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:08.096492    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f23b727c-e88a-42b1-80e8-02384f07c8f6-service-ca-bundle podName:f23b727c-e88a-42b1-80e8-02384f07c8f6 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:09.096468636 +0000 UTC m=+36.714678637 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f23b727c-e88a-42b1-80e8-02384f07c8f6-service-ca-bundle") pod "router-default-7455857b66-nlzjg" (UID: "f23b727c-e88a-42b1-80e8-02384f07c8f6") : configmap references non-existent config key: service-ca.crt
Jul 22 03:34:08.096723 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:08.096525    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f23b727c-e88a-42b1-80e8-02384f07c8f6-metrics-certs\") pod \"router-default-7455857b66-nlzjg\" (UID: \"f23b727c-e88a-42b1-80e8-02384f07c8f6\") " pod="openshift-ingress/router-default-7455857b66-nlzjg"
Jul 22 03:34:08.096723 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:08.096541    2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 22 03:34:08.096723 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:08.096546    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 22 03:34:08.096723 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:08.096561    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7749b46548-cf9gh: secret "image-registry-tls" not found
Jul 22 03:34:08.096723 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:08.096590    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bc03f15-dc0d-4a86-adc6-7ec711363d87-samples-operator-tls podName:7bc03f15-dc0d-4a86-adc6-7ec711363d87 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:09.096576104 +0000 UTC m=+36.714786106 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7bc03f15-dc0d-4a86-adc6-7ec711363d87-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-jds8z" (UID: "7bc03f15-dc0d-4a86-adc6-7ec711363d87") : secret "samples-operator-tls" not found
Jul 22 03:34:08.096723 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:08.096606    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-registry-tls podName:7492beee-6a0b-4b5a-85c6-ec0064b61bbf nodeName:}" failed. No retries permitted until 2026-07-22 03:34:09.096597238 +0000 UTC m=+36.714807224 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-registry-tls") pod "image-registry-7749b46548-cf9gh" (UID: "7492beee-6a0b-4b5a-85c6-ec0064b61bbf") : secret "image-registry-tls" not found
Jul 22 03:34:08.096723 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:08.096681    2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 22 03:34:08.096723 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:08.096724    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f23b727c-e88a-42b1-80e8-02384f07c8f6-metrics-certs podName:f23b727c-e88a-42b1-80e8-02384f07c8f6 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:09.096711663 +0000 UTC m=+36.714921651 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f23b727c-e88a-42b1-80e8-02384f07c8f6-metrics-certs") pod "router-default-7455857b66-nlzjg" (UID: "f23b727c-e88a-42b1-80e8-02384f07c8f6") : secret "router-metrics-certs-default" not found
Jul 22 03:34:08.197237 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:08.197195    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6ced761e-e20b-4b10-b8b1-fe77cd501818-metrics-tls\") pod \"dns-default-g2cfz\" (UID: \"6ced761e-e20b-4b10-b8b1-fe77cd501818\") " pod="openshift-dns/dns-default-g2cfz"
Jul 22 03:34:08.197407 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:08.197367    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 03:34:08.197483 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:08.197447    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ced761e-e20b-4b10-b8b1-fe77cd501818-metrics-tls podName:6ced761e-e20b-4b10-b8b1-fe77cd501818 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:09.197425041 +0000 UTC m=+36.815635043 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6ced761e-e20b-4b10-b8b1-fe77cd501818-metrics-tls") pod "dns-default-g2cfz" (UID: "6ced761e-e20b-4b10-b8b1-fe77cd501818") : secret "dns-default-metrics-tls" not found
Jul 22 03:34:08.965677 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:08.965642    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-ptsm9"]
Jul 22 03:34:08.970000 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:34:08.969971    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee83bd5e_e31f_4049_947f_8f609f93908f.slice/crio-9c168bbea6168afe59ace41cd73bb26f3c6f78254879991df3ed91cfc2030077 WatchSource:0}: Error finding container 9c168bbea6168afe59ace41cd73bb26f3c6f78254879991df3ed91cfc2030077: Status 404 returned error can't find the container with id 9c168bbea6168afe59ace41cd73bb26f3c6f78254879991df3ed91cfc2030077
Jul 22 03:34:08.970000 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:08.969985    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-2p4qp"]
Jul 22 03:34:08.973684 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:34:08.973602    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f0defcc_ccff_43be_8206_54c772885080.slice/crio-a5e6868db55a2c4e846f7e8da027637759df47e39cd4091fb212c637e10f7fa6 WatchSource:0}: Error finding container a5e6868db55a2c4e846f7e8da027637759df47e39cd4091fb212c637e10f7fa6: Status 404 returned error can't find the container with id a5e6868db55a2c4e846f7e8da027637759df47e39cd4091fb212c637e10f7fa6
Jul 22 03:34:08.977278 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:08.977084    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qtwrs"]
Jul 22 03:34:08.980007 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:08.979981    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-lqg9w"]
Jul 22 03:34:08.981009 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:34:08.980729    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff77d659_f9dd_4d0a_8f75_2dbadc25d260.slice/crio-4c176d11fba1492ed50398ce6f95ed780494e8bcd7774238cadaacff0cf9b246 WatchSource:0}: Error finding container 4c176d11fba1492ed50398ce6f95ed780494e8bcd7774238cadaacff0cf9b246: Status 404 returned error can't find the container with id 4c176d11fba1492ed50398ce6f95ed780494e8bcd7774238cadaacff0cf9b246
Jul 22 03:34:08.983769 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:34:08.983704    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cdfda8a_dc86_4da1_b192_4d0ab5cd37f4.slice/crio-11b3f02e95a7dfb33bde8cd7430d1335dab6708d7c877973bab26ec9b8a7184e WatchSource:0}: Error finding container 11b3f02e95a7dfb33bde8cd7430d1335dab6708d7c877973bab26ec9b8a7184e: Status 404 returned error can't find the container with id 11b3f02e95a7dfb33bde8cd7430d1335dab6708d7c877973bab26ec9b8a7184e
Jul 22 03:34:08.987022 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:08.987003    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4hpsp"]
Jul 22 03:34:08.990122 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:34:08.990102    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c29ba98_5a7e_4603_8731_661eeb2b7bf5.slice/crio-c340aa635f85c6a42c08761d82394cda118490d02f5f7c05d6b1b5d25ee02aca WatchSource:0}: Error finding container c340aa635f85c6a42c08761d82394cda118490d02f5f7c05d6b1b5d25ee02aca: Status 404 returned error can't find the container with id c340aa635f85c6a42c08761d82394cda118490d02f5f7c05d6b1b5d25ee02aca
Jul 22 03:34:08.997862 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:08.997839    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fldh9"]
Jul 22 03:34:09.001248 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:34:09.001220    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3317954_0d7e_49df_ae2a_4a4b48382557.slice/crio-9254293b1745b0bd706c99ed45d56c8d981d0cd04f2bc43ba3a0e99fdaef24ed WatchSource:0}: Error finding container 9254293b1745b0bd706c99ed45d56c8d981d0cd04f2bc43ba3a0e99fdaef24ed: Status 404 returned error can't find the container with id 9254293b1745b0bd706c99ed45d56c8d981d0cd04f2bc43ba3a0e99fdaef24ed
Jul 22 03:34:09.005706 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:09.005685    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/367c6e07-f8b3-494b-a12c-3c11e916e61b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-jgckh\" (UID: \"367c6e07-f8b3-494b-a12c-3c11e916e61b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jgckh"
Jul 22 03:34:09.005807 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:09.005795    2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 22 03:34:09.005852 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:09.005842    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/367c6e07-f8b3-494b-a12c-3c11e916e61b-cluster-monitoring-operator-tls podName:367c6e07-f8b3-494b-a12c-3c11e916e61b nodeName:}" failed. No retries permitted until 2026-07-22 03:34:11.005829075 +0000 UTC m=+38.624039060 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/367c6e07-f8b3-494b-a12c-3c11e916e61b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-jgckh" (UID: "367c6e07-f8b3-494b-a12c-3c11e916e61b") : secret "cluster-monitoring-operator-tls" not found
Jul 22 03:34:09.106382 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:09.106345    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f23b727c-e88a-42b1-80e8-02384f07c8f6-service-ca-bundle\") pod \"router-default-7455857b66-nlzjg\" (UID: \"f23b727c-e88a-42b1-80e8-02384f07c8f6\") " pod="openshift-ingress/router-default-7455857b66-nlzjg"
Jul 22 03:34:09.106382 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:09.106385    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-registry-tls\") pod \"image-registry-7749b46548-cf9gh\" (UID: \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\") " pod="openshift-image-registry/image-registry-7749b46548-cf9gh"
Jul 22 03:34:09.106643 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:09.106422    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7bc03f15-dc0d-4a86-adc6-7ec711363d87-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-jds8z\" (UID: \"7bc03f15-dc0d-4a86-adc6-7ec711363d87\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jds8z"
Jul 22 03:34:09.106643 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:09.106471    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f23b727c-e88a-42b1-80e8-02384f07c8f6-metrics-certs\") pod \"router-default-7455857b66-nlzjg\" (UID: \"f23b727c-e88a-42b1-80e8-02384f07c8f6\") " pod="openshift-ingress/router-default-7455857b66-nlzjg"
Jul 22 03:34:09.106643 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:09.106537    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f23b727c-e88a-42b1-80e8-02384f07c8f6-service-ca-bundle podName:f23b727c-e88a-42b1-80e8-02384f07c8f6 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:11.106514055 +0000 UTC m=+38.724724054 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f23b727c-e88a-42b1-80e8-02384f07c8f6-service-ca-bundle") pod "router-default-7455857b66-nlzjg" (UID: "f23b727c-e88a-42b1-80e8-02384f07c8f6") : configmap references non-existent config key: service-ca.crt
Jul 22 03:34:09.106643 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:09.106558    2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 22 03:34:09.106643 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:09.106561    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 22 03:34:09.106643 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:09.106561    2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 22 03:34:09.106643 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:09.106580    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7749b46548-cf9gh: secret "image-registry-tls" not found
Jul 22 03:34:09.106643 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:09.106605    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f23b727c-e88a-42b1-80e8-02384f07c8f6-metrics-certs podName:f23b727c-e88a-42b1-80e8-02384f07c8f6 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:11.106591808 +0000 UTC m=+38.724801795 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f23b727c-e88a-42b1-80e8-02384f07c8f6-metrics-certs") pod "router-default-7455857b66-nlzjg" (UID: "f23b727c-e88a-42b1-80e8-02384f07c8f6") : secret "router-metrics-certs-default" not found
Jul 22 03:34:09.106981 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:09.106652    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bc03f15-dc0d-4a86-adc6-7ec711363d87-samples-operator-tls podName:7bc03f15-dc0d-4a86-adc6-7ec711363d87 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:11.106638478 +0000 UTC m=+38.724848463 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7bc03f15-dc0d-4a86-adc6-7ec711363d87-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-jds8z" (UID: "7bc03f15-dc0d-4a86-adc6-7ec711363d87") : secret "samples-operator-tls" not found
Jul 22 03:34:09.106981 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:09.106671    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-registry-tls podName:7492beee-6a0b-4b5a-85c6-ec0064b61bbf nodeName:}" failed. No retries permitted until 2026-07-22 03:34:11.106663321 +0000 UTC m=+38.724873310 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-registry-tls") pod "image-registry-7749b46548-cf9gh" (UID: "7492beee-6a0b-4b5a-85c6-ec0064b61bbf") : secret "image-registry-tls" not found
Jul 22 03:34:09.157057 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:09.157013    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-lqg9w" event={"ID":"8cdfda8a-dc86-4da1-b192-4d0ab5cd37f4","Type":"ContainerStarted","Data":"11b3f02e95a7dfb33bde8cd7430d1335dab6708d7c877973bab26ec9b8a7184e"}
Jul 22 03:34:09.158246 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:09.158218    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4hpsp" event={"ID":"7c29ba98-5a7e-4603-8731-661eeb2b7bf5","Type":"ContainerStarted","Data":"c340aa635f85c6a42c08761d82394cda118490d02f5f7c05d6b1b5d25ee02aca"}
Jul 22 03:34:09.159348 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:09.159315    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qtwrs" event={"ID":"ff77d659-f9dd-4d0a-8f75-2dbadc25d260","Type":"ContainerStarted","Data":"4c176d11fba1492ed50398ce6f95ed780494e8bcd7774238cadaacff0cf9b246"}
Jul 22 03:34:09.162168 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:09.162142    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6wt45" event={"ID":"26250740-d226-4262-bff6-caf04187cf8d","Type":"ContainerStarted","Data":"64e80406fa387b3de3c158e334150af628f613174fc7de440b9c7bb28b405f35"}
Jul 22 03:34:09.163306 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:09.163273    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fldh9" event={"ID":"f3317954-0d7e-49df-ae2a-4a4b48382557","Type":"ContainerStarted","Data":"9254293b1745b0bd706c99ed45d56c8d981d0cd04f2bc43ba3a0e99fdaef24ed"}
Jul 22 03:34:09.164443 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:09.164415    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-ptsm9" event={"ID":"ee83bd5e-e31f-4049-947f-8f609f93908f","Type":"ContainerStarted","Data":"9c168bbea6168afe59ace41cd73bb26f3c6f78254879991df3ed91cfc2030077"}
Jul 22 03:34:09.165547 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:09.165529    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-2p4qp" event={"ID":"9f0defcc-ccff-43be-8206-54c772885080","Type":"ContainerStarted","Data":"a5e6868db55a2c4e846f7e8da027637759df47e39cd4091fb212c637e10f7fa6"}
Jul 22 03:34:09.207230 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:09.207204    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6ced761e-e20b-4b10-b8b1-fe77cd501818-metrics-tls\") pod \"dns-default-g2cfz\" (UID: \"6ced761e-e20b-4b10-b8b1-fe77cd501818\") " pod="openshift-dns/dns-default-g2cfz"
Jul 22 03:34:09.207410 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:09.207387    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 03:34:09.207487 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:09.207476    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ced761e-e20b-4b10-b8b1-fe77cd501818-metrics-tls podName:6ced761e-e20b-4b10-b8b1-fe77cd501818 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:11.207454164 +0000 UTC m=+38.825664176 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6ced761e-e20b-4b10-b8b1-fe77cd501818-metrics-tls") pod "dns-default-g2cfz" (UID: "6ced761e-e20b-4b10-b8b1-fe77cd501818") : secret "dns-default-metrics-tls" not found
Jul 22 03:34:10.177288 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:10.176140    2570 generic.go:356] "Generic (PLEG): container finished" podID="26250740-d226-4262-bff6-caf04187cf8d" containerID="64e80406fa387b3de3c158e334150af628f613174fc7de440b9c7bb28b405f35" exitCode=0
Jul 22 03:34:10.177288 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:10.177222    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6wt45" event={"ID":"26250740-d226-4262-bff6-caf04187cf8d","Type":"ContainerDied","Data":"64e80406fa387b3de3c158e334150af628f613174fc7de440b9c7bb28b405f35"}
Jul 22 03:34:11.027332 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:11.027291    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/367c6e07-f8b3-494b-a12c-3c11e916e61b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-jgckh\" (UID: \"367c6e07-f8b3-494b-a12c-3c11e916e61b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jgckh"
Jul 22 03:34:11.027535 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:11.027443    2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 22 03:34:11.027535 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:11.027506    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/367c6e07-f8b3-494b-a12c-3c11e916e61b-cluster-monitoring-operator-tls podName:367c6e07-f8b3-494b-a12c-3c11e916e61b nodeName:}" failed. No retries permitted until 2026-07-22 03:34:15.027490752 +0000 UTC m=+42.645700738 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/367c6e07-f8b3-494b-a12c-3c11e916e61b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-jgckh" (UID: "367c6e07-f8b3-494b-a12c-3c11e916e61b") : secret "cluster-monitoring-operator-tls" not found
Jul 22 03:34:11.127906 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:11.127848    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f23b727c-e88a-42b1-80e8-02384f07c8f6-service-ca-bundle\") pod \"router-default-7455857b66-nlzjg\" (UID: \"f23b727c-e88a-42b1-80e8-02384f07c8f6\") " pod="openshift-ingress/router-default-7455857b66-nlzjg"
Jul 22 03:34:11.127906 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:11.127906    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-registry-tls\") pod \"image-registry-7749b46548-cf9gh\" (UID: \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\") " pod="openshift-image-registry/image-registry-7749b46548-cf9gh"
Jul 22 03:34:11.128115 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:11.127971    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7bc03f15-dc0d-4a86-adc6-7ec711363d87-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-jds8z\" (UID: \"7bc03f15-dc0d-4a86-adc6-7ec711363d87\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jds8z"
Jul 22 03:34:11.128115 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:11.128024    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f23b727c-e88a-42b1-80e8-02384f07c8f6-metrics-certs\") pod \"router-default-7455857b66-nlzjg\" (UID: \"f23b727c-e88a-42b1-80e8-02384f07c8f6\") " pod="openshift-ingress/router-default-7455857b66-nlzjg"
Jul 22 03:34:11.128187 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:11.128164    2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 22 03:34:11.128256 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:11.128236    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f23b727c-e88a-42b1-80e8-02384f07c8f6-metrics-certs podName:f23b727c-e88a-42b1-80e8-02384f07c8f6 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:15.128214837 +0000 UTC m=+42.746424845 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f23b727c-e88a-42b1-80e8-02384f07c8f6-metrics-certs") pod "router-default-7455857b66-nlzjg" (UID: "f23b727c-e88a-42b1-80e8-02384f07c8f6") : secret "router-metrics-certs-default" not found
Jul 22 03:34:11.128369 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:11.128346    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 22 03:34:11.128437 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:11.128374    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7749b46548-cf9gh: secret "image-registry-tls" not found
Jul 22 03:34:11.128437 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:11.128430    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-registry-tls podName:7492beee-6a0b-4b5a-85c6-ec0064b61bbf nodeName:}" failed. No retries permitted until 2026-07-22 03:34:15.12841128 +0000 UTC m=+42.746621265 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-registry-tls") pod "image-registry-7749b46548-cf9gh" (UID: "7492beee-6a0b-4b5a-85c6-ec0064b61bbf") : secret "image-registry-tls" not found
Jul 22 03:34:11.128832 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:11.128580    2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 22 03:34:11.128832 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:11.128629    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f23b727c-e88a-42b1-80e8-02384f07c8f6-service-ca-bundle podName:f23b727c-e88a-42b1-80e8-02384f07c8f6 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:15.128593753 +0000 UTC m=+42.746803743 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f23b727c-e88a-42b1-80e8-02384f07c8f6-service-ca-bundle") pod "router-default-7455857b66-nlzjg" (UID: "f23b727c-e88a-42b1-80e8-02384f07c8f6") : configmap references non-existent config key: service-ca.crt
Jul 22 03:34:11.128832 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:11.128788    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bc03f15-dc0d-4a86-adc6-7ec711363d87-samples-operator-tls podName:7bc03f15-dc0d-4a86-adc6-7ec711363d87 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:15.128765943 +0000 UTC m=+42.746975940 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7bc03f15-dc0d-4a86-adc6-7ec711363d87-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-jds8z" (UID: "7bc03f15-dc0d-4a86-adc6-7ec711363d87") : secret "samples-operator-tls" not found
Jul 22 03:34:11.186638 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:11.186223    2570 generic.go:356] "Generic (PLEG): container finished" podID="26250740-d226-4262-bff6-caf04187cf8d" containerID="516d8aff7223d822877718f1448871e5c388b812a584461135d150d5b3178657" exitCode=0
Jul 22 03:34:11.186638 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:11.186309    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6wt45" event={"ID":"26250740-d226-4262-bff6-caf04187cf8d","Type":"ContainerDied","Data":"516d8aff7223d822877718f1448871e5c388b812a584461135d150d5b3178657"}
Jul 22 03:34:11.229336 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:11.229303    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6ced761e-e20b-4b10-b8b1-fe77cd501818-metrics-tls\") pod \"dns-default-g2cfz\" (UID: \"6ced761e-e20b-4b10-b8b1-fe77cd501818\") " pod="openshift-dns/dns-default-g2cfz"
Jul 22 03:34:11.229525 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:11.229470    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 03:34:11.229590 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:11.229543    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ced761e-e20b-4b10-b8b1-fe77cd501818-metrics-tls podName:6ced761e-e20b-4b10-b8b1-fe77cd501818 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:15.229521979 +0000 UTC m=+42.847731984 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6ced761e-e20b-4b10-b8b1-fe77cd501818-metrics-tls") pod "dns-default-g2cfz" (UID: "6ced761e-e20b-4b10-b8b1-fe77cd501818") : secret "dns-default-metrics-tls" not found
Jul 22 03:34:15.067242 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:15.067220    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/367c6e07-f8b3-494b-a12c-3c11e916e61b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-jgckh\" (UID: \"367c6e07-f8b3-494b-a12c-3c11e916e61b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jgckh"
Jul 22 03:34:15.067580 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:15.067388    2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 22 03:34:15.067580 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:15.067470    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/367c6e07-f8b3-494b-a12c-3c11e916e61b-cluster-monitoring-operator-tls podName:367c6e07-f8b3-494b-a12c-3c11e916e61b nodeName:}" failed. No retries permitted until 2026-07-22 03:34:23.06744951 +0000 UTC m=+50.685659513 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/367c6e07-f8b3-494b-a12c-3c11e916e61b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-jgckh" (UID: "367c6e07-f8b3-494b-a12c-3c11e916e61b") : secret "cluster-monitoring-operator-tls" not found
Jul 22 03:34:15.169295 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:15.167767    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f23b727c-e88a-42b1-80e8-02384f07c8f6-service-ca-bundle\") pod \"router-default-7455857b66-nlzjg\" (UID: \"f23b727c-e88a-42b1-80e8-02384f07c8f6\") " pod="openshift-ingress/router-default-7455857b66-nlzjg"
Jul 22 03:34:15.169295 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:15.167808    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-registry-tls\") pod \"image-registry-7749b46548-cf9gh\" (UID: \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\") " pod="openshift-image-registry/image-registry-7749b46548-cf9gh"
Jul 22 03:34:15.169295 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:15.167875    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7bc03f15-dc0d-4a86-adc6-7ec711363d87-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-jds8z\" (UID: \"7bc03f15-dc0d-4a86-adc6-7ec711363d87\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jds8z"
Jul 22 03:34:15.169295 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:15.167944    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f23b727c-e88a-42b1-80e8-02384f07c8f6-metrics-certs\") pod \"router-default-7455857b66-nlzjg\" (UID: \"f23b727c-e88a-42b1-80e8-02384f07c8f6\") " pod="openshift-ingress/router-default-7455857b66-nlzjg"
Jul 22 03:34:15.169295 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:15.168150    2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 22 03:34:15.169295 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:15.168587    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 22 03:34:15.169295 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:15.168603    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7749b46548-cf9gh: secret "image-registry-tls" not found
Jul 22 03:34:15.169295 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:15.168673    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-registry-tls podName:7492beee-6a0b-4b5a-85c6-ec0064b61bbf nodeName:}" failed. No retries permitted until 2026-07-22 03:34:23.168650714 +0000 UTC m=+50.786860716 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-registry-tls") pod "image-registry-7749b46548-cf9gh" (UID: "7492beee-6a0b-4b5a-85c6-ec0064b61bbf") : secret "image-registry-tls" not found
Jul 22 03:34:15.169295 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:15.168751    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f23b727c-e88a-42b1-80e8-02384f07c8f6-service-ca-bundle podName:f23b727c-e88a-42b1-80e8-02384f07c8f6 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:23.168738683 +0000 UTC m=+50.786948692 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f23b727c-e88a-42b1-80e8-02384f07c8f6-service-ca-bundle") pod "router-default-7455857b66-nlzjg" (UID: "f23b727c-e88a-42b1-80e8-02384f07c8f6") : configmap references non-existent config key: service-ca.crt
Jul 22 03:34:15.169295 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:15.168823    2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 22 03:34:15.169295 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:15.168913    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bc03f15-dc0d-4a86-adc6-7ec711363d87-samples-operator-tls podName:7bc03f15-dc0d-4a86-adc6-7ec711363d87 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:23.168899117 +0000 UTC m=+50.787109117 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7bc03f15-dc0d-4a86-adc6-7ec711363d87-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-jds8z" (UID: "7bc03f15-dc0d-4a86-adc6-7ec711363d87") : secret "samples-operator-tls" not found
Jul 22 03:34:15.169295 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:15.168967    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f23b727c-e88a-42b1-80e8-02384f07c8f6-metrics-certs podName:f23b727c-e88a-42b1-80e8-02384f07c8f6 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:23.16894538 +0000 UTC m=+50.787155382 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f23b727c-e88a-42b1-80e8-02384f07c8f6-metrics-certs") pod "router-default-7455857b66-nlzjg" (UID: "f23b727c-e88a-42b1-80e8-02384f07c8f6") : secret "router-metrics-certs-default" not found
Jul 22 03:34:15.201818 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:15.201783    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qtwrs" event={"ID":"ff77d659-f9dd-4d0a-8f75-2dbadc25d260","Type":"ContainerStarted","Data":"62d0181db2aa2813d5837cc23c5d7c753c1876303c8823d2b0a87972f7d37541"}
Jul 22 03:34:15.207917 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:15.207282    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6wt45" event={"ID":"26250740-d226-4262-bff6-caf04187cf8d","Type":"ContainerStarted","Data":"c082ef55149426fdb679a99941e3eddb07c6bfea1e243a070a302420129f85e5"}
Jul 22 03:34:15.209949 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:15.209913    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fldh9" event={"ID":"f3317954-0d7e-49df-ae2a-4a4b48382557","Type":"ContainerStarted","Data":"6cc2fed66a7015f108ee8bac630d72161d16224cd480d8f49986353e2b38c2aa"}
Jul 22 03:34:15.212764 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:15.211779    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-ptsm9" event={"ID":"ee83bd5e-e31f-4049-947f-8f609f93908f","Type":"ContainerStarted","Data":"f53074abe07fe81a59736e19faae8a0247af2587323f6f1dc732d8efe542dab1"}
Jul 22 03:34:15.213732 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:15.213688    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-2p4qp" event={"ID":"9f0defcc-ccff-43be-8206-54c772885080","Type":"ContainerStarted","Data":"ccaa6f2f4871e63b630f53a10be804dadd5d52048c15ce56860110774d0b85a2"}
Jul 22 03:34:15.214923 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:15.214681    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-2p4qp"
Jul 22 03:34:15.215071 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:15.215038    2570 patch_prober.go:28] interesting pod/console-operator-565b7b569d-2p4qp container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.12:8443/readyz\": dial tcp 10.132.0.12:8443: connect: connection refused" start-of-body=
Jul 22 03:34:15.215136 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:15.215094    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-565b7b569d-2p4qp" podUID="9f0defcc-ccff-43be-8206-54c772885080" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.12:8443/readyz\": dial tcp 10.132.0.12:8443: connect: connection refused"
Jul 22 03:34:15.215597 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:15.215553    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-lqg9w" event={"ID":"8cdfda8a-dc86-4da1-b192-4d0ab5cd37f4","Type":"ContainerStarted","Data":"9407bba911145d7fff73f41142ca049c24615f29bd3b58c8f3b300da16ffc086"}
Jul 22 03:34:15.220035 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:15.219531    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4hpsp" event={"ID":"7c29ba98-5a7e-4603-8731-661eeb2b7bf5","Type":"ContainerStarted","Data":"34c681b57f8b59b8d34973008fae270263907aca2b9221dd32f5eb6698939f83"}
Jul 22 03:34:15.223558 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:15.223244    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qtwrs" podStartSLOduration=5.208101527 podStartE2EDuration="11.22322788s" totalImagesPullingTime="6.015126353s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:34:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:34:15.222799015 +0000 UTC m=+42.841009023" watchObservedRunningTime="2026-07-22 03:34:15.22322788 +0000 UTC m=+42.841437888"
Jul 22 03:34:15.242555 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:15.242490    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-ptsm9" podStartSLOduration=5.218817077 podStartE2EDuration="11.242470781s" totalImagesPullingTime="6.023653704s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:34:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:34:15.241804499 +0000 UTC m=+42.860014507" watchObservedRunningTime="2026-07-22 03:34:15.242470781 +0000 UTC m=+42.860680767"
Jul 22 03:34:15.262653 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:15.262024    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-lqg9w" podStartSLOduration=5.250332769 podStartE2EDuration="11.262001438s" totalImagesPullingTime="6.011668669s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:34:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:34:15.260745994 +0000 UTC m=+42.878956029" watchObservedRunningTime="2026-07-22 03:34:15.262001438 +0000 UTC m=+42.880211450"
Jul 22 03:34:15.269647 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:15.269279    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6ced761e-e20b-4b10-b8b1-fe77cd501818-metrics-tls\") pod \"dns-default-g2cfz\" (UID: \"6ced761e-e20b-4b10-b8b1-fe77cd501818\") " pod="openshift-dns/dns-default-g2cfz"
Jul 22 03:34:15.269647 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:15.269401    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 03:34:15.269647 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:15.269454    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ced761e-e20b-4b10-b8b1-fe77cd501818-metrics-tls podName:6ced761e-e20b-4b10-b8b1-fe77cd501818 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:23.269436817 +0000 UTC m=+50.887646808 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6ced761e-e20b-4b10-b8b1-fe77cd501818-metrics-tls") pod "dns-default-g2cfz" (UID: "6ced761e-e20b-4b10-b8b1-fe77cd501818") : secret "dns-default-metrics-tls" not found
Jul 22 03:34:15.302330 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:15.302202    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-2p4qp" podStartSLOduration=5.28034658 podStartE2EDuration="11.302186274s" totalImagesPullingTime="6.021839694s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:34:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:34:15.282698831 +0000 UTC m=+42.900908856" watchObservedRunningTime="2026-07-22 03:34:15.302186274 +0000 UTC m=+42.920396283"
Jul 22 03:34:15.302499 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:15.302441    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-fldh9" podStartSLOduration=2.301329425 podStartE2EDuration="8.302430456s" totalImagesPullingTime="6.001101031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:34:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:34:15.30196133 +0000 UTC m=+42.920171369" watchObservedRunningTime="2026-07-22 03:34:15.302430456 +0000 UTC m=+42.920640466"
Jul 22 03:34:15.329242 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:15.328770    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-6wt45" podStartSLOduration=10.437336351 podStartE2EDuration="42.328749211s" totalImagesPullingTime="31.89141286s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:33:33 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:34:15.326276338 +0000 UTC m=+42.944486347" watchObservedRunningTime="2026-07-22 03:34:15.328749211 +0000 UTC m=+42.946959219"
Jul 22 03:34:15.346258 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:15.346198    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4hpsp" podStartSLOduration=5.340663066 podStartE2EDuration="11.346175815s" totalImagesPullingTime="6.005512749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:34:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:34:15.345098406 +0000 UTC m=+42.963308414" watchObservedRunningTime="2026-07-22 03:34:15.346175815 +0000 UTC m=+42.964385825"
Jul 22 03:34:16.225067 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:16.225032    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2p4qp_9f0defcc-ccff-43be-8206-54c772885080/console-operator/0.log"
Jul 22 03:34:16.225514 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:16.225091    2570 generic.go:356] "Generic (PLEG): container finished" podID="9f0defcc-ccff-43be-8206-54c772885080" containerID="ccaa6f2f4871e63b630f53a10be804dadd5d52048c15ce56860110774d0b85a2" exitCode=255
Jul 22 03:34:16.225514 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:16.225172    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-2p4qp" event={"ID":"9f0defcc-ccff-43be-8206-54c772885080","Type":"ContainerDied","Data":"ccaa6f2f4871e63b630f53a10be804dadd5d52048c15ce56860110774d0b85a2"}
Jul 22 03:34:16.225634 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:16.225556    2570 scope.go:117] "RemoveContainer" containerID="ccaa6f2f4871e63b630f53a10be804dadd5d52048c15ce56860110774d0b85a2"
Jul 22 03:34:17.230519 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:17.230493    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2p4qp_9f0defcc-ccff-43be-8206-54c772885080/console-operator/1.log"
Jul 22 03:34:17.231047 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:17.230938    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2p4qp_9f0defcc-ccff-43be-8206-54c772885080/console-operator/0.log"
Jul 22 03:34:17.231047 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:17.230976    2570 generic.go:356] "Generic (PLEG): container finished" podID="9f0defcc-ccff-43be-8206-54c772885080" containerID="dcdb035e2252e40c1adee1b4bfc18699dc3c75fc9b8ba3f1a67e679ce96c1359" exitCode=255
Jul 22 03:34:17.231047 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:17.231035    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-2p4qp" event={"ID":"9f0defcc-ccff-43be-8206-54c772885080","Type":"ContainerDied","Data":"dcdb035e2252e40c1adee1b4bfc18699dc3c75fc9b8ba3f1a67e679ce96c1359"}
Jul 22 03:34:17.231204 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:17.231093    2570 scope.go:117] "RemoveContainer" containerID="ccaa6f2f4871e63b630f53a10be804dadd5d52048c15ce56860110774d0b85a2"
Jul 22 03:34:17.231317 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:17.231295    2570 scope.go:117] "RemoveContainer" containerID="dcdb035e2252e40c1adee1b4bfc18699dc3c75fc9b8ba3f1a67e679ce96c1359"
Jul 22 03:34:17.231514 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:17.231491    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-2p4qp_openshift-console-operator(9f0defcc-ccff-43be-8206-54c772885080)\"" pod="openshift-console-operator/console-operator-565b7b569d-2p4qp" podUID="9f0defcc-ccff-43be-8206-54c772885080"
Jul 22 03:34:17.389519 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:17.389484    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-j8z8l"]
Jul 22 03:34:17.422721 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:17.422691    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-j8z8l"]
Jul 22 03:34:17.422892 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:17.422807    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-j8z8l"
Jul 22 03:34:17.425393 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:17.425369    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 22 03:34:17.425525 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:17.425370    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 22 03:34:17.425525 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:17.425454    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-cwd77\""
Jul 22 03:34:17.594647 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:17.594540    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b180b221-d3f1-4f70-a258-e680495a3c40-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j8z8l\" (UID: \"b180b221-d3f1-4f70-a258-e680495a3c40\") " pod="openshift-insights/insights-runtime-extractor-j8z8l"
Jul 22 03:34:17.594647 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:17.594585    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vwr4\" (UniqueName: \"kubernetes.io/projected/b180b221-d3f1-4f70-a258-e680495a3c40-kube-api-access-5vwr4\") pod \"insights-runtime-extractor-j8z8l\" (UID: \"b180b221-d3f1-4f70-a258-e680495a3c40\") " pod="openshift-insights/insights-runtime-extractor-j8z8l"
Jul 22 03:34:17.594837 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:17.594662    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b180b221-d3f1-4f70-a258-e680495a3c40-data-volume\") pod \"insights-runtime-extractor-j8z8l\" (UID: \"b180b221-d3f1-4f70-a258-e680495a3c40\") " pod="openshift-insights/insights-runtime-extractor-j8z8l"
Jul 22 03:34:17.594837 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:17.594752    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b180b221-d3f1-4f70-a258-e680495a3c40-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j8z8l\" (UID: \"b180b221-d3f1-4f70-a258-e680495a3c40\") " pod="openshift-insights/insights-runtime-extractor-j8z8l"
Jul 22 03:34:17.594837 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:17.594772    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b180b221-d3f1-4f70-a258-e680495a3c40-crio-socket\") pod \"insights-runtime-extractor-j8z8l\" (UID: \"b180b221-d3f1-4f70-a258-e680495a3c40\") " pod="openshift-insights/insights-runtime-extractor-j8z8l"
Jul 22 03:34:17.695472 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:17.695428    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b180b221-d3f1-4f70-a258-e680495a3c40-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j8z8l\" (UID: \"b180b221-d3f1-4f70-a258-e680495a3c40\") " pod="openshift-insights/insights-runtime-extractor-j8z8l"
Jul 22 03:34:17.695472 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:17.695473    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b180b221-d3f1-4f70-a258-e680495a3c40-crio-socket\") pod \"insights-runtime-extractor-j8z8l\" (UID: \"b180b221-d3f1-4f70-a258-e680495a3c40\") " pod="openshift-insights/insights-runtime-extractor-j8z8l"
Jul 22 03:34:17.695707 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:17.695567    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 03:34:17.695707 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:17.695569    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b180b221-d3f1-4f70-a258-e680495a3c40-crio-socket\") pod \"insights-runtime-extractor-j8z8l\" (UID: \"b180b221-d3f1-4f70-a258-e680495a3c40\") " pod="openshift-insights/insights-runtime-extractor-j8z8l"
Jul 22 03:34:17.695707 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:17.695670    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b180b221-d3f1-4f70-a258-e680495a3c40-kube-rbac-proxy-cm podName:b180b221-d3f1-4f70-a258-e680495a3c40 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:18.19565031 +0000 UTC m=+45.813860308 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b180b221-d3f1-4f70-a258-e680495a3c40-kube-rbac-proxy-cm") pod "insights-runtime-extractor-j8z8l" (UID: "b180b221-d3f1-4f70-a258-e680495a3c40") : configmap "kube-rbac-proxy" not found
Jul 22 03:34:17.695707 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:17.695656    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b180b221-d3f1-4f70-a258-e680495a3c40-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j8z8l\" (UID: \"b180b221-d3f1-4f70-a258-e680495a3c40\") " pod="openshift-insights/insights-runtime-extractor-j8z8l"
Jul 22 03:34:17.695854 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:17.695740    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 03:34:17.695854 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:17.695746    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vwr4\" (UniqueName: \"kubernetes.io/projected/b180b221-d3f1-4f70-a258-e680495a3c40-kube-api-access-5vwr4\") pod \"insights-runtime-extractor-j8z8l\" (UID: \"b180b221-d3f1-4f70-a258-e680495a3c40\") " pod="openshift-insights/insights-runtime-extractor-j8z8l"
Jul 22 03:34:17.695854 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:17.695801    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b180b221-d3f1-4f70-a258-e680495a3c40-data-volume\") pod \"insights-runtime-extractor-j8z8l\" (UID: \"b180b221-d3f1-4f70-a258-e680495a3c40\") " pod="openshift-insights/insights-runtime-extractor-j8z8l"
Jul 22 03:34:17.695854 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:17.695835    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b180b221-d3f1-4f70-a258-e680495a3c40-insights-runtime-extractor-tls podName:b180b221-d3f1-4f70-a258-e680495a3c40 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:18.19580567 +0000 UTC m=+45.814015659 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b180b221-d3f1-4f70-a258-e680495a3c40-insights-runtime-extractor-tls") pod "insights-runtime-extractor-j8z8l" (UID: "b180b221-d3f1-4f70-a258-e680495a3c40") : secret "insights-runtime-extractor-tls" not found
Jul 22 03:34:17.696140 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:17.696123    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b180b221-d3f1-4f70-a258-e680495a3c40-data-volume\") pod \"insights-runtime-extractor-j8z8l\" (UID: \"b180b221-d3f1-4f70-a258-e680495a3c40\") " pod="openshift-insights/insights-runtime-extractor-j8z8l"
Jul 22 03:34:17.707179 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:17.707152    2570 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-2p4qp"
Jul 22 03:34:17.707986 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:17.707959    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vwr4\" (UniqueName: \"kubernetes.io/projected/b180b221-d3f1-4f70-a258-e680495a3c40-kube-api-access-5vwr4\") pod \"insights-runtime-extractor-j8z8l\" (UID: \"b180b221-d3f1-4f70-a258-e680495a3c40\") " pod="openshift-insights/insights-runtime-extractor-j8z8l"
Jul 22 03:34:18.199518 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:18.199473    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b180b221-d3f1-4f70-a258-e680495a3c40-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j8z8l\" (UID: \"b180b221-d3f1-4f70-a258-e680495a3c40\") " pod="openshift-insights/insights-runtime-extractor-j8z8l"
Jul 22 03:34:18.199739 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:18.199595    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b180b221-d3f1-4f70-a258-e680495a3c40-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j8z8l\" (UID: \"b180b221-d3f1-4f70-a258-e680495a3c40\") " pod="openshift-insights/insights-runtime-extractor-j8z8l"
Jul 22 03:34:18.199739 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:18.199629    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 03:34:18.199739 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:18.199698    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b180b221-d3f1-4f70-a258-e680495a3c40-kube-rbac-proxy-cm podName:b180b221-d3f1-4f70-a258-e680495a3c40 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:19.199675658 +0000 UTC m=+46.817885658 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b180b221-d3f1-4f70-a258-e680495a3c40-kube-rbac-proxy-cm") pod "insights-runtime-extractor-j8z8l" (UID: "b180b221-d3f1-4f70-a258-e680495a3c40") : configmap "kube-rbac-proxy" not found
Jul 22 03:34:18.199915 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:18.199756    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 03:34:18.199915 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:18.199829    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b180b221-d3f1-4f70-a258-e680495a3c40-insights-runtime-extractor-tls podName:b180b221-d3f1-4f70-a258-e680495a3c40 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:19.199816903 +0000 UTC m=+46.818026894 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b180b221-d3f1-4f70-a258-e680495a3c40-insights-runtime-extractor-tls") pod "insights-runtime-extractor-j8z8l" (UID: "b180b221-d3f1-4f70-a258-e680495a3c40") : secret "insights-runtime-extractor-tls" not found
Jul 22 03:34:18.236524 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:18.236496    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2p4qp_9f0defcc-ccff-43be-8206-54c772885080/console-operator/1.log"
Jul 22 03:34:18.237002 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:18.236910    2570 scope.go:117] "RemoveContainer" containerID="dcdb035e2252e40c1adee1b4bfc18699dc3c75fc9b8ba3f1a67e679ce96c1359"
Jul 22 03:34:18.237116 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:18.237093    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-2p4qp_openshift-console-operator(9f0defcc-ccff-43be-8206-54c772885080)\"" pod="openshift-console-operator/console-operator-565b7b569d-2p4qp" podUID="9f0defcc-ccff-43be-8206-54c772885080"
Jul 22 03:34:19.207897 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:19.207852    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b180b221-d3f1-4f70-a258-e680495a3c40-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j8z8l\" (UID: \"b180b221-d3f1-4f70-a258-e680495a3c40\") " pod="openshift-insights/insights-runtime-extractor-j8z8l"
Jul 22 03:34:19.208074 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:19.207954    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b180b221-d3f1-4f70-a258-e680495a3c40-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j8z8l\" (UID: \"b180b221-d3f1-4f70-a258-e680495a3c40\") " pod="openshift-insights/insights-runtime-extractor-j8z8l"
Jul 22 03:34:19.208074 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:19.207973    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 03:34:19.208074 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:19.208044    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 03:34:19.208074 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:19.208053    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b180b221-d3f1-4f70-a258-e680495a3c40-kube-rbac-proxy-cm podName:b180b221-d3f1-4f70-a258-e680495a3c40 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:21.208036497 +0000 UTC m=+48.826246482 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b180b221-d3f1-4f70-a258-e680495a3c40-kube-rbac-proxy-cm") pod "insights-runtime-extractor-j8z8l" (UID: "b180b221-d3f1-4f70-a258-e680495a3c40") : configmap "kube-rbac-proxy" not found
Jul 22 03:34:19.208215 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:19.208108    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b180b221-d3f1-4f70-a258-e680495a3c40-insights-runtime-extractor-tls podName:b180b221-d3f1-4f70-a258-e680495a3c40 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:21.208096414 +0000 UTC m=+48.826306400 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b180b221-d3f1-4f70-a258-e680495a3c40-insights-runtime-extractor-tls") pod "insights-runtime-extractor-j8z8l" (UID: "b180b221-d3f1-4f70-a258-e680495a3c40") : secret "insights-runtime-extractor-tls" not found
Jul 22 03:34:19.239887 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:19.239853    2570 scope.go:117] "RemoveContainer" containerID="dcdb035e2252e40c1adee1b4bfc18699dc3c75fc9b8ba3f1a67e679ce96c1359"
Jul 22 03:34:19.240237 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:19.240022    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-2p4qp_openshift-console-operator(9f0defcc-ccff-43be-8206-54c772885080)\"" pod="openshift-console-operator/console-operator-565b7b569d-2p4qp" podUID="9f0defcc-ccff-43be-8206-54c772885080"
Jul 22 03:34:20.076870 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:20.076846    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tpgsf_ed43ef9c-bf6c-4f2c-96b5-fc31d24e5fd1/dns-node-resolver/0.log"
Jul 22 03:34:20.675793 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:20.675765    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7srsk_56a14a96-971e-4dab-a1bb-8e3edb6b73b2/node-ca/0.log"
Jul 22 03:34:21.227653 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:21.227598    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b180b221-d3f1-4f70-a258-e680495a3c40-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j8z8l\" (UID: \"b180b221-d3f1-4f70-a258-e680495a3c40\") " pod="openshift-insights/insights-runtime-extractor-j8z8l"
Jul 22 03:34:21.227826 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:21.227726    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b180b221-d3f1-4f70-a258-e680495a3c40-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j8z8l\" (UID: \"b180b221-d3f1-4f70-a258-e680495a3c40\") " pod="openshift-insights/insights-runtime-extractor-j8z8l"
Jul 22 03:34:21.227826 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:21.227746    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 03:34:21.227826 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:21.227805    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b180b221-d3f1-4f70-a258-e680495a3c40-insights-runtime-extractor-tls podName:b180b221-d3f1-4f70-a258-e680495a3c40 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:25.227791138 +0000 UTC m=+52.846001123 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b180b221-d3f1-4f70-a258-e680495a3c40-insights-runtime-extractor-tls") pod "insights-runtime-extractor-j8z8l" (UID: "b180b221-d3f1-4f70-a258-e680495a3c40") : secret "insights-runtime-extractor-tls" not found
Jul 22 03:34:21.227826 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:21.227814    2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 03:34:21.227963 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:21.227852    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b180b221-d3f1-4f70-a258-e680495a3c40-kube-rbac-proxy-cm podName:b180b221-d3f1-4f70-a258-e680495a3c40 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:25.227841731 +0000 UTC m=+52.846051719 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b180b221-d3f1-4f70-a258-e680495a3c40-kube-rbac-proxy-cm") pod "insights-runtime-extractor-j8z8l" (UID: "b180b221-d3f1-4f70-a258-e680495a3c40") : configmap "kube-rbac-proxy" not found
Jul 22 03:34:21.276168 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:21.276140    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fldh9_f3317954-0d7e-49df-ae2a-4a4b48382557/serve-healthcheck-canary/0.log"
Jul 22 03:34:22.078192 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:22.078162    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-lqg9w_8cdfda8a-dc86-4da1-b192-4d0ab5cd37f4/kube-storage-version-migrator-operator/0.log"
Jul 22 03:34:23.143382 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:23.143343    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/367c6e07-f8b3-494b-a12c-3c11e916e61b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-jgckh\" (UID: \"367c6e07-f8b3-494b-a12c-3c11e916e61b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jgckh"
Jul 22 03:34:23.143871 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:23.143524    2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 22 03:34:23.143871 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:23.143648    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/367c6e07-f8b3-494b-a12c-3c11e916e61b-cluster-monitoring-operator-tls podName:367c6e07-f8b3-494b-a12c-3c11e916e61b nodeName:}" failed. No retries permitted until 2026-07-22 03:34:39.143598071 +0000 UTC m=+66.761808069 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/367c6e07-f8b3-494b-a12c-3c11e916e61b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-jgckh" (UID: "367c6e07-f8b3-494b-a12c-3c11e916e61b") : secret "cluster-monitoring-operator-tls" not found
Jul 22 03:34:23.244025 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:23.243982    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f23b727c-e88a-42b1-80e8-02384f07c8f6-service-ca-bundle\") pod \"router-default-7455857b66-nlzjg\" (UID: \"f23b727c-e88a-42b1-80e8-02384f07c8f6\") " pod="openshift-ingress/router-default-7455857b66-nlzjg"
Jul 22 03:34:23.244025 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:23.244029    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-registry-tls\") pod \"image-registry-7749b46548-cf9gh\" (UID: \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\") " pod="openshift-image-registry/image-registry-7749b46548-cf9gh"
Jul 22 03:34:23.244204 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:23.244088    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7bc03f15-dc0d-4a86-adc6-7ec711363d87-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-jds8z\" (UID: \"7bc03f15-dc0d-4a86-adc6-7ec711363d87\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jds8z"
Jul 22 03:34:23.244204 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:23.244146    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f23b727c-e88a-42b1-80e8-02384f07c8f6-metrics-certs\") pod \"router-default-7455857b66-nlzjg\" (UID: \"f23b727c-e88a-42b1-80e8-02384f07c8f6\") " pod="openshift-ingress/router-default-7455857b66-nlzjg"
Jul 22 03:34:23.244204 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:23.244164    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f23b727c-e88a-42b1-80e8-02384f07c8f6-service-ca-bundle podName:f23b727c-e88a-42b1-80e8-02384f07c8f6 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:39.244141145 +0000 UTC m=+66.862351149 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f23b727c-e88a-42b1-80e8-02384f07c8f6-service-ca-bundle") pod "router-default-7455857b66-nlzjg" (UID: "f23b727c-e88a-42b1-80e8-02384f07c8f6") : configmap references non-existent config key: service-ca.crt
Jul 22 03:34:23.244343 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:23.244216    2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 22 03:34:23.244343 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:23.244242    2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7749b46548-cf9gh: secret "image-registry-tls" not found
Jul 22 03:34:23.244343 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:23.244220    2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 22 03:34:23.244343 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:23.244275    2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 22 03:34:23.244343 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:23.244283    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-registry-tls podName:7492beee-6a0b-4b5a-85c6-ec0064b61bbf nodeName:}" failed. No retries permitted until 2026-07-22 03:34:39.244271584 +0000 UTC m=+66.862481599 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-registry-tls") pod "image-registry-7749b46548-cf9gh" (UID: "7492beee-6a0b-4b5a-85c6-ec0064b61bbf") : secret "image-registry-tls" not found
Jul 22 03:34:23.244343 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:23.244302    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bc03f15-dc0d-4a86-adc6-7ec711363d87-samples-operator-tls podName:7bc03f15-dc0d-4a86-adc6-7ec711363d87 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:39.244291117 +0000 UTC m=+66.862501102 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7bc03f15-dc0d-4a86-adc6-7ec711363d87-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-jds8z" (UID: "7bc03f15-dc0d-4a86-adc6-7ec711363d87") : secret "samples-operator-tls" not found
Jul 22 03:34:23.244343 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:23.244318    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f23b727c-e88a-42b1-80e8-02384f07c8f6-metrics-certs podName:f23b727c-e88a-42b1-80e8-02384f07c8f6 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:39.244309065 +0000 UTC m=+66.862519050 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f23b727c-e88a-42b1-80e8-02384f07c8f6-metrics-certs") pod "router-default-7455857b66-nlzjg" (UID: "f23b727c-e88a-42b1-80e8-02384f07c8f6") : secret "router-metrics-certs-default" not found
Jul 22 03:34:23.345713 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:23.345677    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6ced761e-e20b-4b10-b8b1-fe77cd501818-metrics-tls\") pod \"dns-default-g2cfz\" (UID: \"6ced761e-e20b-4b10-b8b1-fe77cd501818\") " pod="openshift-dns/dns-default-g2cfz"
Jul 22 03:34:23.345872 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:23.345791    2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 03:34:23.345872 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:23.345858    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ced761e-e20b-4b10-b8b1-fe77cd501818-metrics-tls podName:6ced761e-e20b-4b10-b8b1-fe77cd501818 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:39.345843121 +0000 UTC m=+66.964053106 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6ced761e-e20b-4b10-b8b1-fe77cd501818-metrics-tls") pod "dns-default-g2cfz" (UID: "6ced761e-e20b-4b10-b8b1-fe77cd501818") : secret "dns-default-metrics-tls" not found
Jul 22 03:34:25.214874 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:25.214838    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-2p4qp"
Jul 22 03:34:25.215395 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:25.215217    2570 scope.go:117] "RemoveContainer" containerID="dcdb035e2252e40c1adee1b4bfc18699dc3c75fc9b8ba3f1a67e679ce96c1359"
Jul 22 03:34:25.215395 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:25.215366    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-2p4qp_openshift-console-operator(9f0defcc-ccff-43be-8206-54c772885080)\"" pod="openshift-console-operator/console-operator-565b7b569d-2p4qp" podUID="9f0defcc-ccff-43be-8206-54c772885080"
Jul 22 03:34:25.261870 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:25.261832    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b180b221-d3f1-4f70-a258-e680495a3c40-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j8z8l\" (UID: \"b180b221-d3f1-4f70-a258-e680495a3c40\") " pod="openshift-insights/insights-runtime-extractor-j8z8l"
Jul 22 03:34:25.262039 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:25.261988    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b180b221-d3f1-4f70-a258-e680495a3c40-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j8z8l\" (UID: \"b180b221-d3f1-4f70-a258-e680495a3c40\") " pod="openshift-insights/insights-runtime-extractor-j8z8l"
Jul 22 03:34:25.262418 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:25.262167    2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 03:34:25.262418 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:25.262257    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b180b221-d3f1-4f70-a258-e680495a3c40-insights-runtime-extractor-tls podName:b180b221-d3f1-4f70-a258-e680495a3c40 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:33.262235952 +0000 UTC m=+60.880445959 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b180b221-d3f1-4f70-a258-e680495a3c40-insights-runtime-extractor-tls") pod "insights-runtime-extractor-j8z8l" (UID: "b180b221-d3f1-4f70-a258-e680495a3c40") : secret "insights-runtime-extractor-tls" not found
Jul 22 03:34:25.262586 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:25.262431    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b180b221-d3f1-4f70-a258-e680495a3c40-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j8z8l\" (UID: \"b180b221-d3f1-4f70-a258-e680495a3c40\") " pod="openshift-insights/insights-runtime-extractor-j8z8l"
Jul 22 03:34:32.158857 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:32.158830    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ndfqq"
Jul 22 03:34:33.333558 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:33.333510    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b180b221-d3f1-4f70-a258-e680495a3c40-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j8z8l\" (UID: \"b180b221-d3f1-4f70-a258-e680495a3c40\") " pod="openshift-insights/insights-runtime-extractor-j8z8l"
Jul 22 03:34:33.335945 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:33.335913    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b180b221-d3f1-4f70-a258-e680495a3c40-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j8z8l\" (UID: \"b180b221-d3f1-4f70-a258-e680495a3c40\") " pod="openshift-insights/insights-runtime-extractor-j8z8l"
Jul 22 03:34:33.635557 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:33.635467    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-cwd77\""
Jul 22 03:34:33.642859 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:33.642839    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-j8z8l"
Jul 22 03:34:33.781529 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:33.781452    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-j8z8l"]
Jul 22 03:34:33.784025 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:34:33.783995    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb180b221_d3f1_4f70_a258_e680495a3c40.slice/crio-85c9b398a7d44801ee9c33ccc6c48692ffe854ff43598d434e71027bbda8644b WatchSource:0}: Error finding container 85c9b398a7d44801ee9c33ccc6c48692ffe854ff43598d434e71027bbda8644b: Status 404 returned error can't find the container with id 85c9b398a7d44801ee9c33ccc6c48692ffe854ff43598d434e71027bbda8644b
Jul 22 03:34:34.289380 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:34.289338    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-j8z8l" event={"ID":"b180b221-d3f1-4f70-a258-e680495a3c40","Type":"ContainerStarted","Data":"85c9b398a7d44801ee9c33ccc6c48692ffe854ff43598d434e71027bbda8644b"}
Jul 22 03:34:37.471260 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:37.471218    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6adcbd54-1388-4223-8d6d-9423e9ebe09b-metrics-certs\") pod \"network-metrics-daemon-lctg9\" (UID: \"6adcbd54-1388-4223-8d6d-9423e9ebe09b\") " pod="openshift-multus/network-metrics-daemon-lctg9"
Jul 22 03:34:37.474536 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:37.474511    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 22 03:34:37.484484 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:37.484460    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6adcbd54-1388-4223-8d6d-9423e9ebe09b-metrics-certs\") pod \"network-metrics-daemon-lctg9\" (UID: \"6adcbd54-1388-4223-8d6d-9423e9ebe09b\") " pod="openshift-multus/network-metrics-daemon-lctg9"
Jul 22 03:34:37.569078 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:37.569044    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-dzrs8\""
Jul 22 03:34:37.571863 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:37.571837    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc9dm\" (UniqueName: \"kubernetes.io/projected/6963eadb-e7e5-4f25-a6ea-fc0a57c951b8-kube-api-access-hc9dm\") pod \"network-check-target-k2tvl\" (UID: \"6963eadb-e7e5-4f25-a6ea-fc0a57c951b8\") " pod="openshift-network-diagnostics/network-check-target-k2tvl"
Jul 22 03:34:37.571998 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:37.571941    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/16d564e4-f4b5-4293-b5c9-b7bd30239b47-original-pull-secret\") pod \"global-pull-secret-syncer-p867k\" (UID: \"16d564e4-f4b5-4293-b5c9-b7bd30239b47\") " pod="kube-system/global-pull-secret-syncer-p867k"
Jul 22 03:34:37.575626 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:37.575590    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 22 03:34:37.575715 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:37.575658    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 22 03:34:37.576678 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:37.576656    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lctg9"
Jul 22 03:34:37.585409 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:37.585382    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/16d564e4-f4b5-4293-b5c9-b7bd30239b47-original-pull-secret\") pod \"global-pull-secret-syncer-p867k\" (UID: \"16d564e4-f4b5-4293-b5c9-b7bd30239b47\") " pod="kube-system/global-pull-secret-syncer-p867k"
Jul 22 03:34:37.585795 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:37.585767    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 22 03:34:37.596305 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:37.596275    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hc9dm\" (UniqueName: \"kubernetes.io/projected/6963eadb-e7e5-4f25-a6ea-fc0a57c951b8-kube-api-access-hc9dm\") pod \"network-check-target-k2tvl\" (UID: \"6963eadb-e7e5-4f25-a6ea-fc0a57c951b8\") " pod="openshift-network-diagnostics/network-check-target-k2tvl"
Jul 22 03:34:37.846796 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:37.846707    2570 scope.go:117] "RemoveContainer" containerID="dcdb035e2252e40c1adee1b4bfc18699dc3c75fc9b8ba3f1a67e679ce96c1359"
Jul 22 03:34:37.860282 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:37.860258    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-h4vks\""
Jul 22 03:34:37.867656 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:37.867635    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k2tvl"
Jul 22 03:34:37.870945 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:37.870914    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p867k"
Jul 22 03:34:38.422128 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:38.422095    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lctg9"]
Jul 22 03:34:38.711071 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:34:38.711024    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6adcbd54_1388_4223_8d6d_9423e9ebe09b.slice/crio-739d84fc9c0712f1eede38bbc1b84c810878fded350dc6000dc0568ed7ddcbf4 WatchSource:0}: Error finding container 739d84fc9c0712f1eede38bbc1b84c810878fded350dc6000dc0568ed7ddcbf4: Status 404 returned error can't find the container with id 739d84fc9c0712f1eede38bbc1b84c810878fded350dc6000dc0568ed7ddcbf4
Jul 22 03:34:38.854541 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:38.854514    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-k2tvl"]
Jul 22 03:34:38.856142 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:34:38.856116    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6963eadb_e7e5_4f25_a6ea_fc0a57c951b8.slice/crio-b82d3a60fdc4f4b261e7b116e926e7ffacb9f23de434ea049478606f8b3ab43a WatchSource:0}: Error finding container b82d3a60fdc4f4b261e7b116e926e7ffacb9f23de434ea049478606f8b3ab43a: Status 404 returned error can't find the container with id b82d3a60fdc4f4b261e7b116e926e7ffacb9f23de434ea049478606f8b3ab43a
Jul 22 03:34:38.871334 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:38.871308    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-p867k"]
Jul 22 03:34:38.874211 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:34:38.874184    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16d564e4_f4b5_4293_b5c9_b7bd30239b47.slice/crio-49ba795236fc8be3c83f273fd90de5cdd02ef2d16050e53c920dd0299e58057b WatchSource:0}: Error finding container 49ba795236fc8be3c83f273fd90de5cdd02ef2d16050e53c920dd0299e58057b: Status 404 returned error can't find the container with id 49ba795236fc8be3c83f273fd90de5cdd02ef2d16050e53c920dd0299e58057b
Jul 22 03:34:39.187586 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:39.187553    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/367c6e07-f8b3-494b-a12c-3c11e916e61b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-jgckh\" (UID: \"367c6e07-f8b3-494b-a12c-3c11e916e61b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jgckh"
Jul 22 03:34:39.189938 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:39.189909    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/367c6e07-f8b3-494b-a12c-3c11e916e61b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-jgckh\" (UID: \"367c6e07-f8b3-494b-a12c-3c11e916e61b\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jgckh"
Jul 22 03:34:39.288962 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:39.288925    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7bc03f15-dc0d-4a86-adc6-7ec711363d87-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-jds8z\" (UID: \"7bc03f15-dc0d-4a86-adc6-7ec711363d87\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jds8z"
Jul 22 03:34:39.289162 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:39.288972    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f23b727c-e88a-42b1-80e8-02384f07c8f6-metrics-certs\") pod \"router-default-7455857b66-nlzjg\" (UID: \"f23b727c-e88a-42b1-80e8-02384f07c8f6\") " pod="openshift-ingress/router-default-7455857b66-nlzjg"
Jul 22 03:34:39.289162 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:39.289033    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f23b727c-e88a-42b1-80e8-02384f07c8f6-service-ca-bundle\") pod \"router-default-7455857b66-nlzjg\" (UID: \"f23b727c-e88a-42b1-80e8-02384f07c8f6\") " pod="openshift-ingress/router-default-7455857b66-nlzjg"
Jul 22 03:34:39.289162 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:39.289054    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-registry-tls\") pod \"image-registry-7749b46548-cf9gh\" (UID: \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\") " pod="openshift-image-registry/image-registry-7749b46548-cf9gh"
Jul 22 03:34:39.289929 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:39.289900    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f23b727c-e88a-42b1-80e8-02384f07c8f6-service-ca-bundle\") pod \"router-default-7455857b66-nlzjg\" (UID: \"f23b727c-e88a-42b1-80e8-02384f07c8f6\") " pod="openshift-ingress/router-default-7455857b66-nlzjg"
Jul 22 03:34:39.291558 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:39.291537    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-registry-tls\") pod \"image-registry-7749b46548-cf9gh\" (UID: \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\") " pod="openshift-image-registry/image-registry-7749b46548-cf9gh"
Jul 22 03:34:39.292046 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:39.292024    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f23b727c-e88a-42b1-80e8-02384f07c8f6-metrics-certs\") pod \"router-default-7455857b66-nlzjg\" (UID: \"f23b727c-e88a-42b1-80e8-02384f07c8f6\") " pod="openshift-ingress/router-default-7455857b66-nlzjg"
Jul 22 03:34:39.292094 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:39.292031    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7bc03f15-dc0d-4a86-adc6-7ec711363d87-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-jds8z\" (UID: \"7bc03f15-dc0d-4a86-adc6-7ec711363d87\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jds8z"
Jul 22 03:34:39.309539 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:39.309492    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-k2tvl" event={"ID":"6963eadb-e7e5-4f25-a6ea-fc0a57c951b8","Type":"ContainerStarted","Data":"b82d3a60fdc4f4b261e7b116e926e7ffacb9f23de434ea049478606f8b3ab43a"}
Jul 22 03:34:39.310736 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:39.310710    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-p867k" event={"ID":"16d564e4-f4b5-4293-b5c9-b7bd30239b47","Type":"ContainerStarted","Data":"49ba795236fc8be3c83f273fd90de5cdd02ef2d16050e53c920dd0299e58057b"}
Jul 22 03:34:39.311971 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:39.311943    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lctg9" event={"ID":"6adcbd54-1388-4223-8d6d-9423e9ebe09b","Type":"ContainerStarted","Data":"739d84fc9c0712f1eede38bbc1b84c810878fded350dc6000dc0568ed7ddcbf4"}
Jul 22 03:34:39.314169 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:39.314111    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2p4qp_9f0defcc-ccff-43be-8206-54c772885080/console-operator/1.log"
Jul 22 03:34:39.314257 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:39.314210    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-2p4qp" event={"ID":"9f0defcc-ccff-43be-8206-54c772885080","Type":"ContainerStarted","Data":"c25fb237002931c16cfc173035e4b259a27740fec84cd060f086d60d7688ff5a"}
Jul 22 03:34:39.315508 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:39.314788    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-2p4qp"
Jul 22 03:34:39.316146 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:39.316117    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-j8z8l" event={"ID":"b180b221-d3f1-4f70-a258-e680495a3c40","Type":"ContainerStarted","Data":"185647c57a53934831ba6c8275bb896c720f53c3c5473e45e08a3e0b012dcb8b"}
Jul 22 03:34:39.389995 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:39.389948    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6ced761e-e20b-4b10-b8b1-fe77cd501818-metrics-tls\") pod \"dns-default-g2cfz\" (UID: \"6ced761e-e20b-4b10-b8b1-fe77cd501818\") " pod="openshift-dns/dns-default-g2cfz"
Jul 22 03:34:39.392419 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:39.392395    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-tq62s\""
Jul 22 03:34:39.392898 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:39.392879    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6ced761e-e20b-4b10-b8b1-fe77cd501818-metrics-tls\") pod \"dns-default-g2cfz\" (UID: \"6ced761e-e20b-4b10-b8b1-fe77cd501818\") " pod="openshift-dns/dns-default-g2cfz"
Jul 22 03:34:39.400745 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:39.400721    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jgckh"
Jul 22 03:34:39.458666 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:39.458631    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-dfp92\""
Jul 22 03:34:39.467114 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:39.467080    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7749b46548-cf9gh"
Jul 22 03:34:39.489000 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:39.488770    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-jj4kt\""
Jul 22 03:34:39.496961 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:39.496678    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jds8z"
Jul 22 03:34:39.523159 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:39.523067    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-f5f4v\""
Jul 22 03:34:39.532171 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:39.531727    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7455857b66-nlzjg"
Jul 22 03:34:39.556582 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:39.553031    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jn6xf\""
Jul 22 03:34:39.560167 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:39.559933    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-2p4qp"
Jul 22 03:34:39.560654 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:39.560386    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-g2cfz"
Jul 22 03:34:39.592107 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:39.591672    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jgckh"]
Jul 22 03:34:39.609307 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:34:39.609221    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod367c6e07_f8b3_494b_a12c_3c11e916e61b.slice/crio-863832f779d33a99053a009cd6604a6219632214a8edfcad0c2c68bc7616c7a6 WatchSource:0}: Error finding container 863832f779d33a99053a009cd6604a6219632214a8edfcad0c2c68bc7616c7a6: Status 404 returned error can't find the container with id 863832f779d33a99053a009cd6604a6219632214a8edfcad0c2c68bc7616c7a6
Jul 22 03:34:39.699957 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:39.699923    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7749b46548-cf9gh"]
Jul 22 03:34:39.717470 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:34:39.717430    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7492beee_6a0b_4b5a_85c6_ec0064b61bbf.slice/crio-08ecf6a64ed2968ebc68fa35c1d352351edd1101cb0f8893e715e9733f4afcbf WatchSource:0}: Error finding container 08ecf6a64ed2968ebc68fa35c1d352351edd1101cb0f8893e715e9733f4afcbf: Status 404 returned error can't find the container with id 08ecf6a64ed2968ebc68fa35c1d352351edd1101cb0f8893e715e9733f4afcbf
Jul 22 03:34:39.791683 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:39.791562    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jds8z"]
Jul 22 03:34:39.833275 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:39.833167    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7455857b66-nlzjg"]
Jul 22 03:34:39.842276 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:34:39.842246    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf23b727c_e88a_42b1_80e8_02384f07c8f6.slice/crio-b7682d339ff897541800ea6a723a707e25a7f89ac5550f1b76f810eadf63d106 WatchSource:0}: Error finding container b7682d339ff897541800ea6a723a707e25a7f89ac5550f1b76f810eadf63d106: Status 404 returned error can't find the container with id b7682d339ff897541800ea6a723a707e25a7f89ac5550f1b76f810eadf63d106
Jul 22 03:34:39.853484 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:39.853410    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-g2cfz"]
Jul 22 03:34:39.856510 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:34:39.856463    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ced761e_e20b_4b10_b8b1_fe77cd501818.slice/crio-037c677a6780900f3283d81b9ab295bdad5ea137eba69de4df720b72d50980b0 WatchSource:0}: Error finding container 037c677a6780900f3283d81b9ab295bdad5ea137eba69de4df720b72d50980b0: Status 404 returned error can't find the container with id 037c677a6780900f3283d81b9ab295bdad5ea137eba69de4df720b72d50980b0
Jul 22 03:34:40.323726 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:40.323691    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7455857b66-nlzjg" event={"ID":"f23b727c-e88a-42b1-80e8-02384f07c8f6","Type":"ContainerStarted","Data":"4cb140d35e4db3d0002dbc253afb0f4fa08cbb7fab9b55fe16991ed7ab2104e0"}
Jul 22 03:34:40.323939 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:40.323755    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7455857b66-nlzjg" event={"ID":"f23b727c-e88a-42b1-80e8-02384f07c8f6","Type":"ContainerStarted","Data":"b7682d339ff897541800ea6a723a707e25a7f89ac5550f1b76f810eadf63d106"}
Jul 22 03:34:40.331012 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:40.330866    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7749b46548-cf9gh" event={"ID":"7492beee-6a0b-4b5a-85c6-ec0064b61bbf","Type":"ContainerStarted","Data":"7b3c92be71cb1e3214156a24e67780a0c9ce35d926d97d4d4896491ef7dbd224"}
Jul 22 03:34:40.331012 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:40.330924    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7749b46548-cf9gh" event={"ID":"7492beee-6a0b-4b5a-85c6-ec0064b61bbf","Type":"ContainerStarted","Data":"08ecf6a64ed2968ebc68fa35c1d352351edd1101cb0f8893e715e9733f4afcbf"}
Jul 22 03:34:40.331012 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:40.330981    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7749b46548-cf9gh"
Jul 22 03:34:40.333591 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:40.333491    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g2cfz" event={"ID":"6ced761e-e20b-4b10-b8b1-fe77cd501818","Type":"ContainerStarted","Data":"037c677a6780900f3283d81b9ab295bdad5ea137eba69de4df720b72d50980b0"}
Jul 22 03:34:40.335290 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:40.335224    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jgckh" event={"ID":"367c6e07-f8b3-494b-a12c-3c11e916e61b","Type":"ContainerStarted","Data":"863832f779d33a99053a009cd6604a6219632214a8edfcad0c2c68bc7616c7a6"}
Jul 22 03:34:40.337225 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:40.337168    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jds8z" event={"ID":"7bc03f15-dc0d-4a86-adc6-7ec711363d87","Type":"ContainerStarted","Data":"d1a2190550b424bd59d41e4cdd29bfb7ea884925a107aa12ce15c0dab92e9e98"}
Jul 22 03:34:40.378889 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:40.377516    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7455857b66-nlzjg" podStartSLOduration=36.377492276 podStartE2EDuration="36.377492276s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:34:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:34:40.376534236 +0000 UTC m=+67.994744268" watchObservedRunningTime="2026-07-22 03:34:40.377492276 +0000 UTC m=+67.995702301"
Jul 22 03:34:40.521646 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:40.521257    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7749b46548-cf9gh" podStartSLOduration=67.521234746 podStartE2EDuration="1m7.521234746s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:33:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:34:40.414465889 +0000 UTC m=+68.032675929" watchObservedRunningTime="2026-07-22 03:34:40.521234746 +0000 UTC m=+68.139444732"
Jul 22 03:34:40.521646 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:40.521534    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-48tzh"]
Jul 22 03:34:40.528251 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:40.527560    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-48tzh"
Jul 22 03:34:40.531127 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:40.530920    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 22 03:34:40.533471 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:40.533343    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7455857b66-nlzjg"
Jul 22 03:34:40.535623 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:40.535080    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 22 03:34:40.535623 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:40.535338    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-h8fs2\""
Jul 22 03:34:40.536211 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:40.536125    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7455857b66-nlzjg"
Jul 22 03:34:40.554731 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:40.554664    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-48tzh"]
Jul 22 03:34:40.601230 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:40.601201    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdb2q\" (UniqueName: \"kubernetes.io/projected/ada6ca87-7b07-427c-be34-5a8aa1852db5-kube-api-access-fdb2q\") pod \"downloads-5bc67dcd46-48tzh\" (UID: \"ada6ca87-7b07-427c-be34-5a8aa1852db5\") " pod="openshift-console/downloads-5bc67dcd46-48tzh"
Jul 22 03:34:40.703274 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:40.702686    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdb2q\" (UniqueName: \"kubernetes.io/projected/ada6ca87-7b07-427c-be34-5a8aa1852db5-kube-api-access-fdb2q\") pod \"downloads-5bc67dcd46-48tzh\" (UID: \"ada6ca87-7b07-427c-be34-5a8aa1852db5\") " pod="openshift-console/downloads-5bc67dcd46-48tzh"
Jul 22 03:34:40.730644 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:40.729085    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdb2q\" (UniqueName: \"kubernetes.io/projected/ada6ca87-7b07-427c-be34-5a8aa1852db5-kube-api-access-fdb2q\") pod \"downloads-5bc67dcd46-48tzh\" (UID: \"ada6ca87-7b07-427c-be34-5a8aa1852db5\") " pod="openshift-console/downloads-5bc67dcd46-48tzh"
Jul 22 03:34:40.844340 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:40.844026    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-48tzh"
Jul 22 03:34:41.049312 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:41.049235    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-48tzh"]
Jul 22 03:34:41.056472 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:34:41.056436    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podada6ca87_7b07_427c_be34_5a8aa1852db5.slice/crio-d07d0559ef3eac6ba3050ca051ef303481ce5ef814497711ebbdcc43693eafad WatchSource:0}: Error finding container d07d0559ef3eac6ba3050ca051ef303481ce5ef814497711ebbdcc43693eafad: Status 404 returned error can't find the container with id d07d0559ef3eac6ba3050ca051ef303481ce5ef814497711ebbdcc43693eafad
Jul 22 03:34:41.342975 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:41.342933    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-48tzh" event={"ID":"ada6ca87-7b07-427c-be34-5a8aa1852db5","Type":"ContainerStarted","Data":"d07d0559ef3eac6ba3050ca051ef303481ce5ef814497711ebbdcc43693eafad"}
Jul 22 03:34:41.351572 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:41.351535    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lctg9" event={"ID":"6adcbd54-1388-4223-8d6d-9423e9ebe09b","Type":"ContainerStarted","Data":"6b4518438e62f4bfbd33ca4147421c293918f172d5068580f9438bba63352dd8"}
Jul 22 03:34:41.351757 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:41.351599    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lctg9" event={"ID":"6adcbd54-1388-4223-8d6d-9423e9ebe09b","Type":"ContainerStarted","Data":"4ece8ee4a176516b8b0daac04c689f73073374c893cb04e50355c67236dd7c4f"}
Jul 22 03:34:41.352350 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:41.352194    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7455857b66-nlzjg"
Jul 22 03:34:41.353841 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:41.353801    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7455857b66-nlzjg"
Jul 22 03:34:41.370524 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:41.369686    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-lctg9" podStartSLOduration=66.610479691 podStartE2EDuration="1m8.369668675s" totalImagesPullingTime="1.759188984s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:33:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:34:41.368369498 +0000 UTC m=+68.986579539" watchObservedRunningTime="2026-07-22 03:34:41.369668675 +0000 UTC m=+68.987878683"
Jul 22 03:34:50.417190 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:50.417149    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-j8z8l" event={"ID":"b180b221-d3f1-4f70-a258-e680495a3c40","Type":"ContainerStarted","Data":"4111fa25174e0898c7a8c50b94886701d12d0131ad646ecc01bd636ecd2926a7"}
Jul 22 03:34:50.419599 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:50.419511    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g2cfz" event={"ID":"6ced761e-e20b-4b10-b8b1-fe77cd501818","Type":"ContainerStarted","Data":"b637ab6093494dcbe0ca32106742ff6cadbcca8bfc11e6d04a148a3a66aed366"}
Jul 22 03:34:50.422181 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:50.421421    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jgckh" event={"ID":"367c6e07-f8b3-494b-a12c-3c11e916e61b","Type":"ContainerStarted","Data":"4bab954c17b005538ef471ed74974afc2cb04cd30f06e04a712031f5154174f8"}
Jul 22 03:34:50.427789 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:50.427532    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-k2tvl" event={"ID":"6963eadb-e7e5-4f25-a6ea-fc0a57c951b8","Type":"ContainerStarted","Data":"eaf47c6eddda48dc1176a5c4d6418de301494e2270b38949862e1fd67f9cc089"}
Jul 22 03:34:50.428114 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:50.428089    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-k2tvl"
Jul 22 03:34:50.433237 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:50.433209    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-p867k" event={"ID":"16d564e4-f4b5-4293-b5c9-b7bd30239b47","Type":"ContainerStarted","Data":"098b5eba12340f42db0babd8a293252419873602811c098191f6346aa6eb989f"}
Jul 22 03:34:50.436026 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:50.435998    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jds8z" event={"ID":"7bc03f15-dc0d-4a86-adc6-7ec711363d87","Type":"ContainerStarted","Data":"0dca68ea5a35b12cba8543c9a618aa0faabceedb4a38757574479883bed7ef09"}
Jul 22 03:34:50.441220 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:50.441141    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jgckh" podStartSLOduration=35.965695627 podStartE2EDuration="46.441120816s" totalImagesPullingTime="10.475425189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:34:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:34:50.439998887 +0000 UTC m=+78.058208895" watchObservedRunningTime="2026-07-22 03:34:50.441120816 +0000 UTC m=+78.059330823"
Jul 22 03:34:50.456826 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:50.456204    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-k2tvl" podStartSLOduration=66.133571423 podStartE2EDuration="1m17.456181324s" totalImagesPullingTime="11.322609901s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:33:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:34:50.454878469 +0000 UTC m=+78.073088482" watchObservedRunningTime="2026-07-22 03:34:50.456181324 +0000 UTC m=+78.074391332"
Jul 22 03:34:50.472404 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:50.472351    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-p867k" podStartSLOduration=66.166503188 podStartE2EDuration="1m17.472331134s" totalImagesPullingTime="11.305827946s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:33:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:34:50.472221089 +0000 UTC m=+78.090431115" watchObservedRunningTime="2026-07-22 03:34:50.472331134 +0000 UTC m=+78.090541139"
Jul 22 03:34:51.444079 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:51.444038    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g2cfz" event={"ID":"6ced761e-e20b-4b10-b8b1-fe77cd501818","Type":"ContainerStarted","Data":"51d66aa40ee6677a8bb28dc1dbcc5112f0594f587b6b2a9ca7a1695fd76f4bab"}
Jul 22 03:34:51.445434 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:51.444124    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-g2cfz"
Jul 22 03:34:51.446585 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:51.446557    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jds8z" event={"ID":"7bc03f15-dc0d-4a86-adc6-7ec711363d87","Type":"ContainerStarted","Data":"5797a0462136b8b143c683df1d875ed3cf12e683408b9adfcf960af86b4b77bb"}
Jul 22 03:34:51.463226 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:51.463174    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-g2cfz" podStartSLOduration=34.144159792 podStartE2EDuration="44.46315711s" totalImagesPullingTime="10.318997318s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:34:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:34:51.461549757 +0000 UTC m=+79.079759785" watchObservedRunningTime="2026-07-22 03:34:51.46315711 +0000 UTC m=+79.081367111"
Jul 22 03:34:51.479081 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:51.479010    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jds8z" podStartSLOduration=37.182363604 podStartE2EDuration="47.478993615s" totalImagesPullingTime="10.296630011s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:34:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:34:51.478566645 +0000 UTC m=+79.096776664" watchObservedRunningTime="2026-07-22 03:34:51.478993615 +0000 UTC m=+79.097203622"
Jul 22 03:34:51.678832 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:51.678791    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-68d7c7b64c-rlvsn"]
Jul 22 03:34:51.681104 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:51.681077    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68d7c7b64c-rlvsn"
Jul 22 03:34:51.685093 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:51.684093    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 22 03:34:51.685093 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:51.684157    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 22 03:34:51.685093 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:51.684381    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 22 03:34:51.685093 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:51.684578    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 22 03:34:51.685093 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:51.684697    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 22 03:34:51.685549 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:51.685525    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-nq27z\""
Jul 22 03:34:51.690386 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:51.690353    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 22 03:34:51.696301 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:51.696236    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68d7c7b64c-rlvsn"]
Jul 22 03:34:51.797150 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:51.797106    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/da13f849-f973-4a92-85af-808fe7382212-console-config\") pod \"console-68d7c7b64c-rlvsn\" (UID: \"da13f849-f973-4a92-85af-808fe7382212\") " pod="openshift-console/console-68d7c7b64c-rlvsn"
Jul 22 03:34:51.797325 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:51.797162    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/da13f849-f973-4a92-85af-808fe7382212-console-oauth-config\") pod \"console-68d7c7b64c-rlvsn\" (UID: \"da13f849-f973-4a92-85af-808fe7382212\") " pod="openshift-console/console-68d7c7b64c-rlvsn"
Jul 22 03:34:51.797325 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:51.797198    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/da13f849-f973-4a92-85af-808fe7382212-console-serving-cert\") pod \"console-68d7c7b64c-rlvsn\" (UID: \"da13f849-f973-4a92-85af-808fe7382212\") " pod="openshift-console/console-68d7c7b64c-rlvsn"
Jul 22 03:34:51.797325 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:51.797232    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da13f849-f973-4a92-85af-808fe7382212-trusted-ca-bundle\") pod \"console-68d7c7b64c-rlvsn\" (UID: \"da13f849-f973-4a92-85af-808fe7382212\") " pod="openshift-console/console-68d7c7b64c-rlvsn"
Jul 22 03:34:51.797325 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:51.797260    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtnjv\" (UniqueName: \"kubernetes.io/projected/da13f849-f973-4a92-85af-808fe7382212-kube-api-access-wtnjv\") pod \"console-68d7c7b64c-rlvsn\" (UID: \"da13f849-f973-4a92-85af-808fe7382212\") " pod="openshift-console/console-68d7c7b64c-rlvsn"
Jul 22 03:34:51.797521 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:51.797365    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/da13f849-f973-4a92-85af-808fe7382212-service-ca\") pod \"console-68d7c7b64c-rlvsn\" (UID: \"da13f849-f973-4a92-85af-808fe7382212\") " pod="openshift-console/console-68d7c7b64c-rlvsn"
Jul 22 03:34:51.797521 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:51.797409    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/da13f849-f973-4a92-85af-808fe7382212-oauth-serving-cert\") pod \"console-68d7c7b64c-rlvsn\" (UID: \"da13f849-f973-4a92-85af-808fe7382212\") " pod="openshift-console/console-68d7c7b64c-rlvsn"
Jul 22 03:34:51.898267 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:51.898229    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/da13f849-f973-4a92-85af-808fe7382212-console-config\") pod \"console-68d7c7b64c-rlvsn\" (UID: \"da13f849-f973-4a92-85af-808fe7382212\") " pod="openshift-console/console-68d7c7b64c-rlvsn"
Jul 22 03:34:51.898267 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:51.898272    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/da13f849-f973-4a92-85af-808fe7382212-console-oauth-config\") pod \"console-68d7c7b64c-rlvsn\" (UID: \"da13f849-f973-4a92-85af-808fe7382212\") " pod="openshift-console/console-68d7c7b64c-rlvsn"
Jul 22 03:34:51.898512 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:51.898299    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/da13f849-f973-4a92-85af-808fe7382212-console-serving-cert\") pod \"console-68d7c7b64c-rlvsn\" (UID: \"da13f849-f973-4a92-85af-808fe7382212\") " pod="openshift-console/console-68d7c7b64c-rlvsn"
Jul 22 03:34:51.898512 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:51.898331    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da13f849-f973-4a92-85af-808fe7382212-trusted-ca-bundle\") pod \"console-68d7c7b64c-rlvsn\" (UID: \"da13f849-f973-4a92-85af-808fe7382212\") " pod="openshift-console/console-68d7c7b64c-rlvsn"
Jul 22 03:34:51.898512 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:51.898381    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtnjv\" (UniqueName: \"kubernetes.io/projected/da13f849-f973-4a92-85af-808fe7382212-kube-api-access-wtnjv\") pod \"console-68d7c7b64c-rlvsn\" (UID: \"da13f849-f973-4a92-85af-808fe7382212\") " pod="openshift-console/console-68d7c7b64c-rlvsn"
Jul 22 03:34:51.898512 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:51.898421    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/da13f849-f973-4a92-85af-808fe7382212-service-ca\") pod \"console-68d7c7b64c-rlvsn\" (UID: \"da13f849-f973-4a92-85af-808fe7382212\") " pod="openshift-console/console-68d7c7b64c-rlvsn"
Jul 22 03:34:51.898512 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:51.898450    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/da13f849-f973-4a92-85af-808fe7382212-oauth-serving-cert\") pod \"console-68d7c7b64c-rlvsn\" (UID: \"da13f849-f973-4a92-85af-808fe7382212\") " pod="openshift-console/console-68d7c7b64c-rlvsn"
Jul 22 03:34:51.899205 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:51.899182    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/da13f849-f973-4a92-85af-808fe7382212-oauth-serving-cert\") pod \"console-68d7c7b64c-rlvsn\" (UID: \"da13f849-f973-4a92-85af-808fe7382212\") " pod="openshift-console/console-68d7c7b64c-rlvsn"
Jul 22 03:34:51.899835 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:51.899815    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/da13f849-f973-4a92-85af-808fe7382212-console-config\") pod \"console-68d7c7b64c-rlvsn\" (UID: \"da13f849-f973-4a92-85af-808fe7382212\") " pod="openshift-console/console-68d7c7b64c-rlvsn"
Jul 22 03:34:51.900880 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:51.900814    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/da13f849-f973-4a92-85af-808fe7382212-service-ca\") pod \"console-68d7c7b64c-rlvsn\" (UID: \"da13f849-f973-4a92-85af-808fe7382212\") " pod="openshift-console/console-68d7c7b64c-rlvsn"
Jul 22 03:34:51.901249 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:51.901222    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da13f849-f973-4a92-85af-808fe7382212-trusted-ca-bundle\") pod \"console-68d7c7b64c-rlvsn\" (UID: \"da13f849-f973-4a92-85af-808fe7382212\") " pod="openshift-console/console-68d7c7b64c-rlvsn"
Jul 22 03:34:51.903033 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:51.903010    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/da13f849-f973-4a92-85af-808fe7382212-console-serving-cert\") pod \"console-68d7c7b64c-rlvsn\" (UID: \"da13f849-f973-4a92-85af-808fe7382212\") " pod="openshift-console/console-68d7c7b64c-rlvsn"
Jul 22 03:34:51.906309 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:51.906263    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/da13f849-f973-4a92-85af-808fe7382212-console-oauth-config\") pod \"console-68d7c7b64c-rlvsn\" (UID: \"da13f849-f973-4a92-85af-808fe7382212\") " pod="openshift-console/console-68d7c7b64c-rlvsn"
Jul 22 03:34:51.908954 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:51.908930    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtnjv\" (UniqueName: \"kubernetes.io/projected/da13f849-f973-4a92-85af-808fe7382212-kube-api-access-wtnjv\") pod \"console-68d7c7b64c-rlvsn\" (UID: \"da13f849-f973-4a92-85af-808fe7382212\") " pod="openshift-console/console-68d7c7b64c-rlvsn"
Jul 22 03:34:51.994842 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:51.994753    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68d7c7b64c-rlvsn"
Jul 22 03:34:52.173145 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:52.173066    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68d7c7b64c-rlvsn"]
Jul 22 03:34:52.180096 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:34:52.180060    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda13f849_f973_4a92_85af_808fe7382212.slice/crio-6a2204b89aee2635fcb49ec7a5f0b8b9325288fd49bd294eeb606d0e4a1746c4 WatchSource:0}: Error finding container 6a2204b89aee2635fcb49ec7a5f0b8b9325288fd49bd294eeb606d0e4a1746c4: Status 404 returned error can't find the container with id 6a2204b89aee2635fcb49ec7a5f0b8b9325288fd49bd294eeb606d0e4a1746c4
Jul 22 03:34:52.454004 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:52.453938    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68d7c7b64c-rlvsn" event={"ID":"da13f849-f973-4a92-85af-808fe7382212","Type":"ContainerStarted","Data":"6a2204b89aee2635fcb49ec7a5f0b8b9325288fd49bd294eeb606d0e4a1746c4"}
Jul 22 03:34:53.904388 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:53.904259    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-pjlv2"]
Jul 22 03:34:53.919683 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:53.918872    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-pjlv2"
Jul 22 03:34:53.919852 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:53.919824    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-pjlv2"]
Jul 22 03:34:53.923783 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:53.923597    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 22 03:34:53.923783 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:53.923603    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-b7mvt\""
Jul 22 03:34:53.923783 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:53.923709    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 22 03:34:53.923783 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:53.923721    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 22 03:34:54.018833 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:54.018789    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c4a45ddb-ff3b-48ff-a3df-0966e2b5f37b-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-pjlv2\" (UID: \"c4a45ddb-ff3b-48ff-a3df-0966e2b5f37b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-pjlv2"
Jul 22 03:34:54.018989 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:54.018850    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lq89b\" (UniqueName: \"kubernetes.io/projected/c4a45ddb-ff3b-48ff-a3df-0966e2b5f37b-kube-api-access-lq89b\") pod \"prometheus-operator-5945b5cfd-pjlv2\" (UID: \"c4a45ddb-ff3b-48ff-a3df-0966e2b5f37b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-pjlv2"
Jul 22 03:34:54.018989 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:54.018936    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c4a45ddb-ff3b-48ff-a3df-0966e2b5f37b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-pjlv2\" (UID: \"c4a45ddb-ff3b-48ff-a3df-0966e2b5f37b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-pjlv2"
Jul 22 03:34:54.018989 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:54.018977    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c4a45ddb-ff3b-48ff-a3df-0966e2b5f37b-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-pjlv2\" (UID: \"c4a45ddb-ff3b-48ff-a3df-0966e2b5f37b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-pjlv2"
Jul 22 03:34:54.120100 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:54.120059    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c4a45ddb-ff3b-48ff-a3df-0966e2b5f37b-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-pjlv2\" (UID: \"c4a45ddb-ff3b-48ff-a3df-0966e2b5f37b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-pjlv2"
Jul 22 03:34:54.120273 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:54.120135    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lq89b\" (UniqueName: \"kubernetes.io/projected/c4a45ddb-ff3b-48ff-a3df-0966e2b5f37b-kube-api-access-lq89b\") pod \"prometheus-operator-5945b5cfd-pjlv2\" (UID: \"c4a45ddb-ff3b-48ff-a3df-0966e2b5f37b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-pjlv2"
Jul 22 03:34:54.120273 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:54.120184    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c4a45ddb-ff3b-48ff-a3df-0966e2b5f37b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-pjlv2\" (UID: \"c4a45ddb-ff3b-48ff-a3df-0966e2b5f37b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-pjlv2"
Jul 22 03:34:54.120273 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:54.120223    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c4a45ddb-ff3b-48ff-a3df-0966e2b5f37b-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-pjlv2\" (UID: \"c4a45ddb-ff3b-48ff-a3df-0966e2b5f37b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-pjlv2"
Jul 22 03:34:54.120450 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:54.120347    2570 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 22 03:34:54.120450 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:34:54.120415    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4a45ddb-ff3b-48ff-a3df-0966e2b5f37b-prometheus-operator-tls podName:c4a45ddb-ff3b-48ff-a3df-0966e2b5f37b nodeName:}" failed. No retries permitted until 2026-07-22 03:34:54.620393698 +0000 UTC m=+82.238603685 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/c4a45ddb-ff3b-48ff-a3df-0966e2b5f37b-prometheus-operator-tls") pod "prometheus-operator-5945b5cfd-pjlv2" (UID: "c4a45ddb-ff3b-48ff-a3df-0966e2b5f37b") : secret "prometheus-operator-tls" not found
Jul 22 03:34:54.120943 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:54.120914    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c4a45ddb-ff3b-48ff-a3df-0966e2b5f37b-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-pjlv2\" (UID: \"c4a45ddb-ff3b-48ff-a3df-0966e2b5f37b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-pjlv2"
Jul 22 03:34:54.123489 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:54.123467    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c4a45ddb-ff3b-48ff-a3df-0966e2b5f37b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-pjlv2\" (UID: \"c4a45ddb-ff3b-48ff-a3df-0966e2b5f37b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-pjlv2"
Jul 22 03:34:54.128659 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:54.128628    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lq89b\" (UniqueName: \"kubernetes.io/projected/c4a45ddb-ff3b-48ff-a3df-0966e2b5f37b-kube-api-access-lq89b\") pod \"prometheus-operator-5945b5cfd-pjlv2\" (UID: \"c4a45ddb-ff3b-48ff-a3df-0966e2b5f37b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-pjlv2"
Jul 22 03:34:54.625081 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:54.625049    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c4a45ddb-ff3b-48ff-a3df-0966e2b5f37b-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-pjlv2\" (UID: \"c4a45ddb-ff3b-48ff-a3df-0966e2b5f37b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-pjlv2"
Jul 22 03:34:54.628274 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:54.628241    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c4a45ddb-ff3b-48ff-a3df-0966e2b5f37b-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-pjlv2\" (UID: \"c4a45ddb-ff3b-48ff-a3df-0966e2b5f37b\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-pjlv2"
Jul 22 03:34:54.830788 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:34:54.830489    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-pjlv2"
Jul 22 03:35:01.356926 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:01.356895    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7749b46548-cf9gh"
Jul 22 03:35:01.456499 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:01.456473    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-g2cfz"
Jul 22 03:35:02.735979 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:02.735941    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7749b46548-cf9gh"]
Jul 22 03:35:03.454462 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:03.454428    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-pjlv2"]
Jul 22 03:35:03.501670 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:03.501566    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-48tzh" event={"ID":"ada6ca87-7b07-427c-be34-5a8aa1852db5","Type":"ContainerStarted","Data":"ce312b0745f3f609c458705b830ae9a369e54ac27cfc1a08dda62910e00aa70c"}
Jul 22 03:35:03.501670 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:03.501668    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-48tzh"
Jul 22 03:35:03.503560 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:03.503535    2570 patch_prober.go:28] interesting pod/downloads-5bc67dcd46-48tzh container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.18:8080/\": dial tcp 10.132.0.18:8080: connect: connection refused" start-of-body=
Jul 22 03:35:03.503716 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:03.503588    2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-5bc67dcd46-48tzh" podUID="ada6ca87-7b07-427c-be34-5a8aa1852db5" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.18:8080/\": dial tcp 10.132.0.18:8080: connect: connection refused"
Jul 22 03:35:03.504225 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:03.504197    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-j8z8l" event={"ID":"b180b221-d3f1-4f70-a258-e680495a3c40","Type":"ContainerStarted","Data":"02503871d661d0cea26901b8234f58141b26638c20df57ad1e72e236cd27ea58"}
Jul 22 03:35:03.519429 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:03.519375    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-48tzh" podStartSLOduration=1.217840179 podStartE2EDuration="23.519358524s" totalImagesPullingTime="22.301518345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:34:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:35:03.517903406 +0000 UTC m=+91.136113413" watchObservedRunningTime="2026-07-22 03:35:03.519358524 +0000 UTC m=+91.137568581"
Jul 22 03:35:03.539177 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:03.539125    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-j8z8l" podStartSLOduration=17.251733488 podStartE2EDuration="46.539100232s" totalImagesPullingTime="29.287366744s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:34:17 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:35:03.537131331 +0000 UTC m=+91.155341362" watchObservedRunningTime="2026-07-22 03:35:03.539100232 +0000 UTC m=+91.157310239"
Jul 22 03:35:03.543722 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:35:03.543695    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4a45ddb_ff3b_48ff_a3df_0966e2b5f37b.slice/crio-6c113c2e7abf8d8c0a5ab80e8826e82dbeefed0141f0e932c4abb363072918f8 WatchSource:0}: Error finding container 6c113c2e7abf8d8c0a5ab80e8826e82dbeefed0141f0e932c4abb363072918f8: Status 404 returned error can't find the container with id 6c113c2e7abf8d8c0a5ab80e8826e82dbeefed0141f0e932c4abb363072918f8
Jul 22 03:35:04.512643 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:04.511967    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68d7c7b64c-rlvsn" event={"ID":"da13f849-f973-4a92-85af-808fe7382212","Type":"ContainerStarted","Data":"1e683da210b7eaa5bbac429835e1c7815f28cc9ef406c458e5572fdc2b13c172"}
Jul 22 03:35:04.515012 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:04.514902    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-pjlv2" event={"ID":"c4a45ddb-ff3b-48ff-a3df-0966e2b5f37b","Type":"ContainerStarted","Data":"6c113c2e7abf8d8c0a5ab80e8826e82dbeefed0141f0e932c4abb363072918f8"}
Jul 22 03:35:04.528422 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:04.528387    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-48tzh"
Jul 22 03:35:04.561690 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:04.561605    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-68d7c7b64c-rlvsn" podStartSLOduration=2.192597751 podStartE2EDuration="13.561583083s" totalImagesPullingTime="11.368985332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:34:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:35:04.533459025 +0000 UTC m=+92.151669033" watchObservedRunningTime="2026-07-22 03:35:04.561583083 +0000 UTC m=+92.179793090"
Jul 22 03:35:06.526867 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:06.526822    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-pjlv2" event={"ID":"c4a45ddb-ff3b-48ff-a3df-0966e2b5f37b","Type":"ContainerStarted","Data":"501ba87591abedd5f316d9e9470ee168ab1ac9746af6a9a69d978e810e0c5e5f"}
Jul 22 03:35:06.527320 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:06.526940    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-pjlv2" event={"ID":"c4a45ddb-ff3b-48ff-a3df-0966e2b5f37b","Type":"ContainerStarted","Data":"e4f44c61681ecfc784a77088570f4703d14e3f46bc23005a1e8dd6d80ceb3aec"}
Jul 22 03:35:06.547379 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:06.547328    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-pjlv2" podStartSLOduration=11.43335236 podStartE2EDuration="13.547309576s" totalImagesPullingTime="2.113957216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:34:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:35:06.544803491 +0000 UTC m=+94.163013502" watchObservedRunningTime="2026-07-22 03:35:06.547309576 +0000 UTC m=+94.165519582"
Jul 22 03:35:08.292765 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:08.292728    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mlkmd"]
Jul 22 03:35:08.333304 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:08.333270    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mlkmd"
Jul 22 03:35:08.336245 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:08.336219    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 22 03:35:08.336245 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:08.336242    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 22 03:35:08.336440 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:08.336302    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-hd5r7\""
Jul 22 03:35:08.336527 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:08.336230    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 22 03:35:08.355915 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:08.355883    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8e81bffd-5ccd-45f5-b072-ed0b951261ec-root\") pod \"node-exporter-mlkmd\" (UID: \"8e81bffd-5ccd-45f5-b072-ed0b951261ec\") " pod="openshift-monitoring/node-exporter-mlkmd"
Jul 22 03:35:08.356076 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:08.355936    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8e81bffd-5ccd-45f5-b072-ed0b951261ec-node-exporter-tls\") pod \"node-exporter-mlkmd\" (UID: \"8e81bffd-5ccd-45f5-b072-ed0b951261ec\") " pod="openshift-monitoring/node-exporter-mlkmd"
Jul 22 03:35:08.356076 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:08.356001    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e81bffd-5ccd-45f5-b072-ed0b951261ec-metrics-client-ca\") pod \"node-exporter-mlkmd\" (UID: \"8e81bffd-5ccd-45f5-b072-ed0b951261ec\") " pod="openshift-monitoring/node-exporter-mlkmd"
Jul 22 03:35:08.356076 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:08.356052    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8e81bffd-5ccd-45f5-b072-ed0b951261ec-node-exporter-accelerators-collector-config\") pod \"node-exporter-mlkmd\" (UID: \"8e81bffd-5ccd-45f5-b072-ed0b951261ec\") " pod="openshift-monitoring/node-exporter-mlkmd"
Jul 22 03:35:08.356238 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:08.356103    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8e81bffd-5ccd-45f5-b072-ed0b951261ec-node-exporter-textfile\") pod \"node-exporter-mlkmd\" (UID: \"8e81bffd-5ccd-45f5-b072-ed0b951261ec\") " pod="openshift-monitoring/node-exporter-mlkmd"
Jul 22 03:35:08.356238 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:08.356122    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbh6n\" (UniqueName: \"kubernetes.io/projected/8e81bffd-5ccd-45f5-b072-ed0b951261ec-kube-api-access-pbh6n\") pod \"node-exporter-mlkmd\" (UID: \"8e81bffd-5ccd-45f5-b072-ed0b951261ec\") " pod="openshift-monitoring/node-exporter-mlkmd"
Jul 22 03:35:08.356238 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:08.356157    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8e81bffd-5ccd-45f5-b072-ed0b951261ec-node-exporter-wtmp\") pod \"node-exporter-mlkmd\" (UID: \"8e81bffd-5ccd-45f5-b072-ed0b951261ec\") " pod="openshift-monitoring/node-exporter-mlkmd"
Jul 22 03:35:08.356238 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:08.356185    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8e81bffd-5ccd-45f5-b072-ed0b951261ec-sys\") pod \"node-exporter-mlkmd\" (UID: \"8e81bffd-5ccd-45f5-b072-ed0b951261ec\") " pod="openshift-monitoring/node-exporter-mlkmd"
Jul 22 03:35:08.356238 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:08.356209    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8e81bffd-5ccd-45f5-b072-ed0b951261ec-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mlkmd\" (UID: \"8e81bffd-5ccd-45f5-b072-ed0b951261ec\") " pod="openshift-monitoring/node-exporter-mlkmd"
Jul 22 03:35:08.457019 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:08.456985    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8e81bffd-5ccd-45f5-b072-ed0b951261ec-node-exporter-accelerators-collector-config\") pod \"node-exporter-mlkmd\" (UID: \"8e81bffd-5ccd-45f5-b072-ed0b951261ec\") " pod="openshift-monitoring/node-exporter-mlkmd"
Jul 22 03:35:08.457195 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:08.457058    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8e81bffd-5ccd-45f5-b072-ed0b951261ec-node-exporter-textfile\") pod \"node-exporter-mlkmd\" (UID: \"8e81bffd-5ccd-45f5-b072-ed0b951261ec\") " pod="openshift-monitoring/node-exporter-mlkmd"
Jul 22 03:35:08.457195 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:08.457089    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbh6n\" (UniqueName: \"kubernetes.io/projected/8e81bffd-5ccd-45f5-b072-ed0b951261ec-kube-api-access-pbh6n\") pod \"node-exporter-mlkmd\" (UID: \"8e81bffd-5ccd-45f5-b072-ed0b951261ec\") " pod="openshift-monitoring/node-exporter-mlkmd"
Jul 22 03:35:08.457195 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:08.457120    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8e81bffd-5ccd-45f5-b072-ed0b951261ec-node-exporter-wtmp\") pod \"node-exporter-mlkmd\" (UID: \"8e81bffd-5ccd-45f5-b072-ed0b951261ec\") " pod="openshift-monitoring/node-exporter-mlkmd"
Jul 22 03:35:08.457195 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:08.457161    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8e81bffd-5ccd-45f5-b072-ed0b951261ec-sys\") pod \"node-exporter-mlkmd\" (UID: \"8e81bffd-5ccd-45f5-b072-ed0b951261ec\") " pod="openshift-monitoring/node-exporter-mlkmd"
Jul 22 03:35:08.457195 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:08.457190    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8e81bffd-5ccd-45f5-b072-ed0b951261ec-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mlkmd\" (UID: \"8e81bffd-5ccd-45f5-b072-ed0b951261ec\") " pod="openshift-monitoring/node-exporter-mlkmd"
Jul 22 03:35:08.457463 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:08.457234    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8e81bffd-5ccd-45f5-b072-ed0b951261ec-root\") pod \"node-exporter-mlkmd\" (UID: \"8e81bffd-5ccd-45f5-b072-ed0b951261ec\") " pod="openshift-monitoring/node-exporter-mlkmd"
Jul 22 03:35:08.457463 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:08.457266    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8e81bffd-5ccd-45f5-b072-ed0b951261ec-node-exporter-tls\") pod \"node-exporter-mlkmd\" (UID: \"8e81bffd-5ccd-45f5-b072-ed0b951261ec\") " pod="openshift-monitoring/node-exporter-mlkmd"
Jul 22 03:35:08.457463 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:08.457291    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e81bffd-5ccd-45f5-b072-ed0b951261ec-metrics-client-ca\") pod \"node-exporter-mlkmd\" (UID: \"8e81bffd-5ccd-45f5-b072-ed0b951261ec\") " pod="openshift-monitoring/node-exporter-mlkmd"
Jul 22 03:35:08.457463 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:08.457426    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8e81bffd-5ccd-45f5-b072-ed0b951261ec-node-exporter-textfile\") pod \"node-exporter-mlkmd\" (UID: \"8e81bffd-5ccd-45f5-b072-ed0b951261ec\") " pod="openshift-monitoring/node-exporter-mlkmd"
Jul 22 03:35:08.457680 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:08.457497    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8e81bffd-5ccd-45f5-b072-ed0b951261ec-sys\") pod \"node-exporter-mlkmd\" (UID: \"8e81bffd-5ccd-45f5-b072-ed0b951261ec\") " pod="openshift-monitoring/node-exporter-mlkmd"
Jul 22 03:35:08.457680 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:08.457659    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8e81bffd-5ccd-45f5-b072-ed0b951261ec-node-exporter-accelerators-collector-config\") pod \"node-exporter-mlkmd\" (UID: \"8e81bffd-5ccd-45f5-b072-ed0b951261ec\") " pod="openshift-monitoring/node-exporter-mlkmd"
Jul 22 03:35:08.457770 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:08.457713    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8e81bffd-5ccd-45f5-b072-ed0b951261ec-root\") pod \"node-exporter-mlkmd\" (UID: \"8e81bffd-5ccd-45f5-b072-ed0b951261ec\") " pod="openshift-monitoring/node-exporter-mlkmd"
Jul 22 03:35:08.457925 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:08.457902    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e81bffd-5ccd-45f5-b072-ed0b951261ec-metrics-client-ca\") pod \"node-exporter-mlkmd\" (UID: \"8e81bffd-5ccd-45f5-b072-ed0b951261ec\") " pod="openshift-monitoring/node-exporter-mlkmd"
Jul 22 03:35:08.457982 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:08.457911    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8e81bffd-5ccd-45f5-b072-ed0b951261ec-node-exporter-wtmp\") pod \"node-exporter-mlkmd\" (UID: \"8e81bffd-5ccd-45f5-b072-ed0b951261ec\") " pod="openshift-monitoring/node-exporter-mlkmd"
Jul 22 03:35:08.458039 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:35:08.457986    2570 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 22 03:35:08.458084 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:35:08.458043    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e81bffd-5ccd-45f5-b072-ed0b951261ec-node-exporter-tls podName:8e81bffd-5ccd-45f5-b072-ed0b951261ec nodeName:}" failed. No retries permitted until 2026-07-22 03:35:08.958025629 +0000 UTC m=+96.576235631 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/8e81bffd-5ccd-45f5-b072-ed0b951261ec-node-exporter-tls") pod "node-exporter-mlkmd" (UID: "8e81bffd-5ccd-45f5-b072-ed0b951261ec") : secret "node-exporter-tls" not found
Jul 22 03:35:08.460535 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:08.460490    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8e81bffd-5ccd-45f5-b072-ed0b951261ec-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mlkmd\" (UID: \"8e81bffd-5ccd-45f5-b072-ed0b951261ec\") " pod="openshift-monitoring/node-exporter-mlkmd"
Jul 22 03:35:08.477713 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:08.471728    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbh6n\" (UniqueName: \"kubernetes.io/projected/8e81bffd-5ccd-45f5-b072-ed0b951261ec-kube-api-access-pbh6n\") pod \"node-exporter-mlkmd\" (UID: \"8e81bffd-5ccd-45f5-b072-ed0b951261ec\") " pod="openshift-monitoring/node-exporter-mlkmd"
Jul 22 03:35:08.961089 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:08.961050    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8e81bffd-5ccd-45f5-b072-ed0b951261ec-node-exporter-tls\") pod \"node-exporter-mlkmd\" (UID: \"8e81bffd-5ccd-45f5-b072-ed0b951261ec\") " pod="openshift-monitoring/node-exporter-mlkmd"
Jul 22 03:35:08.963890 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:08.963861    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8e81bffd-5ccd-45f5-b072-ed0b951261ec-node-exporter-tls\") pod \"node-exporter-mlkmd\" (UID: \"8e81bffd-5ccd-45f5-b072-ed0b951261ec\") " pod="openshift-monitoring/node-exporter-mlkmd"
Jul 22 03:35:09.245038 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:09.244951    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mlkmd"
Jul 22 03:35:09.299808 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:35:09.299770    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e81bffd_5ccd_45f5_b072_ed0b951261ec.slice/crio-a65a5829e89d5b7525b456d09a3631215757d1a815c57db1794d862fb2b8ad0e WatchSource:0}: Error finding container a65a5829e89d5b7525b456d09a3631215757d1a815c57db1794d862fb2b8ad0e: Status 404 returned error can't find the container with id a65a5829e89d5b7525b456d09a3631215757d1a815c57db1794d862fb2b8ad0e
Jul 22 03:35:09.541771 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:09.541695    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mlkmd" event={"ID":"8e81bffd-5ccd-45f5-b072-ed0b951261ec","Type":"ContainerStarted","Data":"a65a5829e89d5b7525b456d09a3631215757d1a815c57db1794d862fb2b8ad0e"}
Jul 22 03:35:11.551722 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:11.551685    2570 generic.go:356] "Generic (PLEG): container finished" podID="8e81bffd-5ccd-45f5-b072-ed0b951261ec" containerID="08a1eb35be892035bd41af7bb619056595ff17bdf4d278170177f7ac2fdc9a7c" exitCode=0
Jul 22 03:35:11.552107 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:11.551770    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mlkmd" event={"ID":"8e81bffd-5ccd-45f5-b072-ed0b951261ec","Type":"ContainerDied","Data":"08a1eb35be892035bd41af7bb619056595ff17bdf4d278170177f7ac2fdc9a7c"}
Jul 22 03:35:11.995677 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:11.995584    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-68d7c7b64c-rlvsn"
Jul 22 03:35:11.995677 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:11.995677    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-68d7c7b64c-rlvsn"
Jul 22 03:35:12.001359 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:12.001336    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-68d7c7b64c-rlvsn"
Jul 22 03:35:12.559084 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:12.559043    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mlkmd" event={"ID":"8e81bffd-5ccd-45f5-b072-ed0b951261ec","Type":"ContainerStarted","Data":"146849e05d692d772c3c35ea18fd8421464bd86cc70c3f9bfa1abb5e8f84789b"}
Jul 22 03:35:12.559668 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:12.559117    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mlkmd" event={"ID":"8e81bffd-5ccd-45f5-b072-ed0b951261ec","Type":"ContainerStarted","Data":"d161e1e383a7515a2214c5dc5e45904c6b98c8329655215c6f40a5b1f0405a74"}
Jul 22 03:35:12.563952 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:12.563922    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-68d7c7b64c-rlvsn"
Jul 22 03:35:12.581100 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:12.581045    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mlkmd" podStartSLOduration=2.897728185 podStartE2EDuration="4.581025144s" totalImagesPullingTime="1.577482137s" totalInitContainerRuntime="105.814822ms" isStatefulPod=true podCreationTimestamp="2026-07-22 03:35:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:35:12.579588083 +0000 UTC m=+100.197798092" watchObservedRunningTime="2026-07-22 03:35:12.581025144 +0000 UTC m=+100.199235152"
Jul 22 03:35:14.496667 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.496626    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 03:35:14.532764 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.532139    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 03:35:14.532764 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.532308    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.536125 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.536098    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 22 03:35:14.536348 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.536328    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 22 03:35:14.536521 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.536363    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 22 03:35:14.536596 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.536532    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 22 03:35:14.536695 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.536602    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 22 03:35:14.536695 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.536365    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 22 03:35:14.536798 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.536374    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 22 03:35:14.537211 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.537194    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-zzxxd\""
Jul 22 03:35:14.537311 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.537264    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-b2gjpe7o2erh7\""
Jul 22 03:35:14.537716 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.537401    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 22 03:35:14.537716 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.537435    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 22 03:35:14.537716 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.537482    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 22 03:35:14.537716 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.537406    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 22 03:35:14.537716 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.537659    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 22 03:35:14.538055 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.537848    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 22 03:35:14.541259 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.541225    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 22 03:35:14.544527 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.544510    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 22 03:35:14.624428 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.624390    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-config-out\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.624583 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.624463    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.624583 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.624540    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxdbj\" (UniqueName: \"kubernetes.io/projected/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-kube-api-access-gxdbj\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.624583 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.624568    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.624715 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.624589    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-config\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.624715 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.624607    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.624715 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.624692    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.624813 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.624767    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.624813 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.624802    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.624874 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.624825    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.624874 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.624840    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.624935 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.624889    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.624967 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.624933    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.624967 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.624957    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.625037 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.624973    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.625037 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.624992    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.625037 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.625005    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-web-config\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.625037 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.625027    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.625205 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.625060    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.625205 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.625099    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.725859 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.725814    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.725859 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.725870    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.726130 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.725897    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.726130 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.725923    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.726130 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.725961    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.726130 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.725990    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.726130 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.726035    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.726130 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.726062    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.726130 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.726092    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.726130 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.726119    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-web-config\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.726504 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.726166    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.726504 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.726195    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.726504 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.726222    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.726504 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.726260    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-config-out\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.726504 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.726296    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.726504 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.726335    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxdbj\" (UniqueName: \"kubernetes.io/projected/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-kube-api-access-gxdbj\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.726504 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.726365    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.726504 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.726393    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-config\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.726504 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.726417    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.726504 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.726445    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.727241 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.726872    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.727241 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.727172    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.727674 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.727390    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.727856 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.727395    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.729794 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.729424    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.729794 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.727887    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.729965 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.729817    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.730380 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.730352    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.730513 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.730486    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.731269 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.731243    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.731985 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.731956    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.732235 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.732202    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.732299 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.732235    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-config-out\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.732897 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.732846    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.732979 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.732925    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.733273 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.733250    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.733354 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.733332    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.733502 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.733486    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-config\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.733792 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.733766    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-web-config\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.748193 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.748125    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxdbj\" (UniqueName: \"kubernetes.io/projected/fbd30716-48e9-4f2c-9e6c-77df93cb9f23-kube-api-access-gxdbj\") pod \"prometheus-k8s-0\" (UID: \"fbd30716-48e9-4f2c-9e6c-77df93cb9f23\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:14.846085 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:14.846050    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:15.036100 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:15.036069    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 22 03:35:15.037219 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:35:15.037193    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbd30716_48e9_4f2c_9e6c_77df93cb9f23.slice/crio-9186d23e780469fd5c1d2640f046804ce8c47a87c0eaced29a4536c0ab6e76d1 WatchSource:0}: Error finding container 9186d23e780469fd5c1d2640f046804ce8c47a87c0eaced29a4536c0ab6e76d1: Status 404 returned error can't find the container with id 9186d23e780469fd5c1d2640f046804ce8c47a87c0eaced29a4536c0ab6e76d1
Jul 22 03:35:15.572374 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:15.572338    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fbd30716-48e9-4f2c-9e6c-77df93cb9f23","Type":"ContainerStarted","Data":"9186d23e780469fd5c1d2640f046804ce8c47a87c0eaced29a4536c0ab6e76d1"}
Jul 22 03:35:16.579747 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:16.579716    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fbd30716-48e9-4f2c-9e6c-77df93cb9f23","Type":"ContainerStarted","Data":"37b307995313ae8fa027c9b54b3ba33ef286662f86553b1e2d81cc320d8dbc1d"}
Jul 22 03:35:17.585869 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:17.585826    2570 generic.go:356] "Generic (PLEG): container finished" podID="fbd30716-48e9-4f2c-9e6c-77df93cb9f23" containerID="37b307995313ae8fa027c9b54b3ba33ef286662f86553b1e2d81cc320d8dbc1d" exitCode=0
Jul 22 03:35:17.586315 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:17.585897    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fbd30716-48e9-4f2c-9e6c-77df93cb9f23","Type":"ContainerDied","Data":"37b307995313ae8fa027c9b54b3ba33ef286662f86553b1e2d81cc320d8dbc1d"}
Jul 22 03:35:20.604501 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:20.604460    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fbd30716-48e9-4f2c-9e6c-77df93cb9f23","Type":"ContainerStarted","Data":"bd0ed4ca21a34a5083383c481e19fdad4c7d3b921fb303138ad3bb870c8eb2f5"}
Jul 22 03:35:20.604998 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:20.604639    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fbd30716-48e9-4f2c-9e6c-77df93cb9f23","Type":"ContainerStarted","Data":"80e5e585abc70247488b18b1ea99cad7d91d1a95a0738e479b63fee9f9b5bd7b"}
Jul 22 03:35:22.456701 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:22.456667    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-k2tvl"
Jul 22 03:35:23.619160 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:23.619121    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fbd30716-48e9-4f2c-9e6c-77df93cb9f23","Type":"ContainerStarted","Data":"4c14990b1f270a95b1825e5df70a4dd481e870b8a694c1b33a654b61e023e2ad"}
Jul 22 03:35:23.619540 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:23.619218    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fbd30716-48e9-4f2c-9e6c-77df93cb9f23","Type":"ContainerStarted","Data":"ef75a61875b3e7f1166826b5291e4bee76d6adfd31575e49734f46dea53fe72e"}
Jul 22 03:35:23.619540 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:23.619244    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fbd30716-48e9-4f2c-9e6c-77df93cb9f23","Type":"ContainerStarted","Data":"54b9cb22a8ed68a6c1fca4a367c3687346940af60ffc205fbeb13551b14897a6"}
Jul 22 03:35:23.619540 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:23.619268    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fbd30716-48e9-4f2c-9e6c-77df93cb9f23","Type":"ContainerStarted","Data":"86c1239a7b80fba6c115c566dcd2f170482b8b3bc043e5825a703b834b26077e"}
Jul 22 03:35:23.647452 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:23.647383    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.496801159 podStartE2EDuration="9.64736022s" totalImagesPullingTime="7.132510221s" totalInitContainerRuntime="18.04884ms" isStatefulPod=false podCreationTimestamp="2026-07-22 03:35:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:35:23.645198589 +0000 UTC m=+111.263408655" watchObservedRunningTime="2026-07-22 03:35:23.64736022 +0000 UTC m=+111.265570229"
Jul 22 03:35:24.850894 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:24.850864    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:35:25.282233 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:25.282199    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68d7c7b64c-rlvsn"]
Jul 22 03:35:25.629743 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:25.629709    2570 generic.go:356] "Generic (PLEG): container finished" podID="ff77d659-f9dd-4d0a-8f75-2dbadc25d260" containerID="62d0181db2aa2813d5837cc23c5d7c753c1876303c8823d2b0a87972f7d37541" exitCode=0
Jul 22 03:35:25.629903 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:25.629782    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qtwrs" event={"ID":"ff77d659-f9dd-4d0a-8f75-2dbadc25d260","Type":"ContainerDied","Data":"62d0181db2aa2813d5837cc23c5d7c753c1876303c8823d2b0a87972f7d37541"}
Jul 22 03:35:25.630392 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:25.630376    2570 scope.go:117] "RemoveContainer" containerID="62d0181db2aa2813d5837cc23c5d7c753c1876303c8823d2b0a87972f7d37541"
Jul 22 03:35:26.635633 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:26.635579    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-qtwrs" event={"ID":"ff77d659-f9dd-4d0a-8f75-2dbadc25d260","Type":"ContainerStarted","Data":"fc055b6c1ee909d2723515e33b9a5560c80499530e267122b69c36ed526e43e8"}
Jul 22 03:35:27.761450 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:27.761382    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7749b46548-cf9gh" podUID="7492beee-6a0b-4b5a-85c6-ec0064b61bbf" containerName="registry" containerID="cri-o://7b3c92be71cb1e3214156a24e67780a0c9ce35d926d97d4d4896491ef7dbd224" gracePeriod=30
Jul 22 03:35:28.080265 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:28.080240    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7749b46548-cf9gh"
Jul 22 03:35:28.148062 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:28.148028    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-ca-trust-extracted\") pod \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\" (UID: \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\") "
Jul 22 03:35:28.148062 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:28.148077    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9b6bv\" (UniqueName: \"kubernetes.io/projected/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-kube-api-access-9b6bv\") pod \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\" (UID: \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\") "
Jul 22 03:35:28.148328 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:28.148104    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-bound-sa-token\") pod \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\" (UID: \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\") "
Jul 22 03:35:28.148328 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:28.148128    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-registry-certificates\") pod \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\" (UID: \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\") "
Jul 22 03:35:28.148328 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:28.148144    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-registry-tls\") pod \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\" (UID: \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\") "
Jul 22 03:35:28.148328 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:28.148199    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-image-registry-private-configuration\") pod \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\" (UID: \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\") "
Jul 22 03:35:28.148328 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:28.148230    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-trusted-ca\") pod \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\" (UID: \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\") "
Jul 22 03:35:28.148566 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:28.148438    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-installation-pull-secrets\") pod \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\" (UID: \"7492beee-6a0b-4b5a-85c6-ec0064b61bbf\") "
Jul 22 03:35:28.149140 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:28.148706    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "7492beee-6a0b-4b5a-85c6-ec0064b61bbf" (UID: "7492beee-6a0b-4b5a-85c6-ec0064b61bbf"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 03:35:28.149140 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:28.148713    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "7492beee-6a0b-4b5a-85c6-ec0064b61bbf" (UID: "7492beee-6a0b-4b5a-85c6-ec0064b61bbf"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 03:35:28.150939 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:28.150910    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "7492beee-6a0b-4b5a-85c6-ec0064b61bbf" (UID: "7492beee-6a0b-4b5a-85c6-ec0064b61bbf"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 03:35:28.151080 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:28.151011    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-kube-api-access-9b6bv" (OuterVolumeSpecName: "kube-api-access-9b6bv") pod "7492beee-6a0b-4b5a-85c6-ec0064b61bbf" (UID: "7492beee-6a0b-4b5a-85c6-ec0064b61bbf"). InnerVolumeSpecName "kube-api-access-9b6bv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 03:35:28.151148 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:28.151086    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "7492beee-6a0b-4b5a-85c6-ec0064b61bbf" (UID: "7492beee-6a0b-4b5a-85c6-ec0064b61bbf"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 03:35:28.151148 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:28.151112    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "7492beee-6a0b-4b5a-85c6-ec0064b61bbf" (UID: "7492beee-6a0b-4b5a-85c6-ec0064b61bbf"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 03:35:28.151418 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:28.151395    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "7492beee-6a0b-4b5a-85c6-ec0064b61bbf" (UID: "7492beee-6a0b-4b5a-85c6-ec0064b61bbf"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 03:35:28.157102 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:28.157077    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "7492beee-6a0b-4b5a-85c6-ec0064b61bbf" (UID: "7492beee-6a0b-4b5a-85c6-ec0064b61bbf"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 03:35:28.249589 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:28.249547    2570 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-ca-trust-extracted\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 03:35:28.249589 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:28.249581    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9b6bv\" (UniqueName: \"kubernetes.io/projected/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-kube-api-access-9b6bv\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 03:35:28.249589 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:28.249596    2570 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-bound-sa-token\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 03:35:28.249885 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:28.249608    2570 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-registry-certificates\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 03:35:28.249885 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:28.249662    2570 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-registry-tls\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 03:35:28.249885 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:28.249677    2570 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-image-registry-private-configuration\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 03:35:28.249885 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:28.249690    2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-trusted-ca\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 03:35:28.249885 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:28.249703    2570 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7492beee-6a0b-4b5a-85c6-ec0064b61bbf-installation-pull-secrets\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 03:35:28.645518 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:28.645482    2570 generic.go:356] "Generic (PLEG): container finished" podID="7492beee-6a0b-4b5a-85c6-ec0064b61bbf" containerID="7b3c92be71cb1e3214156a24e67780a0c9ce35d926d97d4d4896491ef7dbd224" exitCode=0
Jul 22 03:35:28.645742 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:28.645530    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7749b46548-cf9gh"
Jul 22 03:35:28.645742 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:28.645563    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7749b46548-cf9gh" event={"ID":"7492beee-6a0b-4b5a-85c6-ec0064b61bbf","Type":"ContainerDied","Data":"7b3c92be71cb1e3214156a24e67780a0c9ce35d926d97d4d4896491ef7dbd224"}
Jul 22 03:35:28.645742 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:28.645602    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7749b46548-cf9gh" event={"ID":"7492beee-6a0b-4b5a-85c6-ec0064b61bbf","Type":"ContainerDied","Data":"08ecf6a64ed2968ebc68fa35c1d352351edd1101cb0f8893e715e9733f4afcbf"}
Jul 22 03:35:28.645742 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:28.645637    2570 scope.go:117] "RemoveContainer" containerID="7b3c92be71cb1e3214156a24e67780a0c9ce35d926d97d4d4896491ef7dbd224"
Jul 22 03:35:28.654188 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:28.654171    2570 scope.go:117] "RemoveContainer" containerID="7b3c92be71cb1e3214156a24e67780a0c9ce35d926d97d4d4896491ef7dbd224"
Jul 22 03:35:28.654462 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:35:28.654443    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b3c92be71cb1e3214156a24e67780a0c9ce35d926d97d4d4896491ef7dbd224\": container with ID starting with 7b3c92be71cb1e3214156a24e67780a0c9ce35d926d97d4d4896491ef7dbd224 not found: ID does not exist" containerID="7b3c92be71cb1e3214156a24e67780a0c9ce35d926d97d4d4896491ef7dbd224"
Jul 22 03:35:28.654498 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:28.654472    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b3c92be71cb1e3214156a24e67780a0c9ce35d926d97d4d4896491ef7dbd224"} err="failed to get container status \"7b3c92be71cb1e3214156a24e67780a0c9ce35d926d97d4d4896491ef7dbd224\": rpc error: code = NotFound desc = could not find container \"7b3c92be71cb1e3214156a24e67780a0c9ce35d926d97d4d4896491ef7dbd224\": container with ID starting with 7b3c92be71cb1e3214156a24e67780a0c9ce35d926d97d4d4896491ef7dbd224 not found: ID does not exist"
Jul 22 03:35:28.667897 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:28.667866    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7749b46548-cf9gh"]
Jul 22 03:35:28.682381 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:28.682348    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7749b46548-cf9gh"]
Jul 22 03:35:28.850750 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:28.850716    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7492beee-6a0b-4b5a-85c6-ec0064b61bbf" path="/var/lib/kubelet/pods/7492beee-6a0b-4b5a-85c6-ec0064b61bbf/volumes"
Jul 22 03:35:46.718640 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:46.718533    2570 generic.go:356] "Generic (PLEG): container finished" podID="8cdfda8a-dc86-4da1-b192-4d0ab5cd37f4" containerID="9407bba911145d7fff73f41142ca049c24615f29bd3b58c8f3b300da16ffc086" exitCode=0
Jul 22 03:35:46.719102 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:46.718628    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-lqg9w" event={"ID":"8cdfda8a-dc86-4da1-b192-4d0ab5cd37f4","Type":"ContainerDied","Data":"9407bba911145d7fff73f41142ca049c24615f29bd3b58c8f3b300da16ffc086"}
Jul 22 03:35:46.719102 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:46.718991    2570 scope.go:117] "RemoveContainer" containerID="9407bba911145d7fff73f41142ca049c24615f29bd3b58c8f3b300da16ffc086"
Jul 22 03:35:46.720172 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:46.720150    2570 generic.go:356] "Generic (PLEG): container finished" podID="ee83bd5e-e31f-4049-947f-8f609f93908f" containerID="f53074abe07fe81a59736e19faae8a0247af2587323f6f1dc732d8efe542dab1" exitCode=0
Jul 22 03:35:46.720240 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:46.720221    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-ptsm9" event={"ID":"ee83bd5e-e31f-4049-947f-8f609f93908f","Type":"ContainerDied","Data":"f53074abe07fe81a59736e19faae8a0247af2587323f6f1dc732d8efe542dab1"}
Jul 22 03:35:46.720503 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:46.720491    2570 scope.go:117] "RemoveContainer" containerID="f53074abe07fe81a59736e19faae8a0247af2587323f6f1dc732d8efe542dab1"
Jul 22 03:35:47.729718 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:47.728932    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-lqg9w" event={"ID":"8cdfda8a-dc86-4da1-b192-4d0ab5cd37f4","Type":"ContainerStarted","Data":"fd39fd3b799b8b20816758319f9dbfc8e71c854ca701e4dd1318dc2684b15e6a"}
Jul 22 03:35:47.732997 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:47.732677    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-ptsm9" event={"ID":"ee83bd5e-e31f-4049-947f-8f609f93908f","Type":"ContainerStarted","Data":"d019bf1aeedd710de3da9229b68fd095cba178f70fa96a28cd395b5fffb5ed74"}
Jul 22 03:35:50.310468 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:50.310395    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-68d7c7b64c-rlvsn" podUID="da13f849-f973-4a92-85af-808fe7382212" containerName="console" containerID="cri-o://1e683da210b7eaa5bbac429835e1c7815f28cc9ef406c458e5572fdc2b13c172" gracePeriod=15
Jul 22 03:35:50.591080 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:50.591053    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68d7c7b64c-rlvsn"
Jul 22 03:35:50.639405 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:50.639368    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wtnjv\" (UniqueName: \"kubernetes.io/projected/da13f849-f973-4a92-85af-808fe7382212-kube-api-access-wtnjv\") pod \"da13f849-f973-4a92-85af-808fe7382212\" (UID: \"da13f849-f973-4a92-85af-808fe7382212\") "
Jul 22 03:35:50.639570 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:50.639429    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/da13f849-f973-4a92-85af-808fe7382212-service-ca\") pod \"da13f849-f973-4a92-85af-808fe7382212\" (UID: \"da13f849-f973-4a92-85af-808fe7382212\") "
Jul 22 03:35:50.639570 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:50.639469    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/da13f849-f973-4a92-85af-808fe7382212-oauth-serving-cert\") pod \"da13f849-f973-4a92-85af-808fe7382212\" (UID: \"da13f849-f973-4a92-85af-808fe7382212\") "
Jul 22 03:35:50.639570 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:50.639508    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/da13f849-f973-4a92-85af-808fe7382212-console-config\") pod \"da13f849-f973-4a92-85af-808fe7382212\" (UID: \"da13f849-f973-4a92-85af-808fe7382212\") "
Jul 22 03:35:50.639570 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:50.639549    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/da13f849-f973-4a92-85af-808fe7382212-console-serving-cert\") pod \"da13f849-f973-4a92-85af-808fe7382212\" (UID: \"da13f849-f973-4a92-85af-808fe7382212\") "
Jul 22 03:35:50.639803 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:50.639583    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/da13f849-f973-4a92-85af-808fe7382212-console-oauth-config\") pod \"da13f849-f973-4a92-85af-808fe7382212\" (UID: \"da13f849-f973-4a92-85af-808fe7382212\") "
Jul 22 03:35:50.639803 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:50.639636    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da13f849-f973-4a92-85af-808fe7382212-trusted-ca-bundle\") pod \"da13f849-f973-4a92-85af-808fe7382212\" (UID: \"da13f849-f973-4a92-85af-808fe7382212\") "
Jul 22 03:35:50.639949 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:50.639894    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da13f849-f973-4a92-85af-808fe7382212-service-ca" (OuterVolumeSpecName: "service-ca") pod "da13f849-f973-4a92-85af-808fe7382212" (UID: "da13f849-f973-4a92-85af-808fe7382212"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 03:35:50.639949 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:50.639896    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da13f849-f973-4a92-85af-808fe7382212-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "da13f849-f973-4a92-85af-808fe7382212" (UID: "da13f849-f973-4a92-85af-808fe7382212"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 03:35:50.640206 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:50.639982    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da13f849-f973-4a92-85af-808fe7382212-console-config" (OuterVolumeSpecName: "console-config") pod "da13f849-f973-4a92-85af-808fe7382212" (UID: "da13f849-f973-4a92-85af-808fe7382212"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 03:35:50.640377 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:50.640355    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da13f849-f973-4a92-85af-808fe7382212-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "da13f849-f973-4a92-85af-808fe7382212" (UID: "da13f849-f973-4a92-85af-808fe7382212"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 03:35:50.641936 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:50.641905    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da13f849-f973-4a92-85af-808fe7382212-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "da13f849-f973-4a92-85af-808fe7382212" (UID: "da13f849-f973-4a92-85af-808fe7382212"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 03:35:50.642024 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:50.641958    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da13f849-f973-4a92-85af-808fe7382212-kube-api-access-wtnjv" (OuterVolumeSpecName: "kube-api-access-wtnjv") pod "da13f849-f973-4a92-85af-808fe7382212" (UID: "da13f849-f973-4a92-85af-808fe7382212"). InnerVolumeSpecName "kube-api-access-wtnjv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 03:35:50.642093 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:50.642076    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da13f849-f973-4a92-85af-808fe7382212-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "da13f849-f973-4a92-85af-808fe7382212" (UID: "da13f849-f973-4a92-85af-808fe7382212"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 03:35:50.740312 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:50.740274    2570 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/da13f849-f973-4a92-85af-808fe7382212-oauth-serving-cert\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 03:35:50.740312 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:50.740307    2570 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/da13f849-f973-4a92-85af-808fe7382212-console-config\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 03:35:50.740312 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:50.740317    2570 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/da13f849-f973-4a92-85af-808fe7382212-console-serving-cert\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 03:35:50.740553 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:50.740325    2570 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/da13f849-f973-4a92-85af-808fe7382212-console-oauth-config\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 03:35:50.740553 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:50.740334    2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/da13f849-f973-4a92-85af-808fe7382212-trusted-ca-bundle\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 03:35:50.740553 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:50.740343    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wtnjv\" (UniqueName: \"kubernetes.io/projected/da13f849-f973-4a92-85af-808fe7382212-kube-api-access-wtnjv\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 03:35:50.740553 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:50.740351    2570 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/da13f849-f973-4a92-85af-808fe7382212-service-ca\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 03:35:50.745068 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:50.745042    2570 generic.go:356] "Generic (PLEG): container finished" podID="da13f849-f973-4a92-85af-808fe7382212" containerID="1e683da210b7eaa5bbac429835e1c7815f28cc9ef406c458e5572fdc2b13c172" exitCode=0
Jul 22 03:35:50.745183 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:50.745095    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68d7c7b64c-rlvsn"
Jul 22 03:35:50.745183 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:50.745130    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68d7c7b64c-rlvsn" event={"ID":"da13f849-f973-4a92-85af-808fe7382212","Type":"ContainerDied","Data":"1e683da210b7eaa5bbac429835e1c7815f28cc9ef406c458e5572fdc2b13c172"}
Jul 22 03:35:50.745183 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:50.745180    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68d7c7b64c-rlvsn" event={"ID":"da13f849-f973-4a92-85af-808fe7382212","Type":"ContainerDied","Data":"6a2204b89aee2635fcb49ec7a5f0b8b9325288fd49bd294eeb606d0e4a1746c4"}
Jul 22 03:35:50.745280 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:50.745201    2570 scope.go:117] "RemoveContainer" containerID="1e683da210b7eaa5bbac429835e1c7815f28cc9ef406c458e5572fdc2b13c172"
Jul 22 03:35:50.756269 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:50.756213    2570 scope.go:117] "RemoveContainer" containerID="1e683da210b7eaa5bbac429835e1c7815f28cc9ef406c458e5572fdc2b13c172"
Jul 22 03:35:50.756722 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:35:50.756699    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e683da210b7eaa5bbac429835e1c7815f28cc9ef406c458e5572fdc2b13c172\": container with ID starting with 1e683da210b7eaa5bbac429835e1c7815f28cc9ef406c458e5572fdc2b13c172 not found: ID does not exist" containerID="1e683da210b7eaa5bbac429835e1c7815f28cc9ef406c458e5572fdc2b13c172"
Jul 22 03:35:50.756787 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:50.756730    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e683da210b7eaa5bbac429835e1c7815f28cc9ef406c458e5572fdc2b13c172"} err="failed to get container status \"1e683da210b7eaa5bbac429835e1c7815f28cc9ef406c458e5572fdc2b13c172\": rpc error: code = NotFound desc = could not find container \"1e683da210b7eaa5bbac429835e1c7815f28cc9ef406c458e5572fdc2b13c172\": container with ID starting with 1e683da210b7eaa5bbac429835e1c7815f28cc9ef406c458e5572fdc2b13c172 not found: ID does not exist"
Jul 22 03:35:50.768714 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:50.768685    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68d7c7b64c-rlvsn"]
Jul 22 03:35:50.772814 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:50.772792    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-68d7c7b64c-rlvsn"]
Jul 22 03:35:50.850478 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:35:50.850405    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da13f849-f973-4a92-85af-808fe7382212" path="/var/lib/kubelet/pods/da13f849-f973-4a92-85af-808fe7382212/volumes"
Jul 22 03:36:14.852219 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:36:14.852183    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:36:14.863758 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:36:14.863735    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:36:15.856442 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:36:15.856410    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 22 03:38:32.723717 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:32.723688    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2p4qp_9f0defcc-ccff-43be-8206-54c772885080/console-operator/1.log"
Jul 22 03:38:32.725829 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:32.725804    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2p4qp_9f0defcc-ccff-43be-8206-54c772885080/console-operator/1.log"
Jul 22 03:38:32.734865 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:32.734843    2570 kubelet.go:1648] "Image garbage collection succeeded"
Jul 22 03:38:43.958474 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:43.958434    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-78585bcb5c-xqvkl"]
Jul 22 03:38:43.959813 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:43.958875    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da13f849-f973-4a92-85af-808fe7382212" containerName="console"
Jul 22 03:38:43.959813 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:43.958893    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="da13f849-f973-4a92-85af-808fe7382212" containerName="console"
Jul 22 03:38:43.959813 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:43.958908    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7492beee-6a0b-4b5a-85c6-ec0064b61bbf" containerName="registry"
Jul 22 03:38:43.959813 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:43.958915    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="7492beee-6a0b-4b5a-85c6-ec0064b61bbf" containerName="registry"
Jul 22 03:38:43.959813 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:43.958990    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="7492beee-6a0b-4b5a-85c6-ec0064b61bbf" containerName="registry"
Jul 22 03:38:43.959813 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:43.959000    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="da13f849-f973-4a92-85af-808fe7382212" containerName="console"
Jul 22 03:38:43.960548 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:43.960530    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-78585bcb5c-xqvkl"
Jul 22 03:38:43.964369 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:43.964345    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 22 03:38:43.965478 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:43.965462    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 22 03:38:43.965545 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:43.965511    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 22 03:38:43.965588 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:43.965466    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 22 03:38:43.965588 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:43.965469    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-2zzk4\""
Jul 22 03:38:43.966479 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:43.966460    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 22 03:38:43.989390 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:43.989369    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-78585bcb5c-xqvkl"]
Jul 22 03:38:44.065568 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:44.065535    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a0062912-f644-4a63-ba13-9d4f463ed3aa-metrics-cert\") pod \"lws-controller-manager-78585bcb5c-xqvkl\" (UID: \"a0062912-f644-4a63-ba13-9d4f463ed3aa\") " pod="openshift-lws-operator/lws-controller-manager-78585bcb5c-xqvkl"
Jul 22 03:38:44.065767 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:44.065678    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a0062912-f644-4a63-ba13-9d4f463ed3aa-manager-config\") pod \"lws-controller-manager-78585bcb5c-xqvkl\" (UID: \"a0062912-f644-4a63-ba13-9d4f463ed3aa\") " pod="openshift-lws-operator/lws-controller-manager-78585bcb5c-xqvkl"
Jul 22 03:38:44.065767 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:44.065706    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98k52\" (UniqueName: \"kubernetes.io/projected/a0062912-f644-4a63-ba13-9d4f463ed3aa-kube-api-access-98k52\") pod \"lws-controller-manager-78585bcb5c-xqvkl\" (UID: \"a0062912-f644-4a63-ba13-9d4f463ed3aa\") " pod="openshift-lws-operator/lws-controller-manager-78585bcb5c-xqvkl"
Jul 22 03:38:44.065767 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:44.065731    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a0062912-f644-4a63-ba13-9d4f463ed3aa-cert\") pod \"lws-controller-manager-78585bcb5c-xqvkl\" (UID: \"a0062912-f644-4a63-ba13-9d4f463ed3aa\") " pod="openshift-lws-operator/lws-controller-manager-78585bcb5c-xqvkl"
Jul 22 03:38:44.166583 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:44.166529    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a0062912-f644-4a63-ba13-9d4f463ed3aa-metrics-cert\") pod \"lws-controller-manager-78585bcb5c-xqvkl\" (UID: \"a0062912-f644-4a63-ba13-9d4f463ed3aa\") " pod="openshift-lws-operator/lws-controller-manager-78585bcb5c-xqvkl"
Jul 22 03:38:44.166784 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:44.166643    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a0062912-f644-4a63-ba13-9d4f463ed3aa-manager-config\") pod \"lws-controller-manager-78585bcb5c-xqvkl\" (UID: \"a0062912-f644-4a63-ba13-9d4f463ed3aa\") " pod="openshift-lws-operator/lws-controller-manager-78585bcb5c-xqvkl"
Jul 22 03:38:44.166784 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:44.166664    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98k52\" (UniqueName: \"kubernetes.io/projected/a0062912-f644-4a63-ba13-9d4f463ed3aa-kube-api-access-98k52\") pod \"lws-controller-manager-78585bcb5c-xqvkl\" (UID: \"a0062912-f644-4a63-ba13-9d4f463ed3aa\") " pod="openshift-lws-operator/lws-controller-manager-78585bcb5c-xqvkl"
Jul 22 03:38:44.166784 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:44.166681    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a0062912-f644-4a63-ba13-9d4f463ed3aa-cert\") pod \"lws-controller-manager-78585bcb5c-xqvkl\" (UID: \"a0062912-f644-4a63-ba13-9d4f463ed3aa\") " pod="openshift-lws-operator/lws-controller-manager-78585bcb5c-xqvkl"
Jul 22 03:38:44.167407 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:44.167335    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a0062912-f644-4a63-ba13-9d4f463ed3aa-manager-config\") pod \"lws-controller-manager-78585bcb5c-xqvkl\" (UID: \"a0062912-f644-4a63-ba13-9d4f463ed3aa\") " pod="openshift-lws-operator/lws-controller-manager-78585bcb5c-xqvkl"
Jul 22 03:38:44.169092 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:44.169071    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a0062912-f644-4a63-ba13-9d4f463ed3aa-cert\") pod \"lws-controller-manager-78585bcb5c-xqvkl\" (UID: \"a0062912-f644-4a63-ba13-9d4f463ed3aa\") " pod="openshift-lws-operator/lws-controller-manager-78585bcb5c-xqvkl"
Jul 22 03:38:44.169182 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:44.169129    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a0062912-f644-4a63-ba13-9d4f463ed3aa-metrics-cert\") pod \"lws-controller-manager-78585bcb5c-xqvkl\" (UID: \"a0062912-f644-4a63-ba13-9d4f463ed3aa\") " pod="openshift-lws-operator/lws-controller-manager-78585bcb5c-xqvkl"
Jul 22 03:38:44.177995 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:44.177975    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98k52\" (UniqueName: \"kubernetes.io/projected/a0062912-f644-4a63-ba13-9d4f463ed3aa-kube-api-access-98k52\") pod \"lws-controller-manager-78585bcb5c-xqvkl\" (UID: \"a0062912-f644-4a63-ba13-9d4f463ed3aa\") " pod="openshift-lws-operator/lws-controller-manager-78585bcb5c-xqvkl"
Jul 22 03:38:44.269953 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:44.269870    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-78585bcb5c-xqvkl"
Jul 22 03:38:44.414535 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:44.414504    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-78585bcb5c-xqvkl"]
Jul 22 03:38:44.417901 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:38:44.417871    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0062912_f644_4a63_ba13_9d4f463ed3aa.slice/crio-6ac5229bcdba36164a644384d7fcdccc54de418828aca00bd01210d405c0dc64 WatchSource:0}: Error finding container 6ac5229bcdba36164a644384d7fcdccc54de418828aca00bd01210d405c0dc64: Status 404 returned error can't find the container with id 6ac5229bcdba36164a644384d7fcdccc54de418828aca00bd01210d405c0dc64
Jul 22 03:38:44.419834 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:44.419819    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 03:38:44.933983 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:44.933943    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5fd47b996b-fh5t6"]
Jul 22 03:38:44.936575 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:44.936547    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-fh5t6"
Jul 22 03:38:44.939761 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:44.939537    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 22 03:38:44.939761 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:44.939559    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 22 03:38:44.940591 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:44.940209    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 22 03:38:44.940591 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:44.940288    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 22 03:38:44.940591 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:44.940450    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-qfwz9\""
Jul 22 03:38:44.980852 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:44.980813    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5fd47b996b-fh5t6"]
Jul 22 03:38:45.075845 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:45.075806    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e7801170-6a68-4c68-bfcb-f7ef0acc2fae-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5fd47b996b-fh5t6\" (UID: \"e7801170-6a68-4c68-bfcb-f7ef0acc2fae\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-fh5t6"
Jul 22 03:38:45.076039 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:45.075896    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wr8v\" (UniqueName: \"kubernetes.io/projected/e7801170-6a68-4c68-bfcb-f7ef0acc2fae-kube-api-access-9wr8v\") pod \"opendatahub-operator-controller-manager-5fd47b996b-fh5t6\" (UID: \"e7801170-6a68-4c68-bfcb-f7ef0acc2fae\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-fh5t6"
Jul 22 03:38:45.076039 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:45.075928    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e7801170-6a68-4c68-bfcb-f7ef0acc2fae-webhook-cert\") pod \"opendatahub-operator-controller-manager-5fd47b996b-fh5t6\" (UID: \"e7801170-6a68-4c68-bfcb-f7ef0acc2fae\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-fh5t6"
Jul 22 03:38:45.076039 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:45.075973    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7801170-6a68-4c68-bfcb-f7ef0acc2fae-tmp\") pod \"opendatahub-operator-controller-manager-5fd47b996b-fh5t6\" (UID: \"e7801170-6a68-4c68-bfcb-f7ef0acc2fae\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-fh5t6"
Jul 22 03:38:45.076039 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:45.076026    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e7801170-6a68-4c68-bfcb-f7ef0acc2fae-manifests\") pod \"opendatahub-operator-controller-manager-5fd47b996b-fh5t6\" (UID: \"e7801170-6a68-4c68-bfcb-f7ef0acc2fae\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-fh5t6"
Jul 22 03:38:45.176881 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:45.176849    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wr8v\" (UniqueName: \"kubernetes.io/projected/e7801170-6a68-4c68-bfcb-f7ef0acc2fae-kube-api-access-9wr8v\") pod \"opendatahub-operator-controller-manager-5fd47b996b-fh5t6\" (UID: \"e7801170-6a68-4c68-bfcb-f7ef0acc2fae\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-fh5t6"
Jul 22 03:38:45.177059 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:45.176896    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e7801170-6a68-4c68-bfcb-f7ef0acc2fae-webhook-cert\") pod \"opendatahub-operator-controller-manager-5fd47b996b-fh5t6\" (UID: \"e7801170-6a68-4c68-bfcb-f7ef0acc2fae\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-fh5t6"
Jul 22 03:38:45.177059 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:45.176941    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7801170-6a68-4c68-bfcb-f7ef0acc2fae-tmp\") pod \"opendatahub-operator-controller-manager-5fd47b996b-fh5t6\" (UID: \"e7801170-6a68-4c68-bfcb-f7ef0acc2fae\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-fh5t6"
Jul 22 03:38:45.177059 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:45.176980    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e7801170-6a68-4c68-bfcb-f7ef0acc2fae-manifests\") pod \"opendatahub-operator-controller-manager-5fd47b996b-fh5t6\" (UID: \"e7801170-6a68-4c68-bfcb-f7ef0acc2fae\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-fh5t6"
Jul 22 03:38:45.177059 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:45.177022    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e7801170-6a68-4c68-bfcb-f7ef0acc2fae-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5fd47b996b-fh5t6\" (UID: \"e7801170-6a68-4c68-bfcb-f7ef0acc2fae\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-fh5t6"
Jul 22 03:38:45.178095 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:45.178066    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e7801170-6a68-4c68-bfcb-f7ef0acc2fae-manifests\") pod \"opendatahub-operator-controller-manager-5fd47b996b-fh5t6\" (UID: \"e7801170-6a68-4c68-bfcb-f7ef0acc2fae\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-fh5t6"
Jul 22 03:38:45.180481 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:45.180423    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e7801170-6a68-4c68-bfcb-f7ef0acc2fae-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5fd47b996b-fh5t6\" (UID: \"e7801170-6a68-4c68-bfcb-f7ef0acc2fae\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-fh5t6"
Jul 22 03:38:45.180604 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:45.180582    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e7801170-6a68-4c68-bfcb-f7ef0acc2fae-webhook-cert\") pod \"opendatahub-operator-controller-manager-5fd47b996b-fh5t6\" (UID: \"e7801170-6a68-4c68-bfcb-f7ef0acc2fae\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-fh5t6"
Jul 22 03:38:45.181520 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:45.181497    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7801170-6a68-4c68-bfcb-f7ef0acc2fae-tmp\") pod \"opendatahub-operator-controller-manager-5fd47b996b-fh5t6\" (UID: \"e7801170-6a68-4c68-bfcb-f7ef0acc2fae\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-fh5t6"
Jul 22 03:38:45.187046 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:45.186966    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wr8v\" (UniqueName: \"kubernetes.io/projected/e7801170-6a68-4c68-bfcb-f7ef0acc2fae-kube-api-access-9wr8v\") pod \"opendatahub-operator-controller-manager-5fd47b996b-fh5t6\" (UID: \"e7801170-6a68-4c68-bfcb-f7ef0acc2fae\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-fh5t6"
Jul 22 03:38:45.251086 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:45.251054    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-fh5t6"
Jul 22 03:38:45.399640 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:45.399585    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-78585bcb5c-xqvkl" event={"ID":"a0062912-f644-4a63-ba13-9d4f463ed3aa","Type":"ContainerStarted","Data":"6ac5229bcdba36164a644384d7fcdccc54de418828aca00bd01210d405c0dc64"}
Jul 22 03:38:45.414748 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:45.414724    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5fd47b996b-fh5t6"]
Jul 22 03:38:45.417820 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:38:45.417765    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7801170_6a68_4c68_bfcb_f7ef0acc2fae.slice/crio-ac2345ad20406b86ef3e2f20523449bd5741aceb78e25ee4142bb7e80209c30a WatchSource:0}: Error finding container ac2345ad20406b86ef3e2f20523449bd5741aceb78e25ee4142bb7e80209c30a: Status 404 returned error can't find the container with id ac2345ad20406b86ef3e2f20523449bd5741aceb78e25ee4142bb7e80209c30a
Jul 22 03:38:46.406371 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:46.406331    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-fh5t6" event={"ID":"e7801170-6a68-4c68-bfcb-f7ef0acc2fae","Type":"ContainerStarted","Data":"ac2345ad20406b86ef3e2f20523449bd5741aceb78e25ee4142bb7e80209c30a"}
Jul 22 03:38:47.415649 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:47.415590    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-78585bcb5c-xqvkl" event={"ID":"a0062912-f644-4a63-ba13-9d4f463ed3aa","Type":"ContainerStarted","Data":"3ac708b2e240dc516f86067dfcd0983098226ab1c0761f8af0f7cdd6c78da40a"}
Jul 22 03:38:47.416123 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:47.415924    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-78585bcb5c-xqvkl"
Jul 22 03:38:47.460627 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:47.460555    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-78585bcb5c-xqvkl" podStartSLOduration=2.3106280200000002 podStartE2EDuration="4.460530123s" totalImagesPullingTime="2.149902103s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:38:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:38:47.458535065 +0000 UTC m=+315.076745075" watchObservedRunningTime="2026-07-22 03:38:47.460530123 +0000 UTC m=+315.078740131"
Jul 22 03:38:49.426044 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:49.426009    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-fh5t6" event={"ID":"e7801170-6a68-4c68-bfcb-f7ef0acc2fae","Type":"ContainerStarted","Data":"cc4a55d4fa94a9be8d6484eaef4733479c948842ffedc1e8dd03c2fb30c3f7d8"}
Jul 22 03:38:50.432174 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:50.432142    2570 generic.go:356] "Generic (PLEG): container finished" podID="e7801170-6a68-4c68-bfcb-f7ef0acc2fae" containerID="cc4a55d4fa94a9be8d6484eaef4733479c948842ffedc1e8dd03c2fb30c3f7d8" exitCode=0
Jul 22 03:38:50.432664 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:50.432208    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-fh5t6" event={"ID":"e7801170-6a68-4c68-bfcb-f7ef0acc2fae","Type":"ContainerDied","Data":"cc4a55d4fa94a9be8d6484eaef4733479c948842ffedc1e8dd03c2fb30c3f7d8"}
Jul 22 03:38:51.438693 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:51.438663    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-fh5t6" event={"ID":"e7801170-6a68-4c68-bfcb-f7ef0acc2fae","Type":"ContainerStarted","Data":"e5f28f347a36dac2fd2e7cd2fa234ce956aef556605e026cdeaff14190d47211"}
Jul 22 03:38:51.439061 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:51.438803    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-fh5t6"
Jul 22 03:38:51.462753 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:51.462696    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-fh5t6" podStartSLOduration=2.496240593 podStartE2EDuration="7.462677859s" totalImagesPullingTime="3.91336704s" totalInitContainerRuntime="1.053070226s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:38:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:38:51.46075312 +0000 UTC m=+319.078963143" watchObservedRunningTime="2026-07-22 03:38:51.462677859 +0000 UTC m=+319.080887866"
Jul 22 03:38:58.422939 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:38:58.422906    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-78585bcb5c-xqvkl"
Jul 22 03:39:02.445453 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:39:02.445422    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-fh5t6"
Jul 22 03:39:10.418035 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:39:10.417993    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5bd5997d8b-cpbs7"]
Jul 22 03:39:10.434060 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:39:10.434031    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5bd5997d8b-cpbs7"
Jul 22 03:39:10.437098 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:39:10.437021    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-5dp79\""
Jul 22 03:39:10.437788 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:39:10.437765    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 22 03:39:10.437920 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:39:10.437765    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 22 03:39:10.438877 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:39:10.438854    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5bd5997d8b-cpbs7"]
Jul 22 03:39:10.495754 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:39:10.495716    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9ccr\" (UniqueName: \"kubernetes.io/projected/3eb33ff7-d4d6-4141-b23e-69d3cafb29b2-kube-api-access-n9ccr\") pod \"kube-auth-proxy-5bd5997d8b-cpbs7\" (UID: \"3eb33ff7-d4d6-4141-b23e-69d3cafb29b2\") " pod="openshift-ingress/kube-auth-proxy-5bd5997d8b-cpbs7"
Jul 22 03:39:10.495754 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:39:10.495751    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3eb33ff7-d4d6-4141-b23e-69d3cafb29b2-tmp\") pod \"kube-auth-proxy-5bd5997d8b-cpbs7\" (UID: \"3eb33ff7-d4d6-4141-b23e-69d3cafb29b2\") " pod="openshift-ingress/kube-auth-proxy-5bd5997d8b-cpbs7"
Jul 22 03:39:10.495960 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:39:10.495811    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3eb33ff7-d4d6-4141-b23e-69d3cafb29b2-tls-certs\") pod \"kube-auth-proxy-5bd5997d8b-cpbs7\" (UID: \"3eb33ff7-d4d6-4141-b23e-69d3cafb29b2\") " pod="openshift-ingress/kube-auth-proxy-5bd5997d8b-cpbs7"
Jul 22 03:39:10.597011 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:39:10.596968    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9ccr\" (UniqueName: \"kubernetes.io/projected/3eb33ff7-d4d6-4141-b23e-69d3cafb29b2-kube-api-access-n9ccr\") pod \"kube-auth-proxy-5bd5997d8b-cpbs7\" (UID: \"3eb33ff7-d4d6-4141-b23e-69d3cafb29b2\") " pod="openshift-ingress/kube-auth-proxy-5bd5997d8b-cpbs7"
Jul 22 03:39:10.597011 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:39:10.597006    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3eb33ff7-d4d6-4141-b23e-69d3cafb29b2-tmp\") pod \"kube-auth-proxy-5bd5997d8b-cpbs7\" (UID: \"3eb33ff7-d4d6-4141-b23e-69d3cafb29b2\") " pod="openshift-ingress/kube-auth-proxy-5bd5997d8b-cpbs7"
Jul 22 03:39:10.597216 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:39:10.597190    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3eb33ff7-d4d6-4141-b23e-69d3cafb29b2-tls-certs\") pod \"kube-auth-proxy-5bd5997d8b-cpbs7\" (UID: \"3eb33ff7-d4d6-4141-b23e-69d3cafb29b2\") " pod="openshift-ingress/kube-auth-proxy-5bd5997d8b-cpbs7"
Jul 22 03:39:10.599389 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:39:10.599363    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3eb33ff7-d4d6-4141-b23e-69d3cafb29b2-tmp\") pod \"kube-auth-proxy-5bd5997d8b-cpbs7\" (UID: \"3eb33ff7-d4d6-4141-b23e-69d3cafb29b2\") " pod="openshift-ingress/kube-auth-proxy-5bd5997d8b-cpbs7"
Jul 22 03:39:10.599683 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:39:10.599667    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3eb33ff7-d4d6-4141-b23e-69d3cafb29b2-tls-certs\") pod \"kube-auth-proxy-5bd5997d8b-cpbs7\" (UID: \"3eb33ff7-d4d6-4141-b23e-69d3cafb29b2\") " pod="openshift-ingress/kube-auth-proxy-5bd5997d8b-cpbs7"
Jul 22 03:39:10.605322 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:39:10.605300    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9ccr\" (UniqueName: \"kubernetes.io/projected/3eb33ff7-d4d6-4141-b23e-69d3cafb29b2-kube-api-access-n9ccr\") pod \"kube-auth-proxy-5bd5997d8b-cpbs7\" (UID: \"3eb33ff7-d4d6-4141-b23e-69d3cafb29b2\") " pod="openshift-ingress/kube-auth-proxy-5bd5997d8b-cpbs7"
Jul 22 03:39:10.744556 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:39:10.744462    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5bd5997d8b-cpbs7"
Jul 22 03:39:10.880948 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:39:10.880916    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5bd5997d8b-cpbs7"]
Jul 22 03:39:10.883944 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:39:10.883919    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3eb33ff7_d4d6_4141_b23e_69d3cafb29b2.slice/crio-f89f3fed16765fea1ced1bcb507663806b99d69b77dd993c2a58e4e83fe3b5c6 WatchSource:0}: Error finding container f89f3fed16765fea1ced1bcb507663806b99d69b77dd993c2a58e4e83fe3b5c6: Status 404 returned error can't find the container with id f89f3fed16765fea1ced1bcb507663806b99d69b77dd993c2a58e4e83fe3b5c6
Jul 22 03:39:11.530772 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:39:11.530726    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5bd5997d8b-cpbs7" event={"ID":"3eb33ff7-d4d6-4141-b23e-69d3cafb29b2","Type":"ContainerStarted","Data":"f89f3fed16765fea1ced1bcb507663806b99d69b77dd993c2a58e4e83fe3b5c6"}
Jul 22 03:39:13.541340 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:39:13.541305    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5bd5997d8b-cpbs7" event={"ID":"3eb33ff7-d4d6-4141-b23e-69d3cafb29b2","Type":"ContainerStarted","Data":"2f3d6af5e20420e6edfbfba2d96d84a782f70c88c7000e0b5f91c4bda0110046"}
Jul 22 03:39:13.559789 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:39:13.559736    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5bd5997d8b-cpbs7" podStartSLOduration=1.399265692 podStartE2EDuration="3.559720648s" totalImagesPullingTime="2.160454956s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:39:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:39:13.558685178 +0000 UTC m=+341.176895185" watchObservedRunningTime="2026-07-22 03:39:13.559720648 +0000 UTC m=+341.177930655"
Jul 22 03:40:45.693083 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:40:45.693044    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-csccb"]
Jul 22 03:40:45.696507 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:40:45.696484    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-csccb"
Jul 22 03:40:45.699138 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:40:45.699122    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 22 03:40:45.699231 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:40:45.699156    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 22 03:40:45.699443 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:40:45.699425    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 22 03:40:45.700251 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:40:45.700238    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-zl8mn\""
Jul 22 03:40:45.709804 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:40:45.709782    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-csccb"]
Jul 22 03:40:45.796529 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:40:45.796491    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqbpx\" (UniqueName: \"kubernetes.io/projected/a05e14f1-2573-4358-b15b-09a094fa010c-kube-api-access-kqbpx\") pod \"dns-operator-controller-manager-864dd78c4c-csccb\" (UID: \"a05e14f1-2573-4358-b15b-09a094fa010c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-csccb"
Jul 22 03:40:45.897799 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:40:45.897759    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqbpx\" (UniqueName: \"kubernetes.io/projected/a05e14f1-2573-4358-b15b-09a094fa010c-kube-api-access-kqbpx\") pod \"dns-operator-controller-manager-864dd78c4c-csccb\" (UID: \"a05e14f1-2573-4358-b15b-09a094fa010c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-csccb"
Jul 22 03:40:45.906141 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:40:45.906104    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqbpx\" (UniqueName: \"kubernetes.io/projected/a05e14f1-2573-4358-b15b-09a094fa010c-kube-api-access-kqbpx\") pod \"dns-operator-controller-manager-864dd78c4c-csccb\" (UID: \"a05e14f1-2573-4358-b15b-09a094fa010c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-csccb"
Jul 22 03:40:46.006749 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:40:46.006650    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-csccb"
Jul 22 03:40:46.138493 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:40:46.138464    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-csccb"]
Jul 22 03:40:46.142081 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:40:46.142056    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda05e14f1_2573_4358_b15b_09a094fa010c.slice/crio-7be36f6ea1ef846fed2be4c5caef17d8ced1c83af50850be25cc87bab660cc6f WatchSource:0}: Error finding container 7be36f6ea1ef846fed2be4c5caef17d8ced1c83af50850be25cc87bab660cc6f: Status 404 returned error can't find the container with id 7be36f6ea1ef846fed2be4c5caef17d8ced1c83af50850be25cc87bab660cc6f
Jul 22 03:40:46.910908 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:40:46.910881    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-2cb7r"]
Jul 22 03:40:46.914376 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:40:46.914355    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2cb7r"
Jul 22 03:40:46.917517 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:40:46.917094    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 22 03:40:46.917517 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:40:46.917366    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 22 03:40:46.917897 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:40:46.917577    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-bbdd8\""
Jul 22 03:40:46.926148 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:40:46.926124    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-2cb7r"]
Jul 22 03:40:46.937818 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:40:46.937787    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-csccb" event={"ID":"a05e14f1-2573-4358-b15b-09a094fa010c","Type":"ContainerStarted","Data":"7be36f6ea1ef846fed2be4c5caef17d8ced1c83af50850be25cc87bab660cc6f"}
Jul 22 03:40:47.007924 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:40:47.007890    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c709ad99-a734-47a6-853d-22ec32915ab6-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-2cb7r\" (UID: \"c709ad99-a734-47a6-853d-22ec32915ab6\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2cb7r"
Jul 22 03:40:47.007924 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:40:47.007938    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c709ad99-a734-47a6-853d-22ec32915ab6-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-2cb7r\" (UID: \"c709ad99-a734-47a6-853d-22ec32915ab6\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2cb7r"
Jul 22 03:40:47.008184 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:40:47.007955    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwslh\" (UniqueName: \"kubernetes.io/projected/c709ad99-a734-47a6-853d-22ec32915ab6-kube-api-access-cwslh\") pod \"kuadrant-console-plugin-66b5774985-2cb7r\" (UID: \"c709ad99-a734-47a6-853d-22ec32915ab6\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2cb7r"
Jul 22 03:40:47.109159 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:40:47.109122    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c709ad99-a734-47a6-853d-22ec32915ab6-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-2cb7r\" (UID: \"c709ad99-a734-47a6-853d-22ec32915ab6\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2cb7r"
Jul 22 03:40:47.109306 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:40:47.109167    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c709ad99-a734-47a6-853d-22ec32915ab6-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-2cb7r\" (UID: \"c709ad99-a734-47a6-853d-22ec32915ab6\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2cb7r"
Jul 22 03:40:47.109306 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:40:47.109190    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwslh\" (UniqueName: \"kubernetes.io/projected/c709ad99-a734-47a6-853d-22ec32915ab6-kube-api-access-cwslh\") pod \"kuadrant-console-plugin-66b5774985-2cb7r\" (UID: \"c709ad99-a734-47a6-853d-22ec32915ab6\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2cb7r"
Jul 22 03:40:47.109306 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:40:47.109287    2570 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 22 03:40:47.109404 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:40:47.109356    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c709ad99-a734-47a6-853d-22ec32915ab6-plugin-serving-cert podName:c709ad99-a734-47a6-853d-22ec32915ab6 nodeName:}" failed. No retries permitted until 2026-07-22 03:40:47.609336208 +0000 UTC m=+435.227546197 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/c709ad99-a734-47a6-853d-22ec32915ab6-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-2cb7r" (UID: "c709ad99-a734-47a6-853d-22ec32915ab6") : secret "plugin-serving-cert" not found
Jul 22 03:40:47.109735 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:40:47.109716    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c709ad99-a734-47a6-853d-22ec32915ab6-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-2cb7r\" (UID: \"c709ad99-a734-47a6-853d-22ec32915ab6\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2cb7r"
Jul 22 03:40:47.145303 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:40:47.145276    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwslh\" (UniqueName: \"kubernetes.io/projected/c709ad99-a734-47a6-853d-22ec32915ab6-kube-api-access-cwslh\") pod \"kuadrant-console-plugin-66b5774985-2cb7r\" (UID: \"c709ad99-a734-47a6-853d-22ec32915ab6\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2cb7r"
Jul 22 03:40:47.614229 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:40:47.614190    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c709ad99-a734-47a6-853d-22ec32915ab6-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-2cb7r\" (UID: \"c709ad99-a734-47a6-853d-22ec32915ab6\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2cb7r"
Jul 22 03:40:47.616747 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:40:47.616723    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c709ad99-a734-47a6-853d-22ec32915ab6-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-2cb7r\" (UID: \"c709ad99-a734-47a6-853d-22ec32915ab6\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2cb7r"
Jul 22 03:40:47.832141 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:40:47.832088    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2cb7r"
Jul 22 03:40:47.982263 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:40:47.982232    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-2cb7r"]
Jul 22 03:40:48.275450 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:40:48.275410    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc709ad99_a734_47a6_853d_22ec32915ab6.slice/crio-6f35e152cfd402e4845803a2c02e1e200178586abc571924037906013fb3b80a WatchSource:0}: Error finding container 6f35e152cfd402e4845803a2c02e1e200178586abc571924037906013fb3b80a: Status 404 returned error can't find the container with id 6f35e152cfd402e4845803a2c02e1e200178586abc571924037906013fb3b80a
Jul 22 03:40:48.951733 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:40:48.951690    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-csccb" event={"ID":"a05e14f1-2573-4358-b15b-09a094fa010c","Type":"ContainerStarted","Data":"026e436812e1fc4e5d5b1f0f8c761f0bed614e1e728a29dbbf00372c57363901"}
Jul 22 03:40:48.951923 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:40:48.951813    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-csccb"
Jul 22 03:40:48.953117 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:40:48.953090    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2cb7r" event={"ID":"c709ad99-a734-47a6-853d-22ec32915ab6","Type":"ContainerStarted","Data":"6f35e152cfd402e4845803a2c02e1e200178586abc571924037906013fb3b80a"}
Jul 22 03:40:48.969946 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:40:48.969890    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-csccb" podStartSLOduration=1.785247322 podStartE2EDuration="3.969870906s" totalImagesPullingTime="2.184623584s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:40:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:40:48.968950021 +0000 UTC m=+436.587160053" watchObservedRunningTime="2026-07-22 03:40:48.969870906 +0000 UTC m=+436.588080904"
Jul 22 03:40:59.962109 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:40:59.962021    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-csccb"
Jul 22 03:41:11.070273 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:41:11.070237    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2cb7r" event={"ID":"c709ad99-a734-47a6-853d-22ec32915ab6","Type":"ContainerStarted","Data":"79e9f7fa80aedd49d68596244060b8bfc09015782b3a81cf70eae9467713f253"}
Jul 22 03:41:11.090861 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:41:11.090806    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2cb7r" podStartSLOduration=2.5094370980000003 podStartE2EDuration="25.090789257s" totalImagesPullingTime="22.581352159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:40:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:41:11.08833336 +0000 UTC m=+458.706543361" watchObservedRunningTime="2026-07-22 03:41:11.090789257 +0000 UTC m=+458.708999264"
Jul 22 03:41:30.471041 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:41:30.471001    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:41:30.594963 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:41:30.594915    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:41:30.595129 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:41:30.594982    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:41:30.595129 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:41:30.595042    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-2js4d"
Jul 22 03:41:30.599712 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:41:30.599677    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 22 03:41:30.613742 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:41:30.613717    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7bfg\" (UniqueName: \"kubernetes.io/projected/3707c428-6025-4ada-b4c4-684dc735790e-kube-api-access-k7bfg\") pod \"limitador-limitador-54d6d988d5-2js4d\" (UID: \"3707c428-6025-4ada-b4c4-684dc735790e\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2js4d"
Jul 22 03:41:30.613889 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:41:30.613767    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3707c428-6025-4ada-b4c4-684dc735790e-config-file\") pod \"limitador-limitador-54d6d988d5-2js4d\" (UID: \"3707c428-6025-4ada-b4c4-684dc735790e\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2js4d"
Jul 22 03:41:30.714464 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:41:30.714418    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7bfg\" (UniqueName: \"kubernetes.io/projected/3707c428-6025-4ada-b4c4-684dc735790e-kube-api-access-k7bfg\") pod \"limitador-limitador-54d6d988d5-2js4d\" (UID: \"3707c428-6025-4ada-b4c4-684dc735790e\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2js4d"
Jul 22 03:41:30.714697 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:41:30.714529    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3707c428-6025-4ada-b4c4-684dc735790e-config-file\") pod \"limitador-limitador-54d6d988d5-2js4d\" (UID: \"3707c428-6025-4ada-b4c4-684dc735790e\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2js4d"
Jul 22 03:41:30.715188 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:41:30.715165    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3707c428-6025-4ada-b4c4-684dc735790e-config-file\") pod \"limitador-limitador-54d6d988d5-2js4d\" (UID: \"3707c428-6025-4ada-b4c4-684dc735790e\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2js4d"
Jul 22 03:41:30.723695 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:41:30.723646    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7bfg\" (UniqueName: \"kubernetes.io/projected/3707c428-6025-4ada-b4c4-684dc735790e-kube-api-access-k7bfg\") pod \"limitador-limitador-54d6d988d5-2js4d\" (UID: \"3707c428-6025-4ada-b4c4-684dc735790e\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-2js4d"
Jul 22 03:41:30.909409 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:41:30.909375    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-2js4d"
Jul 22 03:41:31.075020 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:41:31.074996    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:41:31.077726 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:41:31.077701    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3707c428_6025_4ada_b4c4_684dc735790e.slice/crio-289fc150202653e5d73e59d9d3f6212f2e849df69dda2bd95e96f867dc66e91c WatchSource:0}: Error finding container 289fc150202653e5d73e59d9d3f6212f2e849df69dda2bd95e96f867dc66e91c: Status 404 returned error can't find the container with id 289fc150202653e5d73e59d9d3f6212f2e849df69dda2bd95e96f867dc66e91c
Jul 22 03:41:31.157518 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:41:31.157476    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-2js4d" event={"ID":"3707c428-6025-4ada-b4c4-684dc735790e","Type":"ContainerStarted","Data":"289fc150202653e5d73e59d9d3f6212f2e849df69dda2bd95e96f867dc66e91c"}
Jul 22 03:41:35.181850 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:41:35.181810    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-2js4d" event={"ID":"3707c428-6025-4ada-b4c4-684dc735790e","Type":"ContainerStarted","Data":"5a41a81f2c59a92d8c8d924bb92c53a2e2756f4f42acf7d3bdace4055d9ee33b"}
Jul 22 03:41:35.182251 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:41:35.181947    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-2js4d"
Jul 22 03:41:35.202216 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:41:35.202163    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-2js4d" podStartSLOduration=1.950481091 podStartE2EDuration="5.202147279s" totalImagesPullingTime="3.251666188s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:41:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:41:35.200013452 +0000 UTC m=+482.818223437" watchObservedRunningTime="2026-07-22 03:41:35.202147279 +0000 UTC m=+482.820357286"
Jul 22 03:41:46.188490 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:41:46.188458    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-2js4d"
Jul 22 03:42:46.304080 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:42:46.303989    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:42:46.329263 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:42:46.329227    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-vd2dl"]
Jul 22 03:42:46.332434 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:42:46.332417    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-vd2dl"
Jul 22 03:42:46.335344 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:42:46.335312    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-tkpz6\""
Jul 22 03:42:46.335484 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:42:46.335465    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 22 03:42:46.350061 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:42:46.350030    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-vd2dl"]
Jul 22 03:42:46.461300 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:42:46.461269    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26c42593-c5c2-497c-b0ff-52938d7447df-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-vd2dl\" (UID: \"26c42593-c5c2-497c-b0ff-52938d7447df\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-vd2dl"
Jul 22 03:42:46.461467 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:42:46.461334    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46z2w\" (UniqueName: \"kubernetes.io/projected/26c42593-c5c2-497c-b0ff-52938d7447df-kube-api-access-46z2w\") pod \"payload-pre-processing-7677b774bb-vd2dl\" (UID: \"26c42593-c5c2-497c-b0ff-52938d7447df\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-vd2dl"
Jul 22 03:42:46.562468 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:42:46.562383    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46z2w\" (UniqueName: \"kubernetes.io/projected/26c42593-c5c2-497c-b0ff-52938d7447df-kube-api-access-46z2w\") pod \"payload-pre-processing-7677b774bb-vd2dl\" (UID: \"26c42593-c5c2-497c-b0ff-52938d7447df\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-vd2dl"
Jul 22 03:42:46.562640 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:42:46.562468    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26c42593-c5c2-497c-b0ff-52938d7447df-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-vd2dl\" (UID: \"26c42593-c5c2-497c-b0ff-52938d7447df\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-vd2dl"
Jul 22 03:42:46.563134 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:42:46.563115    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26c42593-c5c2-497c-b0ff-52938d7447df-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-vd2dl\" (UID: \"26c42593-c5c2-497c-b0ff-52938d7447df\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-vd2dl"
Jul 22 03:42:46.573982 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:42:46.573952    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46z2w\" (UniqueName: \"kubernetes.io/projected/26c42593-c5c2-497c-b0ff-52938d7447df-kube-api-access-46z2w\") pod \"payload-pre-processing-7677b774bb-vd2dl\" (UID: \"26c42593-c5c2-497c-b0ff-52938d7447df\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-vd2dl"
Jul 22 03:42:46.642781 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:42:46.642743    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-vd2dl"
Jul 22 03:42:46.776766 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:42:46.776736    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-vd2dl"]
Jul 22 03:42:46.780010 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:42:46.779969    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26c42593_c5c2_497c_b0ff_52938d7447df.slice/crio-dff6629d2081671af65631005c806e8584cf68d1baa99987e389111a96ebb622 WatchSource:0}: Error finding container dff6629d2081671af65631005c806e8584cf68d1baa99987e389111a96ebb622: Status 404 returned error can't find the container with id dff6629d2081671af65631005c806e8584cf68d1baa99987e389111a96ebb622
Jul 22 03:42:47.500559 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:42:47.500520    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-vd2dl" event={"ID":"26c42593-c5c2-497c-b0ff-52938d7447df","Type":"ContainerStarted","Data":"dff6629d2081671af65631005c806e8584cf68d1baa99987e389111a96ebb622"}
Jul 22 03:42:49.511624 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:42:49.511584    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-vd2dl" event={"ID":"26c42593-c5c2-497c-b0ff-52938d7447df","Type":"ContainerStarted","Data":"f78a79ecc6cb8bc4c29177b078d9d4165204fb7eeb3acaad9784316d0414588f"}
Jul 22 03:42:49.512035 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:42:49.511710    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-vd2dl"
Jul 22 03:42:49.531521 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:42:49.531474    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-vd2dl" podStartSLOduration=1.320552594 podStartE2EDuration="3.531459188s" totalImagesPullingTime="2.210906594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:42:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:42:49.529279819 +0000 UTC m=+557.147489848" watchObservedRunningTime="2026-07-22 03:42:49.531459188 +0000 UTC m=+557.149669172"
Jul 22 03:43:00.520002 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:00.519965    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-vd2dl"
Jul 22 03:43:03.573232 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:03.573201    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:43:10.570198 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:10.570162    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:43:17.074387 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:17.074358    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-lpgt9"]
Jul 22 03:43:17.078770 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:17.078754    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lpgt9"
Jul 22 03:43:17.081838 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:17.081809    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-vmmxq\""
Jul 22 03:43:17.082993 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:17.082970    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 22 03:43:17.083091 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:17.082970    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 22 03:43:17.083157 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:17.082981    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 22 03:43:17.092204 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:17.092176    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-lpgt9"]
Jul 22 03:43:17.225935 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:17.225899    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e321c227-20dc-4359-9ab0-b63f341699d7-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lpgt9\" (UID: \"e321c227-20dc-4359-9ab0-b63f341699d7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lpgt9"
Jul 22 03:43:17.226179 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:17.225942    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e321c227-20dc-4359-9ab0-b63f341699d7-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lpgt9\" (UID: \"e321c227-20dc-4359-9ab0-b63f341699d7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lpgt9"
Jul 22 03:43:17.226179 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:17.226008    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e321c227-20dc-4359-9ab0-b63f341699d7-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lpgt9\" (UID: \"e321c227-20dc-4359-9ab0-b63f341699d7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lpgt9"
Jul 22 03:43:17.226179 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:17.226030    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e321c227-20dc-4359-9ab0-b63f341699d7-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lpgt9\" (UID: \"e321c227-20dc-4359-9ab0-b63f341699d7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lpgt9"
Jul 22 03:43:17.226179 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:17.226067    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e321c227-20dc-4359-9ab0-b63f341699d7-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lpgt9\" (UID: \"e321c227-20dc-4359-9ab0-b63f341699d7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lpgt9"
Jul 22 03:43:17.226179 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:17.226091    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7czn\" (UniqueName: \"kubernetes.io/projected/e321c227-20dc-4359-9ab0-b63f341699d7-kube-api-access-g7czn\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lpgt9\" (UID: \"e321c227-20dc-4359-9ab0-b63f341699d7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lpgt9"
Jul 22 03:43:17.327002 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:17.326906    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e321c227-20dc-4359-9ab0-b63f341699d7-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lpgt9\" (UID: \"e321c227-20dc-4359-9ab0-b63f341699d7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lpgt9"
Jul 22 03:43:17.327002 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:17.326943    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e321c227-20dc-4359-9ab0-b63f341699d7-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lpgt9\" (UID: \"e321c227-20dc-4359-9ab0-b63f341699d7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lpgt9"
Jul 22 03:43:17.327002 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:17.326969    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e321c227-20dc-4359-9ab0-b63f341699d7-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lpgt9\" (UID: \"e321c227-20dc-4359-9ab0-b63f341699d7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lpgt9"
Jul 22 03:43:17.327002 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:17.326994    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7czn\" (UniqueName: \"kubernetes.io/projected/e321c227-20dc-4359-9ab0-b63f341699d7-kube-api-access-g7czn\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lpgt9\" (UID: \"e321c227-20dc-4359-9ab0-b63f341699d7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lpgt9"
Jul 22 03:43:17.327327 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:17.327035    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e321c227-20dc-4359-9ab0-b63f341699d7-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lpgt9\" (UID: \"e321c227-20dc-4359-9ab0-b63f341699d7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lpgt9"
Jul 22 03:43:17.327327 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:17.327058    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e321c227-20dc-4359-9ab0-b63f341699d7-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lpgt9\" (UID: \"e321c227-20dc-4359-9ab0-b63f341699d7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lpgt9"
Jul 22 03:43:17.327428 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:17.327373    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e321c227-20dc-4359-9ab0-b63f341699d7-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lpgt9\" (UID: \"e321c227-20dc-4359-9ab0-b63f341699d7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lpgt9"
Jul 22 03:43:17.327428 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:17.327406    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e321c227-20dc-4359-9ab0-b63f341699d7-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lpgt9\" (UID: \"e321c227-20dc-4359-9ab0-b63f341699d7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lpgt9"
Jul 22 03:43:17.327428 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:17.327376    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e321c227-20dc-4359-9ab0-b63f341699d7-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lpgt9\" (UID: \"e321c227-20dc-4359-9ab0-b63f341699d7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lpgt9"
Jul 22 03:43:17.329514 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:17.329483    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e321c227-20dc-4359-9ab0-b63f341699d7-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lpgt9\" (UID: \"e321c227-20dc-4359-9ab0-b63f341699d7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lpgt9"
Jul 22 03:43:17.329745 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:17.329726    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e321c227-20dc-4359-9ab0-b63f341699d7-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lpgt9\" (UID: \"e321c227-20dc-4359-9ab0-b63f341699d7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lpgt9"
Jul 22 03:43:17.336226 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:17.336200    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7czn\" (UniqueName: \"kubernetes.io/projected/e321c227-20dc-4359-9ab0-b63f341699d7-kube-api-access-g7czn\") pod \"e2e-trlp-test-simulated-kserve-598466548d-lpgt9\" (UID: \"e321c227-20dc-4359-9ab0-b63f341699d7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lpgt9"
Jul 22 03:43:17.389062 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:17.389019    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lpgt9"
Jul 22 03:43:17.532376 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:17.532337    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-lpgt9"]
Jul 22 03:43:17.536229 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:43:17.536202    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode321c227_20dc_4359_9ab0_b63f341699d7.slice/crio-872528e28c6bc8d675360a5e5c687a15589efb5136f15a24223cb42239cb4490 WatchSource:0}: Error finding container 872528e28c6bc8d675360a5e5c687a15589efb5136f15a24223cb42239cb4490: Status 404 returned error can't find the container with id 872528e28c6bc8d675360a5e5c687a15589efb5136f15a24223cb42239cb4490
Jul 22 03:43:17.636963 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:17.636884    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lpgt9" event={"ID":"e321c227-20dc-4359-9ab0-b63f341699d7","Type":"ContainerStarted","Data":"872528e28c6bc8d675360a5e5c687a15589efb5136f15a24223cb42239cb4490"}
Jul 22 03:43:18.162231 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:18.162197    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:43:18.642926 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:18.642895    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lpgt9" event={"ID":"e321c227-20dc-4359-9ab0-b63f341699d7","Type":"ContainerStarted","Data":"6b493b53fdf670648107f68b9058c07e68814356479ec625f38247e5fe571267"}
Jul 22 03:43:18.664835 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:18.664785    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lpgt9" podStartSLOduration=0.644927804 podStartE2EDuration="1.664767353s" totalImagesPullingTime="1.019839549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:43:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:43:18.662699127 +0000 UTC m=+586.280909157" watchObservedRunningTime="2026-07-22 03:43:18.664767353 +0000 UTC m=+586.282977416"
Jul 22 03:43:25.068348 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:25.068309    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:43:27.390221 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:27.390182    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lpgt9"
Jul 22 03:43:27.390604 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:27.390253    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lpgt9"
Jul 22 03:43:27.402847 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:27.402821    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lpgt9"
Jul 22 03:43:27.694173 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:27.694146    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-lpgt9"
Jul 22 03:43:32.753508 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:32.753478    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2p4qp_9f0defcc-ccff-43be-8206-54c772885080/console-operator/1.log"
Jul 22 03:43:32.753946 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:32.753890    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2p4qp_9f0defcc-ccff-43be-8206-54c772885080/console-operator/1.log"
Jul 22 03:43:47.967379 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:47.967340    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:43:48.449630 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:48.449566    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-8p52g"]
Jul 22 03:43:48.451683 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:48.451668    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8p52g"
Jul 22 03:43:48.454524 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:48.454502    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 22 03:43:48.467010 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:48.466983    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-8p52g"]
Jul 22 03:43:48.594002 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:48.593967    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tccx\" (UniqueName: \"kubernetes.io/projected/25c4f355-e8a5-4b0a-8eb1-b37487417b20-kube-api-access-2tccx\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8p52g\" (UID: \"25c4f355-e8a5-4b0a-8eb1-b37487417b20\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8p52g"
Jul 22 03:43:48.594002 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:48.594006    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25c4f355-e8a5-4b0a-8eb1-b37487417b20-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8p52g\" (UID: \"25c4f355-e8a5-4b0a-8eb1-b37487417b20\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8p52g"
Jul 22 03:43:48.594221 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:48.594025    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25c4f355-e8a5-4b0a-8eb1-b37487417b20-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8p52g\" (UID: \"25c4f355-e8a5-4b0a-8eb1-b37487417b20\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8p52g"
Jul 22 03:43:48.594221 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:48.594113    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25c4f355-e8a5-4b0a-8eb1-b37487417b20-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8p52g\" (UID: \"25c4f355-e8a5-4b0a-8eb1-b37487417b20\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8p52g"
Jul 22 03:43:48.594221 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:48.594144    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25c4f355-e8a5-4b0a-8eb1-b37487417b20-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8p52g\" (UID: \"25c4f355-e8a5-4b0a-8eb1-b37487417b20\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8p52g"
Jul 22 03:43:48.594221 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:48.594178    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25c4f355-e8a5-4b0a-8eb1-b37487417b20-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8p52g\" (UID: \"25c4f355-e8a5-4b0a-8eb1-b37487417b20\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8p52g"
Jul 22 03:43:48.695054 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:48.695012    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tccx\" (UniqueName: \"kubernetes.io/projected/25c4f355-e8a5-4b0a-8eb1-b37487417b20-kube-api-access-2tccx\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8p52g\" (UID: \"25c4f355-e8a5-4b0a-8eb1-b37487417b20\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8p52g"
Jul 22 03:43:48.695054 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:48.695052    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25c4f355-e8a5-4b0a-8eb1-b37487417b20-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8p52g\" (UID: \"25c4f355-e8a5-4b0a-8eb1-b37487417b20\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8p52g"
Jul 22 03:43:48.695297 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:48.695070    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25c4f355-e8a5-4b0a-8eb1-b37487417b20-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8p52g\" (UID: \"25c4f355-e8a5-4b0a-8eb1-b37487417b20\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8p52g"
Jul 22 03:43:48.695297 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:48.695097    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25c4f355-e8a5-4b0a-8eb1-b37487417b20-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8p52g\" (UID: \"25c4f355-e8a5-4b0a-8eb1-b37487417b20\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8p52g"
Jul 22 03:43:48.695297 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:48.695115    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25c4f355-e8a5-4b0a-8eb1-b37487417b20-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8p52g\" (UID: \"25c4f355-e8a5-4b0a-8eb1-b37487417b20\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8p52g"
Jul 22 03:43:48.695297 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:48.695238    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25c4f355-e8a5-4b0a-8eb1-b37487417b20-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8p52g\" (UID: \"25c4f355-e8a5-4b0a-8eb1-b37487417b20\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8p52g"
Jul 22 03:43:48.695468 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:48.695443    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25c4f355-e8a5-4b0a-8eb1-b37487417b20-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8p52g\" (UID: \"25c4f355-e8a5-4b0a-8eb1-b37487417b20\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8p52g"
Jul 22 03:43:48.695563 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:48.695537    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25c4f355-e8a5-4b0a-8eb1-b37487417b20-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8p52g\" (UID: \"25c4f355-e8a5-4b0a-8eb1-b37487417b20\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8p52g"
Jul 22 03:43:48.695648 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:48.695601    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25c4f355-e8a5-4b0a-8eb1-b37487417b20-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8p52g\" (UID: \"25c4f355-e8a5-4b0a-8eb1-b37487417b20\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8p52g"
Jul 22 03:43:48.697497 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:48.697479    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25c4f355-e8a5-4b0a-8eb1-b37487417b20-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8p52g\" (UID: \"25c4f355-e8a5-4b0a-8eb1-b37487417b20\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8p52g"
Jul 22 03:43:48.697896 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:48.697874    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25c4f355-e8a5-4b0a-8eb1-b37487417b20-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8p52g\" (UID: \"25c4f355-e8a5-4b0a-8eb1-b37487417b20\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8p52g"
Jul 22 03:43:48.704317 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:48.704264    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tccx\" (UniqueName: \"kubernetes.io/projected/25c4f355-e8a5-4b0a-8eb1-b37487417b20-kube-api-access-2tccx\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8p52g\" (UID: \"25c4f355-e8a5-4b0a-8eb1-b37487417b20\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8p52g"
Jul 22 03:43:48.762317 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:48.762270    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8p52g"
Jul 22 03:43:48.904169 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:48.904142    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-8p52g"]
Jul 22 03:43:48.905889 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:43:48.905853    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod25c4f355_e8a5_4b0a_8eb1_b37487417b20.slice/crio-79f9a4a470e68c1db1fcb5bb1e48a8f2416a31f8cbc09b9938aaf822ae975ac9 WatchSource:0}: Error finding container 79f9a4a470e68c1db1fcb5bb1e48a8f2416a31f8cbc09b9938aaf822ae975ac9: Status 404 returned error can't find the container with id 79f9a4a470e68c1db1fcb5bb1e48a8f2416a31f8cbc09b9938aaf822ae975ac9
Jul 22 03:43:48.907395 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:48.907378    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 03:43:49.788953 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:49.788917    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8p52g" event={"ID":"25c4f355-e8a5-4b0a-8eb1-b37487417b20","Type":"ContainerStarted","Data":"7cd82d601edd3f343fdd790c32a6edc58fa9ea356922038ce1f3a036e5ff8c17"}
Jul 22 03:43:49.788953 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:49.788959    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8p52g" event={"ID":"25c4f355-e8a5-4b0a-8eb1-b37487417b20","Type":"ContainerStarted","Data":"79f9a4a470e68c1db1fcb5bb1e48a8f2416a31f8cbc09b9938aaf822ae975ac9"}
Jul 22 03:43:49.809598 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:49.809550    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8p52g" podStartSLOduration=1.625069571 podStartE2EDuration="1.80953596s" totalImagesPullingTime="184.466389ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:43:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:43:49.808680669 +0000 UTC m=+617.426890677" watchObservedRunningTime="2026-07-22 03:43:49.80953596 +0000 UTC m=+617.427745967"
Jul 22 03:43:58.762573 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:58.762485    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8p52g"
Jul 22 03:43:58.762573 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:58.762543    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8p52g"
Jul 22 03:43:58.775153 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:58.775123    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8p52g"
Jul 22 03:43:58.840816 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:43:58.840789    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8p52g"
Jul 22 03:44:00.156109 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:44:00.156080    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:45:27.376525 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:45:27.376446    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:45:36.760493 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:45:36.760456    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:45:49.162258 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:45:49.162224    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:47:31.363825 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:47:31.363788    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:47:40.463035 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:47:40.462997    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:47:54.761430 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:47:54.761389    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:48:05.962471 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:48:05.962435    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:48:15.266882 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:48:15.266849    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:48:23.063787 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:48:23.063751    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:48:32.780368 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:48:32.780291    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2p4qp_9f0defcc-ccff-43be-8206-54c772885080/console-operator/1.log"
Jul 22 03:48:32.781378 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:48:32.781360    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2p4qp_9f0defcc-ccff-43be-8206-54c772885080/console-operator/1.log"
Jul 22 03:48:33.458916 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:48:33.458878    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:48:40.962049 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:48:40.962018    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:48:51.666248 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:48:51.666212    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:49:01.860833 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:49:01.860798    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:49:12.560894 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:49:12.560862    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:49:21.663559 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:49:21.663523    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:49:42.976526 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:49:42.976484    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:49:54.961397 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:49:54.961319    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:50:18.467088 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:50:18.467054    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:50:27.161745 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:50:27.161708    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:50:45.469009 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:50:45.468967    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:50:57.465025 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:50:57.464995    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:51:11.961999 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:51:11.961956    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:53:11.661686 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:53:11.661651    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:53:21.263210 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:53:21.263179    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:53:32.808414 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:53:32.808388    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2p4qp_9f0defcc-ccff-43be-8206-54c772885080/console-operator/1.log"
Jul 22 03:53:32.809567 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:53:32.809541    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2p4qp_9f0defcc-ccff-43be-8206-54c772885080/console-operator/1.log"
Jul 22 03:53:34.119996 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:53:34.119964    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:53:47.366533 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:53:47.366495    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:53:57.155310 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:53:57.155266    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:54:06.665078 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:54:06.665043    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:54:15.964794 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:54:15.964759    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:54:26.965284 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:54:26.965209    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:54:38.030112 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:54:38.030075    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:54:46.563059 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:54:46.563023    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:54:57.379460 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:54:57.379423    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:55:19.072767 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:55:19.072731    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:55:28.281116 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:55:28.281079    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:55:33.391992 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:55:33.391957    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f"]
Jul 22 03:55:33.395080 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:55:33.395063    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f"
Jul 22 03:55:33.398948 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:55:33.398934    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-d5e0b9c5-kserve-self-signed-certs\""
Jul 22 03:55:33.422910 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:55:33.422877    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f"]
Jul 22 03:55:33.446149 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:55:33.446117    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c59b63ec-4c55-4435-9045-be5ebf763dd8-model-cache\") pod \"e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f\" (UID: \"c59b63ec-4c55-4435-9045-be5ebf763dd8\") " pod="llm/e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f"
Jul 22 03:55:33.446149 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:55:33.446151    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c59b63ec-4c55-4435-9045-be5ebf763dd8-dshm\") pod \"e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f\" (UID: \"c59b63ec-4c55-4435-9045-be5ebf763dd8\") " pod="llm/e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f"
Jul 22 03:55:33.446332 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:55:33.446174    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c59b63ec-4c55-4435-9045-be5ebf763dd8-home\") pod \"e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f\" (UID: \"c59b63ec-4c55-4435-9045-be5ebf763dd8\") " pod="llm/e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f"
Jul 22 03:55:33.446332 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:55:33.446265    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c59b63ec-4c55-4435-9045-be5ebf763dd8-tls-certs\") pod \"e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f\" (UID: \"c59b63ec-4c55-4435-9045-be5ebf763dd8\") " pod="llm/e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f"
Jul 22 03:55:33.446332 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:55:33.446291    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvnz5\" (UniqueName: \"kubernetes.io/projected/c59b63ec-4c55-4435-9045-be5ebf763dd8-kube-api-access-lvnz5\") pod \"e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f\" (UID: \"c59b63ec-4c55-4435-9045-be5ebf763dd8\") " pod="llm/e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f"
Jul 22 03:55:33.446332 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:55:33.446318    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c59b63ec-4c55-4435-9045-be5ebf763dd8-tmp-dir\") pod \"e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f\" (UID: \"c59b63ec-4c55-4435-9045-be5ebf763dd8\") " pod="llm/e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f"
Jul 22 03:55:33.547367 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:55:33.547332    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c59b63ec-4c55-4435-9045-be5ebf763dd8-model-cache\") pod \"e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f\" (UID: \"c59b63ec-4c55-4435-9045-be5ebf763dd8\") " pod="llm/e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f"
Jul 22 03:55:33.547367 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:55:33.547374    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c59b63ec-4c55-4435-9045-be5ebf763dd8-dshm\") pod \"e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f\" (UID: \"c59b63ec-4c55-4435-9045-be5ebf763dd8\") " pod="llm/e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f"
Jul 22 03:55:33.547603 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:55:33.547393    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c59b63ec-4c55-4435-9045-be5ebf763dd8-home\") pod \"e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f\" (UID: \"c59b63ec-4c55-4435-9045-be5ebf763dd8\") " pod="llm/e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f"
Jul 22 03:55:33.547603 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:55:33.547447    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c59b63ec-4c55-4435-9045-be5ebf763dd8-tls-certs\") pod \"e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f\" (UID: \"c59b63ec-4c55-4435-9045-be5ebf763dd8\") " pod="llm/e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f"
Jul 22 03:55:33.547603 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:55:33.547467    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvnz5\" (UniqueName: \"kubernetes.io/projected/c59b63ec-4c55-4435-9045-be5ebf763dd8-kube-api-access-lvnz5\") pod \"e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f\" (UID: \"c59b63ec-4c55-4435-9045-be5ebf763dd8\") " pod="llm/e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f"
Jul 22 03:55:33.547603 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:55:33.547496    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c59b63ec-4c55-4435-9045-be5ebf763dd8-tmp-dir\") pod \"e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f\" (UID: \"c59b63ec-4c55-4435-9045-be5ebf763dd8\") " pod="llm/e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f"
Jul 22 03:55:33.547897 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:55:33.547820    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c59b63ec-4c55-4435-9045-be5ebf763dd8-model-cache\") pod \"e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f\" (UID: \"c59b63ec-4c55-4435-9045-be5ebf763dd8\") " pod="llm/e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f"
Jul 22 03:55:33.547897 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:55:33.547854    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c59b63ec-4c55-4435-9045-be5ebf763dd8-home\") pod \"e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f\" (UID: \"c59b63ec-4c55-4435-9045-be5ebf763dd8\") " pod="llm/e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f"
Jul 22 03:55:33.548009 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:55:33.547903    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c59b63ec-4c55-4435-9045-be5ebf763dd8-tmp-dir\") pod \"e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f\" (UID: \"c59b63ec-4c55-4435-9045-be5ebf763dd8\") " pod="llm/e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f"
Jul 22 03:55:33.550019 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:55:33.550001    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c59b63ec-4c55-4435-9045-be5ebf763dd8-dshm\") pod \"e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f\" (UID: \"c59b63ec-4c55-4435-9045-be5ebf763dd8\") " pod="llm/e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f"
Jul 22 03:55:33.550068 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:55:33.550046    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c59b63ec-4c55-4435-9045-be5ebf763dd8-tls-certs\") pod \"e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f\" (UID: \"c59b63ec-4c55-4435-9045-be5ebf763dd8\") " pod="llm/e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f"
Jul 22 03:55:33.556547 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:55:33.556528    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvnz5\" (UniqueName: \"kubernetes.io/projected/c59b63ec-4c55-4435-9045-be5ebf763dd8-kube-api-access-lvnz5\") pod \"e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f\" (UID: \"c59b63ec-4c55-4435-9045-be5ebf763dd8\") " pod="llm/e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f"
Jul 22 03:55:33.710101 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:55:33.710042    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f"
Jul 22 03:55:33.877964 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:55:33.877929    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f"]
Jul 22 03:55:33.885770 ip-10-0-132-89 kubenswrapper[2570]: W0722 03:55:33.885744    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc59b63ec_4c55_4435_9045_be5ebf763dd8.slice/crio-8db4f6bbe7d78783cbb9faba6eeb216b99c42d47598974c131f83bc3ca5ff832 WatchSource:0}: Error finding container 8db4f6bbe7d78783cbb9faba6eeb216b99c42d47598974c131f83bc3ca5ff832: Status 404 returned error can't find the container with id 8db4f6bbe7d78783cbb9faba6eeb216b99c42d47598974c131f83bc3ca5ff832
Jul 22 03:55:33.984168 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:55:33.984079    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f" event={"ID":"c59b63ec-4c55-4435-9045-be5ebf763dd8","Type":"ContainerStarted","Data":"9d38b9cb5b5cecb201c1fb69df0c90fe531bea0cbe7c769e11ac4e22f845b5c9"}
Jul 22 03:55:33.984168 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:55:33.984122    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f" event={"ID":"c59b63ec-4c55-4435-9045-be5ebf763dd8","Type":"ContainerStarted","Data":"8db4f6bbe7d78783cbb9faba6eeb216b99c42d47598974c131f83bc3ca5ff832"}
Jul 22 03:55:34.014513 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:55:34.014459    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f" podStartSLOduration=1.014260501 podStartE2EDuration="1.014260501s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:55:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:55:34.013383824 +0000 UTC m=+1321.631593830" watchObservedRunningTime="2026-07-22 03:55:34.014260501 +0000 UTC m=+1321.632470508"
Jul 22 03:55:43.710404 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:55:43.710307    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f"
Jul 22 03:55:43.710404 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:55:43.710359    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f"
Jul 22 03:55:43.723044 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:55:43.723016    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f"
Jul 22 03:55:44.039666 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:55:44.039548    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f"
Jul 22 03:55:44.074327 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:55:44.074290    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:55:59.721322 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:55:59.721241    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:56:12.556122 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:56:12.556085    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f"]
Jul 22 03:56:16.089803 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:56:16.089765    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:56:27.577073 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:56:27.577006    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f" podUID="c59b63ec-4c55-4435-9045-be5ebf763dd8" containerName="main" containerID="cri-o://9d38b9cb5b5cecb201c1fb69df0c90fe531bea0cbe7c769e11ac4e22f845b5c9" gracePeriod=45
Jul 22 03:56:27.842152 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:56:27.842081    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f"
Jul 22 03:56:27.937580 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:56:27.937541    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c59b63ec-4c55-4435-9045-be5ebf763dd8-model-cache\") pod \"c59b63ec-4c55-4435-9045-be5ebf763dd8\" (UID: \"c59b63ec-4c55-4435-9045-be5ebf763dd8\") "
Jul 22 03:56:27.937812 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:56:27.937595    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c59b63ec-4c55-4435-9045-be5ebf763dd8-home\") pod \"c59b63ec-4c55-4435-9045-be5ebf763dd8\" (UID: \"c59b63ec-4c55-4435-9045-be5ebf763dd8\") "
Jul 22 03:56:27.937812 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:56:27.937654    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c59b63ec-4c55-4435-9045-be5ebf763dd8-tmp-dir\") pod \"c59b63ec-4c55-4435-9045-be5ebf763dd8\" (UID: \"c59b63ec-4c55-4435-9045-be5ebf763dd8\") "
Jul 22 03:56:27.937812 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:56:27.937720    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lvnz5\" (UniqueName: \"kubernetes.io/projected/c59b63ec-4c55-4435-9045-be5ebf763dd8-kube-api-access-lvnz5\") pod \"c59b63ec-4c55-4435-9045-be5ebf763dd8\" (UID: \"c59b63ec-4c55-4435-9045-be5ebf763dd8\") "
Jul 22 03:56:27.937812 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:56:27.937775    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c59b63ec-4c55-4435-9045-be5ebf763dd8-tls-certs\") pod \"c59b63ec-4c55-4435-9045-be5ebf763dd8\" (UID: \"c59b63ec-4c55-4435-9045-be5ebf763dd8\") "
Jul 22 03:56:27.937812 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:56:27.937810    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c59b63ec-4c55-4435-9045-be5ebf763dd8-dshm\") pod \"c59b63ec-4c55-4435-9045-be5ebf763dd8\" (UID: \"c59b63ec-4c55-4435-9045-be5ebf763dd8\") "
Jul 22 03:56:27.938055 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:56:27.937911    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c59b63ec-4c55-4435-9045-be5ebf763dd8-home" (OuterVolumeSpecName: "home") pod "c59b63ec-4c55-4435-9045-be5ebf763dd8" (UID: "c59b63ec-4c55-4435-9045-be5ebf763dd8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 03:56:27.938055 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:56:27.937917    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c59b63ec-4c55-4435-9045-be5ebf763dd8-model-cache" (OuterVolumeSpecName: "model-cache") pod "c59b63ec-4c55-4435-9045-be5ebf763dd8" (UID: "c59b63ec-4c55-4435-9045-be5ebf763dd8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 03:56:27.938055 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:56:27.937994    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c59b63ec-4c55-4435-9045-be5ebf763dd8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c59b63ec-4c55-4435-9045-be5ebf763dd8" (UID: "c59b63ec-4c55-4435-9045-be5ebf763dd8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 03:56:27.938225 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:56:27.938204    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c59b63ec-4c55-4435-9045-be5ebf763dd8-model-cache\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 03:56:27.938225 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:56:27.938223    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c59b63ec-4c55-4435-9045-be5ebf763dd8-home\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 03:56:27.938339 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:56:27.938231    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c59b63ec-4c55-4435-9045-be5ebf763dd8-tmp-dir\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 03:56:27.940354 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:56:27.940315    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c59b63ec-4c55-4435-9045-be5ebf763dd8-dshm" (OuterVolumeSpecName: "dshm") pod "c59b63ec-4c55-4435-9045-be5ebf763dd8" (UID: "c59b63ec-4c55-4435-9045-be5ebf763dd8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 03:56:27.940493 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:56:27.940357    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c59b63ec-4c55-4435-9045-be5ebf763dd8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c59b63ec-4c55-4435-9045-be5ebf763dd8" (UID: "c59b63ec-4c55-4435-9045-be5ebf763dd8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 03:56:27.940493 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:56:27.940377    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c59b63ec-4c55-4435-9045-be5ebf763dd8-kube-api-access-lvnz5" (OuterVolumeSpecName: "kube-api-access-lvnz5") pod "c59b63ec-4c55-4435-9045-be5ebf763dd8" (UID: "c59b63ec-4c55-4435-9045-be5ebf763dd8"). InnerVolumeSpecName "kube-api-access-lvnz5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 03:56:28.038743 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:56:28.038704    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lvnz5\" (UniqueName: \"kubernetes.io/projected/c59b63ec-4c55-4435-9045-be5ebf763dd8-kube-api-access-lvnz5\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 03:56:28.038743 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:56:28.038737    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c59b63ec-4c55-4435-9045-be5ebf763dd8-tls-certs\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 03:56:28.038743 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:56:28.038750    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c59b63ec-4c55-4435-9045-be5ebf763dd8-dshm\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 03:56:28.232901 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:56:28.232866    2570 generic.go:356] "Generic (PLEG): container finished" podID="c59b63ec-4c55-4435-9045-be5ebf763dd8" containerID="9d38b9cb5b5cecb201c1fb69df0c90fe531bea0cbe7c769e11ac4e22f845b5c9" exitCode=0
Jul 22 03:56:28.233071 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:56:28.232910    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f"
Jul 22 03:56:28.233071 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:56:28.232946    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f" event={"ID":"c59b63ec-4c55-4435-9045-be5ebf763dd8","Type":"ContainerDied","Data":"9d38b9cb5b5cecb201c1fb69df0c90fe531bea0cbe7c769e11ac4e22f845b5c9"}
Jul 22 03:56:28.233071 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:56:28.232984    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f" event={"ID":"c59b63ec-4c55-4435-9045-be5ebf763dd8","Type":"ContainerDied","Data":"8db4f6bbe7d78783cbb9faba6eeb216b99c42d47598974c131f83bc3ca5ff832"}
Jul 22 03:56:28.233071 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:56:28.233001    2570 scope.go:117] "RemoveContainer" containerID="9d38b9cb5b5cecb201c1fb69df0c90fe531bea0cbe7c769e11ac4e22f845b5c9"
Jul 22 03:56:28.241830 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:56:28.241812    2570 scope.go:117] "RemoveContainer" containerID="9d38b9cb5b5cecb201c1fb69df0c90fe531bea0cbe7c769e11ac4e22f845b5c9"
Jul 22 03:56:28.242086 ip-10-0-132-89 kubenswrapper[2570]: E0722 03:56:28.242067    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d38b9cb5b5cecb201c1fb69df0c90fe531bea0cbe7c769e11ac4e22f845b5c9\": container with ID starting with 9d38b9cb5b5cecb201c1fb69df0c90fe531bea0cbe7c769e11ac4e22f845b5c9 not found: ID does not exist" containerID="9d38b9cb5b5cecb201c1fb69df0c90fe531bea0cbe7c769e11ac4e22f845b5c9"
Jul 22 03:56:28.242140 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:56:28.242097    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d38b9cb5b5cecb201c1fb69df0c90fe531bea0cbe7c769e11ac4e22f845b5c9"} err="failed to get container status \"9d38b9cb5b5cecb201c1fb69df0c90fe531bea0cbe7c769e11ac4e22f845b5c9\": rpc error: code = NotFound desc = could not find container \"9d38b9cb5b5cecb201c1fb69df0c90fe531bea0cbe7c769e11ac4e22f845b5c9\": container with ID starting with 9d38b9cb5b5cecb201c1fb69df0c90fe531bea0cbe7c769e11ac4e22f845b5c9 not found: ID does not exist"
Jul 22 03:56:28.254685 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:56:28.254661    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f"]
Jul 22 03:56:28.262397 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:56:28.262374    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-d5e0b9c5-kserve-75f5dcffd7-llw7f"]
Jul 22 03:56:28.852091 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:56:28.852057    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c59b63ec-4c55-4435-9045-be5ebf763dd8" path="/var/lib/kubelet/pods/c59b63ec-4c55-4435-9045-be5ebf763dd8/volumes"
Jul 22 03:56:31.220527 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:56:31.220497    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:56:41.281925 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:56:41.281893    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:56:53.699327 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:56:53.699286    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:57:08.912096 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:57:08.912057    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:57:19.199762 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:57:19.199724    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:57:32.897146 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:57:32.897074    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:57:42.827796 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:57:42.827762    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:57:53.589480 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:57:53.589443    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:58:10.667487 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:58:10.667444    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:58:20.171091 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:58:20.171050    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:58:29.269604 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:58:29.269571    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:58:32.834108 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:58:32.834075    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2p4qp_9f0defcc-ccff-43be-8206-54c772885080/console-operator/1.log"
Jul 22 03:58:32.836419 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:58:32.836398    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2p4qp_9f0defcc-ccff-43be-8206-54c772885080/console-operator/1.log"
Jul 22 03:58:41.011640 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:58:41.011585    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:58:58.880397 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:58:58.880351    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:59:10.575238 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:59:10.575201    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:59:23.768806 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:59:23.768763    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:59:34.968033 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:59:34.967989    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:59:45.972639 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:59:45.972576    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 03:59:57.513235 ip-10-0-132-89 kubenswrapper[2570]: I0722 03:59:57.513198    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:00:14.571115 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:00:14.571079    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:00:46.270900 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:00:46.270813    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:00:57.667420 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:00:57.667388    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:01:29.967754 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:01:29.967720    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:01:40.665768 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:01:40.665733    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:02:00.471022 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:00.470940    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1bed135-75p22"]
Jul 22 04:02:00.471400 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:00.471303    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c59b63ec-4c55-4435-9045-be5ebf763dd8" containerName="main"
Jul 22 04:02:00.471400 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:00.471319    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="c59b63ec-4c55-4435-9045-be5ebf763dd8" containerName="main"
Jul 22 04:02:00.471400 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:00.471387    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="c59b63ec-4c55-4435-9045-be5ebf763dd8" containerName="main"
Jul 22 04:02:00.474975 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:00.474960    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1bed135-75p22"
Jul 22 04:02:00.477743 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:00.477718    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rbfm6\""
Jul 22 04:02:00.477886 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:00.477747    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 04:02:00.481702 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:00.481673    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1bed135-75p22"]
Jul 22 04:02:00.543105 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:00.543066    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/291b918f-e5a8-46e0-a15e-2ab206dd3843-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a1bed135-75p22\" (UID: \"291b918f-e5a8-46e0-a15e-2ab206dd3843\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1bed135-75p22"
Jul 22 04:02:00.643828 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:00.643784    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/291b918f-e5a8-46e0-a15e-2ab206dd3843-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a1bed135-75p22\" (UID: \"291b918f-e5a8-46e0-a15e-2ab206dd3843\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1bed135-75p22"
Jul 22 04:02:00.644386 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:00.644367    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/291b918f-e5a8-46e0-a15e-2ab206dd3843-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a1bed135-75p22\" (UID: \"291b918f-e5a8-46e0-a15e-2ab206dd3843\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1bed135-75p22"
Jul 22 04:02:00.786505 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:00.786419    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1bed135-75p22"
Jul 22 04:02:00.920342 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:00.920319    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1bed135-75p22"]
Jul 22 04:02:00.922708 ip-10-0-132-89 kubenswrapper[2570]: W0722 04:02:00.922682    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod291b918f_e5a8_46e0_a15e_2ab206dd3843.slice/crio-c16e98a7ed6314db39c625c0da2a7987a297fe28afb93293200513ae567bd729 WatchSource:0}: Error finding container c16e98a7ed6314db39c625c0da2a7987a297fe28afb93293200513ae567bd729: Status 404 returned error can't find the container with id c16e98a7ed6314db39c625c0da2a7987a297fe28afb93293200513ae567bd729
Jul 22 04:02:00.924554 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:00.924538    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 04:02:01.740915 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:01.740876    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1bed135-75p22" event={"ID":"291b918f-e5a8-46e0-a15e-2ab206dd3843","Type":"ContainerStarted","Data":"c16e98a7ed6314db39c625c0da2a7987a297fe28afb93293200513ae567bd729"}
Jul 22 04:02:03.753445 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:03.753406    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1bed135-75p22" event={"ID":"291b918f-e5a8-46e0-a15e-2ab206dd3843","Type":"ContainerStarted","Data":"a4a867858e072cc4e2f45656d5b503d1832a6d572841620863b78cbfb9aebf14"}
Jul 22 04:02:03.777365 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:03.777315    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1bed135-75p22" podStartSLOduration=1.5870912910000001 podStartE2EDuration="3.777299355s" totalImagesPullingTime="2.190208064s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:02:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:02:03.77479699 +0000 UTC m=+1711.393006997" watchObservedRunningTime="2026-07-22 04:02:03.777299355 +0000 UTC m=+1711.395509361"
Jul 22 04:02:04.759942 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:04.759907    2570 generic.go:356] "Generic (PLEG): container finished" podID="291b918f-e5a8-46e0-a15e-2ab206dd3843" containerID="a4a867858e072cc4e2f45656d5b503d1832a6d572841620863b78cbfb9aebf14" exitCode=0
Jul 22 04:02:04.760365 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:04.759997    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1bed135-75p22" event={"ID":"291b918f-e5a8-46e0-a15e-2ab206dd3843","Type":"ContainerDied","Data":"a4a867858e072cc4e2f45656d5b503d1832a6d572841620863b78cbfb9aebf14"}
Jul 22 04:02:05.898243 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:05.898217    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1bed135-75p22"
Jul 22 04:02:05.994937 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:05.994896    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/291b918f-e5a8-46e0-a15e-2ab206dd3843-maas-api-service-ca\") pod \"291b918f-e5a8-46e0-a15e-2ab206dd3843\" (UID: \"291b918f-e5a8-46e0-a15e-2ab206dd3843\") "
Jul 22 04:02:05.995263 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:05.995240    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/291b918f-e5a8-46e0-a15e-2ab206dd3843-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "291b918f-e5a8-46e0-a15e-2ab206dd3843" (UID: "291b918f-e5a8-46e0-a15e-2ab206dd3843"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 04:02:06.095764 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:06.095681    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/291b918f-e5a8-46e0-a15e-2ab206dd3843-maas-api-service-ca\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:02:06.770546 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:06.770512    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1bed135-75p22" event={"ID":"291b918f-e5a8-46e0-a15e-2ab206dd3843","Type":"ContainerDied","Data":"c16e98a7ed6314db39c625c0da2a7987a297fe28afb93293200513ae567bd729"}
Jul 22 04:02:06.770546 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:06.770532    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1bed135-75p22"
Jul 22 04:02:06.770779 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:06.770559    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c16e98a7ed6314db39c625c0da2a7987a297fe28afb93293200513ae567bd729"
Jul 22 04:02:13.562795 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:13.562755    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:02:15.559431 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:15.559397    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1bed135-75p22"]
Jul 22 04:02:15.562098 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:15.562075    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a1bed135-75p22"]
Jul 22 04:02:16.851916 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:16.851880    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="291b918f-e5a8-46e0-a15e-2ab206dd3843" path="/var/lib/kubelet/pods/291b918f-e5a8-46e0-a15e-2ab206dd3843/volumes"
Jul 22 04:02:27.470734 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:27.470694    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:02:44.725905 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:44.725872    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9593c364-l4mqz"]
Jul 22 04:02:44.726371 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:44.726208    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="291b918f-e5a8-46e0-a15e-2ab206dd3843" containerName="revoke-keys"
Jul 22 04:02:44.726371 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:44.726219    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="291b918f-e5a8-46e0-a15e-2ab206dd3843" containerName="revoke-keys"
Jul 22 04:02:44.726371 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:44.726276    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="291b918f-e5a8-46e0-a15e-2ab206dd3843" containerName="revoke-keys"
Jul 22 04:02:44.730079 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:44.730063    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9593c364-l4mqz"
Jul 22 04:02:44.732549 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:44.732512    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 04:02:44.732700 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:44.732593    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rbfm6\""
Jul 22 04:02:44.735984 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:44.735950    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9593c364-l4mqz"]
Jul 22 04:02:44.841266 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:44.841220    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f0fff4d-d200-4fb5-84af-3d88031a8514-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-9593c364-l4mqz\" (UID: \"5f0fff4d-d200-4fb5-84af-3d88031a8514\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9593c364-l4mqz"
Jul 22 04:02:44.942247 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:44.942191    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f0fff4d-d200-4fb5-84af-3d88031a8514-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-9593c364-l4mqz\" (UID: \"5f0fff4d-d200-4fb5-84af-3d88031a8514\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9593c364-l4mqz"
Jul 22 04:02:44.942846 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:44.942825    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f0fff4d-d200-4fb5-84af-3d88031a8514-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-9593c364-l4mqz\" (UID: \"5f0fff4d-d200-4fb5-84af-3d88031a8514\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9593c364-l4mqz"
Jul 22 04:02:45.042149 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:45.042059    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9593c364-l4mqz"
Jul 22 04:02:45.176785 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:45.176761    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9593c364-l4mqz"]
Jul 22 04:02:45.179452 ip-10-0-132-89 kubenswrapper[2570]: W0722 04:02:45.179423    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f0fff4d_d200_4fb5_84af_3d88031a8514.slice/crio-d23ef8630e739f3664020b4d877db730c845b68f165874b86474f317bec6b0d8 WatchSource:0}: Error finding container d23ef8630e739f3664020b4d877db730c845b68f165874b86474f317bec6b0d8: Status 404 returned error can't find the container with id d23ef8630e739f3664020b4d877db730c845b68f165874b86474f317bec6b0d8
Jul 22 04:02:45.947283 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:45.947246    2570 generic.go:356] "Generic (PLEG): container finished" podID="5f0fff4d-d200-4fb5-84af-3d88031a8514" containerID="75e3ffd35f28c6fae075fdb02d62623c0619134630a272b335bcaae07529b322" exitCode=0
Jul 22 04:02:45.947656 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:45.947335    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9593c364-l4mqz" event={"ID":"5f0fff4d-d200-4fb5-84af-3d88031a8514","Type":"ContainerDied","Data":"75e3ffd35f28c6fae075fdb02d62623c0619134630a272b335bcaae07529b322"}
Jul 22 04:02:45.947656 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:45.947384    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9593c364-l4mqz" event={"ID":"5f0fff4d-d200-4fb5-84af-3d88031a8514","Type":"ContainerStarted","Data":"d23ef8630e739f3664020b4d877db730c845b68f165874b86474f317bec6b0d8"}
Jul 22 04:02:47.085058 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:47.085031    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9593c364-l4mqz"
Jul 22 04:02:47.163143 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:47.163113    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f0fff4d-d200-4fb5-84af-3d88031a8514-maas-api-service-ca\") pod \"5f0fff4d-d200-4fb5-84af-3d88031a8514\" (UID: \"5f0fff4d-d200-4fb5-84af-3d88031a8514\") "
Jul 22 04:02:47.163511 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:47.163488    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f0fff4d-d200-4fb5-84af-3d88031a8514-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5f0fff4d-d200-4fb5-84af-3d88031a8514" (UID: "5f0fff4d-d200-4fb5-84af-3d88031a8514"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 04:02:47.264416 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:47.264335    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f0fff4d-d200-4fb5-84af-3d88031a8514-maas-api-service-ca\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:02:47.956923 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:47.956885    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9593c364-l4mqz" event={"ID":"5f0fff4d-d200-4fb5-84af-3d88031a8514","Type":"ContainerDied","Data":"d23ef8630e739f3664020b4d877db730c845b68f165874b86474f317bec6b0d8"}
Jul 22 04:02:47.956923 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:47.956908    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9593c364-l4mqz"
Jul 22 04:02:47.957133 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:47.956948    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d23ef8630e739f3664020b4d877db730c845b68f165874b86474f317bec6b0d8"
Jul 22 04:02:58.382863 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:58.382832    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:02:59.835170 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:59.835135    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9593c364-l4mqz"]
Jul 22 04:02:59.837737 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:02:59.837713    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-9593c364-l4mqz"]
Jul 22 04:03:00.853839 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:03:00.853805    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f0fff4d-d200-4fb5-84af-3d88031a8514" path="/var/lib/kubelet/pods/5f0fff4d-d200-4fb5-84af-3d88031a8514/volumes"
Jul 22 04:03:32.862568 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:03:32.862530    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2p4qp_9f0defcc-ccff-43be-8206-54c772885080/console-operator/1.log"
Jul 22 04:03:32.865148 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:03:32.865128    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2p4qp_9f0defcc-ccff-43be-8206-54c772885080/console-operator/1.log"
Jul 22 04:04:26.573349 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:04:26.573312    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:04:37.273262 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:04:37.273226    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:04:52.064927 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:04:52.064891    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:04:59.272269 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:04:59.272232    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9cb7aec7-dl7bg"]
Jul 22 04:04:59.272802 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:04:59.272579    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f0fff4d-d200-4fb5-84af-3d88031a8514" containerName="revoke-keys"
Jul 22 04:04:59.272802 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:04:59.272590    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f0fff4d-d200-4fb5-84af-3d88031a8514" containerName="revoke-keys"
Jul 22 04:04:59.272802 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:04:59.272676    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f0fff4d-d200-4fb5-84af-3d88031a8514" containerName="revoke-keys"
Jul 22 04:04:59.274517 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:04:59.274502    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9cb7aec7-dl7bg"
Jul 22 04:04:59.277191 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:04:59.277172    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 04:04:59.277267 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:04:59.277249    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rbfm6\""
Jul 22 04:04:59.283970 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:04:59.283935    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9cb7aec7-dl7bg"]
Jul 22 04:04:59.319993 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:04:59.319953    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/575e9acc-78d4-438a-a0a7-a8958cdb6ed0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-9cb7aec7-dl7bg\" (UID: \"575e9acc-78d4-438a-a0a7-a8958cdb6ed0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9cb7aec7-dl7bg"
Jul 22 04:04:59.420922 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:04:59.420887    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/575e9acc-78d4-438a-a0a7-a8958cdb6ed0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-9cb7aec7-dl7bg\" (UID: \"575e9acc-78d4-438a-a0a7-a8958cdb6ed0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9cb7aec7-dl7bg"
Jul 22 04:04:59.421489 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:04:59.421470    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/575e9acc-78d4-438a-a0a7-a8958cdb6ed0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-9cb7aec7-dl7bg\" (UID: \"575e9acc-78d4-438a-a0a7-a8958cdb6ed0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9cb7aec7-dl7bg"
Jul 22 04:04:59.586208 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:04:59.586128    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9cb7aec7-dl7bg"
Jul 22 04:04:59.718599 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:04:59.718469    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9cb7aec7-dl7bg"]
Jul 22 04:04:59.721645 ip-10-0-132-89 kubenswrapper[2570]: W0722 04:04:59.721600    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod575e9acc_78d4_438a_a0a7_a8958cdb6ed0.slice/crio-7eea27c5dbb65651177697a97486f3182d8801514a1491861b49986e889df6b8 WatchSource:0}: Error finding container 7eea27c5dbb65651177697a97486f3182d8801514a1491861b49986e889df6b8: Status 404 returned error can't find the container with id 7eea27c5dbb65651177697a97486f3182d8801514a1491861b49986e889df6b8
Jul 22 04:05:00.575447 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:05:00.575407    2570 generic.go:356] "Generic (PLEG): container finished" podID="575e9acc-78d4-438a-a0a7-a8958cdb6ed0" containerID="a0b313738a3fe08e4aee1968f30e06ca82b8ad3f5a36e7267633da3d32f7bd3e" exitCode=0
Jul 22 04:05:00.575855 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:05:00.575490    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9cb7aec7-dl7bg" event={"ID":"575e9acc-78d4-438a-a0a7-a8958cdb6ed0","Type":"ContainerDied","Data":"a0b313738a3fe08e4aee1968f30e06ca82b8ad3f5a36e7267633da3d32f7bd3e"}
Jul 22 04:05:00.575855 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:05:00.575537    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9cb7aec7-dl7bg" event={"ID":"575e9acc-78d4-438a-a0a7-a8958cdb6ed0","Type":"ContainerStarted","Data":"7eea27c5dbb65651177697a97486f3182d8801514a1491861b49986e889df6b8"}
Jul 22 04:05:01.717354 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:05:01.717330    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9cb7aec7-dl7bg"
Jul 22 04:05:01.739392 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:05:01.739363    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/575e9acc-78d4-438a-a0a7-a8958cdb6ed0-maas-api-service-ca\") pod \"575e9acc-78d4-438a-a0a7-a8958cdb6ed0\" (UID: \"575e9acc-78d4-438a-a0a7-a8958cdb6ed0\") "
Jul 22 04:05:01.739801 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:05:01.739773    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/575e9acc-78d4-438a-a0a7-a8958cdb6ed0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "575e9acc-78d4-438a-a0a7-a8958cdb6ed0" (UID: "575e9acc-78d4-438a-a0a7-a8958cdb6ed0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 04:05:01.840543 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:05:01.840465    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/575e9acc-78d4-438a-a0a7-a8958cdb6ed0-maas-api-service-ca\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:05:02.586937 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:05:02.586902    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9cb7aec7-dl7bg"
Jul 22 04:05:02.587112 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:05:02.586946    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9cb7aec7-dl7bg" event={"ID":"575e9acc-78d4-438a-a0a7-a8958cdb6ed0","Type":"ContainerDied","Data":"7eea27c5dbb65651177697a97486f3182d8801514a1491861b49986e889df6b8"}
Jul 22 04:05:02.587112 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:05:02.586995    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7eea27c5dbb65651177697a97486f3182d8801514a1491861b49986e889df6b8"
Jul 22 04:05:12.663015 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:05:12.662978    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:05:14.357172 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:05:14.357130    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9cb7aec7-dl7bg"]
Jul 22 04:05:14.359182 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:05:14.359156    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-9cb7aec7-dl7bg"]
Jul 22 04:05:14.852233 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:05:14.852196    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="575e9acc-78d4-438a-a0a7-a8958cdb6ed0" path="/var/lib/kubelet/pods/575e9acc-78d4-438a-a0a7-a8958cdb6ed0/volumes"
Jul 22 04:05:48.564535 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:05:48.564500    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:06:01.962465 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:01.962428    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-2f6647b0/e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd"]
Jul 22 04:06:01.963042 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:01.962965    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="575e9acc-78d4-438a-a0a7-a8958cdb6ed0" containerName="revoke-keys"
Jul 22 04:06:01.963042 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:01.962986    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="575e9acc-78d4-438a-a0a7-a8958cdb6ed0" containerName="revoke-keys"
Jul 22 04:06:01.963157 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:01.963082    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="575e9acc-78d4-438a-a0a7-a8958cdb6ed0" containerName="revoke-keys"
Jul 22 04:06:01.964933 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:01.964911    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2f6647b0/e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd"
Jul 22 04:06:01.969144 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:01.969027    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-2f6647b0\"/\"openshift-service-ca.crt\""
Jul 22 04:06:01.969144 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:01.969048    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-2f6647b0\"/\"e2e-lifecycle-model-2f6647b0-kserve-self-signed-certs\""
Jul 22 04:06:01.969144 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:01.969062    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-2f6647b0\"/\"default-dockercfg-qcxhc\""
Jul 22 04:06:01.969144 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:01.969078    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-2f6647b0\"/\"kube-root-ca.crt\""
Jul 22 04:06:01.975165 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:01.975143    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-2f6647b0/e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd"]
Jul 22 04:06:02.078800 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:02.078764    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38d62c67-d484-4113-a278-9055733ce7cc-home\") pod \"e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd\" (UID: \"38d62c67-d484-4113-a278-9055733ce7cc\") " pod="ai-tenant-e2e-mt-2f6647b0/e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd"
Jul 22 04:06:02.078800 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:02.078806    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38d62c67-d484-4113-a278-9055733ce7cc-tmp-dir\") pod \"e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd\" (UID: \"38d62c67-d484-4113-a278-9055733ce7cc\") " pod="ai-tenant-e2e-mt-2f6647b0/e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd"
Jul 22 04:06:02.079108 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:02.078829    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38d62c67-d484-4113-a278-9055733ce7cc-tls-certs\") pod \"e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd\" (UID: \"38d62c67-d484-4113-a278-9055733ce7cc\") " pod="ai-tenant-e2e-mt-2f6647b0/e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd"
Jul 22 04:06:02.079108 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:02.078875    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh9hm\" (UniqueName: \"kubernetes.io/projected/38d62c67-d484-4113-a278-9055733ce7cc-kube-api-access-xh9hm\") pod \"e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd\" (UID: \"38d62c67-d484-4113-a278-9055733ce7cc\") " pod="ai-tenant-e2e-mt-2f6647b0/e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd"
Jul 22 04:06:02.079108 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:02.078944    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38d62c67-d484-4113-a278-9055733ce7cc-dshm\") pod \"e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd\" (UID: \"38d62c67-d484-4113-a278-9055733ce7cc\") " pod="ai-tenant-e2e-mt-2f6647b0/e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd"
Jul 22 04:06:02.079108 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:02.078974    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38d62c67-d484-4113-a278-9055733ce7cc-model-cache\") pod \"e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd\" (UID: \"38d62c67-d484-4113-a278-9055733ce7cc\") " pod="ai-tenant-e2e-mt-2f6647b0/e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd"
Jul 22 04:06:02.179675 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:02.179607    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38d62c67-d484-4113-a278-9055733ce7cc-tmp-dir\") pod \"e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd\" (UID: \"38d62c67-d484-4113-a278-9055733ce7cc\") " pod="ai-tenant-e2e-mt-2f6647b0/e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd"
Jul 22 04:06:02.179675 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:02.179671    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38d62c67-d484-4113-a278-9055733ce7cc-tls-certs\") pod \"e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd\" (UID: \"38d62c67-d484-4113-a278-9055733ce7cc\") " pod="ai-tenant-e2e-mt-2f6647b0/e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd"
Jul 22 04:06:02.179912 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:02.179688    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh9hm\" (UniqueName: \"kubernetes.io/projected/38d62c67-d484-4113-a278-9055733ce7cc-kube-api-access-xh9hm\") pod \"e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd\" (UID: \"38d62c67-d484-4113-a278-9055733ce7cc\") " pod="ai-tenant-e2e-mt-2f6647b0/e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd"
Jul 22 04:06:02.179912 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:02.179724    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38d62c67-d484-4113-a278-9055733ce7cc-dshm\") pod \"e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd\" (UID: \"38d62c67-d484-4113-a278-9055733ce7cc\") " pod="ai-tenant-e2e-mt-2f6647b0/e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd"
Jul 22 04:06:02.179912 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:02.179739    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38d62c67-d484-4113-a278-9055733ce7cc-model-cache\") pod \"e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd\" (UID: \"38d62c67-d484-4113-a278-9055733ce7cc\") " pod="ai-tenant-e2e-mt-2f6647b0/e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd"
Jul 22 04:06:02.179912 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:02.179892    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38d62c67-d484-4113-a278-9055733ce7cc-home\") pod \"e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd\" (UID: \"38d62c67-d484-4113-a278-9055733ce7cc\") " pod="ai-tenant-e2e-mt-2f6647b0/e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd"
Jul 22 04:06:02.180121 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:02.180081    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38d62c67-d484-4113-a278-9055733ce7cc-tmp-dir\") pod \"e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd\" (UID: \"38d62c67-d484-4113-a278-9055733ce7cc\") " pod="ai-tenant-e2e-mt-2f6647b0/e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd"
Jul 22 04:06:02.180121 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:02.180107    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38d62c67-d484-4113-a278-9055733ce7cc-model-cache\") pod \"e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd\" (UID: \"38d62c67-d484-4113-a278-9055733ce7cc\") " pod="ai-tenant-e2e-mt-2f6647b0/e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd"
Jul 22 04:06:02.180193 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:02.180179    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38d62c67-d484-4113-a278-9055733ce7cc-home\") pod \"e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd\" (UID: \"38d62c67-d484-4113-a278-9055733ce7cc\") " pod="ai-tenant-e2e-mt-2f6647b0/e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd"
Jul 22 04:06:02.182152 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:02.182132    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38d62c67-d484-4113-a278-9055733ce7cc-dshm\") pod \"e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd\" (UID: \"38d62c67-d484-4113-a278-9055733ce7cc\") " pod="ai-tenant-e2e-mt-2f6647b0/e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd"
Jul 22 04:06:02.182313 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:02.182297    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38d62c67-d484-4113-a278-9055733ce7cc-tls-certs\") pod \"e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd\" (UID: \"38d62c67-d484-4113-a278-9055733ce7cc\") " pod="ai-tenant-e2e-mt-2f6647b0/e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd"
Jul 22 04:06:02.189025 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:02.189000    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh9hm\" (UniqueName: \"kubernetes.io/projected/38d62c67-d484-4113-a278-9055733ce7cc-kube-api-access-xh9hm\") pod \"e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd\" (UID: \"38d62c67-d484-4113-a278-9055733ce7cc\") " pod="ai-tenant-e2e-mt-2f6647b0/e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd"
Jul 22 04:06:02.275220 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:02.275137    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2f6647b0/e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd"
Jul 22 04:06:02.413752 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:02.413727    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-2f6647b0/e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd"]
Jul 22 04:06:02.416270 ip-10-0-132-89 kubenswrapper[2570]: W0722 04:06:02.416237    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod38d62c67_d484_4113_a278_9055733ce7cc.slice/crio-1efeebacab59fe0c33f24ca43a23e1cb5476dbc20b5749812d022fe7fc5a50f1 WatchSource:0}: Error finding container 1efeebacab59fe0c33f24ca43a23e1cb5476dbc20b5749812d022fe7fc5a50f1: Status 404 returned error can't find the container with id 1efeebacab59fe0c33f24ca43a23e1cb5476dbc20b5749812d022fe7fc5a50f1
Jul 22 04:06:02.867714 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:02.867679    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2f6647b0/e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd" event={"ID":"38d62c67-d484-4113-a278-9055733ce7cc","Type":"ContainerStarted","Data":"d7a172030f3c551fe1f0ce9d2a6e8f9d7e8f3a329bb21f6606026223e890b6a7"}
Jul 22 04:06:02.867714 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:02.867719    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2f6647b0/e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd" event={"ID":"38d62c67-d484-4113-a278-9055733ce7cc","Type":"ContainerStarted","Data":"1efeebacab59fe0c33f24ca43a23e1cb5476dbc20b5749812d022fe7fc5a50f1"}
Jul 22 04:06:02.890075 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:02.890011    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-2f6647b0/e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd" podStartSLOduration=1.8899949390000002 podStartE2EDuration="1.889994939s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:06:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:06:02.888467403 +0000 UTC m=+1950.506677411" watchObservedRunningTime="2026-07-22 04:06:02.889994939 +0000 UTC m=+1950.508204960"
Jul 22 04:06:05.266940 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:05.266900    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:06:12.276291 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:12.276255    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-2f6647b0/e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd"
Jul 22 04:06:12.276716 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:12.276306    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-2f6647b0/e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd"
Jul 22 04:06:12.289011 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:12.288985    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-2f6647b0/e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd"
Jul 22 04:06:12.927933 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:12.927906    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-2f6647b0/e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd"
Jul 22 04:06:19.764966 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:19.764928    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:06:37.359859 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:37.359766    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:06:46.063627 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:46.063584    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:06:50.426000 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:06:50.425963    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-2f6647b0/e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd"]
Jul 22 04:07:05.447066 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:07:05.447000    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-2f6647b0/e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd" podUID="38d62c67-d484-4113-a278-9055733ce7cc" containerName="main" containerID="cri-o://d7a172030f3c551fe1f0ce9d2a6e8f9d7e8f3a329bb21f6606026223e890b6a7" gracePeriod=45
Jul 22 04:07:05.709135 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:07:05.709069    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2f6647b0/e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd"
Jul 22 04:07:05.855774 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:07:05.855743    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xh9hm\" (UniqueName: \"kubernetes.io/projected/38d62c67-d484-4113-a278-9055733ce7cc-kube-api-access-xh9hm\") pod \"38d62c67-d484-4113-a278-9055733ce7cc\" (UID: \"38d62c67-d484-4113-a278-9055733ce7cc\") "
Jul 22 04:07:05.855961 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:07:05.855789    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38d62c67-d484-4113-a278-9055733ce7cc-home\") pod \"38d62c67-d484-4113-a278-9055733ce7cc\" (UID: \"38d62c67-d484-4113-a278-9055733ce7cc\") "
Jul 22 04:07:05.855961 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:07:05.855815    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38d62c67-d484-4113-a278-9055733ce7cc-model-cache\") pod \"38d62c67-d484-4113-a278-9055733ce7cc\" (UID: \"38d62c67-d484-4113-a278-9055733ce7cc\") "
Jul 22 04:07:05.855961 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:07:05.855843    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38d62c67-d484-4113-a278-9055733ce7cc-dshm\") pod \"38d62c67-d484-4113-a278-9055733ce7cc\" (UID: \"38d62c67-d484-4113-a278-9055733ce7cc\") "
Jul 22 04:07:05.855961 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:07:05.855945    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38d62c67-d484-4113-a278-9055733ce7cc-tls-certs\") pod \"38d62c67-d484-4113-a278-9055733ce7cc\" (UID: \"38d62c67-d484-4113-a278-9055733ce7cc\") "
Jul 22 04:07:05.856198 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:07:05.855991    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38d62c67-d484-4113-a278-9055733ce7cc-tmp-dir\") pod \"38d62c67-d484-4113-a278-9055733ce7cc\" (UID: \"38d62c67-d484-4113-a278-9055733ce7cc\") "
Jul 22 04:07:05.856198 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:07:05.856064    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38d62c67-d484-4113-a278-9055733ce7cc-home" (OuterVolumeSpecName: "home") pod "38d62c67-d484-4113-a278-9055733ce7cc" (UID: "38d62c67-d484-4113-a278-9055733ce7cc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 04:07:05.856296 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:07:05.856260    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38d62c67-d484-4113-a278-9055733ce7cc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "38d62c67-d484-4113-a278-9055733ce7cc" (UID: "38d62c67-d484-4113-a278-9055733ce7cc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 04:07:05.856353 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:07:05.856313    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38d62c67-d484-4113-a278-9055733ce7cc-home\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:07:05.856402 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:07:05.856356    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38d62c67-d484-4113-a278-9055733ce7cc-model-cache" (OuterVolumeSpecName: "model-cache") pod "38d62c67-d484-4113-a278-9055733ce7cc" (UID: "38d62c67-d484-4113-a278-9055733ce7cc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 04:07:05.858310 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:07:05.858277    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38d62c67-d484-4113-a278-9055733ce7cc-dshm" (OuterVolumeSpecName: "dshm") pod "38d62c67-d484-4113-a278-9055733ce7cc" (UID: "38d62c67-d484-4113-a278-9055733ce7cc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 04:07:05.858310 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:07:05.858289    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38d62c67-d484-4113-a278-9055733ce7cc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "38d62c67-d484-4113-a278-9055733ce7cc" (UID: "38d62c67-d484-4113-a278-9055733ce7cc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 04:07:05.858474 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:07:05.858365    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38d62c67-d484-4113-a278-9055733ce7cc-kube-api-access-xh9hm" (OuterVolumeSpecName: "kube-api-access-xh9hm") pod "38d62c67-d484-4113-a278-9055733ce7cc" (UID: "38d62c67-d484-4113-a278-9055733ce7cc"). InnerVolumeSpecName "kube-api-access-xh9hm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 04:07:05.958381 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:07:05.958348    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xh9hm\" (UniqueName: \"kubernetes.io/projected/38d62c67-d484-4113-a278-9055733ce7cc-kube-api-access-xh9hm\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:07:05.958381 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:07:05.958377    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38d62c67-d484-4113-a278-9055733ce7cc-model-cache\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:07:05.958381 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:07:05.958387    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38d62c67-d484-4113-a278-9055733ce7cc-dshm\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:07:05.958381 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:07:05.958396    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38d62c67-d484-4113-a278-9055733ce7cc-tls-certs\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:07:05.958799 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:07:05.958408    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38d62c67-d484-4113-a278-9055733ce7cc-tmp-dir\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:07:06.172295 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:07:06.172268    2570 generic.go:356] "Generic (PLEG): container finished" podID="38d62c67-d484-4113-a278-9055733ce7cc" containerID="d7a172030f3c551fe1f0ce9d2a6e8f9d7e8f3a329bb21f6606026223e890b6a7" exitCode=0
Jul 22 04:07:06.172457 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:07:06.172317    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2f6647b0/e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd"
Jul 22 04:07:06.172457 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:07:06.172343    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2f6647b0/e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd" event={"ID":"38d62c67-d484-4113-a278-9055733ce7cc","Type":"ContainerDied","Data":"d7a172030f3c551fe1f0ce9d2a6e8f9d7e8f3a329bb21f6606026223e890b6a7"}
Jul 22 04:07:06.172457 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:07:06.172373    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2f6647b0/e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd" event={"ID":"38d62c67-d484-4113-a278-9055733ce7cc","Type":"ContainerDied","Data":"1efeebacab59fe0c33f24ca43a23e1cb5476dbc20b5749812d022fe7fc5a50f1"}
Jul 22 04:07:06.172457 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:07:06.172388    2570 scope.go:117] "RemoveContainer" containerID="d7a172030f3c551fe1f0ce9d2a6e8f9d7e8f3a329bb21f6606026223e890b6a7"
Jul 22 04:07:06.181475 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:07:06.181458    2570 scope.go:117] "RemoveContainer" containerID="d7a172030f3c551fe1f0ce9d2a6e8f9d7e8f3a329bb21f6606026223e890b6a7"
Jul 22 04:07:06.181728 ip-10-0-132-89 kubenswrapper[2570]: E0722 04:07:06.181707    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7a172030f3c551fe1f0ce9d2a6e8f9d7e8f3a329bb21f6606026223e890b6a7\": container with ID starting with d7a172030f3c551fe1f0ce9d2a6e8f9d7e8f3a329bb21f6606026223e890b6a7 not found: ID does not exist" containerID="d7a172030f3c551fe1f0ce9d2a6e8f9d7e8f3a329bb21f6606026223e890b6a7"
Jul 22 04:07:06.181798 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:07:06.181736    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7a172030f3c551fe1f0ce9d2a6e8f9d7e8f3a329bb21f6606026223e890b6a7"} err="failed to get container status \"d7a172030f3c551fe1f0ce9d2a6e8f9d7e8f3a329bb21f6606026223e890b6a7\": rpc error: code = NotFound desc = could not find container \"d7a172030f3c551fe1f0ce9d2a6e8f9d7e8f3a329bb21f6606026223e890b6a7\": container with ID starting with d7a172030f3c551fe1f0ce9d2a6e8f9d7e8f3a329bb21f6606026223e890b6a7 not found: ID does not exist"
Jul 22 04:07:06.194421 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:07:06.194395    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-2f6647b0/e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd"]
Jul 22 04:07:06.201412 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:07:06.201388    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-2f6647b0/e2e-lifecycle-model-2f6647b0-kserve-56c8866c47-vc4kd"]
Jul 22 04:07:06.851296 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:07:06.851263    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38d62c67-d484-4113-a278-9055733ce7cc" path="/var/lib/kubelet/pods/38d62c67-d484-4113-a278-9055733ce7cc/volumes"
Jul 22 04:07:37.368384 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:07:37.368349    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:07:50.075559 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:07:50.075523    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:08:32.890412 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:08:32.890377    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2p4qp_9f0defcc-ccff-43be-8206-54c772885080/console-operator/1.log"
Jul 22 04:08:32.894860 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:08:32.894828    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2p4qp_9f0defcc-ccff-43be-8206-54c772885080/console-operator/1.log"
Jul 22 04:08:32.955224 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:08:32.955194    2570 scope.go:117] "RemoveContainer" containerID="a4a867858e072cc4e2f45656d5b503d1832a6d572841620863b78cbfb9aebf14"
Jul 22 04:08:36.856030 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:08:36.855983    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:08:51.168105 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:08:51.168071    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-41e8d3/test-model-41e8d3-kserve-7f74c49875-g7n6w"]
Jul 22 04:08:51.168568 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:08:51.168407    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38d62c67-d484-4113-a278-9055733ce7cc" containerName="main"
Jul 22 04:08:51.168568 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:08:51.168417    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="38d62c67-d484-4113-a278-9055733ce7cc" containerName="main"
Jul 22 04:08:51.168568 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:08:51.168504    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="38d62c67-d484-4113-a278-9055733ce7cc" containerName="main"
Jul 22 04:08:51.171600 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:08:51.171583    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-41e8d3/test-model-41e8d3-kserve-7f74c49875-g7n6w"
Jul 22 04:08:51.175275 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:08:51.175254    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-41e8d3\"/\"openshift-service-ca.crt\""
Jul 22 04:08:51.175395 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:08:51.175296    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-41e8d3\"/\"test-model-41e8d3-kserve-self-signed-certs\""
Jul 22 04:08:51.175395 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:08:51.175356    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-41e8d3\"/\"default-dockercfg-hs2q4\""
Jul 22 04:08:51.175514 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:08:51.175397    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-41e8d3\"/\"kube-root-ca.crt\""
Jul 22 04:08:51.181807 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:08:51.181782    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-41e8d3/test-model-41e8d3-kserve-7f74c49875-g7n6w"]
Jul 22 04:08:51.289342 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:08:51.289306    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e20e71d9-d551-45e4-9787-0fc662ab5725-tmp-dir\") pod \"test-model-41e8d3-kserve-7f74c49875-g7n6w\" (UID: \"e20e71d9-d551-45e4-9787-0fc662ab5725\") " pod="ai-tenant-e2e-inf-a-41e8d3/test-model-41e8d3-kserve-7f74c49875-g7n6w"
Jul 22 04:08:51.289535 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:08:51.289365    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e20e71d9-d551-45e4-9787-0fc662ab5725-home\") pod \"test-model-41e8d3-kserve-7f74c49875-g7n6w\" (UID: \"e20e71d9-d551-45e4-9787-0fc662ab5725\") " pod="ai-tenant-e2e-inf-a-41e8d3/test-model-41e8d3-kserve-7f74c49875-g7n6w"
Jul 22 04:08:51.289535 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:08:51.289446    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e20e71d9-d551-45e4-9787-0fc662ab5725-dshm\") pod \"test-model-41e8d3-kserve-7f74c49875-g7n6w\" (UID: \"e20e71d9-d551-45e4-9787-0fc662ab5725\") " pod="ai-tenant-e2e-inf-a-41e8d3/test-model-41e8d3-kserve-7f74c49875-g7n6w"
Jul 22 04:08:51.289535 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:08:51.289483    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e20e71d9-d551-45e4-9787-0fc662ab5725-model-cache\") pod \"test-model-41e8d3-kserve-7f74c49875-g7n6w\" (UID: \"e20e71d9-d551-45e4-9787-0fc662ab5725\") " pod="ai-tenant-e2e-inf-a-41e8d3/test-model-41e8d3-kserve-7f74c49875-g7n6w"
Jul 22 04:08:51.289535 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:08:51.289506    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e20e71d9-d551-45e4-9787-0fc662ab5725-tls-certs\") pod \"test-model-41e8d3-kserve-7f74c49875-g7n6w\" (UID: \"e20e71d9-d551-45e4-9787-0fc662ab5725\") " pod="ai-tenant-e2e-inf-a-41e8d3/test-model-41e8d3-kserve-7f74c49875-g7n6w"
Jul 22 04:08:51.289729 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:08:51.289536    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpsv9\" (UniqueName: \"kubernetes.io/projected/e20e71d9-d551-45e4-9787-0fc662ab5725-kube-api-access-wpsv9\") pod \"test-model-41e8d3-kserve-7f74c49875-g7n6w\" (UID: \"e20e71d9-d551-45e4-9787-0fc662ab5725\") " pod="ai-tenant-e2e-inf-a-41e8d3/test-model-41e8d3-kserve-7f74c49875-g7n6w"
Jul 22 04:08:51.390301 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:08:51.390270    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e20e71d9-d551-45e4-9787-0fc662ab5725-model-cache\") pod \"test-model-41e8d3-kserve-7f74c49875-g7n6w\" (UID: \"e20e71d9-d551-45e4-9787-0fc662ab5725\") " pod="ai-tenant-e2e-inf-a-41e8d3/test-model-41e8d3-kserve-7f74c49875-g7n6w"
Jul 22 04:08:51.390301 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:08:51.390306    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e20e71d9-d551-45e4-9787-0fc662ab5725-tls-certs\") pod \"test-model-41e8d3-kserve-7f74c49875-g7n6w\" (UID: \"e20e71d9-d551-45e4-9787-0fc662ab5725\") " pod="ai-tenant-e2e-inf-a-41e8d3/test-model-41e8d3-kserve-7f74c49875-g7n6w"
Jul 22 04:08:51.390553 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:08:51.390327    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpsv9\" (UniqueName: \"kubernetes.io/projected/e20e71d9-d551-45e4-9787-0fc662ab5725-kube-api-access-wpsv9\") pod \"test-model-41e8d3-kserve-7f74c49875-g7n6w\" (UID: \"e20e71d9-d551-45e4-9787-0fc662ab5725\") " pod="ai-tenant-e2e-inf-a-41e8d3/test-model-41e8d3-kserve-7f74c49875-g7n6w"
Jul 22 04:08:51.390553 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:08:51.390368    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e20e71d9-d551-45e4-9787-0fc662ab5725-tmp-dir\") pod \"test-model-41e8d3-kserve-7f74c49875-g7n6w\" (UID: \"e20e71d9-d551-45e4-9787-0fc662ab5725\") " pod="ai-tenant-e2e-inf-a-41e8d3/test-model-41e8d3-kserve-7f74c49875-g7n6w"
Jul 22 04:08:51.390553 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:08:51.390419    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e20e71d9-d551-45e4-9787-0fc662ab5725-home\") pod \"test-model-41e8d3-kserve-7f74c49875-g7n6w\" (UID: \"e20e71d9-d551-45e4-9787-0fc662ab5725\") " pod="ai-tenant-e2e-inf-a-41e8d3/test-model-41e8d3-kserve-7f74c49875-g7n6w"
Jul 22 04:08:51.390553 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:08:51.390464    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e20e71d9-d551-45e4-9787-0fc662ab5725-dshm\") pod \"test-model-41e8d3-kserve-7f74c49875-g7n6w\" (UID: \"e20e71d9-d551-45e4-9787-0fc662ab5725\") " pod="ai-tenant-e2e-inf-a-41e8d3/test-model-41e8d3-kserve-7f74c49875-g7n6w"
Jul 22 04:08:51.390786 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:08:51.390707    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e20e71d9-d551-45e4-9787-0fc662ab5725-model-cache\") pod \"test-model-41e8d3-kserve-7f74c49875-g7n6w\" (UID: \"e20e71d9-d551-45e4-9787-0fc662ab5725\") " pod="ai-tenant-e2e-inf-a-41e8d3/test-model-41e8d3-kserve-7f74c49875-g7n6w"
Jul 22 04:08:51.390786 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:08:51.390770    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e20e71d9-d551-45e4-9787-0fc662ab5725-tmp-dir\") pod \"test-model-41e8d3-kserve-7f74c49875-g7n6w\" (UID: \"e20e71d9-d551-45e4-9787-0fc662ab5725\") " pod="ai-tenant-e2e-inf-a-41e8d3/test-model-41e8d3-kserve-7f74c49875-g7n6w"
Jul 22 04:08:51.390879 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:08:51.390860    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e20e71d9-d551-45e4-9787-0fc662ab5725-home\") pod \"test-model-41e8d3-kserve-7f74c49875-g7n6w\" (UID: \"e20e71d9-d551-45e4-9787-0fc662ab5725\") " pod="ai-tenant-e2e-inf-a-41e8d3/test-model-41e8d3-kserve-7f74c49875-g7n6w"
Jul 22 04:08:51.392890 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:08:51.392870    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e20e71d9-d551-45e4-9787-0fc662ab5725-dshm\") pod \"test-model-41e8d3-kserve-7f74c49875-g7n6w\" (UID: \"e20e71d9-d551-45e4-9787-0fc662ab5725\") " pod="ai-tenant-e2e-inf-a-41e8d3/test-model-41e8d3-kserve-7f74c49875-g7n6w"
Jul 22 04:08:51.393051 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:08:51.393032    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e20e71d9-d551-45e4-9787-0fc662ab5725-tls-certs\") pod \"test-model-41e8d3-kserve-7f74c49875-g7n6w\" (UID: \"e20e71d9-d551-45e4-9787-0fc662ab5725\") " pod="ai-tenant-e2e-inf-a-41e8d3/test-model-41e8d3-kserve-7f74c49875-g7n6w"
Jul 22 04:08:51.399844 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:08:51.399823    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpsv9\" (UniqueName: \"kubernetes.io/projected/e20e71d9-d551-45e4-9787-0fc662ab5725-kube-api-access-wpsv9\") pod \"test-model-41e8d3-kserve-7f74c49875-g7n6w\" (UID: \"e20e71d9-d551-45e4-9787-0fc662ab5725\") " pod="ai-tenant-e2e-inf-a-41e8d3/test-model-41e8d3-kserve-7f74c49875-g7n6w"
Jul 22 04:08:51.481822 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:08:51.481734    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-41e8d3/test-model-41e8d3-kserve-7f74c49875-g7n6w"
Jul 22 04:08:51.620068 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:08:51.620036    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-41e8d3/test-model-41e8d3-kserve-7f74c49875-g7n6w"]
Jul 22 04:08:51.623552 ip-10-0-132-89 kubenswrapper[2570]: W0722 04:08:51.623521    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode20e71d9_d551_45e4_9787_0fc662ab5725.slice/crio-4be4972544017de1e57a52e26f03f955636a27cbcd2b03cb069fca2765de5513 WatchSource:0}: Error finding container 4be4972544017de1e57a52e26f03f955636a27cbcd2b03cb069fca2765de5513: Status 404 returned error can't find the container with id 4be4972544017de1e57a52e26f03f955636a27cbcd2b03cb069fca2765de5513
Jul 22 04:08:51.662483 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:08:51.662454    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-41e8d3/test-model-41e8d3-kserve-7f74c49875-g7n6w" event={"ID":"e20e71d9-d551-45e4-9787-0fc662ab5725","Type":"ContainerStarted","Data":"4be4972544017de1e57a52e26f03f955636a27cbcd2b03cb069fca2765de5513"}
Jul 22 04:08:52.376594 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:08:52.376561    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:08:52.669234 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:08:52.669199    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-41e8d3/test-model-41e8d3-kserve-7f74c49875-g7n6w" event={"ID":"e20e71d9-d551-45e4-9787-0fc662ab5725","Type":"ContainerStarted","Data":"63caddd4ff8c409b08439df192f90d5f6394202ac758286e1330875ab583dc86"}
Jul 22 04:08:52.687336 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:08:52.687289    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-41e8d3/test-model-41e8d3-kserve-7f74c49875-g7n6w" podStartSLOduration=1.6872733869999998 podStartE2EDuration="1.687273387s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:08:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:08:52.685494883 +0000 UTC m=+2120.303704892" watchObservedRunningTime="2026-07-22 04:08:52.687273387 +0000 UTC m=+2120.305483393"
Jul 22 04:09:01.482481 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:01.482442    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-41e8d3/test-model-41e8d3-kserve-7f74c49875-g7n6w"
Jul 22 04:09:01.483035 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:01.482495    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-41e8d3/test-model-41e8d3-kserve-7f74c49875-g7n6w"
Jul 22 04:09:01.495005 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:01.494979    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-41e8d3/test-model-41e8d3-kserve-7f74c49875-g7n6w"
Jul 22 04:09:01.725854 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:01.725826    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-41e8d3/test-model-41e8d3-kserve-7f74c49875-g7n6w"
Jul 22 04:09:03.771154 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:03.771122    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:09:31.274575 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:31.274541    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-b9a16f/test-model-b9a16f-kserve-584f66899f-vcr4f"]
Jul 22 04:09:31.278386 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:31.278369    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-b9a16f/test-model-b9a16f-kserve-584f66899f-vcr4f"
Jul 22 04:09:31.281157 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:31.281139    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-b9a16f\"/\"kube-root-ca.crt\""
Jul 22 04:09:31.282212 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:31.282186    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-b9a16f\"/\"default-dockercfg-x9hc2\""
Jul 22 04:09:31.282212 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:31.282208    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-b9a16f\"/\"openshift-service-ca.crt\""
Jul 22 04:09:31.282374 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:31.282186    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-b9a16f\"/\"test-model-b9a16f-kserve-self-signed-certs\""
Jul 22 04:09:31.287313 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:31.287290    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-b9a16f/test-model-b9a16f-kserve-584f66899f-vcr4f"]
Jul 22 04:09:31.357633 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:31.357587    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c-home\") pod \"test-model-b9a16f-kserve-584f66899f-vcr4f\" (UID: \"21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c\") " pod="ai-tenant-e2e-inf-b-b9a16f/test-model-b9a16f-kserve-584f66899f-vcr4f"
Jul 22 04:09:31.357893 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:31.357641    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c-tls-certs\") pod \"test-model-b9a16f-kserve-584f66899f-vcr4f\" (UID: \"21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c\") " pod="ai-tenant-e2e-inf-b-b9a16f/test-model-b9a16f-kserve-584f66899f-vcr4f"
Jul 22 04:09:31.357893 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:31.357758    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c-dshm\") pod \"test-model-b9a16f-kserve-584f66899f-vcr4f\" (UID: \"21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c\") " pod="ai-tenant-e2e-inf-b-b9a16f/test-model-b9a16f-kserve-584f66899f-vcr4f"
Jul 22 04:09:31.357893 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:31.357800    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c-tmp-dir\") pod \"test-model-b9a16f-kserve-584f66899f-vcr4f\" (UID: \"21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c\") " pod="ai-tenant-e2e-inf-b-b9a16f/test-model-b9a16f-kserve-584f66899f-vcr4f"
Jul 22 04:09:31.357893 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:31.357830    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cc9jj\" (UniqueName: \"kubernetes.io/projected/21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c-kube-api-access-cc9jj\") pod \"test-model-b9a16f-kserve-584f66899f-vcr4f\" (UID: \"21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c\") " pod="ai-tenant-e2e-inf-b-b9a16f/test-model-b9a16f-kserve-584f66899f-vcr4f"
Jul 22 04:09:31.357893 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:31.357876    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c-model-cache\") pod \"test-model-b9a16f-kserve-584f66899f-vcr4f\" (UID: \"21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c\") " pod="ai-tenant-e2e-inf-b-b9a16f/test-model-b9a16f-kserve-584f66899f-vcr4f"
Jul 22 04:09:31.459124 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:31.459091    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c-model-cache\") pod \"test-model-b9a16f-kserve-584f66899f-vcr4f\" (UID: \"21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c\") " pod="ai-tenant-e2e-inf-b-b9a16f/test-model-b9a16f-kserve-584f66899f-vcr4f"
Jul 22 04:09:31.459307 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:31.459151    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c-home\") pod \"test-model-b9a16f-kserve-584f66899f-vcr4f\" (UID: \"21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c\") " pod="ai-tenant-e2e-inf-b-b9a16f/test-model-b9a16f-kserve-584f66899f-vcr4f"
Jul 22 04:09:31.459307 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:31.459181    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c-tls-certs\") pod \"test-model-b9a16f-kserve-584f66899f-vcr4f\" (UID: \"21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c\") " pod="ai-tenant-e2e-inf-b-b9a16f/test-model-b9a16f-kserve-584f66899f-vcr4f"
Jul 22 04:09:31.459307 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:31.459239    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c-dshm\") pod \"test-model-b9a16f-kserve-584f66899f-vcr4f\" (UID: \"21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c\") " pod="ai-tenant-e2e-inf-b-b9a16f/test-model-b9a16f-kserve-584f66899f-vcr4f"
Jul 22 04:09:31.459307 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:31.459263    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c-tmp-dir\") pod \"test-model-b9a16f-kserve-584f66899f-vcr4f\" (UID: \"21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c\") " pod="ai-tenant-e2e-inf-b-b9a16f/test-model-b9a16f-kserve-584f66899f-vcr4f"
Jul 22 04:09:31.459307 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:31.459290    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cc9jj\" (UniqueName: \"kubernetes.io/projected/21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c-kube-api-access-cc9jj\") pod \"test-model-b9a16f-kserve-584f66899f-vcr4f\" (UID: \"21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c\") " pod="ai-tenant-e2e-inf-b-b9a16f/test-model-b9a16f-kserve-584f66899f-vcr4f"
Jul 22 04:09:31.459565 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:31.459539    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c-model-cache\") pod \"test-model-b9a16f-kserve-584f66899f-vcr4f\" (UID: \"21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c\") " pod="ai-tenant-e2e-inf-b-b9a16f/test-model-b9a16f-kserve-584f66899f-vcr4f"
Jul 22 04:09:31.459661 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:31.459593    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c-home\") pod \"test-model-b9a16f-kserve-584f66899f-vcr4f\" (UID: \"21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c\") " pod="ai-tenant-e2e-inf-b-b9a16f/test-model-b9a16f-kserve-584f66899f-vcr4f"
Jul 22 04:09:31.459661 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:31.459607    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c-tmp-dir\") pod \"test-model-b9a16f-kserve-584f66899f-vcr4f\" (UID: \"21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c\") " pod="ai-tenant-e2e-inf-b-b9a16f/test-model-b9a16f-kserve-584f66899f-vcr4f"
Jul 22 04:09:31.461790 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:31.461767    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c-dshm\") pod \"test-model-b9a16f-kserve-584f66899f-vcr4f\" (UID: \"21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c\") " pod="ai-tenant-e2e-inf-b-b9a16f/test-model-b9a16f-kserve-584f66899f-vcr4f"
Jul 22 04:09:31.461989 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:31.461975    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c-tls-certs\") pod \"test-model-b9a16f-kserve-584f66899f-vcr4f\" (UID: \"21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c\") " pod="ai-tenant-e2e-inf-b-b9a16f/test-model-b9a16f-kserve-584f66899f-vcr4f"
Jul 22 04:09:31.468084 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:31.468061    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc9jj\" (UniqueName: \"kubernetes.io/projected/21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c-kube-api-access-cc9jj\") pod \"test-model-b9a16f-kserve-584f66899f-vcr4f\" (UID: \"21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c\") " pod="ai-tenant-e2e-inf-b-b9a16f/test-model-b9a16f-kserve-584f66899f-vcr4f"
Jul 22 04:09:31.588354 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:31.588265    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-b9a16f/test-model-b9a16f-kserve-584f66899f-vcr4f"
Jul 22 04:09:31.723233 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:31.723153    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-b9a16f/test-model-b9a16f-kserve-584f66899f-vcr4f"]
Jul 22 04:09:31.725668 ip-10-0-132-89 kubenswrapper[2570]: W0722 04:09:31.725640    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod21d9afb2_babb_4fc4_92a2_e4b5bd4ef07c.slice/crio-178656fa5d9520a063ea90f63bf6ab66f6547706fc7ba3137ebf9e0bf63fe735 WatchSource:0}: Error finding container 178656fa5d9520a063ea90f63bf6ab66f6547706fc7ba3137ebf9e0bf63fe735: Status 404 returned error can't find the container with id 178656fa5d9520a063ea90f63bf6ab66f6547706fc7ba3137ebf9e0bf63fe735
Jul 22 04:09:31.853567 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:31.853474    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-b9a16f/test-model-b9a16f-kserve-584f66899f-vcr4f" event={"ID":"21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c","Type":"ContainerStarted","Data":"4867281eed4c0cabcb56dbc19a9881222c4630ca00e14b6e5c13456d1503a319"}
Jul 22 04:09:31.853567 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:31.853523    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-b9a16f/test-model-b9a16f-kserve-584f66899f-vcr4f" event={"ID":"21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c","Type":"ContainerStarted","Data":"178656fa5d9520a063ea90f63bf6ab66f6547706fc7ba3137ebf9e0bf63fe735"}
Jul 22 04:09:31.875119 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:31.875061    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-b9a16f/test-model-b9a16f-kserve-584f66899f-vcr4f" podStartSLOduration=0.875043745 podStartE2EDuration="875.043745ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:09:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:09:31.873198759 +0000 UTC m=+2159.491408766" watchObservedRunningTime="2026-07-22 04:09:31.875043745 +0000 UTC m=+2159.493253751"
Jul 22 04:09:32.974378 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:32.974348    2570 scope.go:117] "RemoveContainer" containerID="75e3ffd35f28c6fae075fdb02d62623c0619134630a272b335bcaae07529b322"
Jul 22 04:09:35.758269 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:35.758230    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:09:41.588771 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:41.588729    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-b9a16f/test-model-b9a16f-kserve-584f66899f-vcr4f"
Jul 22 04:09:41.588771 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:41.588779    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-b9a16f/test-model-b9a16f-kserve-584f66899f-vcr4f"
Jul 22 04:09:41.601516 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:41.601483    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-b9a16f/test-model-b9a16f-kserve-584f66899f-vcr4f"
Jul 22 04:09:41.909810 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:41.909717    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-b9a16f/test-model-b9a16f-kserve-584f66899f-vcr4f"
Jul 22 04:09:50.374688 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:09:50.374653    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:10:11.060190 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:11.060154    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-41e8d3/test-model-41e8d3-kserve-7f74c49875-g7n6w"]
Jul 22 04:10:12.065203 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:12.065164    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-b9a16f/test-model-b9a16f-kserve-584f66899f-vcr4f"]
Jul 22 04:10:15.767670 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:15.767630    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:10:26.080869 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:26.080820    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-41e8d3/test-model-41e8d3-kserve-7f74c49875-g7n6w" podUID="e20e71d9-d551-45e4-9787-0fc662ab5725" containerName="main" containerID="cri-o://63caddd4ff8c409b08439df192f90d5f6394202ac758286e1330875ab583dc86" gracePeriod=45
Jul 22 04:10:26.340722 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:26.340659    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-41e8d3/test-model-41e8d3-kserve-7f74c49875-g7n6w"
Jul 22 04:10:26.448337 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:26.448299    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e20e71d9-d551-45e4-9787-0fc662ab5725-dshm\") pod \"e20e71d9-d551-45e4-9787-0fc662ab5725\" (UID: \"e20e71d9-d551-45e4-9787-0fc662ab5725\") "
Jul 22 04:10:26.448513 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:26.448352    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e20e71d9-d551-45e4-9787-0fc662ab5725-tmp-dir\") pod \"e20e71d9-d551-45e4-9787-0fc662ab5725\" (UID: \"e20e71d9-d551-45e4-9787-0fc662ab5725\") "
Jul 22 04:10:26.448513 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:26.448386    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wpsv9\" (UniqueName: \"kubernetes.io/projected/e20e71d9-d551-45e4-9787-0fc662ab5725-kube-api-access-wpsv9\") pod \"e20e71d9-d551-45e4-9787-0fc662ab5725\" (UID: \"e20e71d9-d551-45e4-9787-0fc662ab5725\") "
Jul 22 04:10:26.448513 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:26.448411    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e20e71d9-d551-45e4-9787-0fc662ab5725-tls-certs\") pod \"e20e71d9-d551-45e4-9787-0fc662ab5725\" (UID: \"e20e71d9-d551-45e4-9787-0fc662ab5725\") "
Jul 22 04:10:26.448513 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:26.448506    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e20e71d9-d551-45e4-9787-0fc662ab5725-model-cache\") pod \"e20e71d9-d551-45e4-9787-0fc662ab5725\" (UID: \"e20e71d9-d551-45e4-9787-0fc662ab5725\") "
Jul 22 04:10:26.448759 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:26.448532    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e20e71d9-d551-45e4-9787-0fc662ab5725-home\") pod \"e20e71d9-d551-45e4-9787-0fc662ab5725\" (UID: \"e20e71d9-d551-45e4-9787-0fc662ab5725\") "
Jul 22 04:10:26.448759 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:26.448673    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e20e71d9-d551-45e4-9787-0fc662ab5725-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e20e71d9-d551-45e4-9787-0fc662ab5725" (UID: "e20e71d9-d551-45e4-9787-0fc662ab5725"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 04:10:26.448883 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:26.448858    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e20e71d9-d551-45e4-9787-0fc662ab5725-model-cache" (OuterVolumeSpecName: "model-cache") pod "e20e71d9-d551-45e4-9787-0fc662ab5725" (UID: "e20e71d9-d551-45e4-9787-0fc662ab5725"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 04:10:26.448883 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:26.448866    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e20e71d9-d551-45e4-9787-0fc662ab5725-home" (OuterVolumeSpecName: "home") pod "e20e71d9-d551-45e4-9787-0fc662ab5725" (UID: "e20e71d9-d551-45e4-9787-0fc662ab5725"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 04:10:26.448883 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:26.448880    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e20e71d9-d551-45e4-9787-0fc662ab5725-tmp-dir\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:10:26.450875 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:26.450840    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e20e71d9-d551-45e4-9787-0fc662ab5725-kube-api-access-wpsv9" (OuterVolumeSpecName: "kube-api-access-wpsv9") pod "e20e71d9-d551-45e4-9787-0fc662ab5725" (UID: "e20e71d9-d551-45e4-9787-0fc662ab5725"). InnerVolumeSpecName "kube-api-access-wpsv9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 04:10:26.450989 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:26.450887    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e20e71d9-d551-45e4-9787-0fc662ab5725-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e20e71d9-d551-45e4-9787-0fc662ab5725" (UID: "e20e71d9-d551-45e4-9787-0fc662ab5725"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 04:10:26.450989 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:26.450895    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e20e71d9-d551-45e4-9787-0fc662ab5725-dshm" (OuterVolumeSpecName: "dshm") pod "e20e71d9-d551-45e4-9787-0fc662ab5725" (UID: "e20e71d9-d551-45e4-9787-0fc662ab5725"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 04:10:26.549773 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:26.549741    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e20e71d9-d551-45e4-9787-0fc662ab5725-dshm\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:10:26.549773 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:26.549769    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpsv9\" (UniqueName: \"kubernetes.io/projected/e20e71d9-d551-45e4-9787-0fc662ab5725-kube-api-access-wpsv9\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:10:26.549773 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:26.549780    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e20e71d9-d551-45e4-9787-0fc662ab5725-tls-certs\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:10:26.550002 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:26.549789    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e20e71d9-d551-45e4-9787-0fc662ab5725-model-cache\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:10:26.550002 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:26.549797    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e20e71d9-d551-45e4-9787-0fc662ab5725-home\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:10:27.086030 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:27.085994    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-b9a16f/test-model-b9a16f-kserve-584f66899f-vcr4f" podUID="21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c" containerName="main" containerID="cri-o://4867281eed4c0cabcb56dbc19a9881222c4630ca00e14b6e5c13456d1503a319" gracePeriod=45
Jul 22 04:10:27.113932 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:27.113856    2570 generic.go:356] "Generic (PLEG): container finished" podID="e20e71d9-d551-45e4-9787-0fc662ab5725" containerID="63caddd4ff8c409b08439df192f90d5f6394202ac758286e1330875ab583dc86" exitCode=0
Jul 22 04:10:27.113932 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:27.113892    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-41e8d3/test-model-41e8d3-kserve-7f74c49875-g7n6w"
Jul 22 04:10:27.113932 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:27.113920    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-41e8d3/test-model-41e8d3-kserve-7f74c49875-g7n6w" event={"ID":"e20e71d9-d551-45e4-9787-0fc662ab5725","Type":"ContainerDied","Data":"63caddd4ff8c409b08439df192f90d5f6394202ac758286e1330875ab583dc86"}
Jul 22 04:10:27.114155 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:27.113950    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-41e8d3/test-model-41e8d3-kserve-7f74c49875-g7n6w" event={"ID":"e20e71d9-d551-45e4-9787-0fc662ab5725","Type":"ContainerDied","Data":"4be4972544017de1e57a52e26f03f955636a27cbcd2b03cb069fca2765de5513"}
Jul 22 04:10:27.114155 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:27.113966    2570 scope.go:117] "RemoveContainer" containerID="63caddd4ff8c409b08439df192f90d5f6394202ac758286e1330875ab583dc86"
Jul 22 04:10:27.160392 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:27.160361    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-41e8d3/test-model-41e8d3-kserve-7f74c49875-g7n6w"]
Jul 22 04:10:27.166584 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:27.166558    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-41e8d3/test-model-41e8d3-kserve-7f74c49875-g7n6w"]
Jul 22 04:10:27.197740 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:27.197717    2570 scope.go:117] "RemoveContainer" containerID="63caddd4ff8c409b08439df192f90d5f6394202ac758286e1330875ab583dc86"
Jul 22 04:10:27.198112 ip-10-0-132-89 kubenswrapper[2570]: E0722 04:10:27.198091    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63caddd4ff8c409b08439df192f90d5f6394202ac758286e1330875ab583dc86\": container with ID starting with 63caddd4ff8c409b08439df192f90d5f6394202ac758286e1330875ab583dc86 not found: ID does not exist" containerID="63caddd4ff8c409b08439df192f90d5f6394202ac758286e1330875ab583dc86"
Jul 22 04:10:27.198222 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:27.198119    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63caddd4ff8c409b08439df192f90d5f6394202ac758286e1330875ab583dc86"} err="failed to get container status \"63caddd4ff8c409b08439df192f90d5f6394202ac758286e1330875ab583dc86\": rpc error: code = NotFound desc = could not find container \"63caddd4ff8c409b08439df192f90d5f6394202ac758286e1330875ab583dc86\": container with ID starting with 63caddd4ff8c409b08439df192f90d5f6394202ac758286e1330875ab583dc86 not found: ID does not exist"
Jul 22 04:10:27.344059 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:27.344029    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-b9a16f/test-model-b9a16f-kserve-584f66899f-vcr4f"
Jul 22 04:10:27.457143 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:27.457112    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c-dshm\") pod \"21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c\" (UID: \"21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c\") "
Jul 22 04:10:27.457336 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:27.457156    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c-home\") pod \"21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c\" (UID: \"21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c\") "
Jul 22 04:10:27.457336 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:27.457177    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c-tls-certs\") pod \"21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c\" (UID: \"21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c\") "
Jul 22 04:10:27.457336 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:27.457217    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c-model-cache\") pod \"21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c\" (UID: \"21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c\") "
Jul 22 04:10:27.457336 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:27.457262    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c-tmp-dir\") pod \"21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c\" (UID: \"21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c\") "
Jul 22 04:10:27.457336 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:27.457282    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cc9jj\" (UniqueName: \"kubernetes.io/projected/21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c-kube-api-access-cc9jj\") pod \"21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c\" (UID: \"21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c\") "
Jul 22 04:10:27.457583 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:27.457458    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c-home" (OuterVolumeSpecName: "home") pod "21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c" (UID: "21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 04:10:27.457583 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:27.457509    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c-model-cache" (OuterVolumeSpecName: "model-cache") pod "21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c" (UID: "21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 04:10:27.457583 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:27.457527    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c" (UID: "21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 04:10:27.459591 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:27.459567    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c-dshm" (OuterVolumeSpecName: "dshm") pod "21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c" (UID: "21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 04:10:27.459694 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:27.459632    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c-kube-api-access-cc9jj" (OuterVolumeSpecName: "kube-api-access-cc9jj") pod "21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c" (UID: "21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c"). InnerVolumeSpecName "kube-api-access-cc9jj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 04:10:27.459694 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:27.459647    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c" (UID: "21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 04:10:27.558385 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:27.558347    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c-tmp-dir\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:10:27.558385 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:27.558379    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cc9jj\" (UniqueName: \"kubernetes.io/projected/21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c-kube-api-access-cc9jj\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:10:27.558385 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:27.558390    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c-dshm\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:10:27.558711 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:27.558401    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c-home\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:10:27.558711 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:27.558408    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c-tls-certs\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:10:27.558711 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:27.558416    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c-model-cache\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:10:28.120663 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:28.120604    2570 generic.go:356] "Generic (PLEG): container finished" podID="21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c" containerID="4867281eed4c0cabcb56dbc19a9881222c4630ca00e14b6e5c13456d1503a319" exitCode=0
Jul 22 04:10:28.120663 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:28.120651    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-b9a16f/test-model-b9a16f-kserve-584f66899f-vcr4f" event={"ID":"21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c","Type":"ContainerDied","Data":"4867281eed4c0cabcb56dbc19a9881222c4630ca00e14b6e5c13456d1503a319"}
Jul 22 04:10:28.120663 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:28.120667    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-b9a16f/test-model-b9a16f-kserve-584f66899f-vcr4f"
Jul 22 04:10:28.121187 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:28.120692    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-b9a16f/test-model-b9a16f-kserve-584f66899f-vcr4f" event={"ID":"21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c","Type":"ContainerDied","Data":"178656fa5d9520a063ea90f63bf6ab66f6547706fc7ba3137ebf9e0bf63fe735"}
Jul 22 04:10:28.121187 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:28.120710    2570 scope.go:117] "RemoveContainer" containerID="4867281eed4c0cabcb56dbc19a9881222c4630ca00e14b6e5c13456d1503a319"
Jul 22 04:10:28.129287 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:28.129265    2570 scope.go:117] "RemoveContainer" containerID="4867281eed4c0cabcb56dbc19a9881222c4630ca00e14b6e5c13456d1503a319"
Jul 22 04:10:28.129551 ip-10-0-132-89 kubenswrapper[2570]: E0722 04:10:28.129532    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4867281eed4c0cabcb56dbc19a9881222c4630ca00e14b6e5c13456d1503a319\": container with ID starting with 4867281eed4c0cabcb56dbc19a9881222c4630ca00e14b6e5c13456d1503a319 not found: ID does not exist" containerID="4867281eed4c0cabcb56dbc19a9881222c4630ca00e14b6e5c13456d1503a319"
Jul 22 04:10:28.129606 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:28.129564    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4867281eed4c0cabcb56dbc19a9881222c4630ca00e14b6e5c13456d1503a319"} err="failed to get container status \"4867281eed4c0cabcb56dbc19a9881222c4630ca00e14b6e5c13456d1503a319\": rpc error: code = NotFound desc = could not find container \"4867281eed4c0cabcb56dbc19a9881222c4630ca00e14b6e5c13456d1503a319\": container with ID starting with 4867281eed4c0cabcb56dbc19a9881222c4630ca00e14b6e5c13456d1503a319 not found: ID does not exist"
Jul 22 04:10:28.143042 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:28.143008    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-b9a16f/test-model-b9a16f-kserve-584f66899f-vcr4f"]
Jul 22 04:10:28.146769 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:28.146747    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-b9a16f/test-model-b9a16f-kserve-584f66899f-vcr4f"]
Jul 22 04:10:28.391290 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:28.391206    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:10:28.851578 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:28.851544    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c" path="/var/lib/kubelet/pods/21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c/volumes"
Jul 22 04:10:28.851970 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:28.851955    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e20e71d9-d551-45e4-9787-0fc662ab5725" path="/var/lib/kubelet/pods/e20e71d9-d551-45e4-9787-0fc662ab5725/volumes"
Jul 22 04:10:34.495133 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:34.495101    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b9a16f-rbvvh"]
Jul 22 04:10:34.495536 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:34.495450    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c" containerName="main"
Jul 22 04:10:34.495536 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:34.495460    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c" containerName="main"
Jul 22 04:10:34.495536 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:34.495469    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e20e71d9-d551-45e4-9787-0fc662ab5725" containerName="main"
Jul 22 04:10:34.495536 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:34.495474    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e20e71d9-d551-45e4-9787-0fc662ab5725" containerName="main"
Jul 22 04:10:34.495536 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:34.495537    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="21d9afb2-babb-4fc4-92a2-e4b5bd4ef07c" containerName="main"
Jul 22 04:10:34.495727 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:34.495549    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="e20e71d9-d551-45e4-9787-0fc662ab5725" containerName="main"
Jul 22 04:10:34.500151 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:34.500134    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b9a16f-rbvvh"
Jul 22 04:10:34.502761 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:34.502736    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 04:10:34.502877 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:34.502819    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rbfm6\""
Jul 22 04:10:34.507340 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:34.507315    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b9a16f-rbvvh"]
Jul 22 04:10:34.630230 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:34.630187    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ad568ff-86e7-4d42-881f-7bd6b8177fca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-b9a16f-rbvvh\" (UID: \"5ad568ff-86e7-4d42-881f-7bd6b8177fca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b9a16f-rbvvh"
Jul 22 04:10:34.731747 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:34.731711    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ad568ff-86e7-4d42-881f-7bd6b8177fca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-b9a16f-rbvvh\" (UID: \"5ad568ff-86e7-4d42-881f-7bd6b8177fca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b9a16f-rbvvh"
Jul 22 04:10:34.732368 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:34.732349    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ad568ff-86e7-4d42-881f-7bd6b8177fca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-b9a16f-rbvvh\" (UID: \"5ad568ff-86e7-4d42-881f-7bd6b8177fca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b9a16f-rbvvh"
Jul 22 04:10:34.812800 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:34.812714    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b9a16f-rbvvh"
Jul 22 04:10:34.954463 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:34.954435    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b9a16f-rbvvh"]
Jul 22 04:10:34.957066 ip-10-0-132-89 kubenswrapper[2570]: W0722 04:10:34.957033    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ad568ff_86e7_4d42_881f_7bd6b8177fca.slice/crio-ad1bc8cff4b35058d0b1183149a8c48e29966503cd28e4bb5911f7f4307e25a9 WatchSource:0}: Error finding container ad1bc8cff4b35058d0b1183149a8c48e29966503cd28e4bb5911f7f4307e25a9: Status 404 returned error can't find the container with id ad1bc8cff4b35058d0b1183149a8c48e29966503cd28e4bb5911f7f4307e25a9
Jul 22 04:10:35.154754 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:35.154662    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b9a16f-rbvvh" event={"ID":"5ad568ff-86e7-4d42-881f-7bd6b8177fca","Type":"ContainerStarted","Data":"ad1bc8cff4b35058d0b1183149a8c48e29966503cd28e4bb5911f7f4307e25a9"}
Jul 22 04:10:36.161003 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:36.160915    2570 generic.go:356] "Generic (PLEG): container finished" podID="5ad568ff-86e7-4d42-881f-7bd6b8177fca" containerID="b9edb48268396dc7961e29892f66bfdd9bc8c4a5b5ca7470559dcf2785b876de" exitCode=0
Jul 22 04:10:36.161349 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:36.160999    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b9a16f-rbvvh" event={"ID":"5ad568ff-86e7-4d42-881f-7bd6b8177fca","Type":"ContainerDied","Data":"b9edb48268396dc7961e29892f66bfdd9bc8c4a5b5ca7470559dcf2785b876de"}
Jul 22 04:10:37.300122 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:37.300097    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b9a16f-rbvvh"
Jul 22 04:10:37.462117 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:37.462086    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ad568ff-86e7-4d42-881f-7bd6b8177fca-maas-api-service-ca\") pod \"5ad568ff-86e7-4d42-881f-7bd6b8177fca\" (UID: \"5ad568ff-86e7-4d42-881f-7bd6b8177fca\") "
Jul 22 04:10:37.462417 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:37.462393    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ad568ff-86e7-4d42-881f-7bd6b8177fca-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5ad568ff-86e7-4d42-881f-7bd6b8177fca" (UID: "5ad568ff-86e7-4d42-881f-7bd6b8177fca"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 04:10:37.563661 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:37.563592    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ad568ff-86e7-4d42-881f-7bd6b8177fca-maas-api-service-ca\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:10:38.171197 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:38.171169    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b9a16f-rbvvh"
Jul 22 04:10:38.171197 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:38.171190    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b9a16f-rbvvh" event={"ID":"5ad568ff-86e7-4d42-881f-7bd6b8177fca","Type":"ContainerDied","Data":"ad1bc8cff4b35058d0b1183149a8c48e29966503cd28e4bb5911f7f4307e25a9"}
Jul 22 04:10:38.171426 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:38.171231    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ad1bc8cff4b35058d0b1183149a8c48e29966503cd28e4bb5911f7f4307e25a9"
Jul 22 04:10:48.168020 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:48.167974    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:10:49.576663 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:49.576627    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b9a16f-rbvvh"]
Jul 22 04:10:49.579419 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:49.579393    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-b9a16f-rbvvh"]
Jul 22 04:10:50.851403 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:10:50.851375    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ad568ff-86e7-4d42-881f-7bd6b8177fca" path="/var/lib/kubelet/pods/5ad568ff-86e7-4d42-881f-7bd6b8177fca/volumes"
Jul 22 04:11:07.569879 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:07.569834    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:11:17.759097 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:17.759054    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:11:20.936138 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:20.936104    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b721b6-7qrlm"]
Jul 22 04:11:20.936534 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:20.936433    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ad568ff-86e7-4d42-881f-7bd6b8177fca" containerName="revoke-keys"
Jul 22 04:11:20.936534 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:20.936445    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ad568ff-86e7-4d42-881f-7bd6b8177fca" containerName="revoke-keys"
Jul 22 04:11:20.936534 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:20.936514    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ad568ff-86e7-4d42-881f-7bd6b8177fca" containerName="revoke-keys"
Jul 22 04:11:20.939717 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:20.939702    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b721b6-7qrlm"
Jul 22 04:11:20.942710 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:20.942690    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rbfm6\""
Jul 22 04:11:20.942830 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:20.942691    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 04:11:20.947295 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:20.947268    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b721b6-7qrlm"]
Jul 22 04:11:21.040897 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:21.040855    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/551568eb-8fb8-479f-b4d2-82e140bd656f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b721b6-7qrlm\" (UID: \"551568eb-8fb8-479f-b4d2-82e140bd656f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b721b6-7qrlm"
Jul 22 04:11:21.141560 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:21.141518    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/551568eb-8fb8-479f-b4d2-82e140bd656f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b721b6-7qrlm\" (UID: \"551568eb-8fb8-479f-b4d2-82e140bd656f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b721b6-7qrlm"
Jul 22 04:11:21.142197 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:21.142176    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/551568eb-8fb8-479f-b4d2-82e140bd656f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b721b6-7qrlm\" (UID: \"551568eb-8fb8-479f-b4d2-82e140bd656f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b721b6-7qrlm"
Jul 22 04:11:21.250932 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:21.250849    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b721b6-7qrlm"
Jul 22 04:11:21.397728 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:21.397701    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b721b6-7qrlm"]
Jul 22 04:11:21.399752 ip-10-0-132-89 kubenswrapper[2570]: W0722 04:11:21.399727    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod551568eb_8fb8_479f_b4d2_82e140bd656f.slice/crio-d9485f4c84980c5b78a113a18bf859ff318d29eed3df1d3950e7da25e45a2c88 WatchSource:0}: Error finding container d9485f4c84980c5b78a113a18bf859ff318d29eed3df1d3950e7da25e45a2c88: Status 404 returned error can't find the container with id d9485f4c84980c5b78a113a18bf859ff318d29eed3df1d3950e7da25e45a2c88
Jul 22 04:11:22.378205 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:22.378114    2570 generic.go:356] "Generic (PLEG): container finished" podID="551568eb-8fb8-479f-b4d2-82e140bd656f" containerID="648fe2b80c015e4f8ac0dbefffbad14eef590ff200215d3e2a432f00718e08a1" exitCode=0
Jul 22 04:11:22.378205 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:22.378178    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b721b6-7qrlm" event={"ID":"551568eb-8fb8-479f-b4d2-82e140bd656f","Type":"ContainerDied","Data":"648fe2b80c015e4f8ac0dbefffbad14eef590ff200215d3e2a432f00718e08a1"}
Jul 22 04:11:22.378581 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:22.378215    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b721b6-7qrlm" event={"ID":"551568eb-8fb8-479f-b4d2-82e140bd656f","Type":"ContainerStarted","Data":"d9485f4c84980c5b78a113a18bf859ff318d29eed3df1d3950e7da25e45a2c88"}
Jul 22 04:11:23.514821 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:23.514794    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b721b6-7qrlm"
Jul 22 04:11:23.663844 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:23.663759    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/551568eb-8fb8-479f-b4d2-82e140bd656f-maas-api-service-ca\") pod \"551568eb-8fb8-479f-b4d2-82e140bd656f\" (UID: \"551568eb-8fb8-479f-b4d2-82e140bd656f\") "
Jul 22 04:11:23.664187 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:23.664162    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/551568eb-8fb8-479f-b4d2-82e140bd656f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "551568eb-8fb8-479f-b4d2-82e140bd656f" (UID: "551568eb-8fb8-479f-b4d2-82e140bd656f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 04:11:23.765286 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:23.765248    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/551568eb-8fb8-479f-b4d2-82e140bd656f-maas-api-service-ca\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:11:24.388888 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:24.388855    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b721b6-7qrlm" event={"ID":"551568eb-8fb8-479f-b4d2-82e140bd656f","Type":"ContainerDied","Data":"d9485f4c84980c5b78a113a18bf859ff318d29eed3df1d3950e7da25e45a2c88"}
Jul 22 04:11:24.388888 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:24.388881    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b721b6-7qrlm"
Jul 22 04:11:24.389097 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:24.388905    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d9485f4c84980c5b78a113a18bf859ff318d29eed3df1d3950e7da25e45a2c88"
Jul 22 04:11:33.007442 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:33.007409    2570 scope.go:117] "RemoveContainer" containerID="a0b313738a3fe08e4aee1968f30e06ca82b8ad3f5a36e7267633da3d32f7bd3e"
Jul 22 04:11:41.070000 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:41.069968    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b721b6-7qrlm"]
Jul 22 04:11:41.072831 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:41.072808    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b721b6-7qrlm"]
Jul 22 04:11:42.851685 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:42.851650    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="551568eb-8fb8-479f-b4d2-82e140bd656f" path="/var/lib/kubelet/pods/551568eb-8fb8-479f-b4d2-82e140bd656f/volumes"
Jul 22 04:11:42.962053 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:42.962005    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:11:47.966842 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:47.966809    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-656735-gdcqq"]
Jul 22 04:11:47.967292 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:47.967201    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="551568eb-8fb8-479f-b4d2-82e140bd656f" containerName="revoke-keys"
Jul 22 04:11:47.967292 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:47.967215    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="551568eb-8fb8-479f-b4d2-82e140bd656f" containerName="revoke-keys"
Jul 22 04:11:47.967292 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:47.967288    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="551568eb-8fb8-479f-b4d2-82e140bd656f" containerName="revoke-keys"
Jul 22 04:11:47.970427 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:47.970411    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-656735-gdcqq"
Jul 22 04:11:47.972821 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:47.972799    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 04:11:47.972925 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:47.972844    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rbfm6\""
Jul 22 04:11:47.977016 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:47.976990    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-656735-gdcqq"]
Jul 22 04:11:48.074372 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:48.074332    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/34789126-cd6d-4ad2-85ce-19eb76bf2a2b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-656735-gdcqq\" (UID: \"34789126-cd6d-4ad2-85ce-19eb76bf2a2b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-656735-gdcqq"
Jul 22 04:11:48.175147 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:48.175097    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/34789126-cd6d-4ad2-85ce-19eb76bf2a2b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-656735-gdcqq\" (UID: \"34789126-cd6d-4ad2-85ce-19eb76bf2a2b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-656735-gdcqq"
Jul 22 04:11:48.175817 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:48.175799    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/34789126-cd6d-4ad2-85ce-19eb76bf2a2b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-656735-gdcqq\" (UID: \"34789126-cd6d-4ad2-85ce-19eb76bf2a2b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-656735-gdcqq"
Jul 22 04:11:48.282467 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:48.282378    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-656735-gdcqq"
Jul 22 04:11:48.420367 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:48.420331    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-656735-gdcqq"]
Jul 22 04:11:48.423399 ip-10-0-132-89 kubenswrapper[2570]: W0722 04:11:48.423371    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34789126_cd6d_4ad2_85ce_19eb76bf2a2b.slice/crio-6680610d3b222a2946ea6caa7ca6c20c6d39cc370faa324c8b76fbb0ffd136b3 WatchSource:0}: Error finding container 6680610d3b222a2946ea6caa7ca6c20c6d39cc370faa324c8b76fbb0ffd136b3: Status 404 returned error can't find the container with id 6680610d3b222a2946ea6caa7ca6c20c6d39cc370faa324c8b76fbb0ffd136b3
Jul 22 04:11:48.499718 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:48.499685    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-656735-gdcqq" event={"ID":"34789126-cd6d-4ad2-85ce-19eb76bf2a2b","Type":"ContainerStarted","Data":"6680610d3b222a2946ea6caa7ca6c20c6d39cc370faa324c8b76fbb0ffd136b3"}
Jul 22 04:11:49.505944 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:49.505910    2570 generic.go:356] "Generic (PLEG): container finished" podID="34789126-cd6d-4ad2-85ce-19eb76bf2a2b" containerID="e950c1c871996ddb39730a7fb3eda92f0ff19fc137fc8f158875047395333077" exitCode=0
Jul 22 04:11:49.506364 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:49.505976    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-656735-gdcqq" event={"ID":"34789126-cd6d-4ad2-85ce-19eb76bf2a2b","Type":"ContainerDied","Data":"e950c1c871996ddb39730a7fb3eda92f0ff19fc137fc8f158875047395333077"}
Jul 22 04:11:50.639934 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:50.639904    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-656735-gdcqq"
Jul 22 04:11:50.698464 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:50.698425    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/34789126-cd6d-4ad2-85ce-19eb76bf2a2b-maas-api-service-ca\") pod \"34789126-cd6d-4ad2-85ce-19eb76bf2a2b\" (UID: \"34789126-cd6d-4ad2-85ce-19eb76bf2a2b\") "
Jul 22 04:11:50.698810 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:50.698788    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34789126-cd6d-4ad2-85ce-19eb76bf2a2b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "34789126-cd6d-4ad2-85ce-19eb76bf2a2b" (UID: "34789126-cd6d-4ad2-85ce-19eb76bf2a2b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 04:11:50.799696 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:50.799581    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/34789126-cd6d-4ad2-85ce-19eb76bf2a2b-maas-api-service-ca\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:11:51.517261 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:51.517227    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-656735-gdcqq" event={"ID":"34789126-cd6d-4ad2-85ce-19eb76bf2a2b","Type":"ContainerDied","Data":"6680610d3b222a2946ea6caa7ca6c20c6d39cc370faa324c8b76fbb0ffd136b3"}
Jul 22 04:11:51.517493 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:51.517274    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6680610d3b222a2946ea6caa7ca6c20c6d39cc370faa324c8b76fbb0ffd136b3"
Jul 22 04:11:51.517493 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:11:51.517239    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-656735-gdcqq"
Jul 22 04:12:08.052049 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:08.052006    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-656735-gdcqq"]
Jul 22 04:12:08.054596 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:08.054573    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-656735-gdcqq"]
Jul 22 04:12:08.517089 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:08.517053    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:12:08.852203 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:08.852124    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34789126-cd6d-4ad2-85ce-19eb76bf2a2b" path="/var/lib/kubelet/pods/34789126-cd6d-4ad2-85ce-19eb76bf2a2b/volumes"
Jul 22 04:12:26.365111 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:26.365025    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:12:37.373604 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:37.373567    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:12:41.579559 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:41.579515    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-5586bf586c-m78fh"]
Jul 22 04:12:41.579947 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:41.579913    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34789126-cd6d-4ad2-85ce-19eb76bf2a2b" containerName="revoke-keys"
Jul 22 04:12:41.579947 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:41.579925    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="34789126-cd6d-4ad2-85ce-19eb76bf2a2b" containerName="revoke-keys"
Jul 22 04:12:41.580030 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:41.580002    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="34789126-cd6d-4ad2-85ce-19eb76bf2a2b" containerName="revoke-keys"
Jul 22 04:12:41.583039 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:41.583022    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-5586bf586c-m78fh"
Jul 22 04:12:41.587292 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:41.587263    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-dd1671\"/\"openshift-service-ca.crt\""
Jul 22 04:12:41.587562 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:41.587313    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-dd1671\"/\"kube-root-ca.crt\""
Jul 22 04:12:41.587562 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:41.587359    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-dd1671\"/\"default-dockercfg-2jjf2\""
Jul 22 04:12:41.587562 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:41.587370    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-dd1671\"/\"auth-test-model-dd1671-kserve-self-signed-certs\""
Jul 22 04:12:41.594310 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:41.594287    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-5586bf586c-m78fh"]
Jul 22 04:12:41.650666 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:41.650602    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-model-cache\") pod \"auth-test-model-dd1671-kserve-5586bf586c-m78fh\" (UID: \"ca5efdfb-d86c-4174-b279-88ae6e1b81e0\") " pod="ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-5586bf586c-m78fh"
Jul 22 04:12:41.650666 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:41.650660    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-tmp-dir\") pod \"auth-test-model-dd1671-kserve-5586bf586c-m78fh\" (UID: \"ca5efdfb-d86c-4174-b279-88ae6e1b81e0\") " pod="ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-5586bf586c-m78fh"
Jul 22 04:12:41.650928 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:41.650738    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-tls-certs\") pod \"auth-test-model-dd1671-kserve-5586bf586c-m78fh\" (UID: \"ca5efdfb-d86c-4174-b279-88ae6e1b81e0\") " pod="ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-5586bf586c-m78fh"
Jul 22 04:12:41.650928 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:41.650782    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxlmh\" (UniqueName: \"kubernetes.io/projected/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-kube-api-access-sxlmh\") pod \"auth-test-model-dd1671-kserve-5586bf586c-m78fh\" (UID: \"ca5efdfb-d86c-4174-b279-88ae6e1b81e0\") " pod="ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-5586bf586c-m78fh"
Jul 22 04:12:41.650928 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:41.650882    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-home\") pod \"auth-test-model-dd1671-kserve-5586bf586c-m78fh\" (UID: \"ca5efdfb-d86c-4174-b279-88ae6e1b81e0\") " pod="ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-5586bf586c-m78fh"
Jul 22 04:12:41.651061 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:41.650947    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-dshm\") pod \"auth-test-model-dd1671-kserve-5586bf586c-m78fh\" (UID: \"ca5efdfb-d86c-4174-b279-88ae6e1b81e0\") " pod="ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-5586bf586c-m78fh"
Jul 22 04:12:41.751939 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:41.751905    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-model-cache\") pod \"auth-test-model-dd1671-kserve-5586bf586c-m78fh\" (UID: \"ca5efdfb-d86c-4174-b279-88ae6e1b81e0\") " pod="ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-5586bf586c-m78fh"
Jul 22 04:12:41.751939 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:41.751938    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-tmp-dir\") pod \"auth-test-model-dd1671-kserve-5586bf586c-m78fh\" (UID: \"ca5efdfb-d86c-4174-b279-88ae6e1b81e0\") " pod="ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-5586bf586c-m78fh"
Jul 22 04:12:41.752202 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:41.751960    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-tls-certs\") pod \"auth-test-model-dd1671-kserve-5586bf586c-m78fh\" (UID: \"ca5efdfb-d86c-4174-b279-88ae6e1b81e0\") " pod="ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-5586bf586c-m78fh"
Jul 22 04:12:41.752202 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:41.751994    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxlmh\" (UniqueName: \"kubernetes.io/projected/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-kube-api-access-sxlmh\") pod \"auth-test-model-dd1671-kserve-5586bf586c-m78fh\" (UID: \"ca5efdfb-d86c-4174-b279-88ae6e1b81e0\") " pod="ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-5586bf586c-m78fh"
Jul 22 04:12:41.752202 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:41.752045    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-home\") pod \"auth-test-model-dd1671-kserve-5586bf586c-m78fh\" (UID: \"ca5efdfb-d86c-4174-b279-88ae6e1b81e0\") " pod="ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-5586bf586c-m78fh"
Jul 22 04:12:41.752202 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:41.752078    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-dshm\") pod \"auth-test-model-dd1671-kserve-5586bf586c-m78fh\" (UID: \"ca5efdfb-d86c-4174-b279-88ae6e1b81e0\") " pod="ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-5586bf586c-m78fh"
Jul 22 04:12:41.752451 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:41.752426    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-tmp-dir\") pod \"auth-test-model-dd1671-kserve-5586bf586c-m78fh\" (UID: \"ca5efdfb-d86c-4174-b279-88ae6e1b81e0\") " pod="ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-5586bf586c-m78fh"
Jul 22 04:12:41.752518 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:41.752438    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-model-cache\") pod \"auth-test-model-dd1671-kserve-5586bf586c-m78fh\" (UID: \"ca5efdfb-d86c-4174-b279-88ae6e1b81e0\") " pod="ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-5586bf586c-m78fh"
Jul 22 04:12:41.752573 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:41.752530    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-home\") pod \"auth-test-model-dd1671-kserve-5586bf586c-m78fh\" (UID: \"ca5efdfb-d86c-4174-b279-88ae6e1b81e0\") " pod="ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-5586bf586c-m78fh"
Jul 22 04:12:41.754645 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:41.754603    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-dshm\") pod \"auth-test-model-dd1671-kserve-5586bf586c-m78fh\" (UID: \"ca5efdfb-d86c-4174-b279-88ae6e1b81e0\") " pod="ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-5586bf586c-m78fh"
Jul 22 04:12:41.754847 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:41.754830    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-tls-certs\") pod \"auth-test-model-dd1671-kserve-5586bf586c-m78fh\" (UID: \"ca5efdfb-d86c-4174-b279-88ae6e1b81e0\") " pod="ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-5586bf586c-m78fh"
Jul 22 04:12:41.761206 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:41.761179    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxlmh\" (UniqueName: \"kubernetes.io/projected/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-kube-api-access-sxlmh\") pod \"auth-test-model-dd1671-kserve-5586bf586c-m78fh\" (UID: \"ca5efdfb-d86c-4174-b279-88ae6e1b81e0\") " pod="ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-5586bf586c-m78fh"
Jul 22 04:12:41.892681 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:41.892563    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-5586bf586c-m78fh"
Jul 22 04:12:42.034340 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:42.034314    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-5586bf586c-m78fh"]
Jul 22 04:12:42.037429 ip-10-0-132-89 kubenswrapper[2570]: W0722 04:12:42.037397    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podca5efdfb_d86c_4174_b279_88ae6e1b81e0.slice/crio-4dc846f9b4188bc2baa5034542bd6bc247a31eaf0e8c80cfd56400756b27680f WatchSource:0}: Error finding container 4dc846f9b4188bc2baa5034542bd6bc247a31eaf0e8c80cfd56400756b27680f: Status 404 returned error can't find the container with id 4dc846f9b4188bc2baa5034542bd6bc247a31eaf0e8c80cfd56400756b27680f
Jul 22 04:12:42.759425 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:42.759387    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-5586bf586c-m78fh" event={"ID":"ca5efdfb-d86c-4174-b279-88ae6e1b81e0","Type":"ContainerStarted","Data":"d7137494cf4bf04a2a04ab5baecb5afaad0566fabacccab15653d1381b3b2f3b"}
Jul 22 04:12:42.760013 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:42.759434    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-5586bf586c-m78fh" event={"ID":"ca5efdfb-d86c-4174-b279-88ae6e1b81e0","Type":"ContainerStarted","Data":"4dc846f9b4188bc2baa5034542bd6bc247a31eaf0e8c80cfd56400756b27680f"}
Jul 22 04:12:42.781209 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:42.781161    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-5586bf586c-m78fh" podStartSLOduration=1.781143348 podStartE2EDuration="1.781143348s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:12:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:12:42.780314522 +0000 UTC m=+2350.398524529" watchObservedRunningTime="2026-07-22 04:12:42.781143348 +0000 UTC m=+2350.399353394"
Jul 22 04:12:46.661773 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:46.661741    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:12:51.893373 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:51.893336    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-5586bf586c-m78fh"
Jul 22 04:12:51.893792 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:51.893395    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-5586bf586c-m78fh"
Jul 22 04:12:51.905940 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:51.905910    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-5586bf586c-m78fh"
Jul 22 04:12:52.817631 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:12:52.817581    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-5586bf586c-m78fh"
Jul 22 04:13:21.585291 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:13:21.585254    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-38610b/auth-test-model-38610b-kserve-76f9f5799-56nhm"]
Jul 22 04:13:21.588834 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:13:21.588817    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-38610b/auth-test-model-38610b-kserve-76f9f5799-56nhm"
Jul 22 04:13:21.592832 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:13:21.592810    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-38610b\"/\"default-dockercfg-j47zf\""
Jul 22 04:13:21.592980 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:13:21.592809    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-38610b\"/\"auth-test-model-38610b-kserve-self-signed-certs\""
Jul 22 04:13:21.592980 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:13:21.592810    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-38610b\"/\"openshift-service-ca.crt\""
Jul 22 04:13:21.592980 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:13:21.592948    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-38610b\"/\"kube-root-ca.crt\""
Jul 22 04:13:21.599104 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:13:21.599084    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-38610b/auth-test-model-38610b-kserve-76f9f5799-56nhm"]
Jul 22 04:13:21.602583 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:13:21.602560    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f-dshm\") pod \"auth-test-model-38610b-kserve-76f9f5799-56nhm\" (UID: \"8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f\") " pod="ai-tenant-e2e-shared-b-38610b/auth-test-model-38610b-kserve-76f9f5799-56nhm"
Jul 22 04:13:21.602697 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:13:21.602602    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f-model-cache\") pod \"auth-test-model-38610b-kserve-76f9f5799-56nhm\" (UID: \"8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f\") " pod="ai-tenant-e2e-shared-b-38610b/auth-test-model-38610b-kserve-76f9f5799-56nhm"
Jul 22 04:13:21.602739 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:13:21.602722    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f-home\") pod \"auth-test-model-38610b-kserve-76f9f5799-56nhm\" (UID: \"8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f\") " pod="ai-tenant-e2e-shared-b-38610b/auth-test-model-38610b-kserve-76f9f5799-56nhm"
Jul 22 04:13:21.602816 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:13:21.602792    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f-tmp-dir\") pod \"auth-test-model-38610b-kserve-76f9f5799-56nhm\" (UID: \"8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f\") " pod="ai-tenant-e2e-shared-b-38610b/auth-test-model-38610b-kserve-76f9f5799-56nhm"
Jul 22 04:13:21.602872 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:13:21.602857    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scffc\" (UniqueName: \"kubernetes.io/projected/8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f-kube-api-access-scffc\") pod \"auth-test-model-38610b-kserve-76f9f5799-56nhm\" (UID: \"8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f\") " pod="ai-tenant-e2e-shared-b-38610b/auth-test-model-38610b-kserve-76f9f5799-56nhm"
Jul 22 04:13:21.602915 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:13:21.602901    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f-tls-certs\") pod \"auth-test-model-38610b-kserve-76f9f5799-56nhm\" (UID: \"8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f\") " pod="ai-tenant-e2e-shared-b-38610b/auth-test-model-38610b-kserve-76f9f5799-56nhm"
Jul 22 04:13:21.704175 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:13:21.704134    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scffc\" (UniqueName: \"kubernetes.io/projected/8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f-kube-api-access-scffc\") pod \"auth-test-model-38610b-kserve-76f9f5799-56nhm\" (UID: \"8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f\") " pod="ai-tenant-e2e-shared-b-38610b/auth-test-model-38610b-kserve-76f9f5799-56nhm"
Jul 22 04:13:21.704175 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:13:21.704176    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f-tls-certs\") pod \"auth-test-model-38610b-kserve-76f9f5799-56nhm\" (UID: \"8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f\") " pod="ai-tenant-e2e-shared-b-38610b/auth-test-model-38610b-kserve-76f9f5799-56nhm"
Jul 22 04:13:21.704400 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:13:21.704198    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f-dshm\") pod \"auth-test-model-38610b-kserve-76f9f5799-56nhm\" (UID: \"8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f\") " pod="ai-tenant-e2e-shared-b-38610b/auth-test-model-38610b-kserve-76f9f5799-56nhm"
Jul 22 04:13:21.704400 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:13:21.704228    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f-model-cache\") pod \"auth-test-model-38610b-kserve-76f9f5799-56nhm\" (UID: \"8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f\") " pod="ai-tenant-e2e-shared-b-38610b/auth-test-model-38610b-kserve-76f9f5799-56nhm"
Jul 22 04:13:21.704400 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:13:21.704272    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f-home\") pod \"auth-test-model-38610b-kserve-76f9f5799-56nhm\" (UID: \"8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f\") " pod="ai-tenant-e2e-shared-b-38610b/auth-test-model-38610b-kserve-76f9f5799-56nhm"
Jul 22 04:13:21.704400 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:13:21.704293    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f-tmp-dir\") pod \"auth-test-model-38610b-kserve-76f9f5799-56nhm\" (UID: \"8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f\") " pod="ai-tenant-e2e-shared-b-38610b/auth-test-model-38610b-kserve-76f9f5799-56nhm"
Jul 22 04:13:21.704767 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:13:21.704744    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f-model-cache\") pod \"auth-test-model-38610b-kserve-76f9f5799-56nhm\" (UID: \"8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f\") " pod="ai-tenant-e2e-shared-b-38610b/auth-test-model-38610b-kserve-76f9f5799-56nhm"
Jul 22 04:13:21.704887 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:13:21.704806    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f-home\") pod \"auth-test-model-38610b-kserve-76f9f5799-56nhm\" (UID: \"8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f\") " pod="ai-tenant-e2e-shared-b-38610b/auth-test-model-38610b-kserve-76f9f5799-56nhm"
Jul 22 04:13:21.704887 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:13:21.704825    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f-tmp-dir\") pod \"auth-test-model-38610b-kserve-76f9f5799-56nhm\" (UID: \"8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f\") " pod="ai-tenant-e2e-shared-b-38610b/auth-test-model-38610b-kserve-76f9f5799-56nhm"
Jul 22 04:13:21.706672 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:13:21.706654    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f-dshm\") pod \"auth-test-model-38610b-kserve-76f9f5799-56nhm\" (UID: \"8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f\") " pod="ai-tenant-e2e-shared-b-38610b/auth-test-model-38610b-kserve-76f9f5799-56nhm"
Jul 22 04:13:21.706958 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:13:21.706941    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f-tls-certs\") pod \"auth-test-model-38610b-kserve-76f9f5799-56nhm\" (UID: \"8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f\") " pod="ai-tenant-e2e-shared-b-38610b/auth-test-model-38610b-kserve-76f9f5799-56nhm"
Jul 22 04:13:21.715293 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:13:21.715269    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-scffc\" (UniqueName: \"kubernetes.io/projected/8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f-kube-api-access-scffc\") pod \"auth-test-model-38610b-kserve-76f9f5799-56nhm\" (UID: \"8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f\") " pod="ai-tenant-e2e-shared-b-38610b/auth-test-model-38610b-kserve-76f9f5799-56nhm"
Jul 22 04:13:21.898964 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:13:21.898869    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-38610b/auth-test-model-38610b-kserve-76f9f5799-56nhm"
Jul 22 04:13:22.040597 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:13:22.040572    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-38610b/auth-test-model-38610b-kserve-76f9f5799-56nhm"]
Jul 22 04:13:22.042496 ip-10-0-132-89 kubenswrapper[2570]: W0722 04:13:22.042465    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8a8f825e_54d6_42b6_a33f_1ab0f5dd0a7f.slice/crio-fb525436940ff813401283df29f6c0522d684c6cc3ce2cd1103574c1b99c88bd WatchSource:0}: Error finding container fb525436940ff813401283df29f6c0522d684c6cc3ce2cd1103574c1b99c88bd: Status 404 returned error can't find the container with id fb525436940ff813401283df29f6c0522d684c6cc3ce2cd1103574c1b99c88bd
Jul 22 04:13:22.957737 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:13:22.957703    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-38610b/auth-test-model-38610b-kserve-76f9f5799-56nhm" event={"ID":"8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f","Type":"ContainerStarted","Data":"0cb290aa5f604be9abc53c21e33c8a85174b23547137329afc591f6c404f7547"}
Jul 22 04:13:22.957737 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:13:22.957748    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-38610b/auth-test-model-38610b-kserve-76f9f5799-56nhm" event={"ID":"8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f","Type":"ContainerStarted","Data":"fb525436940ff813401283df29f6c0522d684c6cc3ce2cd1103574c1b99c88bd"}
Jul 22 04:13:22.975647 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:13:22.975578    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-38610b/auth-test-model-38610b-kserve-76f9f5799-56nhm" podStartSLOduration=1.975564367 podStartE2EDuration="1.975564367s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:13:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:13:22.974330916 +0000 UTC m=+2390.592540923" watchObservedRunningTime="2026-07-22 04:13:22.975564367 +0000 UTC m=+2390.593774373"
Jul 22 04:13:25.260719 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:13:25.260684    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:13:31.899707 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:13:31.899666    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-38610b/auth-test-model-38610b-kserve-76f9f5799-56nhm"
Jul 22 04:13:31.900209 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:13:31.899763    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-38610b/auth-test-model-38610b-kserve-76f9f5799-56nhm"
Jul 22 04:13:31.912473 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:13:31.912447    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-38610b/auth-test-model-38610b-kserve-76f9f5799-56nhm"
Jul 22 04:13:32.013170 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:13:32.013135    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-38610b/auth-test-model-38610b-kserve-76f9f5799-56nhm"
Jul 22 04:13:32.921823 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:13:32.921794    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2p4qp_9f0defcc-ccff-43be-8206-54c772885080/console-operator/1.log"
Jul 22 04:13:32.926718 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:13:32.926690    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2p4qp_9f0defcc-ccff-43be-8206-54c772885080/console-operator/1.log"
Jul 22 04:13:39.179387 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:13:39.179347    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:13:55.570835 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:13:55.570800    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:14:14.170819 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:14:14.170781    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:14:33.271909 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:14:33.268803    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:14:50.364529 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:14:50.364493    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:15:00.169055 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:00.169024    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744895-zxz6c"]
Jul 22 04:15:00.172452 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:00.172426    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744895-zxz6c"
Jul 22 04:15:00.173523 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:00.173497    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-38610b-29744895-kzdjw"]
Jul 22 04:15:00.176778 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:00.176759    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-38610b-29744895-kzdjw"
Jul 22 04:15:00.181324 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:00.181309    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rbfm6\""
Jul 22 04:15:00.182733 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:00.182716    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744895-zxz6c"
Jul 22 04:15:00.187602 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:00.187579    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744895-zxz6c"]
Jul 22 04:15:00.189746 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:00.189728    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-38610b-29744895-kzdjw"
Jul 22 04:15:00.217021 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:00.216973    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-38610b-29744895-kzdjw"]
Jul 22 04:15:00.342589 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:00.342563    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744895-zxz6c"]
Jul 22 04:15:00.343080 ip-10-0-132-89 kubenswrapper[2570]: W0722 04:15:00.343056    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4093e54c_8d9a_4eff_b71e_167208d29dd7.slice/crio-a98df414c2ddb809082d3a7989c40e370803a81863582252a21557eaeabd52b4 WatchSource:0}: Error finding container a98df414c2ddb809082d3a7989c40e370803a81863582252a21557eaeabd52b4: Status 404 returned error can't find the container with id a98df414c2ddb809082d3a7989c40e370803a81863582252a21557eaeabd52b4
Jul 22 04:15:00.369370 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:00.369341    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-38610b-29744895-kzdjw"]
Jul 22 04:15:00.434818 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:00.434728    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-38610b-29744895-kzdjw" event={"ID":"9c01c66b-bf2c-4ac5-a272-0826016b0f39","Type":"ContainerStarted","Data":"42515677e68846d323326fd68e3ef40182a2ea09476ae46de10b9886a5a75b03"}
Jul 22 04:15:00.435981 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:00.435953    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744895-zxz6c" event={"ID":"4093e54c-8d9a-4eff-b71e-167208d29dd7","Type":"ContainerStarted","Data":"a98df414c2ddb809082d3a7989c40e370803a81863582252a21557eaeabd52b4"}
Jul 22 04:15:01.442082 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:01.442041    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744895-zxz6c" event={"ID":"4093e54c-8d9a-4eff-b71e-167208d29dd7","Type":"ContainerStarted","Data":"47d4468f69fb2d633914cebcb3584487f5cbb8333c2aec09effefc9dad31fa4a"}
Jul 22 04:15:01.443700 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:01.443675    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-38610b-29744895-kzdjw" event={"ID":"9c01c66b-bf2c-4ac5-a272-0826016b0f39","Type":"ContainerStarted","Data":"7bfb16cd7fc671879331e9262acb46b759b771a1eb16453c69a43423a504fb46"}
Jul 22 04:15:01.458915 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:01.458857    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744895-zxz6c" podStartSLOduration=1.4588386899999999 podStartE2EDuration="1.45883869s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:15:01.457666707 +0000 UTC m=+2489.075876743" watchObservedRunningTime="2026-07-22 04:15:01.45883869 +0000 UTC m=+2489.077048707"
Jul 22 04:15:01.475325 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:01.475281    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-38610b-29744895-kzdjw" podStartSLOduration=1.475262775 podStartE2EDuration="1.475262775s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:15:01.473856912 +0000 UTC m=+2489.092066939" watchObservedRunningTime="2026-07-22 04:15:01.475262775 +0000 UTC m=+2489.093472783"
Jul 22 04:15:02.449689 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:02.449654    2570 generic.go:356] "Generic (PLEG): container finished" podID="9c01c66b-bf2c-4ac5-a272-0826016b0f39" containerID="7bfb16cd7fc671879331e9262acb46b759b771a1eb16453c69a43423a504fb46" exitCode=0
Jul 22 04:15:02.450109 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:02.449743    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-38610b-29744895-kzdjw" event={"ID":"9c01c66b-bf2c-4ac5-a272-0826016b0f39","Type":"ContainerDied","Data":"7bfb16cd7fc671879331e9262acb46b759b771a1eb16453c69a43423a504fb46"}
Jul 22 04:15:02.451216 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:02.451190    2570 generic.go:356] "Generic (PLEG): container finished" podID="4093e54c-8d9a-4eff-b71e-167208d29dd7" containerID="47d4468f69fb2d633914cebcb3584487f5cbb8333c2aec09effefc9dad31fa4a" exitCode=0
Jul 22 04:15:02.451334 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:02.451253    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744895-zxz6c" event={"ID":"4093e54c-8d9a-4eff-b71e-167208d29dd7","Type":"ContainerDied","Data":"47d4468f69fb2d633914cebcb3584487f5cbb8333c2aec09effefc9dad31fa4a"}
Jul 22 04:15:03.590596 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:03.590566    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744895-zxz6c"
Jul 22 04:15:03.623140 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:03.623115    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-38610b-29744895-kzdjw"
Jul 22 04:15:04.461634 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:04.461594    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744895-zxz6c"
Jul 22 04:15:04.461833 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:04.461656    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744895-zxz6c" event={"ID":"4093e54c-8d9a-4eff-b71e-167208d29dd7","Type":"ContainerDied","Data":"a98df414c2ddb809082d3a7989c40e370803a81863582252a21557eaeabd52b4"}
Jul 22 04:15:04.461833 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:04.461706    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a98df414c2ddb809082d3a7989c40e370803a81863582252a21557eaeabd52b4"
Jul 22 04:15:04.462947 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:04.462921    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-38610b-29744895-kzdjw" event={"ID":"9c01c66b-bf2c-4ac5-a272-0826016b0f39","Type":"ContainerDied","Data":"42515677e68846d323326fd68e3ef40182a2ea09476ae46de10b9886a5a75b03"}
Jul 22 04:15:04.462947 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:04.462941    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-38610b-29744895-kzdjw"
Jul 22 04:15:04.463128 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:04.462962    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="42515677e68846d323326fd68e3ef40182a2ea09476ae46de10b9886a5a75b03"
Jul 22 04:15:08.375749 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:08.375718    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:15:18.919947 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:18.919911    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-5586bf586c-m78fh"]
Jul 22 04:15:18.976463 ip-10-0-132-89 kubenswrapper[2570]: E0722 04:15:18.976425    2570 secret.go:189] Couldn't get secret ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-self-signed-certs: secret "auth-test-model-dd1671-kserve-self-signed-certs" not found
Jul 22 04:15:18.976708 ip-10-0-132-89 kubenswrapper[2570]: E0722 04:15:18.976533    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-tls-certs podName:ca5efdfb-d86c-4174-b279-88ae6e1b81e0 nodeName:}" failed. No retries permitted until 2026-07-22 04:15:19.476508382 +0000 UTC m=+2507.094718386 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-tls-certs") pod "auth-test-model-dd1671-kserve-5586bf586c-m78fh" (UID: "ca5efdfb-d86c-4174-b279-88ae6e1b81e0") : secret "auth-test-model-dd1671-kserve-self-signed-certs" not found
Jul 22 04:15:19.482547 ip-10-0-132-89 kubenswrapper[2570]: E0722 04:15:19.482517    2570 secret.go:189] Couldn't get secret ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-self-signed-certs: secret "auth-test-model-dd1671-kserve-self-signed-certs" not found
Jul 22 04:15:19.482736 ip-10-0-132-89 kubenswrapper[2570]: E0722 04:15:19.482580    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-tls-certs podName:ca5efdfb-d86c-4174-b279-88ae6e1b81e0 nodeName:}" failed. No retries permitted until 2026-07-22 04:15:20.482566907 +0000 UTC m=+2508.100776893 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-tls-certs") pod "auth-test-model-dd1671-kserve-5586bf586c-m78fh" (UID: "ca5efdfb-d86c-4174-b279-88ae6e1b81e0") : secret "auth-test-model-dd1671-kserve-self-signed-certs" not found
Jul 22 04:15:20.016065 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:20.016024    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-38610b/auth-test-model-38610b-kserve-76f9f5799-56nhm"]
Jul 22 04:15:20.492223 ip-10-0-132-89 kubenswrapper[2570]: E0722 04:15:20.492186    2570 secret.go:189] Couldn't get secret ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-self-signed-certs: secret "auth-test-model-dd1671-kserve-self-signed-certs" not found
Jul 22 04:15:20.492417 ip-10-0-132-89 kubenswrapper[2570]: E0722 04:15:20.492265    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-tls-certs podName:ca5efdfb-d86c-4174-b279-88ae6e1b81e0 nodeName:}" failed. No retries permitted until 2026-07-22 04:15:22.492246686 +0000 UTC m=+2510.110456671 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-tls-certs") pod "auth-test-model-dd1671-kserve-5586bf586c-m78fh" (UID: "ca5efdfb-d86c-4174-b279-88ae6e1b81e0") : secret "auth-test-model-dd1671-kserve-self-signed-certs" not found
Jul 22 04:15:22.513419 ip-10-0-132-89 kubenswrapper[2570]: E0722 04:15:22.513368    2570 secret.go:189] Couldn't get secret ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-self-signed-certs: secret "auth-test-model-dd1671-kserve-self-signed-certs" not found
Jul 22 04:15:22.513927 ip-10-0-132-89 kubenswrapper[2570]: E0722 04:15:22.513459    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-tls-certs podName:ca5efdfb-d86c-4174-b279-88ae6e1b81e0 nodeName:}" failed. No retries permitted until 2026-07-22 04:15:26.513442895 +0000 UTC m=+2514.131652881 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-tls-certs") pod "auth-test-model-dd1671-kserve-5586bf586c-m78fh" (UID: "ca5efdfb-d86c-4174-b279-88ae6e1b81e0") : secret "auth-test-model-dd1671-kserve-self-signed-certs" not found
Jul 22 04:15:23.472070 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:23.472037    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:15:26.554649 ip-10-0-132-89 kubenswrapper[2570]: E0722 04:15:26.554593    2570 secret.go:189] Couldn't get secret ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-self-signed-certs: secret "auth-test-model-dd1671-kserve-self-signed-certs" not found
Jul 22 04:15:26.555036 ip-10-0-132-89 kubenswrapper[2570]: E0722 04:15:26.554693    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-tls-certs podName:ca5efdfb-d86c-4174-b279-88ae6e1b81e0 nodeName:}" failed. No retries permitted until 2026-07-22 04:15:34.554677565 +0000 UTC m=+2522.172887550 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-tls-certs") pod "auth-test-model-dd1671-kserve-5586bf586c-m78fh" (UID: "ca5efdfb-d86c-4174-b279-88ae6e1b81e0") : secret "auth-test-model-dd1671-kserve-self-signed-certs" not found
Jul 22 04:15:34.554373 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:34.554304    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-5586bf586c-m78fh" podUID="ca5efdfb-d86c-4174-b279-88ae6e1b81e0" containerName="main" containerID="cri-o://d7137494cf4bf04a2a04ab5baecb5afaad0566fabacccab15653d1381b3b2f3b" gracePeriod=45
Jul 22 04:15:34.630727 ip-10-0-132-89 kubenswrapper[2570]: E0722 04:15:34.630691    2570 secret.go:189] Couldn't get secret ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-self-signed-certs: secret "auth-test-model-dd1671-kserve-self-signed-certs" not found
Jul 22 04:15:34.630886 ip-10-0-132-89 kubenswrapper[2570]: E0722 04:15:34.630780    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-tls-certs podName:ca5efdfb-d86c-4174-b279-88ae6e1b81e0 nodeName:}" failed. No retries permitted until 2026-07-22 04:15:50.630758612 +0000 UTC m=+2538.248968598 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-tls-certs") pod "auth-test-model-dd1671-kserve-5586bf586c-m78fh" (UID: "ca5efdfb-d86c-4174-b279-88ae6e1b81e0") : secret "auth-test-model-dd1671-kserve-self-signed-certs" not found
Jul 22 04:15:34.808138 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:34.808067    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-5586bf586c-m78fh"
Jul 22 04:15:34.832434 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:34.832407    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-tmp-dir\") pod \"ca5efdfb-d86c-4174-b279-88ae6e1b81e0\" (UID: \"ca5efdfb-d86c-4174-b279-88ae6e1b81e0\") "
Jul 22 04:15:34.832592 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:34.832463    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-model-cache\") pod \"ca5efdfb-d86c-4174-b279-88ae6e1b81e0\" (UID: \"ca5efdfb-d86c-4174-b279-88ae6e1b81e0\") "
Jul 22 04:15:34.832592 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:34.832484    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-dshm\") pod \"ca5efdfb-d86c-4174-b279-88ae6e1b81e0\" (UID: \"ca5efdfb-d86c-4174-b279-88ae6e1b81e0\") "
Jul 22 04:15:34.832592 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:34.832519    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-home\") pod \"ca5efdfb-d86c-4174-b279-88ae6e1b81e0\" (UID: \"ca5efdfb-d86c-4174-b279-88ae6e1b81e0\") "
Jul 22 04:15:34.832592 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:34.832589    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sxlmh\" (UniqueName: \"kubernetes.io/projected/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-kube-api-access-sxlmh\") pod \"ca5efdfb-d86c-4174-b279-88ae6e1b81e0\" (UID: \"ca5efdfb-d86c-4174-b279-88ae6e1b81e0\") "
Jul 22 04:15:34.832832 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:34.832645    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-tls-certs\") pod \"ca5efdfb-d86c-4174-b279-88ae6e1b81e0\" (UID: \"ca5efdfb-d86c-4174-b279-88ae6e1b81e0\") "
Jul 22 04:15:34.832832 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:34.832747    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ca5efdfb-d86c-4174-b279-88ae6e1b81e0" (UID: "ca5efdfb-d86c-4174-b279-88ae6e1b81e0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 04:15:34.832832 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:34.832804    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-model-cache" (OuterVolumeSpecName: "model-cache") pod "ca5efdfb-d86c-4174-b279-88ae6e1b81e0" (UID: "ca5efdfb-d86c-4174-b279-88ae6e1b81e0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 04:15:34.832832 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:34.832815    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-home" (OuterVolumeSpecName: "home") pod "ca5efdfb-d86c-4174-b279-88ae6e1b81e0" (UID: "ca5efdfb-d86c-4174-b279-88ae6e1b81e0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 04:15:34.833011 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:34.832937    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-home\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:15:34.833011 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:34.832961    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-tmp-dir\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:15:34.833011 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:34.832979    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-model-cache\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:15:34.834918 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:34.834894    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-dshm" (OuterVolumeSpecName: "dshm") pod "ca5efdfb-d86c-4174-b279-88ae6e1b81e0" (UID: "ca5efdfb-d86c-4174-b279-88ae6e1b81e0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 04:15:34.835055 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:34.835005    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ca5efdfb-d86c-4174-b279-88ae6e1b81e0" (UID: "ca5efdfb-d86c-4174-b279-88ae6e1b81e0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 04:15:34.835055 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:34.835024    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-kube-api-access-sxlmh" (OuterVolumeSpecName: "kube-api-access-sxlmh") pod "ca5efdfb-d86c-4174-b279-88ae6e1b81e0" (UID: "ca5efdfb-d86c-4174-b279-88ae6e1b81e0"). InnerVolumeSpecName "kube-api-access-sxlmh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 04:15:34.933779 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:34.933739    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxlmh\" (UniqueName: \"kubernetes.io/projected/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-kube-api-access-sxlmh\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:15:34.933779 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:34.933771    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-tls-certs\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:15:34.933779 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:34.933782    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca5efdfb-d86c-4174-b279-88ae6e1b81e0-dshm\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:15:35.036596 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:35.036539    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-38610b/auth-test-model-38610b-kserve-76f9f5799-56nhm" podUID="8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f" containerName="main" containerID="cri-o://0cb290aa5f604be9abc53c21e33c8a85174b23547137329afc591f6c404f7547" gracePeriod=45
Jul 22 04:15:35.278740 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:35.278716    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-38610b/auth-test-model-38610b-kserve-76f9f5799-56nhm"
Jul 22 04:15:35.337251 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:35.337215    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f-home\") pod \"8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f\" (UID: \"8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f\") "
Jul 22 04:15:35.337448 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:35.337281    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f-tmp-dir\") pod \"8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f\" (UID: \"8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f\") "
Jul 22 04:15:35.337448 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:35.337339    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f-model-cache\") pod \"8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f\" (UID: \"8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f\") "
Jul 22 04:15:35.337448 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:35.337394    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f-tls-certs\") pod \"8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f\" (UID: \"8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f\") "
Jul 22 04:15:35.337448 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:35.337439    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-scffc\" (UniqueName: \"kubernetes.io/projected/8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f-kube-api-access-scffc\") pod \"8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f\" (UID: \"8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f\") "
Jul 22 04:15:35.337686 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:35.337469    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f-dshm\") pod \"8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f\" (UID: \"8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f\") "
Jul 22 04:15:35.337686 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:35.337546    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f-home" (OuterVolumeSpecName: "home") pod "8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f" (UID: "8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 04:15:35.337686 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:35.337592    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f" (UID: "8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 04:15:35.337787 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:35.337681    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f-model-cache" (OuterVolumeSpecName: "model-cache") pod "8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f" (UID: "8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 04:15:35.337828 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:35.337805    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f-home\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:15:35.337828 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:35.337825    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f-tmp-dir\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:15:35.337895 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:35.337837    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f-model-cache\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:15:35.339846 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:35.339816    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f-dshm" (OuterVolumeSpecName: "dshm") pod "8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f" (UID: "8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 04:15:35.339846 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:35.339835    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f-kube-api-access-scffc" (OuterVolumeSpecName: "kube-api-access-scffc") pod "8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f" (UID: "8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f"). InnerVolumeSpecName "kube-api-access-scffc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 04:15:35.340007 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:35.339888    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f" (UID: "8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 04:15:35.438682 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:35.438651    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f-tls-certs\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:15:35.438682 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:35.438677    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-scffc\" (UniqueName: \"kubernetes.io/projected/8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f-kube-api-access-scffc\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:15:35.438682 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:35.438686    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f-dshm\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:15:35.615509 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:35.615473    2570 generic.go:356] "Generic (PLEG): container finished" podID="8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f" containerID="0cb290aa5f604be9abc53c21e33c8a85174b23547137329afc591f6c404f7547" exitCode=0
Jul 22 04:15:35.615967 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:35.615524    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-38610b/auth-test-model-38610b-kserve-76f9f5799-56nhm"
Jul 22 04:15:35.615967 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:35.615562    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-38610b/auth-test-model-38610b-kserve-76f9f5799-56nhm" event={"ID":"8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f","Type":"ContainerDied","Data":"0cb290aa5f604be9abc53c21e33c8a85174b23547137329afc591f6c404f7547"}
Jul 22 04:15:35.615967 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:35.615604    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-38610b/auth-test-model-38610b-kserve-76f9f5799-56nhm" event={"ID":"8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f","Type":"ContainerDied","Data":"fb525436940ff813401283df29f6c0522d684c6cc3ce2cd1103574c1b99c88bd"}
Jul 22 04:15:35.615967 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:35.615648    2570 scope.go:117] "RemoveContainer" containerID="0cb290aa5f604be9abc53c21e33c8a85174b23547137329afc591f6c404f7547"
Jul 22 04:15:35.617006 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:35.616900    2570 generic.go:356] "Generic (PLEG): container finished" podID="ca5efdfb-d86c-4174-b279-88ae6e1b81e0" containerID="d7137494cf4bf04a2a04ab5baecb5afaad0566fabacccab15653d1381b3b2f3b" exitCode=0
Jul 22 04:15:35.617006 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:35.616938    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-5586bf586c-m78fh"
Jul 22 04:15:35.617006 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:35.616986    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-5586bf586c-m78fh" event={"ID":"ca5efdfb-d86c-4174-b279-88ae6e1b81e0","Type":"ContainerDied","Data":"d7137494cf4bf04a2a04ab5baecb5afaad0566fabacccab15653d1381b3b2f3b"}
Jul 22 04:15:35.617154 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:35.617026    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-5586bf586c-m78fh" event={"ID":"ca5efdfb-d86c-4174-b279-88ae6e1b81e0","Type":"ContainerDied","Data":"4dc846f9b4188bc2baa5034542bd6bc247a31eaf0e8c80cfd56400756b27680f"}
Jul 22 04:15:35.625803 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:35.625786    2570 scope.go:117] "RemoveContainer" containerID="0cb290aa5f604be9abc53c21e33c8a85174b23547137329afc591f6c404f7547"
Jul 22 04:15:35.626060 ip-10-0-132-89 kubenswrapper[2570]: E0722 04:15:35.626043    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cb290aa5f604be9abc53c21e33c8a85174b23547137329afc591f6c404f7547\": container with ID starting with 0cb290aa5f604be9abc53c21e33c8a85174b23547137329afc591f6c404f7547 not found: ID does not exist" containerID="0cb290aa5f604be9abc53c21e33c8a85174b23547137329afc591f6c404f7547"
Jul 22 04:15:35.626124 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:35.626067    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cb290aa5f604be9abc53c21e33c8a85174b23547137329afc591f6c404f7547"} err="failed to get container status \"0cb290aa5f604be9abc53c21e33c8a85174b23547137329afc591f6c404f7547\": rpc error: code = NotFound desc = could not find container \"0cb290aa5f604be9abc53c21e33c8a85174b23547137329afc591f6c404f7547\": container with ID starting with 0cb290aa5f604be9abc53c21e33c8a85174b23547137329afc591f6c404f7547 not found: ID does not exist"
Jul 22 04:15:35.626124 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:35.626083    2570 scope.go:117] "RemoveContainer" containerID="d7137494cf4bf04a2a04ab5baecb5afaad0566fabacccab15653d1381b3b2f3b"
Jul 22 04:15:35.632877 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:35.632851    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-5586bf586c-m78fh"]
Jul 22 04:15:35.635012 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:35.634994    2570 scope.go:117] "RemoveContainer" containerID="d7137494cf4bf04a2a04ab5baecb5afaad0566fabacccab15653d1381b3b2f3b"
Jul 22 04:15:35.635300 ip-10-0-132-89 kubenswrapper[2570]: E0722 04:15:35.635278    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7137494cf4bf04a2a04ab5baecb5afaad0566fabacccab15653d1381b3b2f3b\": container with ID starting with d7137494cf4bf04a2a04ab5baecb5afaad0566fabacccab15653d1381b3b2f3b not found: ID does not exist" containerID="d7137494cf4bf04a2a04ab5baecb5afaad0566fabacccab15653d1381b3b2f3b"
Jul 22 04:15:35.635347 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:35.635312    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7137494cf4bf04a2a04ab5baecb5afaad0566fabacccab15653d1381b3b2f3b"} err="failed to get container status \"d7137494cf4bf04a2a04ab5baecb5afaad0566fabacccab15653d1381b3b2f3b\": rpc error: code = NotFound desc = could not find container \"d7137494cf4bf04a2a04ab5baecb5afaad0566fabacccab15653d1381b3b2f3b\": container with ID starting with d7137494cf4bf04a2a04ab5baecb5afaad0566fabacccab15653d1381b3b2f3b not found: ID does not exist"
Jul 22 04:15:35.635708 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:35.635690    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-dd1671/auth-test-model-dd1671-kserve-5586bf586c-m78fh"]
Jul 22 04:15:35.647062 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:35.647010    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-38610b/auth-test-model-38610b-kserve-76f9f5799-56nhm"]
Jul 22 04:15:35.648963 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:35.648943    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-38610b/auth-test-model-38610b-kserve-76f9f5799-56nhm"]
Jul 22 04:15:36.851920 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:36.851886    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f" path="/var/lib/kubelet/pods/8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f/volumes"
Jul 22 04:15:36.852313 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:36.852271    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca5efdfb-d86c-4174-b279-88ae6e1b81e0" path="/var/lib/kubelet/pods/ca5efdfb-d86c-4174-b279-88ae6e1b81e0/volumes"
Jul 22 04:15:39.567548 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:39.567520    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-dd1671/sub-test-model-dd1671-kserve-77cc75454f-ww77g"]
Jul 22 04:15:39.567964 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:39.567892    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c01c66b-bf2c-4ac5-a272-0826016b0f39" containerName="cleanup"
Jul 22 04:15:39.567964 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:39.567905    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c01c66b-bf2c-4ac5-a272-0826016b0f39" containerName="cleanup"
Jul 22 04:15:39.567964 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:39.567912    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f" containerName="main"
Jul 22 04:15:39.567964 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:39.567917    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f" containerName="main"
Jul 22 04:15:39.567964 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:39.567931    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca5efdfb-d86c-4174-b279-88ae6e1b81e0" containerName="main"
Jul 22 04:15:39.567964 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:39.567936    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca5efdfb-d86c-4174-b279-88ae6e1b81e0" containerName="main"
Jul 22 04:15:39.567964 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:39.567945    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4093e54c-8d9a-4eff-b71e-167208d29dd7" containerName="cleanup"
Jul 22 04:15:39.567964 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:39.567950    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="4093e54c-8d9a-4eff-b71e-167208d29dd7" containerName="cleanup"
Jul 22 04:15:39.568223 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:39.568018    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a8f825e-54d6-42b6-a33f-1ab0f5dd0a7f" containerName="main"
Jul 22 04:15:39.568223 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:39.568029    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="4093e54c-8d9a-4eff-b71e-167208d29dd7" containerName="cleanup"
Jul 22 04:15:39.568223 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:39.568035    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c01c66b-bf2c-4ac5-a272-0826016b0f39" containerName="cleanup"
Jul 22 04:15:39.568223 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:39.568040    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca5efdfb-d86c-4174-b279-88ae6e1b81e0" containerName="main"
Jul 22 04:15:39.570889 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:39.570870    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-dd1671/sub-test-model-dd1671-kserve-77cc75454f-ww77g"
Jul 22 04:15:39.574875 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:39.574853    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-dd1671\"/\"sub-test-model-dd1671-kserve-self-signed-certs\""
Jul 22 04:15:39.575036 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:39.574876    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-dd1671\"/\"kube-root-ca.crt\""
Jul 22 04:15:39.575036 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:39.574859    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-dd1671\"/\"default-dockercfg-2jjf2\""
Jul 22 04:15:39.575036 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:39.574853    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-dd1671\"/\"openshift-service-ca.crt\""
Jul 22 04:15:39.583111 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:39.583089    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-dd1671/sub-test-model-dd1671-kserve-77cc75454f-ww77g"]
Jul 22 04:15:39.674455 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:39.674411    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88c3a6df-756b-43f0-8313-2346af0de6bc-dshm\") pod \"sub-test-model-dd1671-kserve-77cc75454f-ww77g\" (UID: \"88c3a6df-756b-43f0-8313-2346af0de6bc\") " pod="ai-tenant-e2e-shared-a-dd1671/sub-test-model-dd1671-kserve-77cc75454f-ww77g"
Jul 22 04:15:39.674455 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:39.674450    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88c3a6df-756b-43f0-8313-2346af0de6bc-tmp-dir\") pod \"sub-test-model-dd1671-kserve-77cc75454f-ww77g\" (UID: \"88c3a6df-756b-43f0-8313-2346af0de6bc\") " pod="ai-tenant-e2e-shared-a-dd1671/sub-test-model-dd1671-kserve-77cc75454f-ww77g"
Jul 22 04:15:39.674715 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:39.674474    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88c3a6df-756b-43f0-8313-2346af0de6bc-tls-certs\") pod \"sub-test-model-dd1671-kserve-77cc75454f-ww77g\" (UID: \"88c3a6df-756b-43f0-8313-2346af0de6bc\") " pod="ai-tenant-e2e-shared-a-dd1671/sub-test-model-dd1671-kserve-77cc75454f-ww77g"
Jul 22 04:15:39.674715 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:39.674578    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btfsc\" (UniqueName: \"kubernetes.io/projected/88c3a6df-756b-43f0-8313-2346af0de6bc-kube-api-access-btfsc\") pod \"sub-test-model-dd1671-kserve-77cc75454f-ww77g\" (UID: \"88c3a6df-756b-43f0-8313-2346af0de6bc\") " pod="ai-tenant-e2e-shared-a-dd1671/sub-test-model-dd1671-kserve-77cc75454f-ww77g"
Jul 22 04:15:39.674715 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:39.674679    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88c3a6df-756b-43f0-8313-2346af0de6bc-home\") pod \"sub-test-model-dd1671-kserve-77cc75454f-ww77g\" (UID: \"88c3a6df-756b-43f0-8313-2346af0de6bc\") " pod="ai-tenant-e2e-shared-a-dd1671/sub-test-model-dd1671-kserve-77cc75454f-ww77g"
Jul 22 04:15:39.674715 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:39.674714    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88c3a6df-756b-43f0-8313-2346af0de6bc-model-cache\") pod \"sub-test-model-dd1671-kserve-77cc75454f-ww77g\" (UID: \"88c3a6df-756b-43f0-8313-2346af0de6bc\") " pod="ai-tenant-e2e-shared-a-dd1671/sub-test-model-dd1671-kserve-77cc75454f-ww77g"
Jul 22 04:15:39.775388 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:39.775350    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88c3a6df-756b-43f0-8313-2346af0de6bc-home\") pod \"sub-test-model-dd1671-kserve-77cc75454f-ww77g\" (UID: \"88c3a6df-756b-43f0-8313-2346af0de6bc\") " pod="ai-tenant-e2e-shared-a-dd1671/sub-test-model-dd1671-kserve-77cc75454f-ww77g"
Jul 22 04:15:39.775388 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:39.775389    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88c3a6df-756b-43f0-8313-2346af0de6bc-model-cache\") pod \"sub-test-model-dd1671-kserve-77cc75454f-ww77g\" (UID: \"88c3a6df-756b-43f0-8313-2346af0de6bc\") " pod="ai-tenant-e2e-shared-a-dd1671/sub-test-model-dd1671-kserve-77cc75454f-ww77g"
Jul 22 04:15:39.775601 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:39.775421    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88c3a6df-756b-43f0-8313-2346af0de6bc-dshm\") pod \"sub-test-model-dd1671-kserve-77cc75454f-ww77g\" (UID: \"88c3a6df-756b-43f0-8313-2346af0de6bc\") " pod="ai-tenant-e2e-shared-a-dd1671/sub-test-model-dd1671-kserve-77cc75454f-ww77g"
Jul 22 04:15:39.775601 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:39.775541    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88c3a6df-756b-43f0-8313-2346af0de6bc-tmp-dir\") pod \"sub-test-model-dd1671-kserve-77cc75454f-ww77g\" (UID: \"88c3a6df-756b-43f0-8313-2346af0de6bc\") " pod="ai-tenant-e2e-shared-a-dd1671/sub-test-model-dd1671-kserve-77cc75454f-ww77g"
Jul 22 04:15:39.775601 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:39.775580    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88c3a6df-756b-43f0-8313-2346af0de6bc-tls-certs\") pod \"sub-test-model-dd1671-kserve-77cc75454f-ww77g\" (UID: \"88c3a6df-756b-43f0-8313-2346af0de6bc\") " pod="ai-tenant-e2e-shared-a-dd1671/sub-test-model-dd1671-kserve-77cc75454f-ww77g"
Jul 22 04:15:39.775749 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:39.775711    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btfsc\" (UniqueName: \"kubernetes.io/projected/88c3a6df-756b-43f0-8313-2346af0de6bc-kube-api-access-btfsc\") pod \"sub-test-model-dd1671-kserve-77cc75454f-ww77g\" (UID: \"88c3a6df-756b-43f0-8313-2346af0de6bc\") " pod="ai-tenant-e2e-shared-a-dd1671/sub-test-model-dd1671-kserve-77cc75454f-ww77g"
Jul 22 04:15:39.775859 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:39.775840    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88c3a6df-756b-43f0-8313-2346af0de6bc-home\") pod \"sub-test-model-dd1671-kserve-77cc75454f-ww77g\" (UID: \"88c3a6df-756b-43f0-8313-2346af0de6bc\") " pod="ai-tenant-e2e-shared-a-dd1671/sub-test-model-dd1671-kserve-77cc75454f-ww77g"
Jul 22 04:15:39.775903 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:39.775862    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88c3a6df-756b-43f0-8313-2346af0de6bc-model-cache\") pod \"sub-test-model-dd1671-kserve-77cc75454f-ww77g\" (UID: \"88c3a6df-756b-43f0-8313-2346af0de6bc\") " pod="ai-tenant-e2e-shared-a-dd1671/sub-test-model-dd1671-kserve-77cc75454f-ww77g"
Jul 22 04:15:39.775954 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:39.775933    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88c3a6df-756b-43f0-8313-2346af0de6bc-tmp-dir\") pod \"sub-test-model-dd1671-kserve-77cc75454f-ww77g\" (UID: \"88c3a6df-756b-43f0-8313-2346af0de6bc\") " pod="ai-tenant-e2e-shared-a-dd1671/sub-test-model-dd1671-kserve-77cc75454f-ww77g"
Jul 22 04:15:39.777877 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:39.777859    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88c3a6df-756b-43f0-8313-2346af0de6bc-dshm\") pod \"sub-test-model-dd1671-kserve-77cc75454f-ww77g\" (UID: \"88c3a6df-756b-43f0-8313-2346af0de6bc\") " pod="ai-tenant-e2e-shared-a-dd1671/sub-test-model-dd1671-kserve-77cc75454f-ww77g"
Jul 22 04:15:39.778142 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:39.778123    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88c3a6df-756b-43f0-8313-2346af0de6bc-tls-certs\") pod \"sub-test-model-dd1671-kserve-77cc75454f-ww77g\" (UID: \"88c3a6df-756b-43f0-8313-2346af0de6bc\") " pod="ai-tenant-e2e-shared-a-dd1671/sub-test-model-dd1671-kserve-77cc75454f-ww77g"
Jul 22 04:15:39.786684 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:39.786656    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btfsc\" (UniqueName: \"kubernetes.io/projected/88c3a6df-756b-43f0-8313-2346af0de6bc-kube-api-access-btfsc\") pod \"sub-test-model-dd1671-kserve-77cc75454f-ww77g\" (UID: \"88c3a6df-756b-43f0-8313-2346af0de6bc\") " pod="ai-tenant-e2e-shared-a-dd1671/sub-test-model-dd1671-kserve-77cc75454f-ww77g"
Jul 22 04:15:39.879930 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:39.879837    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-dd1671/sub-test-model-dd1671-kserve-77cc75454f-ww77g"
Jul 22 04:15:40.024940 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:40.024820    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-dd1671/sub-test-model-dd1671-kserve-77cc75454f-ww77g"]
Jul 22 04:15:40.027685 ip-10-0-132-89 kubenswrapper[2570]: W0722 04:15:40.027654    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod88c3a6df_756b_43f0_8313_2346af0de6bc.slice/crio-e4b70f3ef0a686f6d32bcb6fd37b5047a9baafa2c66403d8a1f9a3cd60e0faac WatchSource:0}: Error finding container e4b70f3ef0a686f6d32bcb6fd37b5047a9baafa2c66403d8a1f9a3cd60e0faac: Status 404 returned error can't find the container with id e4b70f3ef0a686f6d32bcb6fd37b5047a9baafa2c66403d8a1f9a3cd60e0faac
Jul 22 04:15:40.648042 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:40.648011    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-dd1671/sub-test-model-dd1671-kserve-77cc75454f-ww77g" event={"ID":"88c3a6df-756b-43f0-8313-2346af0de6bc","Type":"ContainerStarted","Data":"651d80a2e3dde1871135a73153b5afa66aad7fa64fa2774ea0e75bcfd4255388"}
Jul 22 04:15:40.648422 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:40.648053    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-dd1671/sub-test-model-dd1671-kserve-77cc75454f-ww77g" event={"ID":"88c3a6df-756b-43f0-8313-2346af0de6bc","Type":"ContainerStarted","Data":"e4b70f3ef0a686f6d32bcb6fd37b5047a9baafa2c66403d8a1f9a3cd60e0faac"}
Jul 22 04:15:40.668669 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:40.668597    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-dd1671/sub-test-model-dd1671-kserve-77cc75454f-ww77g" podStartSLOduration=1.668580594 podStartE2EDuration="1.668580594s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:15:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:15:40.666707115 +0000 UTC m=+2528.284917121" watchObservedRunningTime="2026-07-22 04:15:40.668580594 +0000 UTC m=+2528.286790601"
Jul 22 04:15:43.077759 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:43.077725    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:15:49.880127 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:49.880087    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-dd1671/sub-test-model-dd1671-kserve-77cc75454f-ww77g"
Jul 22 04:15:49.880488 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:49.880166    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-dd1671/sub-test-model-dd1671-kserve-77cc75454f-ww77g"
Jul 22 04:15:49.892900 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:49.892874    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-dd1671/sub-test-model-dd1671-kserve-77cc75454f-ww77g"
Jul 22 04:15:50.706908 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:15:50.706877    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-dd1671/sub-test-model-dd1671-kserve-77cc75454f-ww77g"
Jul 22 04:16:00.961746 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:00.961711    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-38610b/sub-test-model-38610b-kserve-7596f5f87f-8lbzs"]
Jul 22 04:16:00.965265 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:00.965248    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-38610b/sub-test-model-38610b-kserve-7596f5f87f-8lbzs"
Jul 22 04:16:00.969042 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:00.969014    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-38610b\"/\"kube-root-ca.crt\""
Jul 22 04:16:00.969163 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:00.969052    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-38610b\"/\"openshift-service-ca.crt\""
Jul 22 04:16:00.969163 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:00.969117    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-38610b\"/\"sub-test-model-38610b-kserve-self-signed-certs\""
Jul 22 04:16:00.969270 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:00.969169    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-38610b\"/\"default-dockercfg-j47zf\""
Jul 22 04:16:00.978580 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:00.978559    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-38610b/sub-test-model-38610b-kserve-7596f5f87f-8lbzs"]
Jul 22 04:16:01.073856 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:01.073818    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2769e9b6-a4fe-4df3-bc8c-16768d544ab2-dshm\") pod \"sub-test-model-38610b-kserve-7596f5f87f-8lbzs\" (UID: \"2769e9b6-a4fe-4df3-bc8c-16768d544ab2\") " pod="ai-tenant-e2e-shared-b-38610b/sub-test-model-38610b-kserve-7596f5f87f-8lbzs"
Jul 22 04:16:01.074056 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:01.073863    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2769e9b6-a4fe-4df3-bc8c-16768d544ab2-model-cache\") pod \"sub-test-model-38610b-kserve-7596f5f87f-8lbzs\" (UID: \"2769e9b6-a4fe-4df3-bc8c-16768d544ab2\") " pod="ai-tenant-e2e-shared-b-38610b/sub-test-model-38610b-kserve-7596f5f87f-8lbzs"
Jul 22 04:16:01.074056 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:01.073887    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2769e9b6-a4fe-4df3-bc8c-16768d544ab2-tls-certs\") pod \"sub-test-model-38610b-kserve-7596f5f87f-8lbzs\" (UID: \"2769e9b6-a4fe-4df3-bc8c-16768d544ab2\") " pod="ai-tenant-e2e-shared-b-38610b/sub-test-model-38610b-kserve-7596f5f87f-8lbzs"
Jul 22 04:16:01.074056 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:01.074001    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2769e9b6-a4fe-4df3-bc8c-16768d544ab2-home\") pod \"sub-test-model-38610b-kserve-7596f5f87f-8lbzs\" (UID: \"2769e9b6-a4fe-4df3-bc8c-16768d544ab2\") " pod="ai-tenant-e2e-shared-b-38610b/sub-test-model-38610b-kserve-7596f5f87f-8lbzs"
Jul 22 04:16:01.074056 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:01.074039    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2769e9b6-a4fe-4df3-bc8c-16768d544ab2-tmp-dir\") pod \"sub-test-model-38610b-kserve-7596f5f87f-8lbzs\" (UID: \"2769e9b6-a4fe-4df3-bc8c-16768d544ab2\") " pod="ai-tenant-e2e-shared-b-38610b/sub-test-model-38610b-kserve-7596f5f87f-8lbzs"
Jul 22 04:16:01.074192 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:01.074059    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkbcs\" (UniqueName: \"kubernetes.io/projected/2769e9b6-a4fe-4df3-bc8c-16768d544ab2-kube-api-access-bkbcs\") pod \"sub-test-model-38610b-kserve-7596f5f87f-8lbzs\" (UID: \"2769e9b6-a4fe-4df3-bc8c-16768d544ab2\") " pod="ai-tenant-e2e-shared-b-38610b/sub-test-model-38610b-kserve-7596f5f87f-8lbzs"
Jul 22 04:16:01.174558 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:01.174513    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2769e9b6-a4fe-4df3-bc8c-16768d544ab2-tmp-dir\") pod \"sub-test-model-38610b-kserve-7596f5f87f-8lbzs\" (UID: \"2769e9b6-a4fe-4df3-bc8c-16768d544ab2\") " pod="ai-tenant-e2e-shared-b-38610b/sub-test-model-38610b-kserve-7596f5f87f-8lbzs"
Jul 22 04:16:01.174558 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:01.174560    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkbcs\" (UniqueName: \"kubernetes.io/projected/2769e9b6-a4fe-4df3-bc8c-16768d544ab2-kube-api-access-bkbcs\") pod \"sub-test-model-38610b-kserve-7596f5f87f-8lbzs\" (UID: \"2769e9b6-a4fe-4df3-bc8c-16768d544ab2\") " pod="ai-tenant-e2e-shared-b-38610b/sub-test-model-38610b-kserve-7596f5f87f-8lbzs"
Jul 22 04:16:01.174849 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:01.174655    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2769e9b6-a4fe-4df3-bc8c-16768d544ab2-dshm\") pod \"sub-test-model-38610b-kserve-7596f5f87f-8lbzs\" (UID: \"2769e9b6-a4fe-4df3-bc8c-16768d544ab2\") " pod="ai-tenant-e2e-shared-b-38610b/sub-test-model-38610b-kserve-7596f5f87f-8lbzs"
Jul 22 04:16:01.174849 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:01.174687    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2769e9b6-a4fe-4df3-bc8c-16768d544ab2-model-cache\") pod \"sub-test-model-38610b-kserve-7596f5f87f-8lbzs\" (UID: \"2769e9b6-a4fe-4df3-bc8c-16768d544ab2\") " pod="ai-tenant-e2e-shared-b-38610b/sub-test-model-38610b-kserve-7596f5f87f-8lbzs"
Jul 22 04:16:01.174849 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:01.174711    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2769e9b6-a4fe-4df3-bc8c-16768d544ab2-tls-certs\") pod \"sub-test-model-38610b-kserve-7596f5f87f-8lbzs\" (UID: \"2769e9b6-a4fe-4df3-bc8c-16768d544ab2\") " pod="ai-tenant-e2e-shared-b-38610b/sub-test-model-38610b-kserve-7596f5f87f-8lbzs"
Jul 22 04:16:01.174849 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:01.174788    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2769e9b6-a4fe-4df3-bc8c-16768d544ab2-home\") pod \"sub-test-model-38610b-kserve-7596f5f87f-8lbzs\" (UID: \"2769e9b6-a4fe-4df3-bc8c-16768d544ab2\") " pod="ai-tenant-e2e-shared-b-38610b/sub-test-model-38610b-kserve-7596f5f87f-8lbzs"
Jul 22 04:16:01.175051 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:01.175024    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2769e9b6-a4fe-4df3-bc8c-16768d544ab2-tmp-dir\") pod \"sub-test-model-38610b-kserve-7596f5f87f-8lbzs\" (UID: \"2769e9b6-a4fe-4df3-bc8c-16768d544ab2\") " pod="ai-tenant-e2e-shared-b-38610b/sub-test-model-38610b-kserve-7596f5f87f-8lbzs"
Jul 22 04:16:01.175110 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:01.175061    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2769e9b6-a4fe-4df3-bc8c-16768d544ab2-model-cache\") pod \"sub-test-model-38610b-kserve-7596f5f87f-8lbzs\" (UID: \"2769e9b6-a4fe-4df3-bc8c-16768d544ab2\") " pod="ai-tenant-e2e-shared-b-38610b/sub-test-model-38610b-kserve-7596f5f87f-8lbzs"
Jul 22 04:16:01.175462 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:01.175416    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2769e9b6-a4fe-4df3-bc8c-16768d544ab2-home\") pod \"sub-test-model-38610b-kserve-7596f5f87f-8lbzs\" (UID: \"2769e9b6-a4fe-4df3-bc8c-16768d544ab2\") " pod="ai-tenant-e2e-shared-b-38610b/sub-test-model-38610b-kserve-7596f5f87f-8lbzs"
Jul 22 04:16:01.177347 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:01.177325    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2769e9b6-a4fe-4df3-bc8c-16768d544ab2-dshm\") pod \"sub-test-model-38610b-kserve-7596f5f87f-8lbzs\" (UID: \"2769e9b6-a4fe-4df3-bc8c-16768d544ab2\") " pod="ai-tenant-e2e-shared-b-38610b/sub-test-model-38610b-kserve-7596f5f87f-8lbzs"
Jul 22 04:16:01.177524 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:01.177505    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2769e9b6-a4fe-4df3-bc8c-16768d544ab2-tls-certs\") pod \"sub-test-model-38610b-kserve-7596f5f87f-8lbzs\" (UID: \"2769e9b6-a4fe-4df3-bc8c-16768d544ab2\") " pod="ai-tenant-e2e-shared-b-38610b/sub-test-model-38610b-kserve-7596f5f87f-8lbzs"
Jul 22 04:16:01.183888 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:01.183861    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkbcs\" (UniqueName: \"kubernetes.io/projected/2769e9b6-a4fe-4df3-bc8c-16768d544ab2-kube-api-access-bkbcs\") pod \"sub-test-model-38610b-kserve-7596f5f87f-8lbzs\" (UID: \"2769e9b6-a4fe-4df3-bc8c-16768d544ab2\") " pod="ai-tenant-e2e-shared-b-38610b/sub-test-model-38610b-kserve-7596f5f87f-8lbzs"
Jul 22 04:16:01.274441 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:01.274350    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-38610b/sub-test-model-38610b-kserve-7596f5f87f-8lbzs"
Jul 22 04:16:01.420582 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:01.420553    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-38610b/sub-test-model-38610b-kserve-7596f5f87f-8lbzs"]
Jul 22 04:16:01.422834 ip-10-0-132-89 kubenswrapper[2570]: W0722 04:16:01.422791    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2769e9b6_a4fe_4df3_bc8c_16768d544ab2.slice/crio-f377d61406139653773c0ff357c3de0434882f17134cfb66b92fd335dfa9e1d6 WatchSource:0}: Error finding container f377d61406139653773c0ff357c3de0434882f17134cfb66b92fd335dfa9e1d6: Status 404 returned error can't find the container with id f377d61406139653773c0ff357c3de0434882f17134cfb66b92fd335dfa9e1d6
Jul 22 04:16:01.749255 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:01.749215    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-38610b/sub-test-model-38610b-kserve-7596f5f87f-8lbzs" event={"ID":"2769e9b6-a4fe-4df3-bc8c-16768d544ab2","Type":"ContainerStarted","Data":"768503791b7a780c121ef02828b7dc36ebf606cea1c931f493a6bb3c480ccf88"}
Jul 22 04:16:01.749255 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:01.749257    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-38610b/sub-test-model-38610b-kserve-7596f5f87f-8lbzs" event={"ID":"2769e9b6-a4fe-4df3-bc8c-16768d544ab2","Type":"ContainerStarted","Data":"f377d61406139653773c0ff357c3de0434882f17134cfb66b92fd335dfa9e1d6"}
Jul 22 04:16:01.770214 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:01.770162    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-38610b/sub-test-model-38610b-kserve-7596f5f87f-8lbzs" podStartSLOduration=1.770146834 podStartE2EDuration="1.770146834s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:16:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:16:01.76848745 +0000 UTC m=+2549.386697457" watchObservedRunningTime="2026-07-22 04:16:01.770146834 +0000 UTC m=+2549.388356840"
Jul 22 04:16:04.673603 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:04.673571    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:16:11.275131 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:11.275088    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-38610b/sub-test-model-38610b-kserve-7596f5f87f-8lbzs"
Jul 22 04:16:11.275131 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:11.275136    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-38610b/sub-test-model-38610b-kserve-7596f5f87f-8lbzs"
Jul 22 04:16:11.287543 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:11.287519    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-38610b/sub-test-model-38610b-kserve-7596f5f87f-8lbzs"
Jul 22 04:16:11.809177 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:11.809146    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-38610b/sub-test-model-38610b-kserve-7596f5f87f-8lbzs"
Jul 22 04:16:22.864450 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:22.864420    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:16:39.806377 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:39.806344    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-dd1671/sub-test-model-dd1671-kserve-77cc75454f-ww77g"]
Jul 22 04:16:40.646777 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:40.646748    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-38610b/sub-test-model-38610b-kserve-7596f5f87f-8lbzs"]
Jul 22 04:16:42.259653 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:42.259603    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:16:54.834665 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:54.834560    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-dd1671/sub-test-model-dd1671-kserve-77cc75454f-ww77g" podUID="88c3a6df-756b-43f0-8313-2346af0de6bc" containerName="main" containerID="cri-o://651d80a2e3dde1871135a73153b5afa66aad7fa64fa2774ea0e75bcfd4255388" gracePeriod=45
Jul 22 04:16:55.011979 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:55.011942    2570 generic.go:356] "Generic (PLEG): container finished" podID="88c3a6df-756b-43f0-8313-2346af0de6bc" containerID="651d80a2e3dde1871135a73153b5afa66aad7fa64fa2774ea0e75bcfd4255388" exitCode=0
Jul 22 04:16:55.012142 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:55.012018    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-dd1671/sub-test-model-dd1671-kserve-77cc75454f-ww77g" event={"ID":"88c3a6df-756b-43f0-8313-2346af0de6bc","Type":"ContainerDied","Data":"651d80a2e3dde1871135a73153b5afa66aad7fa64fa2774ea0e75bcfd4255388"}
Jul 22 04:16:55.086341 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:55.086278    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-dd1671/sub-test-model-dd1671-kserve-77cc75454f-ww77g"
Jul 22 04:16:55.247159 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:55.247119    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-btfsc\" (UniqueName: \"kubernetes.io/projected/88c3a6df-756b-43f0-8313-2346af0de6bc-kube-api-access-btfsc\") pod \"88c3a6df-756b-43f0-8313-2346af0de6bc\" (UID: \"88c3a6df-756b-43f0-8313-2346af0de6bc\") "
Jul 22 04:16:55.247331 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:55.247188    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88c3a6df-756b-43f0-8313-2346af0de6bc-tmp-dir\") pod \"88c3a6df-756b-43f0-8313-2346af0de6bc\" (UID: \"88c3a6df-756b-43f0-8313-2346af0de6bc\") "
Jul 22 04:16:55.247331 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:55.247210    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88c3a6df-756b-43f0-8313-2346af0de6bc-home\") pod \"88c3a6df-756b-43f0-8313-2346af0de6bc\" (UID: \"88c3a6df-756b-43f0-8313-2346af0de6bc\") "
Jul 22 04:16:55.247331 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:55.247250    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88c3a6df-756b-43f0-8313-2346af0de6bc-dshm\") pod \"88c3a6df-756b-43f0-8313-2346af0de6bc\" (UID: \"88c3a6df-756b-43f0-8313-2346af0de6bc\") "
Jul 22 04:16:55.247331 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:55.247279    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88c3a6df-756b-43f0-8313-2346af0de6bc-model-cache\") pod \"88c3a6df-756b-43f0-8313-2346af0de6bc\" (UID: \"88c3a6df-756b-43f0-8313-2346af0de6bc\") "
Jul 22 04:16:55.247331 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:55.247318    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88c3a6df-756b-43f0-8313-2346af0de6bc-tls-certs\") pod \"88c3a6df-756b-43f0-8313-2346af0de6bc\" (UID: \"88c3a6df-756b-43f0-8313-2346af0de6bc\") "
Jul 22 04:16:55.247592 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:55.247489    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88c3a6df-756b-43f0-8313-2346af0de6bc-home" (OuterVolumeSpecName: "home") pod "88c3a6df-756b-43f0-8313-2346af0de6bc" (UID: "88c3a6df-756b-43f0-8313-2346af0de6bc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 04:16:55.247592 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:55.247504    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88c3a6df-756b-43f0-8313-2346af0de6bc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "88c3a6df-756b-43f0-8313-2346af0de6bc" (UID: "88c3a6df-756b-43f0-8313-2346af0de6bc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 04:16:55.247592 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:55.247564    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88c3a6df-756b-43f0-8313-2346af0de6bc-model-cache" (OuterVolumeSpecName: "model-cache") pod "88c3a6df-756b-43f0-8313-2346af0de6bc" (UID: "88c3a6df-756b-43f0-8313-2346af0de6bc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 04:16:55.247776 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:55.247725    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88c3a6df-756b-43f0-8313-2346af0de6bc-tmp-dir\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:16:55.247776 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:55.247745    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88c3a6df-756b-43f0-8313-2346af0de6bc-home\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:16:55.247776 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:55.247756    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88c3a6df-756b-43f0-8313-2346af0de6bc-model-cache\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:16:55.249544 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:55.249520    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88c3a6df-756b-43f0-8313-2346af0de6bc-dshm" (OuterVolumeSpecName: "dshm") pod "88c3a6df-756b-43f0-8313-2346af0de6bc" (UID: "88c3a6df-756b-43f0-8313-2346af0de6bc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 04:16:55.249681 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:55.249574    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88c3a6df-756b-43f0-8313-2346af0de6bc-kube-api-access-btfsc" (OuterVolumeSpecName: "kube-api-access-btfsc") pod "88c3a6df-756b-43f0-8313-2346af0de6bc" (UID: "88c3a6df-756b-43f0-8313-2346af0de6bc"). InnerVolumeSpecName "kube-api-access-btfsc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 04:16:55.249681 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:55.249632    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88c3a6df-756b-43f0-8313-2346af0de6bc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "88c3a6df-756b-43f0-8313-2346af0de6bc" (UID: "88c3a6df-756b-43f0-8313-2346af0de6bc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 04:16:55.348601 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:55.348507    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btfsc\" (UniqueName: \"kubernetes.io/projected/88c3a6df-756b-43f0-8313-2346af0de6bc-kube-api-access-btfsc\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:16:55.348601 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:55.348545    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88c3a6df-756b-43f0-8313-2346af0de6bc-dshm\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:16:55.348601 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:55.348556    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88c3a6df-756b-43f0-8313-2346af0de6bc-tls-certs\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:16:55.668382 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:55.668318    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-38610b/sub-test-model-38610b-kserve-7596f5f87f-8lbzs" podUID="2769e9b6-a4fe-4df3-bc8c-16768d544ab2" containerName="main" containerID="cri-o://768503791b7a780c121ef02828b7dc36ebf606cea1c931f493a6bb3c480ccf88" gracePeriod=45
Jul 22 04:16:55.930416 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:55.930353    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-38610b/sub-test-model-38610b-kserve-7596f5f87f-8lbzs"
Jul 22 04:16:56.018756 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:56.018726    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-dd1671/sub-test-model-dd1671-kserve-77cc75454f-ww77g"
Jul 22 04:16:56.018944 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:56.018751    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-dd1671/sub-test-model-dd1671-kserve-77cc75454f-ww77g" event={"ID":"88c3a6df-756b-43f0-8313-2346af0de6bc","Type":"ContainerDied","Data":"e4b70f3ef0a686f6d32bcb6fd37b5047a9baafa2c66403d8a1f9a3cd60e0faac"}
Jul 22 04:16:56.018944 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:56.018813    2570 scope.go:117] "RemoveContainer" containerID="651d80a2e3dde1871135a73153b5afa66aad7fa64fa2774ea0e75bcfd4255388"
Jul 22 04:16:56.020115 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:56.020093    2570 generic.go:356] "Generic (PLEG): container finished" podID="2769e9b6-a4fe-4df3-bc8c-16768d544ab2" containerID="768503791b7a780c121ef02828b7dc36ebf606cea1c931f493a6bb3c480ccf88" exitCode=0
Jul 22 04:16:56.020269 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:56.020129    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-38610b/sub-test-model-38610b-kserve-7596f5f87f-8lbzs"
Jul 22 04:16:56.020269 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:56.020180    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-38610b/sub-test-model-38610b-kserve-7596f5f87f-8lbzs" event={"ID":"2769e9b6-a4fe-4df3-bc8c-16768d544ab2","Type":"ContainerDied","Data":"768503791b7a780c121ef02828b7dc36ebf606cea1c931f493a6bb3c480ccf88"}
Jul 22 04:16:56.020269 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:56.020224    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-38610b/sub-test-model-38610b-kserve-7596f5f87f-8lbzs" event={"ID":"2769e9b6-a4fe-4df3-bc8c-16768d544ab2","Type":"ContainerDied","Data":"f377d61406139653773c0ff357c3de0434882f17134cfb66b92fd335dfa9e1d6"}
Jul 22 04:16:56.028319 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:56.028300    2570 scope.go:117] "RemoveContainer" containerID="768503791b7a780c121ef02828b7dc36ebf606cea1c931f493a6bb3c480ccf88"
Jul 22 04:16:56.036061 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:56.036045    2570 scope.go:117] "RemoveContainer" containerID="768503791b7a780c121ef02828b7dc36ebf606cea1c931f493a6bb3c480ccf88"
Jul 22 04:16:56.036317 ip-10-0-132-89 kubenswrapper[2570]: E0722 04:16:56.036301    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"768503791b7a780c121ef02828b7dc36ebf606cea1c931f493a6bb3c480ccf88\": container with ID starting with 768503791b7a780c121ef02828b7dc36ebf606cea1c931f493a6bb3c480ccf88 not found: ID does not exist" containerID="768503791b7a780c121ef02828b7dc36ebf606cea1c931f493a6bb3c480ccf88"
Jul 22 04:16:56.036380 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:56.036329    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"768503791b7a780c121ef02828b7dc36ebf606cea1c931f493a6bb3c480ccf88"} err="failed to get container status \"768503791b7a780c121ef02828b7dc36ebf606cea1c931f493a6bb3c480ccf88\": rpc error: code = NotFound desc = could not find container \"768503791b7a780c121ef02828b7dc36ebf606cea1c931f493a6bb3c480ccf88\": container with ID starting with 768503791b7a780c121ef02828b7dc36ebf606cea1c931f493a6bb3c480ccf88 not found: ID does not exist"
Jul 22 04:16:56.042969 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:56.042947    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-dd1671/sub-test-model-dd1671-kserve-77cc75454f-ww77g"]
Jul 22 04:16:56.047553 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:56.047530    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-dd1671/sub-test-model-dd1671-kserve-77cc75454f-ww77g"]
Jul 22 04:16:56.054018 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:56.054000    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2769e9b6-a4fe-4df3-bc8c-16768d544ab2-home\") pod \"2769e9b6-a4fe-4df3-bc8c-16768d544ab2\" (UID: \"2769e9b6-a4fe-4df3-bc8c-16768d544ab2\") "
Jul 22 04:16:56.054113 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:56.054099    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2769e9b6-a4fe-4df3-bc8c-16768d544ab2-tls-certs\") pod \"2769e9b6-a4fe-4df3-bc8c-16768d544ab2\" (UID: \"2769e9b6-a4fe-4df3-bc8c-16768d544ab2\") "
Jul 22 04:16:56.054155 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:56.054123    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bkbcs\" (UniqueName: \"kubernetes.io/projected/2769e9b6-a4fe-4df3-bc8c-16768d544ab2-kube-api-access-bkbcs\") pod \"2769e9b6-a4fe-4df3-bc8c-16768d544ab2\" (UID: \"2769e9b6-a4fe-4df3-bc8c-16768d544ab2\") "
Jul 22 04:16:56.054189 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:56.054160    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2769e9b6-a4fe-4df3-bc8c-16768d544ab2-model-cache\") pod \"2769e9b6-a4fe-4df3-bc8c-16768d544ab2\" (UID: \"2769e9b6-a4fe-4df3-bc8c-16768d544ab2\") "
Jul 22 04:16:56.054228 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:56.054192    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2769e9b6-a4fe-4df3-bc8c-16768d544ab2-dshm\") pod \"2769e9b6-a4fe-4df3-bc8c-16768d544ab2\" (UID: \"2769e9b6-a4fe-4df3-bc8c-16768d544ab2\") "
Jul 22 04:16:56.054228 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:56.054221    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2769e9b6-a4fe-4df3-bc8c-16768d544ab2-tmp-dir\") pod \"2769e9b6-a4fe-4df3-bc8c-16768d544ab2\" (UID: \"2769e9b6-a4fe-4df3-bc8c-16768d544ab2\") "
Jul 22 04:16:56.054339 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:56.054249    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2769e9b6-a4fe-4df3-bc8c-16768d544ab2-home" (OuterVolumeSpecName: "home") pod "2769e9b6-a4fe-4df3-bc8c-16768d544ab2" (UID: "2769e9b6-a4fe-4df3-bc8c-16768d544ab2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 04:16:56.054413 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:56.054401    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2769e9b6-a4fe-4df3-bc8c-16768d544ab2-home\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:16:56.054494 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:56.054474    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2769e9b6-a4fe-4df3-bc8c-16768d544ab2-model-cache" (OuterVolumeSpecName: "model-cache") pod "2769e9b6-a4fe-4df3-bc8c-16768d544ab2" (UID: "2769e9b6-a4fe-4df3-bc8c-16768d544ab2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 04:16:56.054544 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:56.054513    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2769e9b6-a4fe-4df3-bc8c-16768d544ab2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2769e9b6-a4fe-4df3-bc8c-16768d544ab2" (UID: "2769e9b6-a4fe-4df3-bc8c-16768d544ab2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 04:16:56.056515 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:56.056487    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2769e9b6-a4fe-4df3-bc8c-16768d544ab2-dshm" (OuterVolumeSpecName: "dshm") pod "2769e9b6-a4fe-4df3-bc8c-16768d544ab2" (UID: "2769e9b6-a4fe-4df3-bc8c-16768d544ab2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 04:16:56.056515 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:56.056505    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2769e9b6-a4fe-4df3-bc8c-16768d544ab2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2769e9b6-a4fe-4df3-bc8c-16768d544ab2" (UID: "2769e9b6-a4fe-4df3-bc8c-16768d544ab2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 04:16:56.056670 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:56.056528    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2769e9b6-a4fe-4df3-bc8c-16768d544ab2-kube-api-access-bkbcs" (OuterVolumeSpecName: "kube-api-access-bkbcs") pod "2769e9b6-a4fe-4df3-bc8c-16768d544ab2" (UID: "2769e9b6-a4fe-4df3-bc8c-16768d544ab2"). InnerVolumeSpecName "kube-api-access-bkbcs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 04:16:56.155089 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:56.155051    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2769e9b6-a4fe-4df3-bc8c-16768d544ab2-tls-certs\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:16:56.155089 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:56.155082    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bkbcs\" (UniqueName: \"kubernetes.io/projected/2769e9b6-a4fe-4df3-bc8c-16768d544ab2-kube-api-access-bkbcs\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:16:56.155089 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:56.155092    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2769e9b6-a4fe-4df3-bc8c-16768d544ab2-model-cache\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:16:56.155089 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:56.155101    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2769e9b6-a4fe-4df3-bc8c-16768d544ab2-dshm\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:16:56.155352 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:56.155109    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2769e9b6-a4fe-4df3-bc8c-16768d544ab2-tmp-dir\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:16:56.263485 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:56.263396    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:16:56.343429 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:56.343398    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-38610b/sub-test-model-38610b-kserve-7596f5f87f-8lbzs"]
Jul 22 04:16:56.346909 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:56.346882    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-38610b/sub-test-model-38610b-kserve-7596f5f87f-8lbzs"]
Jul 22 04:16:56.851782 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:56.851753    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2769e9b6-a4fe-4df3-bc8c-16768d544ab2" path="/var/lib/kubelet/pods/2769e9b6-a4fe-4df3-bc8c-16768d544ab2/volumes"
Jul 22 04:16:56.852178 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:16:56.852164    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88c3a6df-756b-43f0-8313-2346af0de6bc" path="/var/lib/kubelet/pods/88c3a6df-756b-43f0-8313-2346af0de6bc/volumes"
Jul 22 04:17:22.366685 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:22.366646    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-38610b/rate-test-model-38610b-kserve-6d98dc9cc9-7klhf"]
Jul 22 04:17:22.367100 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:22.367000    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88c3a6df-756b-43f0-8313-2346af0de6bc" containerName="main"
Jul 22 04:17:22.367100 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:22.367010    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="88c3a6df-756b-43f0-8313-2346af0de6bc" containerName="main"
Jul 22 04:17:22.367100 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:22.367026    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2769e9b6-a4fe-4df3-bc8c-16768d544ab2" containerName="main"
Jul 22 04:17:22.367100 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:22.367031    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="2769e9b6-a4fe-4df3-bc8c-16768d544ab2" containerName="main"
Jul 22 04:17:22.367100 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:22.367094    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="2769e9b6-a4fe-4df3-bc8c-16768d544ab2" containerName="main"
Jul 22 04:17:22.367100 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:22.367102    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="88c3a6df-756b-43f0-8313-2346af0de6bc" containerName="main"
Jul 22 04:17:22.369740 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:22.369722    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-38610b/rate-test-model-38610b-kserve-6d98dc9cc9-7klhf"
Jul 22 04:17:22.373516 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:22.373491    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-38610b\"/\"kube-root-ca.crt\""
Jul 22 04:17:22.373694 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:22.373550    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-38610b\"/\"default-dockercfg-j47zf\""
Jul 22 04:17:22.373694 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:22.373575    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-38610b\"/\"openshift-service-ca.crt\""
Jul 22 04:17:22.373694 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:22.373574    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-38610b\"/\"rate-test-model-38610b-kserve-self-signed-certs\""
Jul 22 04:17:22.381388 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:22.381364    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-38610b/rate-test-model-38610b-kserve-6d98dc9cc9-7klhf"]
Jul 22 04:17:22.386278 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:22.386257    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c466798-bf18-4c82-9497-4c6afcbe60bb-dshm\") pod \"rate-test-model-38610b-kserve-6d98dc9cc9-7klhf\" (UID: \"3c466798-bf18-4c82-9497-4c6afcbe60bb\") " pod="ai-tenant-e2e-shared-b-38610b/rate-test-model-38610b-kserve-6d98dc9cc9-7klhf"
Jul 22 04:17:22.386393 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:22.386293    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c466798-bf18-4c82-9497-4c6afcbe60bb-tls-certs\") pod \"rate-test-model-38610b-kserve-6d98dc9cc9-7klhf\" (UID: \"3c466798-bf18-4c82-9497-4c6afcbe60bb\") " pod="ai-tenant-e2e-shared-b-38610b/rate-test-model-38610b-kserve-6d98dc9cc9-7klhf"
Jul 22 04:17:22.386440 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:22.386393    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c466798-bf18-4c82-9497-4c6afcbe60bb-home\") pod \"rate-test-model-38610b-kserve-6d98dc9cc9-7klhf\" (UID: \"3c466798-bf18-4c82-9497-4c6afcbe60bb\") " pod="ai-tenant-e2e-shared-b-38610b/rate-test-model-38610b-kserve-6d98dc9cc9-7klhf"
Jul 22 04:17:22.386440 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:22.386419    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c466798-bf18-4c82-9497-4c6afcbe60bb-model-cache\") pod \"rate-test-model-38610b-kserve-6d98dc9cc9-7klhf\" (UID: \"3c466798-bf18-4c82-9497-4c6afcbe60bb\") " pod="ai-tenant-e2e-shared-b-38610b/rate-test-model-38610b-kserve-6d98dc9cc9-7klhf"
Jul 22 04:17:22.386537 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:22.386437    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smrcg\" (UniqueName: \"kubernetes.io/projected/3c466798-bf18-4c82-9497-4c6afcbe60bb-kube-api-access-smrcg\") pod \"rate-test-model-38610b-kserve-6d98dc9cc9-7klhf\" (UID: \"3c466798-bf18-4c82-9497-4c6afcbe60bb\") " pod="ai-tenant-e2e-shared-b-38610b/rate-test-model-38610b-kserve-6d98dc9cc9-7klhf"
Jul 22 04:17:22.386537 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:22.386462    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c466798-bf18-4c82-9497-4c6afcbe60bb-tmp-dir\") pod \"rate-test-model-38610b-kserve-6d98dc9cc9-7klhf\" (UID: \"3c466798-bf18-4c82-9497-4c6afcbe60bb\") " pod="ai-tenant-e2e-shared-b-38610b/rate-test-model-38610b-kserve-6d98dc9cc9-7klhf"
Jul 22 04:17:22.487928 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:22.487892    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c466798-bf18-4c82-9497-4c6afcbe60bb-home\") pod \"rate-test-model-38610b-kserve-6d98dc9cc9-7klhf\" (UID: \"3c466798-bf18-4c82-9497-4c6afcbe60bb\") " pod="ai-tenant-e2e-shared-b-38610b/rate-test-model-38610b-kserve-6d98dc9cc9-7klhf"
Jul 22 04:17:22.487928 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:22.487929    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c466798-bf18-4c82-9497-4c6afcbe60bb-model-cache\") pod \"rate-test-model-38610b-kserve-6d98dc9cc9-7klhf\" (UID: \"3c466798-bf18-4c82-9497-4c6afcbe60bb\") " pod="ai-tenant-e2e-shared-b-38610b/rate-test-model-38610b-kserve-6d98dc9cc9-7klhf"
Jul 22 04:17:22.488173 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:22.487948    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smrcg\" (UniqueName: \"kubernetes.io/projected/3c466798-bf18-4c82-9497-4c6afcbe60bb-kube-api-access-smrcg\") pod \"rate-test-model-38610b-kserve-6d98dc9cc9-7klhf\" (UID: \"3c466798-bf18-4c82-9497-4c6afcbe60bb\") " pod="ai-tenant-e2e-shared-b-38610b/rate-test-model-38610b-kserve-6d98dc9cc9-7klhf"
Jul 22 04:17:22.488173 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:22.487989    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c466798-bf18-4c82-9497-4c6afcbe60bb-tmp-dir\") pod \"rate-test-model-38610b-kserve-6d98dc9cc9-7klhf\" (UID: \"3c466798-bf18-4c82-9497-4c6afcbe60bb\") " pod="ai-tenant-e2e-shared-b-38610b/rate-test-model-38610b-kserve-6d98dc9cc9-7klhf"
Jul 22 04:17:22.488173 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:22.488034    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c466798-bf18-4c82-9497-4c6afcbe60bb-dshm\") pod \"rate-test-model-38610b-kserve-6d98dc9cc9-7klhf\" (UID: \"3c466798-bf18-4c82-9497-4c6afcbe60bb\") " pod="ai-tenant-e2e-shared-b-38610b/rate-test-model-38610b-kserve-6d98dc9cc9-7klhf"
Jul 22 04:17:22.488173 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:22.488069    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c466798-bf18-4c82-9497-4c6afcbe60bb-tls-certs\") pod \"rate-test-model-38610b-kserve-6d98dc9cc9-7klhf\" (UID: \"3c466798-bf18-4c82-9497-4c6afcbe60bb\") " pod="ai-tenant-e2e-shared-b-38610b/rate-test-model-38610b-kserve-6d98dc9cc9-7klhf"
Jul 22 04:17:22.488359 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:22.488338    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c466798-bf18-4c82-9497-4c6afcbe60bb-home\") pod \"rate-test-model-38610b-kserve-6d98dc9cc9-7klhf\" (UID: \"3c466798-bf18-4c82-9497-4c6afcbe60bb\") " pod="ai-tenant-e2e-shared-b-38610b/rate-test-model-38610b-kserve-6d98dc9cc9-7klhf"
Jul 22 04:17:22.488404 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:22.488377    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c466798-bf18-4c82-9497-4c6afcbe60bb-model-cache\") pod \"rate-test-model-38610b-kserve-6d98dc9cc9-7klhf\" (UID: \"3c466798-bf18-4c82-9497-4c6afcbe60bb\") " pod="ai-tenant-e2e-shared-b-38610b/rate-test-model-38610b-kserve-6d98dc9cc9-7klhf"
Jul 22 04:17:22.488441 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:22.488397    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c466798-bf18-4c82-9497-4c6afcbe60bb-tmp-dir\") pod \"rate-test-model-38610b-kserve-6d98dc9cc9-7klhf\" (UID: \"3c466798-bf18-4c82-9497-4c6afcbe60bb\") " pod="ai-tenant-e2e-shared-b-38610b/rate-test-model-38610b-kserve-6d98dc9cc9-7klhf"
Jul 22 04:17:22.490536 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:22.490513    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c466798-bf18-4c82-9497-4c6afcbe60bb-dshm\") pod \"rate-test-model-38610b-kserve-6d98dc9cc9-7klhf\" (UID: \"3c466798-bf18-4c82-9497-4c6afcbe60bb\") " pod="ai-tenant-e2e-shared-b-38610b/rate-test-model-38610b-kserve-6d98dc9cc9-7klhf"
Jul 22 04:17:22.490801 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:22.490785    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c466798-bf18-4c82-9497-4c6afcbe60bb-tls-certs\") pod \"rate-test-model-38610b-kserve-6d98dc9cc9-7klhf\" (UID: \"3c466798-bf18-4c82-9497-4c6afcbe60bb\") " pod="ai-tenant-e2e-shared-b-38610b/rate-test-model-38610b-kserve-6d98dc9cc9-7klhf"
Jul 22 04:17:22.496259 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:22.496241    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smrcg\" (UniqueName: \"kubernetes.io/projected/3c466798-bf18-4c82-9497-4c6afcbe60bb-kube-api-access-smrcg\") pod \"rate-test-model-38610b-kserve-6d98dc9cc9-7klhf\" (UID: \"3c466798-bf18-4c82-9497-4c6afcbe60bb\") " pod="ai-tenant-e2e-shared-b-38610b/rate-test-model-38610b-kserve-6d98dc9cc9-7klhf"
Jul 22 04:17:22.679458 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:22.679420    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-38610b/rate-test-model-38610b-kserve-6d98dc9cc9-7klhf"
Jul 22 04:17:22.819955 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:22.819930    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-38610b/rate-test-model-38610b-kserve-6d98dc9cc9-7klhf"]
Jul 22 04:17:22.822380 ip-10-0-132-89 kubenswrapper[2570]: W0722 04:17:22.822350    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3c466798_bf18_4c82_9497_4c6afcbe60bb.slice/crio-be730ea5c3027fe0884db9f7b2055a25f738ca1088b478e778c58b92b5fd6abf WatchSource:0}: Error finding container be730ea5c3027fe0884db9f7b2055a25f738ca1088b478e778c58b92b5fd6abf: Status 404 returned error can't find the container with id be730ea5c3027fe0884db9f7b2055a25f738ca1088b478e778c58b92b5fd6abf
Jul 22 04:17:23.155884 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:23.155794    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-38610b/rate-test-model-38610b-kserve-6d98dc9cc9-7klhf" event={"ID":"3c466798-bf18-4c82-9497-4c6afcbe60bb","Type":"ContainerStarted","Data":"ff581c8a1473bbeafc3d497bfefcb06d8e57a1d4b8ffb9a06fac579ad39d1682"}
Jul 22 04:17:23.155884 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:23.155840    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-38610b/rate-test-model-38610b-kserve-6d98dc9cc9-7klhf" event={"ID":"3c466798-bf18-4c82-9497-4c6afcbe60bb","Type":"ContainerStarted","Data":"be730ea5c3027fe0884db9f7b2055a25f738ca1088b478e778c58b92b5fd6abf"}
Jul 22 04:17:23.174684 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:23.174636    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-38610b/rate-test-model-38610b-kserve-6d98dc9cc9-7klhf" podStartSLOduration=1.174604548 podStartE2EDuration="1.174604548s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:17:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:17:23.17329803 +0000 UTC m=+2630.791508036" watchObservedRunningTime="2026-07-22 04:17:23.174604548 +0000 UTC m=+2630.792814588"
Jul 22 04:17:25.865462 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:25.865424    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:17:32.680267 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:32.680223    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-38610b/rate-test-model-38610b-kserve-6d98dc9cc9-7klhf"
Jul 22 04:17:32.680689 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:32.680290    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-38610b/rate-test-model-38610b-kserve-6d98dc9cc9-7klhf"
Jul 22 04:17:32.693500 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:32.693474    2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-38610b/rate-test-model-38610b-kserve-6d98dc9cc9-7klhf"
Jul 22 04:17:33.071186 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:33.071105    2570 scope.go:117] "RemoveContainer" containerID="648fe2b80c015e4f8ac0dbefffbad14eef590ff200215d3e2a432f00718e08a1"
Jul 22 04:17:33.079494 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:33.079475    2570 scope.go:117] "RemoveContainer" containerID="b9edb48268396dc7961e29892f66bfdd9bc8c4a5b5ca7470559dcf2785b876de"
Jul 22 04:17:33.214582 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:33.214555    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-38610b/rate-test-model-38610b-kserve-6d98dc9cc9-7klhf"
Jul 22 04:17:42.571769 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:17:42.571734    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:18:06.558195 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:06.558157    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:18:11.489940 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:11.489904    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-38610b/rate-test-model-38610b-kserve-6d98dc9cc9-7klhf"]
Jul 22 04:18:12.482126 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:12.482094    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7464-139878922509616"]
Jul 22 04:18:12.485424 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:12.485409    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-139878922509616"
Jul 22 04:18:12.487959 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:12.487939    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-5t2t4\""
Jul 22 04:18:12.493087 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:12.493065    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7464-139878922509616"]
Jul 22 04:18:12.530103 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:12.530067    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksl6k\" (UniqueName: \"kubernetes.io/projected/ed8b9cd2-0439-4db4-a652-89b51a05f70f-kube-api-access-ksl6k\") pod \"test-curl-7464-139878922509616\" (UID: \"ed8b9cd2-0439-4db4-a652-89b51a05f70f\") " pod="opendatahub/test-curl-7464-139878922509616"
Jul 22 04:18:12.631514 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:12.631474    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksl6k\" (UniqueName: \"kubernetes.io/projected/ed8b9cd2-0439-4db4-a652-89b51a05f70f-kube-api-access-ksl6k\") pod \"test-curl-7464-139878922509616\" (UID: \"ed8b9cd2-0439-4db4-a652-89b51a05f70f\") " pod="opendatahub/test-curl-7464-139878922509616"
Jul 22 04:18:12.640034 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:12.640008    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksl6k\" (UniqueName: \"kubernetes.io/projected/ed8b9cd2-0439-4db4-a652-89b51a05f70f-kube-api-access-ksl6k\") pod \"test-curl-7464-139878922509616\" (UID: \"ed8b9cd2-0439-4db4-a652-89b51a05f70f\") " pod="opendatahub/test-curl-7464-139878922509616"
Jul 22 04:18:12.795226 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:12.795129    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-139878922509616"
Jul 22 04:18:12.931086 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:12.931060    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7464-139878922509616"]
Jul 22 04:18:12.933648 ip-10-0-132-89 kubenswrapper[2570]: W0722 04:18:12.933601    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poded8b9cd2_0439_4db4_a652_89b51a05f70f.slice/crio-3afa3f83bf4793699d3900ad7afad7c8c6a3276615557bd2dfa79997b908008f WatchSource:0}: Error finding container 3afa3f83bf4793699d3900ad7afad7c8c6a3276615557bd2dfa79997b908008f: Status 404 returned error can't find the container with id 3afa3f83bf4793699d3900ad7afad7c8c6a3276615557bd2dfa79997b908008f
Jul 22 04:18:12.934950 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:12.934935    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 04:18:13.389946 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:13.389905    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7464-139878922509616" event={"ID":"ed8b9cd2-0439-4db4-a652-89b51a05f70f","Type":"ContainerStarted","Data":"3afa3f83bf4793699d3900ad7afad7c8c6a3276615557bd2dfa79997b908008f"}
Jul 22 04:18:14.396800 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:14.396770    2570 generic.go:356] "Generic (PLEG): container finished" podID="ed8b9cd2-0439-4db4-a652-89b51a05f70f" containerID="52e4ac90090afb045e9df62b3993fa67e521f3da21ce1787804038d430432a09" exitCode=0
Jul 22 04:18:14.397184 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:14.396856    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7464-139878922509616" event={"ID":"ed8b9cd2-0439-4db4-a652-89b51a05f70f","Type":"ContainerDied","Data":"52e4ac90090afb045e9df62b3993fa67e521f3da21ce1787804038d430432a09"}
Jul 22 04:18:15.538217 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:15.538194    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-139878922509616"
Jul 22 04:18:15.555102 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:15.555078    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ksl6k\" (UniqueName: \"kubernetes.io/projected/ed8b9cd2-0439-4db4-a652-89b51a05f70f-kube-api-access-ksl6k\") pod \"ed8b9cd2-0439-4db4-a652-89b51a05f70f\" (UID: \"ed8b9cd2-0439-4db4-a652-89b51a05f70f\") "
Jul 22 04:18:15.557859 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:15.557829    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed8b9cd2-0439-4db4-a652-89b51a05f70f-kube-api-access-ksl6k" (OuterVolumeSpecName: "kube-api-access-ksl6k") pod "ed8b9cd2-0439-4db4-a652-89b51a05f70f" (UID: "ed8b9cd2-0439-4db4-a652-89b51a05f70f"). InnerVolumeSpecName "kube-api-access-ksl6k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 04:18:15.575904 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:15.575869    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7464-139878922509616_ed8b9cd2-0439-4db4-a652-89b51a05f70f/test-curl-7464-139878922509616/0.log"
Jul 22 04:18:15.599448 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:15.599412    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7464-139878922509616"]
Jul 22 04:18:15.603352 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:15.603325    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7464-139878922509616"]
Jul 22 04:18:15.656405 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:15.656324    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ksl6k\" (UniqueName: \"kubernetes.io/projected/ed8b9cd2-0439-4db4-a652-89b51a05f70f-kube-api-access-ksl6k\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:18:15.725389 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:15.725351    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7464-139878922509872"]
Jul 22 04:18:15.725757 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:15.725743    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed8b9cd2-0439-4db4-a652-89b51a05f70f" containerName="test-curl-7464-139878922509616"
Jul 22 04:18:15.725815 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:15.725759    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed8b9cd2-0439-4db4-a652-89b51a05f70f" containerName="test-curl-7464-139878922509616"
Jul 22 04:18:15.725851 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:15.725816    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed8b9cd2-0439-4db4-a652-89b51a05f70f" containerName="test-curl-7464-139878922509616"
Jul 22 04:18:15.728726 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:15.728711    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-139878922509872"
Jul 22 04:18:15.734566 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:15.734545    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7464-139878922509872"]
Jul 22 04:18:15.756957 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:15.756924    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5jbh\" (UniqueName: \"kubernetes.io/projected/527cc23d-888a-480d-a2df-c22595b1f289-kube-api-access-p5jbh\") pod \"test-curl-7464-139878922509872\" (UID: \"527cc23d-888a-480d-a2df-c22595b1f289\") " pod="opendatahub/test-curl-7464-139878922509872"
Jul 22 04:18:15.857963 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:15.857930    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5jbh\" (UniqueName: \"kubernetes.io/projected/527cc23d-888a-480d-a2df-c22595b1f289-kube-api-access-p5jbh\") pod \"test-curl-7464-139878922509872\" (UID: \"527cc23d-888a-480d-a2df-c22595b1f289\") " pod="opendatahub/test-curl-7464-139878922509872"
Jul 22 04:18:15.867192 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:15.867168    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5jbh\" (UniqueName: \"kubernetes.io/projected/527cc23d-888a-480d-a2df-c22595b1f289-kube-api-access-p5jbh\") pod \"test-curl-7464-139878922509872\" (UID: \"527cc23d-888a-480d-a2df-c22595b1f289\") " pod="opendatahub/test-curl-7464-139878922509872"
Jul 22 04:18:16.039420 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:16.039385    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-139878922509872"
Jul 22 04:18:16.172370 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:16.172347    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7464-139878922509872"]
Jul 22 04:18:16.177340 ip-10-0-132-89 kubenswrapper[2570]: W0722 04:18:16.175717    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod527cc23d_888a_480d_a2df_c22595b1f289.slice/crio-7e3b18c9eaefd0e10a6e6189b4206b608194ad77d645bd9689758686d01a0111 WatchSource:0}: Error finding container 7e3b18c9eaefd0e10a6e6189b4206b608194ad77d645bd9689758686d01a0111: Status 404 returned error can't find the container with id 7e3b18c9eaefd0e10a6e6189b4206b608194ad77d645bd9689758686d01a0111
Jul 22 04:18:16.407826 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:16.407727    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7464-139878922509872" event={"ID":"527cc23d-888a-480d-a2df-c22595b1f289","Type":"ContainerStarted","Data":"7e3b18c9eaefd0e10a6e6189b4206b608194ad77d645bd9689758686d01a0111"}
Jul 22 04:18:16.408864 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:16.408844    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3afa3f83bf4793699d3900ad7afad7c8c6a3276615557bd2dfa79997b908008f"
Jul 22 04:18:16.408864 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:16.408855    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-139878922509616"
Jul 22 04:18:16.419336 ip-10-0-132-89 kubenswrapper[2570]: E0722 04:18:16.419305    2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7464-139878922509616\" is forbidden: User \"system:node:ip-10-0-132-89.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-89.ec2.internal' and this object" podUID="ed8b9cd2-0439-4db4-a652-89b51a05f70f" pod="opendatahub/test-curl-7464-139878922509616"
Jul 22 04:18:16.852452 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:16.852417    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed8b9cd2-0439-4db4-a652-89b51a05f70f" path="/var/lib/kubelet/pods/ed8b9cd2-0439-4db4-a652-89b51a05f70f/volumes"
Jul 22 04:18:17.415931 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:17.415898    2570 generic.go:356] "Generic (PLEG): container finished" podID="527cc23d-888a-480d-a2df-c22595b1f289" containerID="635fa563ee1888b734c58f4b6ee67d7ed2e47fe3b63f68e73ead90668ddbef47" exitCode=0
Jul 22 04:18:17.416131 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:17.415951    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7464-139878922509872" event={"ID":"527cc23d-888a-480d-a2df-c22595b1f289","Type":"ContainerDied","Data":"635fa563ee1888b734c58f4b6ee67d7ed2e47fe3b63f68e73ead90668ddbef47"}
Jul 22 04:18:18.552234 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:18.552206    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-139878922509872"
Jul 22 04:18:18.572861 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:18.572837    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7464-139878922509872_527cc23d-888a-480d-a2df-c22595b1f289/test-curl-7464-139878922509872/0.log"
Jul 22 04:18:18.580122 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:18.580098    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p5jbh\" (UniqueName: \"kubernetes.io/projected/527cc23d-888a-480d-a2df-c22595b1f289-kube-api-access-p5jbh\") pod \"527cc23d-888a-480d-a2df-c22595b1f289\" (UID: \"527cc23d-888a-480d-a2df-c22595b1f289\") "
Jul 22 04:18:18.582317 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:18.582298    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/527cc23d-888a-480d-a2df-c22595b1f289-kube-api-access-p5jbh" (OuterVolumeSpecName: "kube-api-access-p5jbh") pod "527cc23d-888a-480d-a2df-c22595b1f289" (UID: "527cc23d-888a-480d-a2df-c22595b1f289"). InnerVolumeSpecName "kube-api-access-p5jbh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 04:18:18.597601 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:18.597568    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7464-139878922509872"]
Jul 22 04:18:18.599118 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:18.599099    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7464-139878922509872"]
Jul 22 04:18:18.681461 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:18.681385    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5jbh\" (UniqueName: \"kubernetes.io/projected/527cc23d-888a-480d-a2df-c22595b1f289-kube-api-access-p5jbh\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:18:18.768214 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:18.768183    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7464-139878922449808"]
Jul 22 04:18:18.768542 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:18.768531    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="527cc23d-888a-480d-a2df-c22595b1f289" containerName="test-curl-7464-139878922509872"
Jul 22 04:18:18.768595 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:18.768544    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="527cc23d-888a-480d-a2df-c22595b1f289" containerName="test-curl-7464-139878922509872"
Jul 22 04:18:18.768639 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:18.768602    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="527cc23d-888a-480d-a2df-c22595b1f289" containerName="test-curl-7464-139878922509872"
Jul 22 04:18:18.771399 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:18.771385    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-139878922449808"
Jul 22 04:18:18.779947 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:18.779920    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7464-139878922449808"]
Jul 22 04:18:18.852388 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:18.852356    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="527cc23d-888a-480d-a2df-c22595b1f289" path="/var/lib/kubelet/pods/527cc23d-888a-480d-a2df-c22595b1f289/volumes"
Jul 22 04:18:18.884285 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:18.884249    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kczbl\" (UniqueName: \"kubernetes.io/projected/36dd465c-4ea6-43c9-b58a-4b03bcd2effe-kube-api-access-kczbl\") pod \"test-curl-7464-139878922449808\" (UID: \"36dd465c-4ea6-43c9-b58a-4b03bcd2effe\") " pod="opendatahub/test-curl-7464-139878922449808"
Jul 22 04:18:18.985657 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:18.985552    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kczbl\" (UniqueName: \"kubernetes.io/projected/36dd465c-4ea6-43c9-b58a-4b03bcd2effe-kube-api-access-kczbl\") pod \"test-curl-7464-139878922449808\" (UID: \"36dd465c-4ea6-43c9-b58a-4b03bcd2effe\") " pod="opendatahub/test-curl-7464-139878922449808"
Jul 22 04:18:18.994651 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:18.994599    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kczbl\" (UniqueName: \"kubernetes.io/projected/36dd465c-4ea6-43c9-b58a-4b03bcd2effe-kube-api-access-kczbl\") pod \"test-curl-7464-139878922449808\" (UID: \"36dd465c-4ea6-43c9-b58a-4b03bcd2effe\") " pod="opendatahub/test-curl-7464-139878922449808"
Jul 22 04:18:19.088668 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:19.088634    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-139878922449808"
Jul 22 04:18:19.220322 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:19.220273    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7464-139878922449808"]
Jul 22 04:18:19.222474 ip-10-0-132-89 kubenswrapper[2570]: W0722 04:18:19.222450    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod36dd465c_4ea6_43c9_b58a_4b03bcd2effe.slice/crio-8b7223197e69c6949f4fd5ddc166a9c03de038970b9a13bddcce4f98fe4ee84c WatchSource:0}: Error finding container 8b7223197e69c6949f4fd5ddc166a9c03de038970b9a13bddcce4f98fe4ee84c: Status 404 returned error can't find the container with id 8b7223197e69c6949f4fd5ddc166a9c03de038970b9a13bddcce4f98fe4ee84c
Jul 22 04:18:19.426510 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:19.426480    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-139878922509872"
Jul 22 04:18:19.426702 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:19.426531    2570 scope.go:117] "RemoveContainer" containerID="635fa563ee1888b734c58f4b6ee67d7ed2e47fe3b63f68e73ead90668ddbef47"
Jul 22 04:18:19.427928 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:19.427789    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7464-139878922449808" event={"ID":"36dd465c-4ea6-43c9-b58a-4b03bcd2effe","Type":"ContainerStarted","Data":"8b7223197e69c6949f4fd5ddc166a9c03de038970b9a13bddcce4f98fe4ee84c"}
Jul 22 04:18:20.435254 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:20.435216    2570 generic.go:356] "Generic (PLEG): container finished" podID="36dd465c-4ea6-43c9-b58a-4b03bcd2effe" containerID="9ad2e050f2caef4e60d5beb247a4938b066f4ca5da46452b4d3f963e13bb96c4" exitCode=0
Jul 22 04:18:20.435715 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:20.435259    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7464-139878922449808" event={"ID":"36dd465c-4ea6-43c9-b58a-4b03bcd2effe","Type":"ContainerDied","Data":"9ad2e050f2caef4e60d5beb247a4938b066f4ca5da46452b4d3f963e13bb96c4"}
Jul 22 04:18:21.578393 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:21.578370    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-139878922449808"
Jul 22 04:18:21.600347 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:21.600313    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7464-139878922449808_36dd465c-4ea6-43c9-b58a-4b03bcd2effe/test-curl-7464-139878922449808/0.log"
Jul 22 04:18:21.607523 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:21.607499    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kczbl\" (UniqueName: \"kubernetes.io/projected/36dd465c-4ea6-43c9-b58a-4b03bcd2effe-kube-api-access-kczbl\") pod \"36dd465c-4ea6-43c9-b58a-4b03bcd2effe\" (UID: \"36dd465c-4ea6-43c9-b58a-4b03bcd2effe\") "
Jul 22 04:18:21.609768 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:21.609743    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36dd465c-4ea6-43c9-b58a-4b03bcd2effe-kube-api-access-kczbl" (OuterVolumeSpecName: "kube-api-access-kczbl") pod "36dd465c-4ea6-43c9-b58a-4b03bcd2effe" (UID: "36dd465c-4ea6-43c9-b58a-4b03bcd2effe"). InnerVolumeSpecName "kube-api-access-kczbl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 04:18:21.627079 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:21.627043    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7464-139878922449808"]
Jul 22 04:18:21.629304 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:21.629283    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7464-139878922449808"]
Jul 22 04:18:21.709048 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:21.708962    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kczbl\" (UniqueName: \"kubernetes.io/projected/36dd465c-4ea6-43c9-b58a-4b03bcd2effe-kube-api-access-kczbl\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:18:21.755825 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:21.755787    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7464-139878922449808"]
Jul 22 04:18:21.756146 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:21.756132    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36dd465c-4ea6-43c9-b58a-4b03bcd2effe" containerName="test-curl-7464-139878922449808"
Jul 22 04:18:21.756186 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:21.756148    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="36dd465c-4ea6-43c9-b58a-4b03bcd2effe" containerName="test-curl-7464-139878922449808"
Jul 22 04:18:21.756226 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:21.756210    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="36dd465c-4ea6-43c9-b58a-4b03bcd2effe" containerName="test-curl-7464-139878922449808"
Jul 22 04:18:21.759128 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:21.759113    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-139878922449808"
Jul 22 04:18:21.766683 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:21.766663    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7464-139878922449808"]
Jul 22 04:18:21.810239 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:21.810204    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrn56\" (UniqueName: \"kubernetes.io/projected/345b6e00-bd9a-46ac-a196-2a7aa3d584b0-kube-api-access-lrn56\") pod \"test-curl-7464-139878922449808\" (UID: \"345b6e00-bd9a-46ac-a196-2a7aa3d584b0\") " pod="opendatahub/test-curl-7464-139878922449808"
Jul 22 04:18:21.911660 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:21.911597    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrn56\" (UniqueName: \"kubernetes.io/projected/345b6e00-bd9a-46ac-a196-2a7aa3d584b0-kube-api-access-lrn56\") pod \"test-curl-7464-139878922449808\" (UID: \"345b6e00-bd9a-46ac-a196-2a7aa3d584b0\") " pod="opendatahub/test-curl-7464-139878922449808"
Jul 22 04:18:21.921169 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:21.921145    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrn56\" (UniqueName: \"kubernetes.io/projected/345b6e00-bd9a-46ac-a196-2a7aa3d584b0-kube-api-access-lrn56\") pod \"test-curl-7464-139878922449808\" (UID: \"345b6e00-bd9a-46ac-a196-2a7aa3d584b0\") " pod="opendatahub/test-curl-7464-139878922449808"
Jul 22 04:18:22.068959 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:22.068858    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-139878922449808"
Jul 22 04:18:22.203973 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:22.203946    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7464-139878922449808"]
Jul 22 04:18:22.206340 ip-10-0-132-89 kubenswrapper[2570]: W0722 04:18:22.206316    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod345b6e00_bd9a_46ac_a196_2a7aa3d584b0.slice/crio-bd670d7ccc8423f3856c3319b4510efad26b07bc507f667b6abec2655f6199e4 WatchSource:0}: Error finding container bd670d7ccc8423f3856c3319b4510efad26b07bc507f667b6abec2655f6199e4: Status 404 returned error can't find the container with id bd670d7ccc8423f3856c3319b4510efad26b07bc507f667b6abec2655f6199e4
Jul 22 04:18:22.446188 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:22.446152    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7464-139878922449808" event={"ID":"345b6e00-bd9a-46ac-a196-2a7aa3d584b0","Type":"ContainerStarted","Data":"bd670d7ccc8423f3856c3319b4510efad26b07bc507f667b6abec2655f6199e4"}
Jul 22 04:18:22.447385 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:22.447359    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8b7223197e69c6949f4fd5ddc166a9c03de038970b9a13bddcce4f98fe4ee84c"
Jul 22 04:18:22.447385 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:22.447366    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-139878922449808"
Jul 22 04:18:22.459911 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:22.459871    2570 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7464-139878922449808" oldPodUID="36dd465c-4ea6-43c9-b58a-4b03bcd2effe" podUID="345b6e00-bd9a-46ac-a196-2a7aa3d584b0"
Jul 22 04:18:22.569208 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:22.569182    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:18:22.851399 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:22.851319    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36dd465c-4ea6-43c9-b58a-4b03bcd2effe" path="/var/lib/kubelet/pods/36dd465c-4ea6-43c9-b58a-4b03bcd2effe/volumes"
Jul 22 04:18:23.454924 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:23.454881    2570 generic.go:356] "Generic (PLEG): container finished" podID="345b6e00-bd9a-46ac-a196-2a7aa3d584b0" containerID="b18d2248c671bd8c73ee155c4f23950c7ec8fe96a2d6852e369c2355de7b5bc9" exitCode=0
Jul 22 04:18:23.455114 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:23.454940    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7464-139878922449808" event={"ID":"345b6e00-bd9a-46ac-a196-2a7aa3d584b0","Type":"ContainerDied","Data":"b18d2248c671bd8c73ee155c4f23950c7ec8fe96a2d6852e369c2355de7b5bc9"}
Jul 22 04:18:24.600686 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:24.600662    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-139878922449808"
Jul 22 04:18:24.621217 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:24.621192    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7464-139878922449808_345b6e00-bd9a-46ac-a196-2a7aa3d584b0/test-curl-7464-139878922449808/0.log"
Jul 22 04:18:24.639468 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:24.639441    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrn56\" (UniqueName: \"kubernetes.io/projected/345b6e00-bd9a-46ac-a196-2a7aa3d584b0-kube-api-access-lrn56\") pod \"345b6e00-bd9a-46ac-a196-2a7aa3d584b0\" (UID: \"345b6e00-bd9a-46ac-a196-2a7aa3d584b0\") "
Jul 22 04:18:24.641760 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:24.641725    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/345b6e00-bd9a-46ac-a196-2a7aa3d584b0-kube-api-access-lrn56" (OuterVolumeSpecName: "kube-api-access-lrn56") pod "345b6e00-bd9a-46ac-a196-2a7aa3d584b0" (UID: "345b6e00-bd9a-46ac-a196-2a7aa3d584b0"). InnerVolumeSpecName "kube-api-access-lrn56". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 04:18:24.644274 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:24.644250    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7464-139878922449808"]
Jul 22 04:18:24.649117 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:24.649096    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7464-139878922449808"]
Jul 22 04:18:24.740475 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:24.740392    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrn56\" (UniqueName: \"kubernetes.io/projected/345b6e00-bd9a-46ac-a196-2a7aa3d584b0-kube-api-access-lrn56\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:18:24.772847 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:24.772816    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7464-139878922450816"]
Jul 22 04:18:24.773200 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:24.773187    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="345b6e00-bd9a-46ac-a196-2a7aa3d584b0" containerName="test-curl-7464-139878922449808"
Jul 22 04:18:24.773239 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:24.773203    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="345b6e00-bd9a-46ac-a196-2a7aa3d584b0" containerName="test-curl-7464-139878922449808"
Jul 22 04:18:24.773273 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:24.773252    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="345b6e00-bd9a-46ac-a196-2a7aa3d584b0" containerName="test-curl-7464-139878922449808"
Jul 22 04:18:24.775282 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:24.775267    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-139878922450816"
Jul 22 04:18:24.783257 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:24.783234    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7464-139878922450816"]
Jul 22 04:18:24.841422 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:24.841382    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8nd6\" (UniqueName: \"kubernetes.io/projected/079c80ec-37dc-4fb4-a1ad-7315383e85ad-kube-api-access-p8nd6\") pod \"test-curl-7464-139878922450816\" (UID: \"079c80ec-37dc-4fb4-a1ad-7315383e85ad\") " pod="opendatahub/test-curl-7464-139878922450816"
Jul 22 04:18:24.852881 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:24.852849    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="345b6e00-bd9a-46ac-a196-2a7aa3d584b0" path="/var/lib/kubelet/pods/345b6e00-bd9a-46ac-a196-2a7aa3d584b0/volumes"
Jul 22 04:18:24.942247 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:24.942212    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8nd6\" (UniqueName: \"kubernetes.io/projected/079c80ec-37dc-4fb4-a1ad-7315383e85ad-kube-api-access-p8nd6\") pod \"test-curl-7464-139878922450816\" (UID: \"079c80ec-37dc-4fb4-a1ad-7315383e85ad\") " pod="opendatahub/test-curl-7464-139878922450816"
Jul 22 04:18:24.950861 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:24.950832    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8nd6\" (UniqueName: \"kubernetes.io/projected/079c80ec-37dc-4fb4-a1ad-7315383e85ad-kube-api-access-p8nd6\") pod \"test-curl-7464-139878922450816\" (UID: \"079c80ec-37dc-4fb4-a1ad-7315383e85ad\") " pod="opendatahub/test-curl-7464-139878922450816"
Jul 22 04:18:25.086008 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:25.085919    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-139878922450816"
Jul 22 04:18:25.221982 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:25.221799    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7464-139878922450816"]
Jul 22 04:18:25.225226 ip-10-0-132-89 kubenswrapper[2570]: W0722 04:18:25.225197    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod079c80ec_37dc_4fb4_a1ad_7315383e85ad.slice/crio-d55743b34613393855ac8750f11d3abee9f22062edac33677e5aa7990c5616ec WatchSource:0}: Error finding container d55743b34613393855ac8750f11d3abee9f22062edac33677e5aa7990c5616ec: Status 404 returned error can't find the container with id d55743b34613393855ac8750f11d3abee9f22062edac33677e5aa7990c5616ec
Jul 22 04:18:25.466851 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:25.466823    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-139878922449808"
Jul 22 04:18:25.467063 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:25.466827    2570 scope.go:117] "RemoveContainer" containerID="b18d2248c671bd8c73ee155c4f23950c7ec8fe96a2d6852e369c2355de7b5bc9"
Jul 22 04:18:25.468214 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:25.468188    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7464-139878922450816" event={"ID":"079c80ec-37dc-4fb4-a1ad-7315383e85ad","Type":"ContainerStarted","Data":"d55743b34613393855ac8750f11d3abee9f22062edac33677e5aa7990c5616ec"}
Jul 22 04:18:26.475784 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:26.475743    2570 generic.go:356] "Generic (PLEG): container finished" podID="079c80ec-37dc-4fb4-a1ad-7315383e85ad" containerID="276b6b3fdae94804cb36265e1a89e1e004d55502cdb6f0a46344144f46fe8317" exitCode=0
Jul 22 04:18:26.476204 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:26.475832    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7464-139878922450816" event={"ID":"079c80ec-37dc-4fb4-a1ad-7315383e85ad","Type":"ContainerDied","Data":"276b6b3fdae94804cb36265e1a89e1e004d55502cdb6f0a46344144f46fe8317"}
Jul 22 04:18:26.510481 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:26.510438    2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-38610b/rate-test-model-38610b-kserve-6d98dc9cc9-7klhf" podUID="3c466798-bf18-4c82-9497-4c6afcbe60bb" containerName="main" containerID="cri-o://ff581c8a1473bbeafc3d497bfefcb06d8e57a1d4b8ffb9a06fac579ad39d1682" gracePeriod=45
Jul 22 04:18:26.776229 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:26.776208    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-38610b/rate-test-model-38610b-kserve-6d98dc9cc9-7klhf"
Jul 22 04:18:26.859594 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:26.859567    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c466798-bf18-4c82-9497-4c6afcbe60bb-tmp-dir\") pod \"3c466798-bf18-4c82-9497-4c6afcbe60bb\" (UID: \"3c466798-bf18-4c82-9497-4c6afcbe60bb\") "
Jul 22 04:18:26.859767 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:26.859626    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c466798-bf18-4c82-9497-4c6afcbe60bb-tls-certs\") pod \"3c466798-bf18-4c82-9497-4c6afcbe60bb\" (UID: \"3c466798-bf18-4c82-9497-4c6afcbe60bb\") "
Jul 22 04:18:26.859767 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:26.859665    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c466798-bf18-4c82-9497-4c6afcbe60bb-home\") pod \"3c466798-bf18-4c82-9497-4c6afcbe60bb\" (UID: \"3c466798-bf18-4c82-9497-4c6afcbe60bb\") "
Jul 22 04:18:26.859767 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:26.859711    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-smrcg\" (UniqueName: \"kubernetes.io/projected/3c466798-bf18-4c82-9497-4c6afcbe60bb-kube-api-access-smrcg\") pod \"3c466798-bf18-4c82-9497-4c6afcbe60bb\" (UID: \"3c466798-bf18-4c82-9497-4c6afcbe60bb\") "
Jul 22 04:18:26.859904 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:26.859782    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c466798-bf18-4c82-9497-4c6afcbe60bb-model-cache\") pod \"3c466798-bf18-4c82-9497-4c6afcbe60bb\" (UID: \"3c466798-bf18-4c82-9497-4c6afcbe60bb\") "
Jul 22 04:18:26.859904 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:26.859828    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c466798-bf18-4c82-9497-4c6afcbe60bb-dshm\") pod \"3c466798-bf18-4c82-9497-4c6afcbe60bb\" (UID: \"3c466798-bf18-4c82-9497-4c6afcbe60bb\") "
Jul 22 04:18:26.859904 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:26.859857    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c466798-bf18-4c82-9497-4c6afcbe60bb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3c466798-bf18-4c82-9497-4c6afcbe60bb" (UID: "3c466798-bf18-4c82-9497-4c6afcbe60bb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 04:18:26.860080 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:26.859948    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c466798-bf18-4c82-9497-4c6afcbe60bb-home" (OuterVolumeSpecName: "home") pod "3c466798-bf18-4c82-9497-4c6afcbe60bb" (UID: "3c466798-bf18-4c82-9497-4c6afcbe60bb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 04:18:26.860133 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:26.860070    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c466798-bf18-4c82-9497-4c6afcbe60bb-model-cache" (OuterVolumeSpecName: "model-cache") pod "3c466798-bf18-4c82-9497-4c6afcbe60bb" (UID: "3c466798-bf18-4c82-9497-4c6afcbe60bb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 04:18:26.860196 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:26.860176    2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c466798-bf18-4c82-9497-4c6afcbe60bb-home\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:18:26.860248 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:26.860203    2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c466798-bf18-4c82-9497-4c6afcbe60bb-model-cache\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:18:26.860248 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:26.860221    2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c466798-bf18-4c82-9497-4c6afcbe60bb-tmp-dir\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:18:26.862003 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:26.861984    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c466798-bf18-4c82-9497-4c6afcbe60bb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3c466798-bf18-4c82-9497-4c6afcbe60bb" (UID: "3c466798-bf18-4c82-9497-4c6afcbe60bb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 04:18:26.862063 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:26.861991    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c466798-bf18-4c82-9497-4c6afcbe60bb-dshm" (OuterVolumeSpecName: "dshm") pod "3c466798-bf18-4c82-9497-4c6afcbe60bb" (UID: "3c466798-bf18-4c82-9497-4c6afcbe60bb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 04:18:26.862063 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:26.862004    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c466798-bf18-4c82-9497-4c6afcbe60bb-kube-api-access-smrcg" (OuterVolumeSpecName: "kube-api-access-smrcg") pod "3c466798-bf18-4c82-9497-4c6afcbe60bb" (UID: "3c466798-bf18-4c82-9497-4c6afcbe60bb"). InnerVolumeSpecName "kube-api-access-smrcg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 04:18:26.961628 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:26.961589    2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c466798-bf18-4c82-9497-4c6afcbe60bb-dshm\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:18:26.961628 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:26.961634    2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c466798-bf18-4c82-9497-4c6afcbe60bb-tls-certs\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:18:26.961837 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:26.961647    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smrcg\" (UniqueName: \"kubernetes.io/projected/3c466798-bf18-4c82-9497-4c6afcbe60bb-kube-api-access-smrcg\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:18:27.482334 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:27.482299    2570 generic.go:356] "Generic (PLEG): container finished" podID="3c466798-bf18-4c82-9497-4c6afcbe60bb" containerID="ff581c8a1473bbeafc3d497bfefcb06d8e57a1d4b8ffb9a06fac579ad39d1682" exitCode=0
Jul 22 04:18:27.482788 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:27.482343    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-38610b/rate-test-model-38610b-kserve-6d98dc9cc9-7klhf"
Jul 22 04:18:27.482788 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:27.482381    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-38610b/rate-test-model-38610b-kserve-6d98dc9cc9-7klhf" event={"ID":"3c466798-bf18-4c82-9497-4c6afcbe60bb","Type":"ContainerDied","Data":"ff581c8a1473bbeafc3d497bfefcb06d8e57a1d4b8ffb9a06fac579ad39d1682"}
Jul 22 04:18:27.482788 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:27.482416    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-38610b/rate-test-model-38610b-kserve-6d98dc9cc9-7klhf" event={"ID":"3c466798-bf18-4c82-9497-4c6afcbe60bb","Type":"ContainerDied","Data":"be730ea5c3027fe0884db9f7b2055a25f738ca1088b478e778c58b92b5fd6abf"}
Jul 22 04:18:27.482788 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:27.482434    2570 scope.go:117] "RemoveContainer" containerID="ff581c8a1473bbeafc3d497bfefcb06d8e57a1d4b8ffb9a06fac579ad39d1682"
Jul 22 04:18:27.495391 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:27.495359    2570 scope.go:117] "RemoveContainer" containerID="ff581c8a1473bbeafc3d497bfefcb06d8e57a1d4b8ffb9a06fac579ad39d1682"
Jul 22 04:18:27.495697 ip-10-0-132-89 kubenswrapper[2570]: E0722 04:18:27.495663    2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff581c8a1473bbeafc3d497bfefcb06d8e57a1d4b8ffb9a06fac579ad39d1682\": container with ID starting with ff581c8a1473bbeafc3d497bfefcb06d8e57a1d4b8ffb9a06fac579ad39d1682 not found: ID does not exist" containerID="ff581c8a1473bbeafc3d497bfefcb06d8e57a1d4b8ffb9a06fac579ad39d1682"
Jul 22 04:18:27.495789 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:27.495707    2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff581c8a1473bbeafc3d497bfefcb06d8e57a1d4b8ffb9a06fac579ad39d1682"} err="failed to get container status \"ff581c8a1473bbeafc3d497bfefcb06d8e57a1d4b8ffb9a06fac579ad39d1682\": rpc error: code = NotFound desc = could not find container \"ff581c8a1473bbeafc3d497bfefcb06d8e57a1d4b8ffb9a06fac579ad39d1682\": container with ID starting with ff581c8a1473bbeafc3d497bfefcb06d8e57a1d4b8ffb9a06fac579ad39d1682 not found: ID does not exist"
Jul 22 04:18:27.504274 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:27.504249    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-38610b/rate-test-model-38610b-kserve-6d98dc9cc9-7klhf"]
Jul 22 04:18:27.512417 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:27.510133    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-38610b/rate-test-model-38610b-kserve-6d98dc9cc9-7klhf"]
Jul 22 04:18:27.613428 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:27.613405    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-139878922450816"
Jul 22 04:18:27.632945 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:27.632917    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7464-139878922450816_079c80ec-37dc-4fb4-a1ad-7315383e85ad/test-curl-7464-139878922450816/0.log"
Jul 22 04:18:27.653476 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:27.653439    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7464-139878922450816"]
Jul 22 04:18:27.656135 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:27.656112    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7464-139878922450816"]
Jul 22 04:18:27.666681 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:27.666661    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p8nd6\" (UniqueName: \"kubernetes.io/projected/079c80ec-37dc-4fb4-a1ad-7315383e85ad-kube-api-access-p8nd6\") pod \"079c80ec-37dc-4fb4-a1ad-7315383e85ad\" (UID: \"079c80ec-37dc-4fb4-a1ad-7315383e85ad\") "
Jul 22 04:18:27.668812 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:27.668792    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/079c80ec-37dc-4fb4-a1ad-7315383e85ad-kube-api-access-p8nd6" (OuterVolumeSpecName: "kube-api-access-p8nd6") pod "079c80ec-37dc-4fb4-a1ad-7315383e85ad" (UID: "079c80ec-37dc-4fb4-a1ad-7315383e85ad"). InnerVolumeSpecName "kube-api-access-p8nd6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 04:18:27.767704 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:27.767670    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p8nd6\" (UniqueName: \"kubernetes.io/projected/079c80ec-37dc-4fb4-a1ad-7315383e85ad-kube-api-access-p8nd6\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:18:27.774677 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:27.774646    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7464-139878922450816"]
Jul 22 04:18:27.775069 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:27.775052    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="079c80ec-37dc-4fb4-a1ad-7315383e85ad" containerName="test-curl-7464-139878922450816"
Jul 22 04:18:27.775113 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:27.775072    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="079c80ec-37dc-4fb4-a1ad-7315383e85ad" containerName="test-curl-7464-139878922450816"
Jul 22 04:18:27.775113 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:27.775085    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c466798-bf18-4c82-9497-4c6afcbe60bb" containerName="main"
Jul 22 04:18:27.775113 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:27.775091    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c466798-bf18-4c82-9497-4c6afcbe60bb" containerName="main"
Jul 22 04:18:27.775195 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:27.775167    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="079c80ec-37dc-4fb4-a1ad-7315383e85ad" containerName="test-curl-7464-139878922450816"
Jul 22 04:18:27.775195 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:27.775181    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c466798-bf18-4c82-9497-4c6afcbe60bb" containerName="main"
Jul 22 04:18:27.778158 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:27.778143    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-139878922450816"
Jul 22 04:18:27.784711 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:27.784682    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7464-139878922450816"]
Jul 22 04:18:27.868833 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:27.868794    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sx2fb\" (UniqueName: \"kubernetes.io/projected/283314d3-ebb4-4391-af5e-de57998732a6-kube-api-access-sx2fb\") pod \"test-curl-7464-139878922450816\" (UID: \"283314d3-ebb4-4391-af5e-de57998732a6\") " pod="opendatahub/test-curl-7464-139878922450816"
Jul 22 04:18:27.970249 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:27.970203    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sx2fb\" (UniqueName: \"kubernetes.io/projected/283314d3-ebb4-4391-af5e-de57998732a6-kube-api-access-sx2fb\") pod \"test-curl-7464-139878922450816\" (UID: \"283314d3-ebb4-4391-af5e-de57998732a6\") " pod="opendatahub/test-curl-7464-139878922450816"
Jul 22 04:18:27.979095 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:27.979057    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sx2fb\" (UniqueName: \"kubernetes.io/projected/283314d3-ebb4-4391-af5e-de57998732a6-kube-api-access-sx2fb\") pod \"test-curl-7464-139878922450816\" (UID: \"283314d3-ebb4-4391-af5e-de57998732a6\") " pod="opendatahub/test-curl-7464-139878922450816"
Jul 22 04:18:28.089023 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:28.088926    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-139878922450816"
Jul 22 04:18:28.224033 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:28.223964    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7464-139878922450816"]
Jul 22 04:18:28.226311 ip-10-0-132-89 kubenswrapper[2570]: W0722 04:18:28.226282    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod283314d3_ebb4_4391_af5e_de57998732a6.slice/crio-d30b7a43a3f12bd894561cd2d71f54b9720ab1dc9d5cf9dccad332bda1abbdc6 WatchSource:0}: Error finding container d30b7a43a3f12bd894561cd2d71f54b9720ab1dc9d5cf9dccad332bda1abbdc6: Status 404 returned error can't find the container with id d30b7a43a3f12bd894561cd2d71f54b9720ab1dc9d5cf9dccad332bda1abbdc6
Jul 22 04:18:28.488780 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:28.488744    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7464-139878922450816" event={"ID":"283314d3-ebb4-4391-af5e-de57998732a6","Type":"ContainerStarted","Data":"d30b7a43a3f12bd894561cd2d71f54b9720ab1dc9d5cf9dccad332bda1abbdc6"}
Jul 22 04:18:28.489921 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:28.489902    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d55743b34613393855ac8750f11d3abee9f22062edac33677e5aa7990c5616ec"
Jul 22 04:18:28.489983 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:28.489911    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-139878922450816"
Jul 22 04:18:28.503383 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:28.503357    2570 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7464-139878922450816" oldPodUID="079c80ec-37dc-4fb4-a1ad-7315383e85ad" podUID="283314d3-ebb4-4391-af5e-de57998732a6"
Jul 22 04:18:28.852297 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:28.852215    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="079c80ec-37dc-4fb4-a1ad-7315383e85ad" path="/var/lib/kubelet/pods/079c80ec-37dc-4fb4-a1ad-7315383e85ad/volumes"
Jul 22 04:18:28.852593 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:28.852581    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c466798-bf18-4c82-9497-4c6afcbe60bb" path="/var/lib/kubelet/pods/3c466798-bf18-4c82-9497-4c6afcbe60bb/volumes"
Jul 22 04:18:29.496146 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:29.496108    2570 generic.go:356] "Generic (PLEG): container finished" podID="283314d3-ebb4-4391-af5e-de57998732a6" containerID="34bbe4f7a3a97e22424c5e211548b9ba3ca42401e41658abc2f842e10f4690c3" exitCode=0
Jul 22 04:18:29.496546 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:29.496190    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7464-139878922450816" event={"ID":"283314d3-ebb4-4391-af5e-de57998732a6","Type":"ContainerDied","Data":"34bbe4f7a3a97e22424c5e211548b9ba3ca42401e41658abc2f842e10f4690c3"}
Jul 22 04:18:30.637307 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:30.637280    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-139878922450816"
Jul 22 04:18:30.658771 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:30.658743    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7464-139878922450816_283314d3-ebb4-4391-af5e-de57998732a6/test-curl-7464-139878922450816/0.log"
Jul 22 04:18:30.681197 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:30.681157    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7464-139878922450816"]
Jul 22 04:18:30.683256 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:30.683227    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7464-139878922450816"]
Jul 22 04:18:30.694929 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:30.694902    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sx2fb\" (UniqueName: \"kubernetes.io/projected/283314d3-ebb4-4391-af5e-de57998732a6-kube-api-access-sx2fb\") pod \"283314d3-ebb4-4391-af5e-de57998732a6\" (UID: \"283314d3-ebb4-4391-af5e-de57998732a6\") "
Jul 22 04:18:30.697089 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:30.697068    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/283314d3-ebb4-4391-af5e-de57998732a6-kube-api-access-sx2fb" (OuterVolumeSpecName: "kube-api-access-sx2fb") pod "283314d3-ebb4-4391-af5e-de57998732a6" (UID: "283314d3-ebb4-4391-af5e-de57998732a6"). InnerVolumeSpecName "kube-api-access-sx2fb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 04:18:30.795902 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:30.795815    2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sx2fb\" (UniqueName: \"kubernetes.io/projected/283314d3-ebb4-4391-af5e-de57998732a6-kube-api-access-sx2fb\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:18:30.851630 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:30.851575    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="283314d3-ebb4-4391-af5e-de57998732a6" path="/var/lib/kubelet/pods/283314d3-ebb4-4391-af5e-de57998732a6/volumes"
Jul 22 04:18:30.878574 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:30.878544    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-dd1671-dfq7m"]
Jul 22 04:18:30.878903 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:30.878889    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="283314d3-ebb4-4391-af5e-de57998732a6" containerName="test-curl-7464-139878922450816"
Jul 22 04:18:30.878957 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:30.878905    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="283314d3-ebb4-4391-af5e-de57998732a6" containerName="test-curl-7464-139878922450816"
Jul 22 04:18:30.878990 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:30.878973    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="283314d3-ebb4-4391-af5e-de57998732a6" containerName="test-curl-7464-139878922450816"
Jul 22 04:18:30.882035 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:30.882021    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-dd1671-dfq7m"
Jul 22 04:18:30.884956 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:30.884933    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 04:18:30.885080 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:30.885043    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rbfm6\""
Jul 22 04:18:30.892375 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:30.892349    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-dd1671-dfq7m"]
Jul 22 04:18:30.997558 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:30.997516    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cb2244d-403d-4aa9-b3bc-bb62b41038a2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-dd1671-dfq7m\" (UID: \"3cb2244d-403d-4aa9-b3bc-bb62b41038a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-dd1671-dfq7m"
Jul 22 04:18:31.098733 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:31.098643    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cb2244d-403d-4aa9-b3bc-bb62b41038a2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-dd1671-dfq7m\" (UID: \"3cb2244d-403d-4aa9-b3bc-bb62b41038a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-dd1671-dfq7m"
Jul 22 04:18:31.099271 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:31.099248    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cb2244d-403d-4aa9-b3bc-bb62b41038a2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-dd1671-dfq7m\" (UID: \"3cb2244d-403d-4aa9-b3bc-bb62b41038a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-dd1671-dfq7m"
Jul 22 04:18:31.192747 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:31.192705    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-dd1671-dfq7m"
Jul 22 04:18:31.348493 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:31.348314    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-dd1671-dfq7m"]
Jul 22 04:18:31.351246 ip-10-0-132-89 kubenswrapper[2570]: W0722 04:18:31.351220    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cb2244d_403d_4aa9_b3bc_bb62b41038a2.slice/crio-e13aaf802645085ebc821fa90ca6cf925a832c55e15ff703fe2299eee0b1babc WatchSource:0}: Error finding container e13aaf802645085ebc821fa90ca6cf925a832c55e15ff703fe2299eee0b1babc: Status 404 returned error can't find the container with id e13aaf802645085ebc821fa90ca6cf925a832c55e15ff703fe2299eee0b1babc
Jul 22 04:18:31.507348 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:31.507313    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-dd1671-dfq7m" event={"ID":"3cb2244d-403d-4aa9-b3bc-bb62b41038a2","Type":"ContainerStarted","Data":"e13aaf802645085ebc821fa90ca6cf925a832c55e15ff703fe2299eee0b1babc"}
Jul 22 04:18:31.508497 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:31.508470    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7464-139878922450816"
Jul 22 04:18:31.508497 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:31.508468    2570 scope.go:117] "RemoveContainer" containerID="34bbe4f7a3a97e22424c5e211548b9ba3ca42401e41658abc2f842e10f4690c3"
Jul 22 04:18:32.514360 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:32.514326    2570 generic.go:356] "Generic (PLEG): container finished" podID="3cb2244d-403d-4aa9-b3bc-bb62b41038a2" containerID="41a85c65fc7a61cdb4d9efa609f72c567d4b5e63ecd37f92f6d6a128c8dbe67b" exitCode=0
Jul 22 04:18:32.514854 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:32.514415    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-dd1671-dfq7m" event={"ID":"3cb2244d-403d-4aa9-b3bc-bb62b41038a2","Type":"ContainerDied","Data":"41a85c65fc7a61cdb4d9efa609f72c567d4b5e63ecd37f92f6d6a128c8dbe67b"}
Jul 22 04:18:32.951713 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:32.951684    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2p4qp_9f0defcc-ccff-43be-8206-54c772885080/console-operator/1.log"
Jul 22 04:18:32.956252 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:32.956232    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2p4qp_9f0defcc-ccff-43be-8206-54c772885080/console-operator/1.log"
Jul 22 04:18:33.108854 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:33.108820    2570 scope.go:117] "RemoveContainer" containerID="e950c1c871996ddb39730a7fb3eda92f0ff19fc137fc8f158875047395333077"
Jul 22 04:18:33.658914 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:33.658889    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-dd1671-dfq7m"
Jul 22 04:18:33.722246 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:33.722208    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cb2244d-403d-4aa9-b3bc-bb62b41038a2-maas-api-service-ca\") pod \"3cb2244d-403d-4aa9-b3bc-bb62b41038a2\" (UID: \"3cb2244d-403d-4aa9-b3bc-bb62b41038a2\") "
Jul 22 04:18:33.722578 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:33.722555    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cb2244d-403d-4aa9-b3bc-bb62b41038a2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3cb2244d-403d-4aa9-b3bc-bb62b41038a2" (UID: "3cb2244d-403d-4aa9-b3bc-bb62b41038a2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 04:18:33.823067 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:33.822980    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cb2244d-403d-4aa9-b3bc-bb62b41038a2-maas-api-service-ca\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:18:34.526524 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:34.526490    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-dd1671-dfq7m"
Jul 22 04:18:34.526524 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:34.526518    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-dd1671-dfq7m" event={"ID":"3cb2244d-403d-4aa9-b3bc-bb62b41038a2","Type":"ContainerDied","Data":"e13aaf802645085ebc821fa90ca6cf925a832c55e15ff703fe2299eee0b1babc"}
Jul 22 04:18:34.526744 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:34.526573    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e13aaf802645085ebc821fa90ca6cf925a832c55e15ff703fe2299eee0b1babc"
Jul 22 04:18:44.362931 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:44.362891    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:18:46.043345 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:46.043314    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-dd1671-dfq7m"]
Jul 22 04:18:46.046167 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:46.046143    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-dd1671-dfq7m"]
Jul 22 04:18:46.852082 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:46.852049    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cb2244d-403d-4aa9-b3bc-bb62b41038a2" path="/var/lib/kubelet/pods/3cb2244d-403d-4aa9-b3bc-bb62b41038a2/volumes"
Jul 22 04:18:53.564489 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:53.564450    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-38610b-65qwc"]
Jul 22 04:18:53.564863 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:53.564815    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cb2244d-403d-4aa9-b3bc-bb62b41038a2" containerName="revoke-keys"
Jul 22 04:18:53.564863 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:53.564827    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cb2244d-403d-4aa9-b3bc-bb62b41038a2" containerName="revoke-keys"
Jul 22 04:18:53.564940 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:53.564883    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cb2244d-403d-4aa9-b3bc-bb62b41038a2" containerName="revoke-keys"
Jul 22 04:18:53.569262 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:53.569243    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-38610b-65qwc"
Jul 22 04:18:53.572086 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:53.572061    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rbfm6\""
Jul 22 04:18:53.572194 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:53.572062    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 04:18:53.575984 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:53.575958    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-38610b-65qwc"]
Jul 22 04:18:53.699340 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:53.699305    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c2cc05e-701e-447b-b105-99c81676b003-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-38610b-65qwc\" (UID: \"4c2cc05e-701e-447b-b105-99c81676b003\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-38610b-65qwc"
Jul 22 04:18:53.800214 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:53.800173    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c2cc05e-701e-447b-b105-99c81676b003-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-38610b-65qwc\" (UID: \"4c2cc05e-701e-447b-b105-99c81676b003\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-38610b-65qwc"
Jul 22 04:18:53.800891 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:53.800870    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c2cc05e-701e-447b-b105-99c81676b003-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-38610b-65qwc\" (UID: \"4c2cc05e-701e-447b-b105-99c81676b003\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-38610b-65qwc"
Jul 22 04:18:53.881826 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:53.881735    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-38610b-65qwc"
Jul 22 04:18:54.028145 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:54.028123    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-38610b-65qwc"]
Jul 22 04:18:54.030210 ip-10-0-132-89 kubenswrapper[2570]: W0722 04:18:54.030182    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c2cc05e_701e_447b_b105_99c81676b003.slice/crio-ce5e1ae0e7b70de259dd0b5a79b934b74bb5151b31d08de723e9e444be4abd7a WatchSource:0}: Error finding container ce5e1ae0e7b70de259dd0b5a79b934b74bb5151b31d08de723e9e444be4abd7a: Status 404 returned error can't find the container with id ce5e1ae0e7b70de259dd0b5a79b934b74bb5151b31d08de723e9e444be4abd7a
Jul 22 04:18:54.623064 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:54.622973    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-38610b-65qwc" event={"ID":"4c2cc05e-701e-447b-b105-99c81676b003","Type":"ContainerStarted","Data":"1d89236e3736b6289fe4ee898e6a9e7b77d66a8014757f3beea0b0a2bb8f15f5"}
Jul 22 04:18:54.623064 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:54.623021    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-38610b-65qwc" event={"ID":"4c2cc05e-701e-447b-b105-99c81676b003","Type":"ContainerStarted","Data":"ce5e1ae0e7b70de259dd0b5a79b934b74bb5151b31d08de723e9e444be4abd7a"}
Jul 22 04:18:54.647229 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:54.647179    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-38610b-65qwc" podStartSLOduration=1.647163208 podStartE2EDuration="1.647163208s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:18:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:18:54.646702596 +0000 UTC m=+2722.264912603" watchObservedRunningTime="2026-07-22 04:18:54.647163208 +0000 UTC m=+2722.265373214"
Jul 22 04:18:55.629540 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:55.629497    2570 generic.go:356] "Generic (PLEG): container finished" podID="4c2cc05e-701e-447b-b105-99c81676b003" containerID="1d89236e3736b6289fe4ee898e6a9e7b77d66a8014757f3beea0b0a2bb8f15f5" exitCode=0
Jul 22 04:18:55.629999 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:55.629568    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-38610b-65qwc" event={"ID":"4c2cc05e-701e-447b-b105-99c81676b003","Type":"ContainerDied","Data":"1d89236e3736b6289fe4ee898e6a9e7b77d66a8014757f3beea0b0a2bb8f15f5"}
Jul 22 04:18:56.768957 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:56.768925    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-38610b-65qwc"
Jul 22 04:18:56.926334 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:56.926307    2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c2cc05e-701e-447b-b105-99c81676b003-maas-api-service-ca\") pod \"4c2cc05e-701e-447b-b105-99c81676b003\" (UID: \"4c2cc05e-701e-447b-b105-99c81676b003\") "
Jul 22 04:18:56.926737 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:56.926712    2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c2cc05e-701e-447b-b105-99c81676b003-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4c2cc05e-701e-447b-b105-99c81676b003" (UID: "4c2cc05e-701e-447b-b105-99c81676b003"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 04:18:57.027400 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:57.027361    2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c2cc05e-701e-447b-b105-99c81676b003-maas-api-service-ca\") on node \"ip-10-0-132-89.ec2.internal\" DevicePath \"\""
Jul 22 04:18:57.640925 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:57.640894    2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-38610b-65qwc"
Jul 22 04:18:57.640925 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:57.640924    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-38610b-65qwc" event={"ID":"4c2cc05e-701e-447b-b105-99c81676b003","Type":"ContainerDied","Data":"ce5e1ae0e7b70de259dd0b5a79b934b74bb5151b31d08de723e9e444be4abd7a"}
Jul 22 04:18:57.641148 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:18:57.640966    2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ce5e1ae0e7b70de259dd0b5a79b934b74bb5151b31d08de723e9e444be4abd7a"
Jul 22 04:19:04.373719 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:04.373681    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-38610b-29744895-kzdjw"]
Jul 22 04:19:04.377958 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:04.377934    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-38610b-29744895-kzdjw"]
Jul 22 04:19:04.852875 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:04.852844    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c01c66b-bf2c-4ac5-a272-0826016b0f39" path="/var/lib/kubelet/pods/9c01c66b-bf2c-4ac5-a272-0826016b0f39/volumes"
Jul 22 04:19:07.695409 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:07.695377    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-2js4d"]
Jul 22 04:19:08.716599 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:08.716562    2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-38610b-65qwc"]
Jul 22 04:19:08.721926 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:08.721901    2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-38610b-65qwc"]
Jul 22 04:19:08.852298 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:08.852261    2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c2cc05e-701e-447b-b105-99c81676b003" path="/var/lib/kubelet/pods/4c2cc05e-701e-447b-b105-99c81676b003/volumes"
Jul 22 04:19:22.335470 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:22.335438    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5fd47b996b-fh5t6_e7801170-6a68-4c68-bfcb-f7ef0acc2fae/copy-manifests/0.log"
Jul 22 04:19:22.342349 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:22.342331    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5fd47b996b-fh5t6_e7801170-6a68-4c68-bfcb-f7ef0acc2fae/manager/0.log"
Jul 22 04:19:23.177035 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:23.176998    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29744895-zxz6c_4093e54c-8d9a-4eff-b71e-167208d29dd7/cleanup/0.log"
Jul 22 04:19:24.808867 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:24.808841    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-csccb_a05e14f1-2573-4358-b15b-09a094fa010c/manager/0.log"
Jul 22 04:19:24.918588 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:24.918564    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-2cb7r_c709ad99-a734-47a6-853d-22ec32915ab6/kuadrant-console-plugin/0.log"
Jul 22 04:19:25.297064 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:25.297039    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-2js4d_3707c428-6025-4ada-b4c4-684dc735790e/limitador/0.log"
Jul 22 04:19:26.129558 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:26.129534    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bd5997d8b-cpbs7_3eb33ff7-d4d6-4141-b23e-69d3cafb29b2/kube-auth-proxy/0.log"
Jul 22 04:19:26.360143 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:26.360114    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-vd2dl_26c42593-c5c2-497c-b0ff-52938d7447df/payload-pre-processing/0.log"
Jul 22 04:19:26.593507 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:26.593474    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7455857b66-nlzjg_f23b727c-e88a-42b1-80e8-02384f07c8f6/router/0.log"
Jul 22 04:19:27.180512 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:27.180485    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-lpgt9_e321c227-20dc-4359-9ab0-b63f341699d7/main/0.log"
Jul 22 04:19:27.523842 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:27.523764    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-8p52g_25c4f355-e8a5-4b0a-8eb1-b37487417b20/main/0.log"
Jul 22 04:19:32.836414 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:32.836372    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-m4q6n/must-gather-vtqg5"]
Jul 22 04:19:32.836899 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:32.836758    2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c2cc05e-701e-447b-b105-99c81676b003" containerName="revoke-keys"
Jul 22 04:19:32.836899 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:32.836776    2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c2cc05e-701e-447b-b105-99c81676b003" containerName="revoke-keys"
Jul 22 04:19:32.836899 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:32.836898    2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c2cc05e-701e-447b-b105-99c81676b003" containerName="revoke-keys"
Jul 22 04:19:32.840921 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:32.840896    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-m4q6n/must-gather-vtqg5"
Jul 22 04:19:32.843809 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:32.843783    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-m4q6n\"/\"kube-root-ca.crt\""
Jul 22 04:19:32.843953 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:32.843855    2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-m4q6n\"/\"default-dockercfg-hhvf6\""
Jul 22 04:19:32.844527 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:32.844507    2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-m4q6n\"/\"openshift-service-ca.crt\""
Jul 22 04:19:32.857773 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:32.857741    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-m4q6n/must-gather-vtqg5"]
Jul 22 04:19:32.938176 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:32.938141    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4ef9972c-aeed-496d-bca2-cb5de52d01a0-must-gather-output\") pod \"must-gather-vtqg5\" (UID: \"4ef9972c-aeed-496d-bca2-cb5de52d01a0\") " pod="openshift-must-gather-m4q6n/must-gather-vtqg5"
Jul 22 04:19:32.938375 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:32.938186    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czhgj\" (UniqueName: \"kubernetes.io/projected/4ef9972c-aeed-496d-bca2-cb5de52d01a0-kube-api-access-czhgj\") pod \"must-gather-vtqg5\" (UID: \"4ef9972c-aeed-496d-bca2-cb5de52d01a0\") " pod="openshift-must-gather-m4q6n/must-gather-vtqg5"
Jul 22 04:19:33.039464 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:33.039425    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4ef9972c-aeed-496d-bca2-cb5de52d01a0-must-gather-output\") pod \"must-gather-vtqg5\" (UID: \"4ef9972c-aeed-496d-bca2-cb5de52d01a0\") " pod="openshift-must-gather-m4q6n/must-gather-vtqg5"
Jul 22 04:19:33.039464 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:33.039470    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czhgj\" (UniqueName: \"kubernetes.io/projected/4ef9972c-aeed-496d-bca2-cb5de52d01a0-kube-api-access-czhgj\") pod \"must-gather-vtqg5\" (UID: \"4ef9972c-aeed-496d-bca2-cb5de52d01a0\") " pod="openshift-must-gather-m4q6n/must-gather-vtqg5"
Jul 22 04:19:33.039885 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:33.039863    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4ef9972c-aeed-496d-bca2-cb5de52d01a0-must-gather-output\") pod \"must-gather-vtqg5\" (UID: \"4ef9972c-aeed-496d-bca2-cb5de52d01a0\") " pod="openshift-must-gather-m4q6n/must-gather-vtqg5"
Jul 22 04:19:33.048188 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:33.048168    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czhgj\" (UniqueName: \"kubernetes.io/projected/4ef9972c-aeed-496d-bca2-cb5de52d01a0-kube-api-access-czhgj\") pod \"must-gather-vtqg5\" (UID: \"4ef9972c-aeed-496d-bca2-cb5de52d01a0\") " pod="openshift-must-gather-m4q6n/must-gather-vtqg5"
Jul 22 04:19:33.152746 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:33.152676    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-m4q6n/must-gather-vtqg5"
Jul 22 04:19:33.305751 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:33.295050    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-m4q6n/must-gather-vtqg5"]
Jul 22 04:19:33.305751 ip-10-0-132-89 kubenswrapper[2570]: W0722 04:19:33.297491    2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4ef9972c_aeed_496d_bca2_cb5de52d01a0.slice/crio-bc2dee1adc9be9888e4052042e84e70f205c1e9a48fbfed011afc75c84d22d4b WatchSource:0}: Error finding container bc2dee1adc9be9888e4052042e84e70f205c1e9a48fbfed011afc75c84d22d4b: Status 404 returned error can't find the container with id bc2dee1adc9be9888e4052042e84e70f205c1e9a48fbfed011afc75c84d22d4b
Jul 22 04:19:33.819220 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:33.819179    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-m4q6n/must-gather-vtqg5" event={"ID":"4ef9972c-aeed-496d-bca2-cb5de52d01a0","Type":"ContainerStarted","Data":"bc2dee1adc9be9888e4052042e84e70f205c1e9a48fbfed011afc75c84d22d4b"}
Jul 22 04:19:34.827270 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:34.827234    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-m4q6n/must-gather-vtqg5" event={"ID":"4ef9972c-aeed-496d-bca2-cb5de52d01a0","Type":"ContainerStarted","Data":"9cd4bcbfd5c9ed54284619e2ffe39e189e6ea200385381298a673cb1eda95b30"}
Jul 22 04:19:34.827671 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:34.827281    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-m4q6n/must-gather-vtqg5" event={"ID":"4ef9972c-aeed-496d-bca2-cb5de52d01a0","Type":"ContainerStarted","Data":"bd28a420bbe92bb5a6108416b1a3d93dcc2efc1ee2b3f66231ae91fddb41d61b"}
Jul 22 04:19:34.846740 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:34.846671    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-m4q6n/must-gather-vtqg5" podStartSLOduration=1.9142870840000001 podStartE2EDuration="2.846648197s" totalImagesPullingTime="932.361113ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:19:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:19:34.845094012 +0000 UTC m=+2762.463304019" watchObservedRunningTime="2026-07-22 04:19:34.846648197 +0000 UTC m=+2762.464858205"
Jul 22 04:19:35.922886 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:35.922854    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-p867k_16d564e4-f4b5-4293-b5c9-b7bd30239b47/global-pull-secret-syncer/0.log"
Jul 22 04:19:36.012975 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:36.012947    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-thjp8_22a4fd03-a5a7-4d6e-97fb-fb63e5be241c/konnectivity-agent/0.log"
Jul 22 04:19:36.073872 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:36.073839    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-89.ec2.internal_1decd6d269268ae8653188a0a80fb590/haproxy/0.log"
Jul 22 04:19:41.025715 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:41.025676    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-csccb_a05e14f1-2573-4358-b15b-09a094fa010c/manager/0.log"
Jul 22 04:19:41.052468 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:41.052442    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-2cb7r_c709ad99-a734-47a6-853d-22ec32915ab6/kuadrant-console-plugin/0.log"
Jul 22 04:19:41.326777 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:41.326706    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-2js4d_3707c428-6025-4ada-b4c4-684dc735790e/limitador/0.log"
Jul 22 04:19:43.402244 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:43.402163    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-jgckh_367c6e07-f8b3-494b-a12c-3c11e916e61b/cluster-monitoring-operator/0.log"
Jul 22 04:19:43.731800 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:43.731765    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mlkmd_8e81bffd-5ccd-45f5-b072-ed0b951261ec/node-exporter/0.log"
Jul 22 04:19:43.754960 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:43.754913    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mlkmd_8e81bffd-5ccd-45f5-b072-ed0b951261ec/kube-rbac-proxy/0.log"
Jul 22 04:19:43.775089 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:43.775058    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mlkmd_8e81bffd-5ccd-45f5-b072-ed0b951261ec/init-textfile/0.log"
Jul 22 04:19:43.879711 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:43.879672    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fbd30716-48e9-4f2c-9e6c-77df93cb9f23/prometheus/0.log"
Jul 22 04:19:43.900172 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:43.900144    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fbd30716-48e9-4f2c-9e6c-77df93cb9f23/config-reloader/0.log"
Jul 22 04:19:43.922514 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:43.922489    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fbd30716-48e9-4f2c-9e6c-77df93cb9f23/thanos-sidecar/0.log"
Jul 22 04:19:43.943237 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:43.943207    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fbd30716-48e9-4f2c-9e6c-77df93cb9f23/kube-rbac-proxy-web/0.log"
Jul 22 04:19:43.966403 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:43.966379    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fbd30716-48e9-4f2c-9e6c-77df93cb9f23/kube-rbac-proxy/0.log"
Jul 22 04:19:43.989425 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:43.989346    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fbd30716-48e9-4f2c-9e6c-77df93cb9f23/kube-rbac-proxy-thanos/0.log"
Jul 22 04:19:44.013638 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:44.013164    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fbd30716-48e9-4f2c-9e6c-77df93cb9f23/init-config-reloader/0.log"
Jul 22 04:19:44.061359 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:44.061320    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-pjlv2_c4a45ddb-ff3b-48ff-a3df-0966e2b5f37b/prometheus-operator/0.log"
Jul 22 04:19:44.081730 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:44.081697    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-pjlv2_c4a45ddb-ff3b-48ff-a3df-0966e2b5f37b/kube-rbac-proxy/0.log"
Jul 22 04:19:46.263151 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:46.263115    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2p4qp_9f0defcc-ccff-43be-8206-54c772885080/console-operator/1.log"
Jul 22 04:19:46.269800 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:46.269768    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-2p4qp_9f0defcc-ccff-43be-8206-54c772885080/console-operator/2.log"
Jul 22 04:19:46.498261 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:46.498221    2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-m4q6n/perf-node-gather-daemonset-6zb5c"]
Jul 22 04:19:46.506878 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:46.506846    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-6zb5c"
Jul 22 04:19:46.513110 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:46.513072    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-m4q6n/perf-node-gather-daemonset-6zb5c"]
Jul 22 04:19:46.683697 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:46.683654    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8458a243-57d2-4b17-90dd-054870a0e7d6-sys\") pod \"perf-node-gather-daemonset-6zb5c\" (UID: \"8458a243-57d2-4b17-90dd-054870a0e7d6\") " pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-6zb5c"
Jul 22 04:19:46.683898 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:46.683707    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8458a243-57d2-4b17-90dd-054870a0e7d6-proc\") pod \"perf-node-gather-daemonset-6zb5c\" (UID: \"8458a243-57d2-4b17-90dd-054870a0e7d6\") " pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-6zb5c"
Jul 22 04:19:46.683898 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:46.683765    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8458a243-57d2-4b17-90dd-054870a0e7d6-lib-modules\") pod \"perf-node-gather-daemonset-6zb5c\" (UID: \"8458a243-57d2-4b17-90dd-054870a0e7d6\") " pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-6zb5c"
Jul 22 04:19:46.683898 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:46.683822    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8458a243-57d2-4b17-90dd-054870a0e7d6-podres\") pod \"perf-node-gather-daemonset-6zb5c\" (UID: \"8458a243-57d2-4b17-90dd-054870a0e7d6\") " pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-6zb5c"
Jul 22 04:19:46.683898 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:46.683855    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsrps\" (UniqueName: \"kubernetes.io/projected/8458a243-57d2-4b17-90dd-054870a0e7d6-kube-api-access-hsrps\") pod \"perf-node-gather-daemonset-6zb5c\" (UID: \"8458a243-57d2-4b17-90dd-054870a0e7d6\") " pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-6zb5c"
Jul 22 04:19:46.784774 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:46.784682    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8458a243-57d2-4b17-90dd-054870a0e7d6-sys\") pod \"perf-node-gather-daemonset-6zb5c\" (UID: \"8458a243-57d2-4b17-90dd-054870a0e7d6\") " pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-6zb5c"
Jul 22 04:19:46.784774 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:46.784719    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8458a243-57d2-4b17-90dd-054870a0e7d6-proc\") pod \"perf-node-gather-daemonset-6zb5c\" (UID: \"8458a243-57d2-4b17-90dd-054870a0e7d6\") " pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-6zb5c"
Jul 22 04:19:46.784774 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:46.784764    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8458a243-57d2-4b17-90dd-054870a0e7d6-lib-modules\") pod \"perf-node-gather-daemonset-6zb5c\" (UID: \"8458a243-57d2-4b17-90dd-054870a0e7d6\") " pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-6zb5c"
Jul 22 04:19:46.785051 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:46.784817    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8458a243-57d2-4b17-90dd-054870a0e7d6-podres\") pod \"perf-node-gather-daemonset-6zb5c\" (UID: \"8458a243-57d2-4b17-90dd-054870a0e7d6\") " pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-6zb5c"
Jul 22 04:19:46.785051 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:46.784817    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8458a243-57d2-4b17-90dd-054870a0e7d6-sys\") pod \"perf-node-gather-daemonset-6zb5c\" (UID: \"8458a243-57d2-4b17-90dd-054870a0e7d6\") " pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-6zb5c"
Jul 22 04:19:46.785051 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:46.784896    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8458a243-57d2-4b17-90dd-054870a0e7d6-proc\") pod \"perf-node-gather-daemonset-6zb5c\" (UID: \"8458a243-57d2-4b17-90dd-054870a0e7d6\") " pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-6zb5c"
Jul 22 04:19:46.785051 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:46.784895    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsrps\" (UniqueName: \"kubernetes.io/projected/8458a243-57d2-4b17-90dd-054870a0e7d6-kube-api-access-hsrps\") pod \"perf-node-gather-daemonset-6zb5c\" (UID: \"8458a243-57d2-4b17-90dd-054870a0e7d6\") " pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-6zb5c"
Jul 22 04:19:46.785051 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:46.784959    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8458a243-57d2-4b17-90dd-054870a0e7d6-lib-modules\") pod \"perf-node-gather-daemonset-6zb5c\" (UID: \"8458a243-57d2-4b17-90dd-054870a0e7d6\") " pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-6zb5c"
Jul 22 04:19:46.785051 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:46.784930    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8458a243-57d2-4b17-90dd-054870a0e7d6-podres\") pod \"perf-node-gather-daemonset-6zb5c\" (UID: \"8458a243-57d2-4b17-90dd-054870a0e7d6\") " pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-6zb5c"
Jul 22 04:19:46.794037 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:46.794001    2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsrps\" (UniqueName: \"kubernetes.io/projected/8458a243-57d2-4b17-90dd-054870a0e7d6-kube-api-access-hsrps\") pod \"perf-node-gather-daemonset-6zb5c\" (UID: \"8458a243-57d2-4b17-90dd-054870a0e7d6\") " pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-6zb5c"
Jul 22 04:19:46.824507 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:46.824474    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-6zb5c"
Jul 22 04:19:46.885881 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:46.885855    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-48tzh_ada6ca87-7b07-427c-be34-5a8aa1852db5/download-server/0.log"
Jul 22 04:19:47.005228 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:47.005176    2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-m4q6n/perf-node-gather-daemonset-6zb5c"]
Jul 22 04:19:47.554019 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:47.553990    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-4hpsp_7c29ba98-5a7e-4603-8731-661eeb2b7bf5/volume-data-source-validator/0.log"
Jul 22 04:19:47.968314 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:47.968087    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-6zb5c" event={"ID":"8458a243-57d2-4b17-90dd-054870a0e7d6","Type":"ContainerStarted","Data":"3e238b01b64cb511c423f3ba2a25ac4344bb9fe4e12cd2f41b55fdf588e1edd0"}
Jul 22 04:19:47.968535 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:47.968382    2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-6zb5c" event={"ID":"8458a243-57d2-4b17-90dd-054870a0e7d6","Type":"ContainerStarted","Data":"d8479962d116dfef6b375f943cc5831fe52bd3fa2d47688e2b17dc63583408df"}
Jul 22 04:19:47.968535 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:47.968425    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-6zb5c"
Jul 22 04:19:47.986522 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:47.986453    2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-6zb5c" podStartSLOduration=1.9864306 podStartE2EDuration="1.9864306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 04:19:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:19:47.983407896 +0000 UTC m=+2775.601617925" watchObservedRunningTime="2026-07-22 04:19:47.9864306 +0000 UTC m=+2775.604640607"
Jul 22 04:19:48.569886 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:48.569852    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-g2cfz_6ced761e-e20b-4b10-b8b1-fe77cd501818/dns/0.log"
Jul 22 04:19:48.592217 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:48.592193    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-g2cfz_6ced761e-e20b-4b10-b8b1-fe77cd501818/kube-rbac-proxy/0.log"
Jul 22 04:19:48.748331 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:48.748304    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tpgsf_ed43ef9c-bf6c-4f2c-96b5-fc31d24e5fd1/dns-node-resolver/0.log"
Jul 22 04:19:49.334283 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:49.334257    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7srsk_56a14a96-971e-4dab-a1bb-8e3edb6b73b2/node-ca/0.log"
Jul 22 04:19:50.462725 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:50.462697    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bd5997d8b-cpbs7_3eb33ff7-d4d6-4141-b23e-69d3cafb29b2/kube-auth-proxy/0.log"
Jul 22 04:19:50.554702 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:50.554665    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-vd2dl_26c42593-c5c2-497c-b0ff-52938d7447df/payload-pre-processing/0.log"
Jul 22 04:19:50.607009 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:50.606978    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7455857b66-nlzjg_f23b727c-e88a-42b1-80e8-02384f07c8f6/router/0.log"
Jul 22 04:19:51.336633 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:51.336589    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fldh9_f3317954-0d7e-49df-ae2a-4a4b48382557/serve-healthcheck-canary/0.log"
Jul 22 04:19:51.842822 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:51.842790    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-ptsm9_ee83bd5e-e31f-4049-947f-8f609f93908f/insights-operator/0.log"
Jul 22 04:19:51.843307 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:51.843036    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-ptsm9_ee83bd5e-e31f-4049-947f-8f609f93908f/insights-operator/1.log"
Jul 22 04:19:51.998065 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:51.998039    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-j8z8l_b180b221-d3f1-4f70-a258-e680495a3c40/kube-rbac-proxy/0.log"
Jul 22 04:19:52.019878 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:52.019847    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-j8z8l_b180b221-d3f1-4f70-a258-e680495a3c40/exporter/0.log"
Jul 22 04:19:52.041701 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:52.041678    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-j8z8l_b180b221-d3f1-4f70-a258-e680495a3c40/extractor/0.log"
Jul 22 04:19:53.985379 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:53.985306    2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-6zb5c"
Jul 22 04:19:54.608307 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:54.608261    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5fd47b996b-fh5t6_e7801170-6a68-4c68-bfcb-f7ef0acc2fae/manager/0.log"
Jul 22 04:19:54.629304 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:54.629279    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5fd47b996b-fh5t6_e7801170-6a68-4c68-bfcb-f7ef0acc2fae/copy-manifests/0.log"
Jul 22 04:19:56.149216 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:56.149132    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-78585bcb5c-xqvkl_a0062912-f644-4a63-ba13-9d4f463ed3aa/manager/0.log"
Jul 22 04:19:56.915068 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:56.915045    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bd5997d8b-cpbs7_3eb33ff7-d4d6-4141-b23e-69d3cafb29b2/kube-auth-proxy/0.log"
Jul 22 04:19:56.991078 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:56.990995    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-vd2dl_26c42593-c5c2-497c-b0ff-52938d7447df/payload-pre-processing/0.log"
Jul 22 04:19:57.039676 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:57.039648    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7455857b66-nlzjg_f23b727c-e88a-42b1-80e8-02384f07c8f6/router/0.log"
Jul 22 04:19:58.445897 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:58.445870    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bd5997d8b-cpbs7_3eb33ff7-d4d6-4141-b23e-69d3cafb29b2/kube-auth-proxy/0.log"
Jul 22 04:19:58.519444 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:58.519365    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-vd2dl_26c42593-c5c2-497c-b0ff-52938d7447df/payload-pre-processing/0.log"
Jul 22 04:19:58.553662 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:19:58.553638    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7455857b66-nlzjg_f23b727c-e88a-42b1-80e8-02384f07c8f6/router/0.log"
Jul 22 04:20:01.709474 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:20:01.709439    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-lqg9w_8cdfda8a-dc86-4da1-b192-4d0ab5cd37f4/kube-storage-version-migrator-operator/1.log"
Jul 22 04:20:01.710470 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:20:01.710440    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-lqg9w_8cdfda8a-dc86-4da1-b192-4d0ab5cd37f4/kube-storage-version-migrator-operator/0.log"
Jul 22 04:20:02.739436 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:20:02.739408    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6wt45_26250740-d226-4262-bff6-caf04187cf8d/kube-multus-additional-cni-plugins/0.log"
Jul 22 04:20:02.762688 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:20:02.762660    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6wt45_26250740-d226-4262-bff6-caf04187cf8d/egress-router-binary-copy/0.log"
Jul 22 04:20:02.786679 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:20:02.786653    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6wt45_26250740-d226-4262-bff6-caf04187cf8d/cni-plugins/0.log"
Jul 22 04:20:02.809054 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:20:02.809028    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6wt45_26250740-d226-4262-bff6-caf04187cf8d/bond-cni-plugin/0.log"
Jul 22 04:20:02.831578 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:20:02.831551    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6wt45_26250740-d226-4262-bff6-caf04187cf8d/routeoverride-cni/0.log"
Jul 22 04:20:02.856336 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:20:02.856310    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6wt45_26250740-d226-4262-bff6-caf04187cf8d/whereabouts-cni-bincopy/0.log"
Jul 22 04:20:02.876544 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:20:02.876519    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6wt45_26250740-d226-4262-bff6-caf04187cf8d/whereabouts-cni/0.log"
Jul 22 04:20:03.248015 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:20:03.247980    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-xbhc9_00bf5310-aba0-4e3e-9ac8-5d68e7de24bf/kube-multus/0.log"
Jul 22 04:20:03.268188 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:20:03.268161    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-lctg9_6adcbd54-1388-4223-8d6d-9423e9ebe09b/network-metrics-daemon/0.log"
Jul 22 04:20:03.286830 ip-10-0-132-89 kubenswrapper[2570]: I0722 04:20:03.286807    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-lctg9_6adcbd54-1388-4223-8d6d-9423e9ebe09b/kube-rbac-proxy/0.log"