Jul 13 16:40:13.331596 ip-10-0-128-173 systemd[1]: Starting Kubernetes Kubelet...
Jul 13 16:40:13.743589 ip-10-0-128-173 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 13 16:40:13.743589 ip-10-0-128-173 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 13 16:40:13.743589 ip-10-0-128-173 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 13 16:40:13.743589 ip-10-0-128-173 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 13 16:40:13.743589 ip-10-0-128-173 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 13 16:40:13.744666 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.744564    2566 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 13 16:40:13.749979 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.749950    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 16:40:13.749979 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.749977    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 16:40:13.750053 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.749982    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 16:40:13.750053 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.749986    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 16:40:13.750053 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.749990    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 16:40:13.750053 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.749993    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 16:40:13.750053 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.749996    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 16:40:13.750053 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.749999    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 16:40:13.750053 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750002    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 16:40:13.750053 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750005    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 16:40:13.750053 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750008    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 16:40:13.750053 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750011    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 16:40:13.750053 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750014    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 16:40:13.750053 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750018    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 16:40:13.750053 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750021    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 16:40:13.750053 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750023    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 16:40:13.750053 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750026    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 16:40:13.750053 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750036    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 16:40:13.750053 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750039    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 16:40:13.750053 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750042    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 16:40:13.750053 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750045    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 16:40:13.750053 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750048    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 16:40:13.750584 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750051    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 16:40:13.750584 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750054    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 16:40:13.750584 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750059    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 16:40:13.750584 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750064    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 16:40:13.750584 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750068    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 16:40:13.750584 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750071    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 16:40:13.750584 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750074    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 16:40:13.750584 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750078    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 16:40:13.750584 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750082    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 16:40:13.750584 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750085    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 16:40:13.750584 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750088    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 16:40:13.750584 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750092    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 16:40:13.750584 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750095    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 16:40:13.750584 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750098    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 16:40:13.750584 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750101    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 16:40:13.750584 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750103    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 16:40:13.750584 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750106    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 16:40:13.750584 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750108    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 16:40:13.750584 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750111    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 16:40:13.751090 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750114    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 16:40:13.751090 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750117    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 16:40:13.751090 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750119    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 16:40:13.751090 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750122    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 16:40:13.751090 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750125    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 16:40:13.751090 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750128    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 16:40:13.751090 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750131    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 16:40:13.751090 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750134    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 16:40:13.751090 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750137    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 16:40:13.751090 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750139    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 16:40:13.751090 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750142    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 16:40:13.751090 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750144    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 16:40:13.751090 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750149    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 16:40:13.751090 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750152    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 16:40:13.751090 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750156    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 16:40:13.751090 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750160    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 16:40:13.751090 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750162    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 16:40:13.751090 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750165    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 16:40:13.751090 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750168    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 16:40:13.751090 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750171    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 16:40:13.751614 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750174    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 16:40:13.751614 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750177    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 16:40:13.751614 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750179    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 16:40:13.751614 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750182    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 16:40:13.751614 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750185    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 16:40:13.751614 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750188    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 16:40:13.751614 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750191    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 16:40:13.751614 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750194    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 16:40:13.751614 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750197    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 16:40:13.751614 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750200    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 16:40:13.751614 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750203    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 16:40:13.751614 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750206    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 16:40:13.751614 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750209    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 16:40:13.751614 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750212    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 16:40:13.751614 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750215    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 16:40:13.751614 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750217    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 16:40:13.751614 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750220    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 16:40:13.751614 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750223    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 16:40:13.751614 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750226    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 16:40:13.751614 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750228    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 16:40:13.752109 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750231    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 16:40:13.752109 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750234    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 16:40:13.752109 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750236    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 16:40:13.752109 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750240    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 16:40:13.752109 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750243    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 16:40:13.752109 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750246    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 16:40:13.752109 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750250    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 16:40:13.752109 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750253    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 16:40:13.752109 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750257    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 16:40:13.752109 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750260    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 16:40:13.752109 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750263    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 16:40:13.752109 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750265    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 16:40:13.752109 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750268    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 16:40:13.752109 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750271    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 16:40:13.752109 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750273    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 16:40:13.752109 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750276    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 16:40:13.752109 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750279    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 16:40:13.752109 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.750282    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 16:40:13.752109 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751071    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 16:40:13.752109 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751078    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 16:40:13.752661 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751082    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 16:40:13.752661 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751085    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 16:40:13.752661 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751088    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 16:40:13.752661 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751092    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 16:40:13.752661 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751096    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 16:40:13.752661 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751098    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 16:40:13.752661 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751101    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 16:40:13.752661 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751105    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 16:40:13.752661 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751109    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 16:40:13.752661 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751112    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 16:40:13.752661 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751115    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 16:40:13.752661 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751118    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 16:40:13.752661 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751121    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 16:40:13.752661 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751126    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 16:40:13.752661 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751130    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 16:40:13.752661 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751133    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 16:40:13.752661 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751136    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 16:40:13.752661 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751139    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 16:40:13.752661 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751142    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 16:40:13.753197 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751145    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 16:40:13.753197 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751149    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 16:40:13.753197 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751154    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 16:40:13.753197 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751157    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 16:40:13.753197 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751160    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 16:40:13.753197 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751163    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 16:40:13.753197 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751167    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 16:40:13.753197 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751170    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 16:40:13.753197 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751173    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 16:40:13.753197 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751176    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 16:40:13.753197 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751179    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 16:40:13.753197 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751182    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 16:40:13.753197 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751185    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 16:40:13.753197 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751188    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 16:40:13.753197 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751191    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 16:40:13.753197 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751194    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 16:40:13.753197 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751196    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 16:40:13.753197 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751200    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 16:40:13.753197 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751203    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 16:40:13.753709 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751205    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 16:40:13.753709 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751208    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 16:40:13.753709 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751211    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 16:40:13.753709 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751214    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 16:40:13.753709 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751218    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 16:40:13.753709 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751220    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 16:40:13.753709 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751223    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 16:40:13.753709 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751226    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 16:40:13.753709 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751229    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 16:40:13.753709 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751231    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 16:40:13.753709 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751234    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 16:40:13.753709 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751237    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 16:40:13.753709 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751240    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 16:40:13.753709 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751243    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 16:40:13.753709 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751245    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 16:40:13.753709 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751248    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 16:40:13.753709 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751251    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 16:40:13.753709 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751253    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 16:40:13.753709 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751256    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 16:40:13.754212 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751259    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 16:40:13.754212 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751262    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 16:40:13.754212 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751264    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 16:40:13.754212 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751267    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 16:40:13.754212 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751270    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 16:40:13.754212 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751273    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 16:40:13.754212 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751275    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 16:40:13.754212 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751278    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 16:40:13.754212 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751281    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 16:40:13.754212 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751283    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 16:40:13.754212 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751286    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 16:40:13.754212 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751289    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 16:40:13.754212 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751292    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 16:40:13.754212 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751295    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 16:40:13.754212 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751298    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 16:40:13.754212 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751301    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 16:40:13.754212 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751304    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 16:40:13.754212 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751306    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 16:40:13.754212 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751309    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 16:40:13.754212 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751312    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 16:40:13.754762 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751315    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 16:40:13.754762 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751317    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 16:40:13.754762 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751335    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 16:40:13.754762 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751338    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 16:40:13.754762 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751342    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 16:40:13.754762 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751345    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 16:40:13.754762 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751348    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 16:40:13.754762 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751351    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 16:40:13.754762 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751353    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 16:40:13.754762 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751356    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 16:40:13.754762 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751359    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 16:40:13.754762 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751362    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 16:40:13.754762 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751365    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 16:40:13.754762 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751370    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 16:40:13.754762 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751374    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 16:40:13.754762 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751377    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 16:40:13.754762 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751380    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 16:40:13.754762 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751384    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 16:40:13.754762 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751387    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 16:40:13.754762 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751390    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 16:40:13.755279 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751796    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 16:40:13.755279 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751801    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 16:40:13.755279 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751804    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 16:40:13.755279 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751808    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 16:40:13.755279 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751811    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 16:40:13.755279 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751814    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 16:40:13.755279 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751817    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 16:40:13.755279 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751820    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 16:40:13.755279 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751823    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 16:40:13.755279 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751826    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 16:40:13.755279 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751829    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 16:40:13.755279 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751831    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 16:40:13.755279 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751834    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 16:40:13.755279 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751839    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 16:40:13.755279 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751843    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 16:40:13.755279 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751846    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 16:40:13.755279 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751849    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 16:40:13.755279 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751852    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 16:40:13.755279 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751855    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 16:40:13.755770 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751858    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 16:40:13.755770 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751861    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 16:40:13.755770 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751864    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 16:40:13.755770 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751867    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 16:40:13.755770 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751870    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 16:40:13.755770 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751873    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 16:40:13.755770 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751876    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 16:40:13.755770 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751879    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 16:40:13.755770 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751881    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 16:40:13.755770 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751884    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 16:40:13.755770 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751887    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 16:40:13.755770 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751890    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 16:40:13.755770 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751893    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 16:40:13.755770 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751896    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 16:40:13.755770 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751899    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 16:40:13.755770 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751901    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 16:40:13.755770 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751904    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 16:40:13.755770 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751907    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 16:40:13.755770 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751917    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 16:40:13.755770 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751920    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 16:40:13.756271 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751922    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 16:40:13.756271 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751925    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 16:40:13.756271 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751928    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 16:40:13.756271 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751930    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 16:40:13.756271 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751933    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 16:40:13.756271 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751937    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 16:40:13.756271 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751940    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 16:40:13.756271 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751943    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 16:40:13.756271 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751946    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 16:40:13.756271 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751949    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 16:40:13.756271 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751952    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 16:40:13.756271 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751955    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 16:40:13.756271 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751958    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 16:40:13.756271 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751961    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 16:40:13.756271 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751963    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 16:40:13.756271 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751968    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 16:40:13.756271 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751971    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 16:40:13.756271 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751974    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 16:40:13.756271 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751977    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 16:40:13.756766 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751980    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 16:40:13.756766 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751983    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 16:40:13.756766 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751986    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 16:40:13.756766 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751989    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 16:40:13.756766 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751992    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 16:40:13.756766 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751995    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 16:40:13.756766 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.751998    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 16:40:13.756766 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752000    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 16:40:13.756766 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752003    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 16:40:13.756766 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752006    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 16:40:13.756766 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752009    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 16:40:13.756766 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752011    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 16:40:13.756766 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752015    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 16:40:13.756766 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752018    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 16:40:13.756766 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752021    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 16:40:13.756766 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752023    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 16:40:13.756766 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752027    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 16:40:13.756766 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752030    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 16:40:13.756766 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752032    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 16:40:13.756766 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752038    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 16:40:13.757281 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752041    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 16:40:13.757281 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752044    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 16:40:13.757281 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752047    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 16:40:13.757281 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752050    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 16:40:13.757281 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752053    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 16:40:13.757281 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752056    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 16:40:13.757281 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752059    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 16:40:13.757281 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752061    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 16:40:13.757281 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752064    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 16:40:13.757281 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752067    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 16:40:13.757281 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752070    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 16:40:13.757281 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752072    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 16:40:13.757281 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752075    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 16:40:13.757281 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752078    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 16:40:13.757281 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752080    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 16:40:13.757281 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752083    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 16:40:13.757281 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752086    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 16:40:13.757281 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752088    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 16:40:13.757281 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752091    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 16:40:13.757797 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752094    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 16:40:13.757797 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752096    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 16:40:13.757797 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752178    2566 flags.go:64] FLAG: --address="0.0.0.0"
Jul 13 16:40:13.757797 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752186    2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 13 16:40:13.757797 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752195    2566 flags.go:64] FLAG: --anonymous-auth="true"
Jul 13 16:40:13.757797 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752202    2566 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 13 16:40:13.757797 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752210    2566 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 13 16:40:13.757797 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752215    2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 13 16:40:13.757797 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752222    2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 13 16:40:13.757797 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752227    2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 13 16:40:13.757797 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752230    2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 13 16:40:13.757797 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752234    2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 13 16:40:13.757797 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752239    2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 13 16:40:13.757797 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752243    2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 13 16:40:13.757797 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752246    2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 13 16:40:13.757797 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752250    2566 flags.go:64] FLAG: --cgroup-root=""
Jul 13 16:40:13.757797 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752254    2566 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 13 16:40:13.757797 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752258    2566 flags.go:64] FLAG: --client-ca-file=""
Jul 13 16:40:13.757797 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752262    2566 flags.go:64] FLAG: --cloud-provider="external"
Jul 13 16:40:13.757797 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752265    2566 flags.go:64] FLAG: --cluster-dns="[]"
Jul 13 16:40:13.757797 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752270    2566 flags.go:64] FLAG: --cluster-domain=""
Jul 13 16:40:13.757797 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752273    2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 13 16:40:13.757797 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752276    2566 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 13 16:40:13.757797 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752279    2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 13 16:40:13.758439 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752283    2566 flags.go:64] FLAG: --container-log-max-files="5"
Jul 13 16:40:13.758439 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752287    2566 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 13 16:40:13.758439 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752291    2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 13 16:40:13.758439 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752294    2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 13 16:40:13.758439 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752298    2566 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 13 16:40:13.758439 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752301    2566 flags.go:64] FLAG: --contention-profiling="false"
Jul 13 16:40:13.758439 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752304    2566 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 13 16:40:13.758439 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752307    2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 13 16:40:13.758439 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752311    2566 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 13 16:40:13.758439 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752314    2566 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 13 16:40:13.758439 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752331    2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 13 16:40:13.758439 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752335    2566 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 13 16:40:13.758439 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752339    2566 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 13 16:40:13.758439 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752342    2566 flags.go:64] FLAG: --enable-load-reader="false"
Jul 13 16:40:13.758439 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752345    2566 flags.go:64] FLAG: --enable-server="true"
Jul 13 16:40:13.758439 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752348    2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 13 16:40:13.758439 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752354    2566 flags.go:64] FLAG: --event-burst="100"
Jul 13 16:40:13.758439 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752357    2566 flags.go:64] FLAG: --event-qps="50"
Jul 13 16:40:13.758439 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752360    2566 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 13 16:40:13.758439 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752363    2566 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 13 16:40:13.758439 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752368    2566 flags.go:64] FLAG: --eviction-hard=""
Jul 13 16:40:13.758439 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752373    2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 13 16:40:13.758439 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752376    2566 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 13 16:40:13.758439 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752380    2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 13 16:40:13.758439 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752383    2566 flags.go:64] FLAG: --eviction-soft=""
Jul 13 16:40:13.759117 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752386    2566 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 13 16:40:13.759117 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752389    2566 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 13 16:40:13.759117 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752392    2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 13 16:40:13.759117 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752395    2566 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 13 16:40:13.759117 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752398    2566 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 13 16:40:13.759117 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752401    2566 flags.go:64] FLAG: --fail-swap-on="true"
Jul 13 16:40:13.759117 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752405    2566 flags.go:64] FLAG: --feature-gates=""
Jul 13 16:40:13.759117 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752408    2566 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 13 16:40:13.759117 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752412    2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 13 16:40:13.759117 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752415    2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 13 16:40:13.759117 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752418    2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 13 16:40:13.759117 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752422    2566 flags.go:64] FLAG: --healthz-port="10248"
Jul 13 16:40:13.759117 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752425    2566 flags.go:64] FLAG: --help="false"
Jul 13 16:40:13.759117 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752428    2566 flags.go:64] FLAG: --hostname-override="ip-10-0-128-173.ec2.internal"
Jul 13 16:40:13.759117 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752431    2566 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 13 16:40:13.759117 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752435    2566 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 13 16:40:13.759117 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752438    2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 13 16:40:13.759117 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752441    2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 13 16:40:13.759117 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752445    2566 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 13 16:40:13.759117 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752448    2566 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 13 16:40:13.759117 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752451    2566 flags.go:64] FLAG: --image-service-endpoint=""
Jul 13 16:40:13.759117 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752454    2566 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 13 16:40:13.759117 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752457    2566 flags.go:64] FLAG: --kube-api-burst="100"
Jul 13 16:40:13.759768 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752461    2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 13 16:40:13.759768 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752465    2566 flags.go:64] FLAG: --kube-api-qps="50"
Jul 13 16:40:13.759768 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752468    2566 flags.go:64] FLAG: --kube-reserved=""
Jul 13 16:40:13.759768 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752471    2566 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 13 16:40:13.759768 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752475    2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 13 16:40:13.759768 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752478    2566 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 13 16:40:13.759768 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752482    2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 13 16:40:13.759768 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752485    2566 flags.go:64] FLAG: --lock-file=""
Jul 13 16:40:13.759768 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752487    2566 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 13 16:40:13.759768 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752490    2566 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 13 16:40:13.759768 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752494    2566 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 13 16:40:13.759768 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752500    2566 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 13 16:40:13.759768 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752503    2566 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 13 16:40:13.759768 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752506    2566 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 13 16:40:13.759768 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752509    2566 flags.go:64] FLAG: --logging-format="text"
Jul 13 16:40:13.759768 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752512    2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 13 16:40:13.759768 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752516    2566 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 13 16:40:13.759768 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752519    2566 flags.go:64] FLAG: --manifest-url=""
Jul 13 16:40:13.759768 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752522    2566 flags.go:64] FLAG: --manifest-url-header=""
Jul 13 16:40:13.759768 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752527    2566 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 13 16:40:13.759768 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752530    2566 flags.go:64] FLAG: --max-open-files="1000000"
Jul 13 16:40:13.759768 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752535    2566 flags.go:64] FLAG: --max-pods="110"
Jul 13 16:40:13.759768 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752538    2566 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 13 16:40:13.759768 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752541    2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 13 16:40:13.759768 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752544    2566 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 13 16:40:13.760452 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752547    2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 13 16:40:13.760452 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752551    2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 13 16:40:13.760452 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752554    2566 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 13 16:40:13.760452 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752557    2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 13 16:40:13.760452 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752566    2566 flags.go:64] FLAG: --node-status-max-images="50"
Jul 13 16:40:13.760452 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752569    2566 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 13 16:40:13.760452 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752573    2566 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 13 16:40:13.760452 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752576    2566 flags.go:64] FLAG: --pod-cidr=""
Jul 13 16:40:13.760452 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752580    2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 13 16:40:13.760452 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752586    2566 flags.go:64] FLAG: --pod-manifest-path=""
Jul 13 16:40:13.760452 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752589    2566 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 13 16:40:13.760452 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752595    2566 flags.go:64] FLAG: --pods-per-core="0"
Jul 13 16:40:13.760452 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752598    2566 flags.go:64] FLAG: --port="10250"
Jul 13 16:40:13.760452 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752601    2566 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 13 16:40:13.760452 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752604    2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0bfe3f17c3baea133"
Jul 13 16:40:13.760452 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752608    2566 flags.go:64] FLAG: --qos-reserved=""
Jul 13 16:40:13.760452 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752611    2566 flags.go:64] FLAG: --read-only-port="10255"
Jul 13 16:40:13.760452 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752614    2566 flags.go:64] FLAG: --register-node="true"
Jul 13 16:40:13.760452 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752618    2566 flags.go:64] FLAG: --register-with-taints=""
Jul 13 16:40:13.760452 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752621    2566 flags.go:64] FLAG: --registry-burst="10"
Jul 13 16:40:13.760452 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752624    2566 flags.go:64] FLAG: --registry-qps="5"
Jul 13 16:40:13.760452 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752627    2566 flags.go:64] FLAG: --reserved-cpus=""
Jul 13 16:40:13.760452 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752630    2566 flags.go:64] FLAG: --reserved-memory=""
Jul 13 16:40:13.760452 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752634    2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 13 16:40:13.760452 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752637    2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 13 16:40:13.761120 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752641    2566 flags.go:64] FLAG: --rotate-certificates="false"
Jul 13 16:40:13.761120 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752644    2566 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 13 16:40:13.761120 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752647    2566 flags.go:64] FLAG: --runonce="false"
Jul 13 16:40:13.761120 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752650    2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 13 16:40:13.761120 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752654    2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 13 16:40:13.761120 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752658    2566 flags.go:64] FLAG: --seccomp-default="false"
Jul 13 16:40:13.761120 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752661    2566 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 13 16:40:13.761120 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752664    2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 13 16:40:13.761120 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752667    2566 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 13 16:40:13.761120 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752671    2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 13 16:40:13.761120 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752674    2566 flags.go:64] FLAG: --storage-driver-password="root"
Jul 13 16:40:13.761120 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752678    2566 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 13 16:40:13.761120 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752681    2566 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 13 16:40:13.761120 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752684    2566 flags.go:64] FLAG: --storage-driver-user="root"
Jul 13 16:40:13.761120 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752687    2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 13 16:40:13.761120 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752691    2566 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 13 16:40:13.761120 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752694    2566 flags.go:64] FLAG: --system-cgroups=""
Jul 13 16:40:13.761120 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752698    2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 13 16:40:13.761120 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752705    2566 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 13 16:40:13.761120 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752708    2566 flags.go:64] FLAG: --tls-cert-file=""
Jul 13 16:40:13.761120 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752711    2566 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 13 16:40:13.761120 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752716    2566 flags.go:64] FLAG: --tls-min-version=""
Jul 13 16:40:13.761120 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752719    2566 flags.go:64] FLAG: --tls-private-key-file=""
Jul 13 16:40:13.761120 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752723    2566 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 13 16:40:13.761120 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752726    2566 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 13 16:40:13.761793 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752729    2566 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 13 16:40:13.761793 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752733    2566 flags.go:64] FLAG: --v="2"
Jul 13 16:40:13.761793 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752738    2566 flags.go:64] FLAG: --version="false"
Jul 13 16:40:13.761793 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752742    2566 flags.go:64] FLAG: --vmodule=""
Jul 13 16:40:13.761793 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752746    2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 13 16:40:13.761793 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.752750    2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 13 16:40:13.761793 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752897    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 16:40:13.761793 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752902    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 16:40:13.761793 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752905    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 16:40:13.761793 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752909    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 16:40:13.761793 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752914    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 16:40:13.761793 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752921    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 16:40:13.761793 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752925    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 16:40:13.761793 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752928    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 16:40:13.761793 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752931    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 16:40:13.761793 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752934    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 16:40:13.761793 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752937    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 16:40:13.761793 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752940    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 16:40:13.761793 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752943    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 16:40:13.761793 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752946    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 16:40:13.761793 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752949    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 16:40:13.762552 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752951    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 16:40:13.762552 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752954    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 16:40:13.762552 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752957    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 16:40:13.762552 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752961    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 16:40:13.762552 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752965    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 16:40:13.762552 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752969    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 16:40:13.762552 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752972    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 16:40:13.762552 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752974    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 16:40:13.762552 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752977    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 16:40:13.762552 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752980    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 16:40:13.762552 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752982    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 16:40:13.762552 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752985    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 16:40:13.762552 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752988    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 16:40:13.762552 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752991    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 16:40:13.762552 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752993    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 16:40:13.762552 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752996    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 16:40:13.762552 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.752999    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 16:40:13.762552 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753001    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 16:40:13.762552 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753004    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 16:40:13.762552 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753007    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 16:40:13.763123 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753010    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 16:40:13.763123 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753013    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 16:40:13.763123 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753018    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 16:40:13.763123 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753022    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 16:40:13.763123 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753026    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 16:40:13.763123 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753029    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 16:40:13.763123 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753032    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 16:40:13.763123 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753035    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 16:40:13.763123 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753037    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 16:40:13.763123 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753040    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 16:40:13.763123 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753043    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 16:40:13.763123 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753046    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 16:40:13.763123 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753048    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 16:40:13.763123 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753051    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 16:40:13.763123 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753054    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 16:40:13.763123 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753057    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 16:40:13.763123 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753062    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 16:40:13.763123 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753065    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 16:40:13.763123 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753068    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 16:40:13.763123 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753070    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 16:40:13.763679 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753073    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 16:40:13.763679 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753076    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 16:40:13.763679 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753079    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 16:40:13.763679 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753082    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 16:40:13.763679 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753085    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 16:40:13.763679 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753088    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 16:40:13.763679 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753090    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 16:40:13.763679 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753093    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 16:40:13.763679 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753096    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 16:40:13.763679 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753098    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 16:40:13.763679 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753101    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 16:40:13.763679 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753105    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 16:40:13.763679 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753108    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 16:40:13.763679 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753112    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 16:40:13.763679 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753116    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 16:40:13.763679 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753119    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 16:40:13.763679 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753122    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 16:40:13.763679 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753124    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 16:40:13.763679 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753127    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 16:40:13.764164 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753130    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 16:40:13.764164 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753133    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 16:40:13.764164 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753135    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 16:40:13.764164 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753138    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 16:40:13.764164 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753141    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 16:40:13.764164 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753144    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 16:40:13.764164 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753146    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 16:40:13.764164 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753149    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 16:40:13.764164 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753152    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 16:40:13.764164 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753156    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 16:40:13.764164 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753159    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 16:40:13.764164 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753162    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 16:40:13.764164 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753164    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 16:40:13.764164 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753167    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 16:40:13.764164 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753170    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 16:40:13.764164 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753173    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 16:40:13.764164 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753175    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 16:40:13.764164 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753178    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 16:40:13.764164 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753181    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 16:40:13.764164 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753183    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 16:40:13.764694 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753186    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 16:40:13.764694 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753189    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 16:40:13.764694 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753192    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 16:40:13.764694 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753194    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 16:40:13.764694 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.753197    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 16:40:13.764694 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.754717    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 13 16:40:13.764694 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.761838    2566 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 13 16:40:13.764694 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.761856    2566 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 13 16:40:13.764694 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.762835    2566 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 13 16:40:13.764694 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.762843    2566 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 13 16:40:13.764694 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.762914    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 16:40:13.764694 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.762921    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 16:40:13.764694 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.762925    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 16:40:13.764694 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.762928    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 16:40:13.765065 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.762932    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 16:40:13.765065 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.762935    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 16:40:13.765065 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.762938    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 16:40:13.765065 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.762941    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 16:40:13.765065 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.762944    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 16:40:13.765065 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.762947    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 16:40:13.765065 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.762950    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 16:40:13.765065 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.762953    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 16:40:13.765065 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.762957    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 16:40:13.765065 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.762968    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 16:40:13.765065 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.762977    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 16:40:13.765065 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.762980    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 16:40:13.765065 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.762983    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 16:40:13.765065 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.762986    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 16:40:13.765065 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.762989    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 16:40:13.765065 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.762993    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 16:40:13.765065 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.762995    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 16:40:13.765065 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.762998    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 16:40:13.765065 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763001    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 16:40:13.765065 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763004    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 16:40:13.765585 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763009    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 16:40:13.765585 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763013    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 16:40:13.765585 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763016    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 16:40:13.765585 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763020    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 16:40:13.765585 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763023    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 16:40:13.765585 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763027    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 16:40:13.765585 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763030    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 16:40:13.765585 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763033    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 16:40:13.765585 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763036    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 16:40:13.765585 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763038    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 16:40:13.765585 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763041    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 16:40:13.765585 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763044    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 16:40:13.765585 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763047    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 16:40:13.765585 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763049    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 16:40:13.765585 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763052    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 16:40:13.765585 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763055    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 16:40:13.765585 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763058    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 16:40:13.765585 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763061    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 16:40:13.765585 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763064    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 16:40:13.765585 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763066    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 16:40:13.766116 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763069    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 16:40:13.766116 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763073    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 16:40:13.766116 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763076    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 16:40:13.766116 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763079    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 16:40:13.766116 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763081    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 16:40:13.766116 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763084    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 16:40:13.766116 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763086    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 16:40:13.766116 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763089    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 16:40:13.766116 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763092    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 16:40:13.766116 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763095    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 16:40:13.766116 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763098    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 16:40:13.766116 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763100    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 16:40:13.766116 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763103    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 16:40:13.766116 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763106    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 16:40:13.766116 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763109    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 16:40:13.766116 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763112    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 16:40:13.766116 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763115    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 16:40:13.766116 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763118    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 16:40:13.766116 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763121    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 16:40:13.766694 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763124    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 16:40:13.766694 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763127    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 16:40:13.766694 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763130    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 16:40:13.766694 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763133    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 16:40:13.766694 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763136    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 16:40:13.766694 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763138    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 16:40:13.766694 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763141    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 16:40:13.766694 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763144    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 16:40:13.766694 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763147    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 16:40:13.766694 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763151    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 16:40:13.766694 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763154    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 16:40:13.766694 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763158    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 16:40:13.766694 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763161    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 16:40:13.766694 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763164    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 16:40:13.766694 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763167    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 16:40:13.766694 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763170    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 16:40:13.766694 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763173    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 16:40:13.766694 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763175    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 16:40:13.766694 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763178    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 16:40:13.766694 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763181    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 16:40:13.767202 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763185    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 16:40:13.767202 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763189    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 16:40:13.767202 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763192    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 16:40:13.767202 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763195    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 16:40:13.767202 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763197    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 16:40:13.767202 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763200    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 16:40:13.767202 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763203    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 16:40:13.767202 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763205    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 16:40:13.767202 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763208    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 16:40:13.767202 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763211    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 16:40:13.767202 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763214    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 16:40:13.767202 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763216    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 16:40:13.767202 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763219    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 16:40:13.767202 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763222    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 16:40:13.767202 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763224    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 16:40:13.767202 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763227    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 16:40:13.767662 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.763233    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 13 16:40:13.767662 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763370    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 16:40:13.767662 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763376    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 16:40:13.767662 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763380    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 16:40:13.767662 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763383    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 16:40:13.767662 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763386    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 16:40:13.767662 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763389    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 16:40:13.767662 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763392    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 16:40:13.767662 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763395    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 16:40:13.767662 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763398    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 16:40:13.767662 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763401    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 16:40:13.767662 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763404    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 16:40:13.767662 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763407    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 16:40:13.767662 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763411    2566 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 16:40:13.767662 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763413    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 16:40:13.768058 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763417    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 16:40:13.768058 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763420    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 16:40:13.768058 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763423    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 16:40:13.768058 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763426    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 16:40:13.768058 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763429    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 16:40:13.768058 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763431    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 16:40:13.768058 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763434    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 16:40:13.768058 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763437    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 16:40:13.768058 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763440    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 16:40:13.768058 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763442    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 16:40:13.768058 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763445    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 16:40:13.768058 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763448    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 16:40:13.768058 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763451    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 16:40:13.768058 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763454    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 16:40:13.768058 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763456    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 16:40:13.768058 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763459    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 16:40:13.768058 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763462    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 16:40:13.768058 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763464    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 16:40:13.768058 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763467    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 16:40:13.768058 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763470    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 16:40:13.768600 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763472    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 16:40:13.768600 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763475    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 16:40:13.768600 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763478    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 16:40:13.768600 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763481    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 16:40:13.768600 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763484    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 16:40:13.768600 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763487    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 16:40:13.768600 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763489    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 16:40:13.768600 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763492    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 16:40:13.768600 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763495    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 16:40:13.768600 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763498    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 16:40:13.768600 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763501    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 16:40:13.768600 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763503    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 16:40:13.768600 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763507    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 16:40:13.768600 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763509    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 16:40:13.768600 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763512    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 16:40:13.768600 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763515    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 16:40:13.768600 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763517    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 16:40:13.768600 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763520    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 16:40:13.768600 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763523    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 16:40:13.768600 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763527    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 16:40:13.769111 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763531    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 16:40:13.769111 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763534    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 16:40:13.769111 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763537    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 16:40:13.769111 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763540    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 16:40:13.769111 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763543    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 16:40:13.769111 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763546    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 16:40:13.769111 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763549    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 16:40:13.769111 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763553    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 16:40:13.769111 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763557    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 16:40:13.769111 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763560    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 16:40:13.769111 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763564    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 16:40:13.769111 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763567    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 16:40:13.769111 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763570    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 16:40:13.769111 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763573    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 16:40:13.769111 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763576    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 16:40:13.769111 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763579    2566 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 16:40:13.769111 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763582    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 16:40:13.769111 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763585    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 16:40:13.769111 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763588    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 16:40:13.769616 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763591    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 16:40:13.769616 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763594    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 16:40:13.769616 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763597    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 16:40:13.769616 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763600    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 16:40:13.769616 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763604    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 16:40:13.769616 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763607    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 16:40:13.769616 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763610    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 16:40:13.769616 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763613    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 16:40:13.769616 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763616    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 16:40:13.769616 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763619    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 16:40:13.769616 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763621    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 16:40:13.769616 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763624    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 16:40:13.769616 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763628    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 16:40:13.769616 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763631    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 16:40:13.769616 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763634    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 16:40:13.769616 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763637    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 16:40:13.769616 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763640    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 16:40:13.769616 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763644    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 16:40:13.769616 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763648    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 16:40:13.770410 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763651    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 16:40:13.770410 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763654    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 16:40:13.770410 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763657    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 16:40:13.770410 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763660    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 16:40:13.770410 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763663    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 16:40:13.770410 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763665    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 16:40:13.770410 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:13.763668    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 16:40:13.770410 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.763673    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 13 16:40:13.770410 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.763799    2566 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 13 16:40:13.770410 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.765821    2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 13 16:40:13.770410 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.767235    2566 server.go:1019] "Starting client certificate rotation"
Jul 13 16:40:13.770410 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.767357    2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 13 16:40:13.770410 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.767393    2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 13 16:40:13.771500 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.771480    2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 13 16:40:13.773989 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.773968    2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 13 16:40:13.781397 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.781377    2566 log.go:25] "Validated CRI v1 runtime API"
Jul 13 16:40:13.783579 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.783563    2566 log.go:25] "Validated CRI v1 image API"
Jul 13 16:40:13.786119 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.786102    2566 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 13 16:40:13.790349 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.790326    2566 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 7e683940-1a37-4eae-b776-d5a7da27c979:/dev/nvme0n1p4 8c27ce30-b19f-42e9-97b2-a7aff2676a86:/dev/nvme0n1p3]
Jul 13 16:40:13.790413 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.790349    2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 13 16:40:13.795791 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.795667    2566 manager.go:216] Machine: {Timestamp:2026-07-13 16:40:13.794283931 +0000 UTC m=+0.335664355 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3088055 MemoryCapacity:33164505088 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d73a8c2b5dc3b1c90d0c2af8dcacb SystemUUID:ec2d73a8-c2b5-dc3b-1c90-d0c2af8dcacb BootID:1fad3cc5-0681-43b7-9d8d-c0c384df2ed3 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582254592 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e8:d6:27:1a:cd Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e8:d6:27:1a:cd Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d6:63:7e:c2:df:f1 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164505088 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 13 16:40:13.795791 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.795787    2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 13 16:40:13.795927 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.795873    2566 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 13 16:40:13.798878 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.798859    2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 13 16:40:13.799388 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.799360    2566 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 13 16:40:13.799534 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.799390    2566 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-173.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 13 16:40:13.800176 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.800166    2566 topology_manager.go:138] "Creating topology manager with none policy"
Jul 13 16:40:13.800211 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.800179    2566 container_manager_linux.go:309] "Creating device plugin manager"
Jul 13 16:40:13.800211 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.800187    2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 13 16:40:13.800906 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.800896    2566 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 13 16:40:13.802278 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.802265    2566 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 13 16:40:13.805342 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.805317    2566 state_mem.go:36] "Initialized new in-memory state store"
Jul 13 16:40:13.805467 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.805457    2566 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 13 16:40:13.805510 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.805496    2566 kubelet.go:486] "Attempting to sync node with API server"
Jul 13 16:40:13.805557 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.805515    2566 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 13 16:40:13.805557 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.805531    2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 13 16:40:13.805557 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.805541    2566 kubelet.go:398] "Adding apiserver pod source"
Jul 13 16:40:13.805557 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.805551    2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 13 16:40:13.805963 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.805950    2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 13 16:40:13.806021 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.805971    2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 13 16:40:13.809185 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.809133    2566 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 13 16:40:13.811552 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.811533    2566 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 13 16:40:13.811636 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.811567    2566 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 13 16:40:13.811739 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.811728    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 13 16:40:13.811780 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.811744    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 13 16:40:13.811780 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.811751    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 13 16:40:13.811780 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.811756    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 13 16:40:13.811780 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.811765    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 13 16:40:13.811780 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.811771    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 13 16:40:13.812542 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.812533    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 13 16:40:13.812542 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.812542    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 13 16:40:13.812601 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.812550    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 13 16:40:13.812601 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.812557    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 13 16:40:13.812601 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.812566    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 13 16:40:13.812601 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.812575    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 13 16:40:13.813479 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.813467    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 13 16:40:13.813479 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.813480    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 13 16:40:13.816989 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.816972    2566 server.go:1268] "Started kubelet"
Jul 13 16:40:13.817167 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.817095    2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 13 16:40:13.817219 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.817136    2566 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 13 16:40:13.817219 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.817204    2566 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 13 16:40:13.818048 ip-10-0-128-173 systemd[1]: Started Kubernetes Kubelet.
Jul 13 16:40:13.818733 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.818586    2566 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 13 16:40:13.819319 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.819285    2566 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-173.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 13 16:40:13.819768 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.819752    2566 server.go:310] "Adding debug handlers to kubelet server"
Jul 13 16:40:13.819978 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:13.819940    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-173.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 13 16:40:13.820037 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:13.819940    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 13 16:40:13.823036 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:13.823014    2566 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 13 16:40:13.823128 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:13.822032    2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-173.ec2.internal.18c1e6e888c689a8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-173.ec2.internal,UID:ip-10-0-128-173.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-173.ec2.internal,},FirstTimestamp:2026-07-13 16:40:13.816940968 +0000 UTC m=+0.358321390,LastTimestamp:2026-07-13 16:40:13.816940968 +0000 UTC m=+0.358321390,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-173.ec2.internal,}"
Jul 13 16:40:13.824699 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.824680    2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 13 16:40:13.824699 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.824695    2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 13 16:40:13.824897 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:13.824781    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-173.ec2.internal\" not found"
Jul 13 16:40:13.825559 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.825540    2566 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 13 16:40:13.825559 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.825561    2566 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 13 16:40:13.825676 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.825542    2566 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 13 16:40:13.825734 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.825704    2566 reconstruct.go:97] "Volume reconstruction finished"
Jul 13 16:40:13.825734 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.825711    2566 reconciler.go:29] "Reconciler: start to sync state"
Jul 13 16:40:13.827375 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.827352    2566 factory.go:55] Registering systemd factory
Jul 13 16:40:13.827375 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.827371    2566 factory.go:223] Registration of the systemd container factory successfully
Jul 13 16:40:13.827611 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.827590    2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tpp4x"
Jul 13 16:40:13.827852 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.827831    2566 factory.go:153] Registering CRI-O factory
Jul 13 16:40:13.827937 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.827876    2566 factory.go:223] Registration of the crio container factory successfully
Jul 13 16:40:13.827937 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.827928    2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 13 16:40:13.828046 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.827969    2566 factory.go:103] Registering Raw factory
Jul 13 16:40:13.828046 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.827985    2566 manager.go:1196] Started watching for new ooms in manager
Jul 13 16:40:13.828755 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.828738    2566 manager.go:349] Starting recovery of all containers
Jul 13 16:40:13.835701 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.835672    2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tpp4x"
Jul 13 16:40:13.835886 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:13.835862    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 13 16:40:13.836020 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:13.835991    2566 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-173.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 13 16:40:13.843729 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.843561    2566 manager.go:354] Recovery completed
Jul 13 16:40:13.848466 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.848448    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 16:40:13.850624 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.850602    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 16:40:13.850711 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.850638    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 16:40:13.850711 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.850652    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientPID"
Jul 13 16:40:13.851265 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.851245    2566 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 13 16:40:13.851265 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.851261    2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 13 16:40:13.851406 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.851280    2566 state_mem.go:36] "Initialized new in-memory state store"
Jul 13 16:40:13.851406 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.851251    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 16:40:13.852030 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.852016    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 16:40:13.852072 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.852045    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 16:40:13.852072 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.852057    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientPID"
Jul 13 16:40:13.852488 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.852477    2566 policy_none.go:49] "None policy: Start"
Jul 13 16:40:13.852525 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.852492    2566 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 13 16:40:13.852525 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.852503    2566 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 13 16:40:13.853466 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:13.853375    2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-173.ec2.internal.18c1e6e88ac87927  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-173.ec2.internal,UID:ip-10-0-128-173.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-173.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-173.ec2.internal,},FirstTimestamp:2026-07-13 16:40:13.850622247 +0000 UTC m=+0.392002673,LastTimestamp:2026-07-13 16:40:13.850622247 +0000 UTC m=+0.392002673,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-173.ec2.internal,}"
Jul 13 16:40:13.853727 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.853712    2566 policy_none.go:47] "Start"
Jul 13 16:40:13.886064 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.886045    2566 manager.go:337] "Starting Device Plugin manager"
Jul 13 16:40:13.886180 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:13.886099    2566 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 13 16:40:13.886180 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.886114    2566 server.go:87] "Starting device plugin registration server"
Jul 13 16:40:13.886465 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.886450    2566 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 13 16:40:13.886530 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.886467    2566 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 13 16:40:13.886594 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.886576    2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 13 16:40:13.886690 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.886672    2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 13 16:40:13.886690 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.886690    2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 13 16:40:13.887392 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:13.887369    2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 13 16:40:13.887486 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:13.887428    2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-173.ec2.internal\" not found"
Jul 13 16:40:13.972543 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.972453    2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 13 16:40:13.973845 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.973776    2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 13 16:40:13.973845 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.973821    2566 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 13 16:40:13.973845 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.973846    2566 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 13 16:40:13.974012 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:13.973886    2566 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 13 16:40:13.976474 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.976449    2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 16:40:13.987437 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.987421    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 16:40:13.988357 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.988336    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 16:40:13.988439 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.988371    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 16:40:13.988439 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.988382    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientPID"
Jul 13 16:40:13.988439 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.988407    2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-173.ec2.internal"
Jul 13 16:40:13.997393 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:13.997373    2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-173.ec2.internal"
Jul 13 16:40:13.997457 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:13.997404    2566 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-173.ec2.internal\": node \"ip-10-0-128-173.ec2.internal\" not found"
Jul 13 16:40:14.053962 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:14.053939    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-173.ec2.internal\" not found"
Jul 13 16:40:14.074896 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:14.074857    2566 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-173.ec2.internal"]
Jul 13 16:40:14.074973 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:14.074942    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 16:40:14.075973 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:14.075956    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 16:40:14.076056 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:14.075987    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 16:40:14.076056 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:14.076004    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientPID"
Jul 13 16:40:14.077616 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:14.077600    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 16:40:14.077722 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:14.077704    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal"
Jul 13 16:40:14.077780 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:14.077740    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 16:40:14.078398 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:14.078379    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 16:40:14.078463 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:14.078415    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 16:40:14.078463 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:14.078428    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientPID"
Jul 13 16:40:14.078531 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:14.078383    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 16:40:14.078531 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:14.078488    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 16:40:14.078531 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:14.078499    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientPID"
Jul 13 16:40:14.079575 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:14.079560    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-173.ec2.internal"
Jul 13 16:40:14.079633 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:14.079593    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 16:40:14.080296 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:14.080278    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 16:40:14.080395 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:14.080309    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 16:40:14.080395 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:14.080338    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeHasSufficientPID"
Jul 13 16:40:14.108460 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:14.108427    2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-173.ec2.internal\" not found" node="ip-10-0-128-173.ec2.internal"
Jul 13 16:40:14.113382 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:14.113364    2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-173.ec2.internal\" not found" node="ip-10-0-128-173.ec2.internal"
Jul 13 16:40:14.127461 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:14.127439    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f5c60a2d059f0ab2782f7f7ac54db0b9-config\") pod \"kube-apiserver-proxy-ip-10-0-128-173.ec2.internal\" (UID: \"f5c60a2d059f0ab2782f7f7ac54db0b9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-173.ec2.internal"
Jul 13 16:40:14.127556 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:14.127469    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0016bcdb5f11dd2bf90f9728e967d7e2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal\" (UID: \"0016bcdb5f11dd2bf90f9728e967d7e2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal"
Jul 13 16:40:14.127556 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:14.127504    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0016bcdb5f11dd2bf90f9728e967d7e2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal\" (UID: \"0016bcdb5f11dd2bf90f9728e967d7e2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal"
Jul 13 16:40:14.154791 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:14.154749    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-173.ec2.internal\" not found"
Jul 13 16:40:14.228658 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:14.228572    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0016bcdb5f11dd2bf90f9728e967d7e2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal\" (UID: \"0016bcdb5f11dd2bf90f9728e967d7e2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal"
Jul 13 16:40:14.228658 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:14.228609    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0016bcdb5f11dd2bf90f9728e967d7e2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal\" (UID: \"0016bcdb5f11dd2bf90f9728e967d7e2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal"
Jul 13 16:40:14.228658 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:14.228627    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f5c60a2d059f0ab2782f7f7ac54db0b9-config\") pod \"kube-apiserver-proxy-ip-10-0-128-173.ec2.internal\" (UID: \"f5c60a2d059f0ab2782f7f7ac54db0b9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-173.ec2.internal"
Jul 13 16:40:14.228860 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:14.228677    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f5c60a2d059f0ab2782f7f7ac54db0b9-config\") pod \"kube-apiserver-proxy-ip-10-0-128-173.ec2.internal\" (UID: \"f5c60a2d059f0ab2782f7f7ac54db0b9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-173.ec2.internal"
Jul 13 16:40:14.228860 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:14.228679    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0016bcdb5f11dd2bf90f9728e967d7e2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal\" (UID: \"0016bcdb5f11dd2bf90f9728e967d7e2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal"
Jul 13 16:40:14.228860 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:14.228706    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0016bcdb5f11dd2bf90f9728e967d7e2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal\" (UID: \"0016bcdb5f11dd2bf90f9728e967d7e2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal"
Jul 13 16:40:14.256146 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:14.256106    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-173.ec2.internal\" not found"
Jul 13 16:40:14.358109 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:14.358066    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-173.ec2.internal\" not found"
Jul 13 16:40:14.414550 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:14.414513    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal"
Jul 13 16:40:14.416824 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:14.416804    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-173.ec2.internal"
Jul 13 16:40:14.458762 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:14.458726    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-173.ec2.internal\" not found"
Jul 13 16:40:14.560594 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:14.560496    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-173.ec2.internal\" not found"
Jul 13 16:40:14.663016 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:14.662983    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-173.ec2.internal\" not found"
Jul 13 16:40:14.763739 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:14.763705    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-173.ec2.internal\" not found"
Jul 13 16:40:14.766967 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:14.766946    2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 13 16:40:14.767117 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:14.767091    2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 13 16:40:14.842190 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:14.842096    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-12 16:35:14 +0000 UTC" deadline="2027-12-19 13:02:53.383638425 +0000 UTC"
Jul 13 16:40:14.842190 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:14.842140    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12572h22m38.541503005s"
Jul 13 16:40:14.864916 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:14.864475    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-173.ec2.internal\" not found"
Jul 13 16:40:14.965177 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:14.965140    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-173.ec2.internal\" not found"
Jul 13 16:40:14.975734 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:14.975712    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 16:40:14.976961 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:14.976913    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal" event={"ID":"0016bcdb5f11dd2bf90f9728e967d7e2","Type":"ContainerStarted","Data":"d0eb819a7add69ab94cfbf63e32c8589934fbcfc3370f0c1a3e50327e15d8759"}
Jul 13 16:40:14.977973 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:14.977950    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-173.ec2.internal" event={"ID":"f5c60a2d059f0ab2782f7f7ac54db0b9","Type":"ContainerStarted","Data":"7565cbf26709843413c60311df5d0d3acde498bbf465df89f03d70372f301b31"}
Jul 13 16:40:15.065490 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:15.065448    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-173.ec2.internal\" not found"
Jul 13 16:40:15.168355 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:15.168272    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-173.ec2.internal\" not found"
Jul 13 16:40:15.238866 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.238816    2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 16:40:15.269095 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:15.269052    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-173.ec2.internal\" not found"
Jul 13 16:40:15.313748 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.313719    2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 16:40:15.351002 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.350971    2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 16:40:15.426826 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.426729    2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal"
Jul 13 16:40:15.441132 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.441103    2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 13 16:40:15.442340 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.442320    2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-173.ec2.internal"
Jul 13 16:40:15.452171 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.452146    2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 13 16:40:15.721670 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.721561    2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 16:40:15.807849 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.807816    2566 apiserver.go:52] "Watching apiserver"
Jul 13 16:40:15.815660 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.815613    2566 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 13 16:40:15.816061 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.816031    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-lrq6k","kube-system/kube-apiserver-proxy-ip-10-0-128-173.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7vzvs","openshift-cluster-node-tuning-operator/tuned-ck8t2","openshift-image-registry/node-ca-j2pwp","openshift-multus/multus-4wq9f","openshift-multus/multus-additional-cni-plugins-6lnrp","openshift-dns/node-resolver-m6gh9","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal","openshift-multus/network-metrics-daemon-5hxjw","openshift-network-diagnostics/network-check-target-72twn","openshift-network-operator/iptables-alerter-f8h6c","openshift-ovn-kubernetes/ovnkube-node-gdqcb","kube-system/global-pull-secret-syncer-cx6bw"]
Jul 13 16:40:15.817828 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.817801    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6lnrp"
Jul 13 16:40:15.819293 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.819243    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.820703 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.820679    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7vzvs"
Jul 13 16:40:15.821246 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.821216    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 13 16:40:15.821246 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.821236    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 13 16:40:15.821465 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.821236    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 13 16:40:15.821465 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.821355    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-9zdps\""
Jul 13 16:40:15.821465 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.821431    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 13 16:40:15.821465 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.821439    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 13 16:40:15.822146 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.822114    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.822873 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.822672    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 13 16:40:15.822873 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.822682    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 13 16:40:15.822873 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.822868    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-mr84s\""
Jul 13 16:40:15.823708 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.823675    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-j2pwp"
Jul 13 16:40:15.824408 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.824369    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 13 16:40:15.824500 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.824486    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 13 16:40:15.824563 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.824520    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 13 16:40:15.824563 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.824552    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 13 16:40:15.824673 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.824607    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 13 16:40:15.824796 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.824781    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 13 16:40:15.824868 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.824809    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5qcxg\""
Jul 13 16:40:15.824868 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.824852    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 13 16:40:15.824976 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.824878    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 13 16:40:15.825225 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.825193    2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 13 16:40:15.825647 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.825495    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 13 16:40:15.825770 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.825749    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-trf2l\""
Jul 13 16:40:15.825854 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.825753    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.826521 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.826499    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-pfxhh\""
Jul 13 16:40:15.828289 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.827399    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:40:15.828289 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:15.827511    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cx6bw" podUID="5cdf4449-5e1c-453d-b7f0-ae71276fd373"
Jul 13 16:40:15.828289 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.827830    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 13 16:40:15.829374 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.828580    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 13 16:40:15.829374 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.829057    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-9rw5l\""
Jul 13 16:40:15.829374 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.829349    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 13 16:40:15.831400 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.830809    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lrq6k"
Jul 13 16:40:15.831400 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.831182    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 13 16:40:15.832663 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.832642    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-m6gh9"
Jul 13 16:40:15.834443 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.834418    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:40:15.834559 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:15.834511    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5hxjw" podUID="60216865-6676-407c-bd07-6c87d339c76d"
Jul 13 16:40:15.836039 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.836015    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:40:15.836173 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:15.836142    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72twn" podUID="b9ba2c1f-5b34-4948-b18f-661b82e24e08"
Jul 13 16:40:15.836320 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.836297    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 13 16:40:15.836489 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.836375    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qgh8c\""
Jul 13 16:40:15.836489 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.836421    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 13 16:40:15.836713 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.836692    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 13 16:40:15.836918 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.836897    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-p5hp7\""
Jul 13 16:40:15.837015 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.836998    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 13 16:40:15.837774 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.837754    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-f8h6c"
Jul 13 16:40:15.837868 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.837776    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/650632b5-fb83-42dd-9274-4be5ff84ffe9-system-cni-dir\") pod \"multus-additional-cni-plugins-6lnrp\" (UID: \"650632b5-fb83-42dd-9274-4be5ff84ffe9\") " pod="openshift-multus/multus-additional-cni-plugins-6lnrp"
Jul 13 16:40:15.837868 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.837807    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-system-cni-dir\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.837868 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.837836    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6a33d4fc-2988-4f88-a92c-212d95ec2060-ovnkube-script-lib\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.837868 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.837862    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkrb5\" (UniqueName: \"kubernetes.io/projected/7079d352-3e5d-40bd-a225-a20aff5172ac-kube-api-access-jkrb5\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.838079 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.837887    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxfqp\" (UniqueName: \"kubernetes.io/projected/ebb4c244-b823-48ea-a747-09b0330ce4e7-kube-api-access-sxfqp\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.838079 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.837928    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-log-socket\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.838079 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.837954    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-host-run-ovn-kubernetes\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.838079 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.837978    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/096914c3-b654-44bc-9051-f695fe22f31d-device-dir\") pod \"aws-ebs-csi-driver-node-7vzvs\" (UID: \"096914c3-b654-44bc-9051-f695fe22f31d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7vzvs"
Jul 13 16:40:15.838079 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.838006    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/096914c3-b654-44bc-9051-f695fe22f31d-etc-selinux\") pod \"aws-ebs-csi-driver-node-7vzvs\" (UID: \"096914c3-b654-44bc-9051-f695fe22f31d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7vzvs"
Jul 13 16:40:15.838079 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.838032    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7079d352-3e5d-40bd-a225-a20aff5172ac-etc-sysctl-d\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.838079 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.838057    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/096914c3-b654-44bc-9051-f695fe22f31d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7vzvs\" (UID: \"096914c3-b654-44bc-9051-f695fe22f31d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7vzvs"
Jul 13 16:40:15.838497 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.838081    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ebb4c244-b823-48ea-a747-09b0330ce4e7-cni-binary-copy\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.838497 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.838105    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-host-run-netns\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.838497 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.838133    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-run-systemd\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.838497 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.838156    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-run-ovn\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.838497 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.838219    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzvj4\" (UniqueName: \"kubernetes.io/projected/6a33d4fc-2988-4f88-a92c-212d95ec2060-kube-api-access-kzvj4\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.838497 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.838258    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/559d8901-967f-4100-a1df-0600fd3d305a-host\") pod \"node-ca-j2pwp\" (UID: \"559d8901-967f-4100-a1df-0600fd3d305a\") " pod="openshift-image-registry/node-ca-j2pwp"
Jul 13 16:40:15.838497 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.838315    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-cnibin\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.838497 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.838342    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5cdf4449-5e1c-453d-b7f0-ae71276fd373-dbus\") pod \"global-pull-secret-syncer-cx6bw\" (UID: \"5cdf4449-5e1c-453d-b7f0-ae71276fd373\") " pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:40:15.838497 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.838374    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-host-run-netns\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.838497 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.838398    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-node-log\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.838497 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.838436    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-host-cni-netd\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.838497 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.838473    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6a33d4fc-2988-4f88-a92c-212d95ec2060-ovn-node-metrics-cert\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.838497 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.838500    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-host-var-lib-cni-bin\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.839231 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.838525    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-host-slash\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.839231 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.838549    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-run-openvswitch\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.839231 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.838596    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-os-release\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.839231 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.838644    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-multus-conf-dir\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.839231 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.838673    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-etc-openvswitch\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.839231 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.838697    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7079d352-3e5d-40bd-a225-a20aff5172ac-sys\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.839231 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.838722    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7079d352-3e5d-40bd-a225-a20aff5172ac-var-lib-kubelet\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.839231 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.838747    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ebb4c244-b823-48ea-a747-09b0330ce4e7-multus-daemon-config\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.839231 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.838770    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7079d352-3e5d-40bd-a225-a20aff5172ac-tmp\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.839231 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.838792    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-host-var-lib-kubelet\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.839231 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.838843    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-host-kubelet\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.839231 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.838867    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/096914c3-b654-44bc-9051-f695fe22f31d-socket-dir\") pod \"aws-ebs-csi-driver-node-7vzvs\" (UID: \"096914c3-b654-44bc-9051-f695fe22f31d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7vzvs"
Jul 13 16:40:15.839231 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.838890    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/650632b5-fb83-42dd-9274-4be5ff84ffe9-cnibin\") pod \"multus-additional-cni-plugins-6lnrp\" (UID: \"650632b5-fb83-42dd-9274-4be5ff84ffe9\") " pod="openshift-multus/multus-additional-cni-plugins-6lnrp"
Jul 13 16:40:15.839231 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.838921    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/650632b5-fb83-42dd-9274-4be5ff84ffe9-cni-binary-copy\") pod \"multus-additional-cni-plugins-6lnrp\" (UID: \"650632b5-fb83-42dd-9274-4be5ff84ffe9\") " pod="openshift-multus/multus-additional-cni-plugins-6lnrp"
Jul 13 16:40:15.839231 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.838949    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-multus-cni-dir\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.839231 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.838975    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-multus-socket-dir-parent\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.839231 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.839000    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7079d352-3e5d-40bd-a225-a20aff5172ac-etc-systemd\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.840190 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.839022    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7079d352-3e5d-40bd-a225-a20aff5172ac-host\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.840190 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.839047    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9f58z\" (UniqueName: \"kubernetes.io/projected/650632b5-fb83-42dd-9274-4be5ff84ffe9-kube-api-access-9f58z\") pod \"multus-additional-cni-plugins-6lnrp\" (UID: \"650632b5-fb83-42dd-9274-4be5ff84ffe9\") " pod="openshift-multus/multus-additional-cni-plugins-6lnrp"
Jul 13 16:40:15.840190 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.839092    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbv92\" (UniqueName: \"kubernetes.io/projected/096914c3-b654-44bc-9051-f695fe22f31d-kube-api-access-jbv92\") pod \"aws-ebs-csi-driver-node-7vzvs\" (UID: \"096914c3-b654-44bc-9051-f695fe22f31d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7vzvs"
Jul 13 16:40:15.840190 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.839119    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7079d352-3e5d-40bd-a225-a20aff5172ac-etc-sysconfig\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.840190 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.839142    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7079d352-3e5d-40bd-a225-a20aff5172ac-run\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.840190 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.839165    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/650632b5-fb83-42dd-9274-4be5ff84ffe9-os-release\") pod \"multus-additional-cni-plugins-6lnrp\" (UID: \"650632b5-fb83-42dd-9274-4be5ff84ffe9\") " pod="openshift-multus/multus-additional-cni-plugins-6lnrp"
Jul 13 16:40:15.840190 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.839190    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/650632b5-fb83-42dd-9274-4be5ff84ffe9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6lnrp\" (UID: \"650632b5-fb83-42dd-9274-4be5ff84ffe9\") " pod="openshift-multus/multus-additional-cni-plugins-6lnrp"
Jul 13 16:40:15.840190 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.839215    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-etc-kubernetes\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.840190 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.839239    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-var-lib-openvswitch\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.840190 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.839264    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-host-cni-bin\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.840190 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.839289    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6a33d4fc-2988-4f88-a92c-212d95ec2060-env-overrides\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.840190 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.839333    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7079d352-3e5d-40bd-a225-a20aff5172ac-etc-sysctl-conf\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.840190 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.839378    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7079d352-3e5d-40bd-a225-a20aff5172ac-lib-modules\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.840190 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.839402    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/096914c3-b654-44bc-9051-f695fe22f31d-registration-dir\") pod \"aws-ebs-csi-driver-node-7vzvs\" (UID: \"096914c3-b654-44bc-9051-f695fe22f31d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7vzvs"
Jul 13 16:40:15.840190 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.839425    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-host-run-k8s-cni-cncf-io\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.840190 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.839455    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-host-run-multus-certs\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.841093 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.839481    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.841093 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.839510    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/096914c3-b654-44bc-9051-f695fe22f31d-sys-fs\") pod \"aws-ebs-csi-driver-node-7vzvs\" (UID: \"096914c3-b654-44bc-9051-f695fe22f31d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7vzvs"
Jul 13 16:40:15.841093 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.839536    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7079d352-3e5d-40bd-a225-a20aff5172ac-etc-kubernetes\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.841093 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.839592    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-www66\" (UniqueName: \"kubernetes.io/projected/559d8901-967f-4100-a1df-0600fd3d305a-kube-api-access-www66\") pod \"node-ca-j2pwp\" (UID: \"559d8901-967f-4100-a1df-0600fd3d305a\") " pod="openshift-image-registry/node-ca-j2pwp"
Jul 13 16:40:15.841093 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.839628    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/650632b5-fb83-42dd-9274-4be5ff84ffe9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6lnrp\" (UID: \"650632b5-fb83-42dd-9274-4be5ff84ffe9\") " pod="openshift-multus/multus-additional-cni-plugins-6lnrp"
Jul 13 16:40:15.841093 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.839667    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-host-var-lib-cni-multus\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.841093 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.839710    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/559d8901-967f-4100-a1df-0600fd3d305a-serviceca\") pod \"node-ca-j2pwp\" (UID: \"559d8901-967f-4100-a1df-0600fd3d305a\") " pod="openshift-image-registry/node-ca-j2pwp"
Jul 13 16:40:15.841093 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.839760    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/650632b5-fb83-42dd-9274-4be5ff84ffe9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6lnrp\" (UID: \"650632b5-fb83-42dd-9274-4be5ff84ffe9\") " pod="openshift-multus/multus-additional-cni-plugins-6lnrp"
Jul 13 16:40:15.841093 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.839791    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5cdf4449-5e1c-453d-b7f0-ae71276fd373-kubelet-config\") pod \"global-pull-secret-syncer-cx6bw\" (UID: \"5cdf4449-5e1c-453d-b7f0-ae71276fd373\") " pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:40:15.841093 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.839826    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5cdf4449-5e1c-453d-b7f0-ae71276fd373-original-pull-secret\") pod \"global-pull-secret-syncer-cx6bw\" (UID: \"5cdf4449-5e1c-453d-b7f0-ae71276fd373\") " pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:40:15.841093 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.839850    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7079d352-3e5d-40bd-a225-a20aff5172ac-etc-modprobe-d\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.841093 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.839874    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7079d352-3e5d-40bd-a225-a20aff5172ac-etc-tuned\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.841093 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.839914    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6a33d4fc-2988-4f88-a92c-212d95ec2060-ovnkube-config\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.841093 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.839943    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-hostroot\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.841093 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.839968    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-systemd-units\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.841997 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.841156    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 13 16:40:15.841997 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.841224    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-fl5bb\""
Jul 13 16:40:15.841997 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.841326    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 13 16:40:15.841997 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.841446    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 13 16:40:15.841997 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.841934    2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 13 16:40:15.871385 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.871356    2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-swppl"
Jul 13 16:40:15.880780 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.880753    2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-swppl"
Jul 13 16:40:15.927572 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.927538    2566 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 13 16:40:15.940554 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.940515    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-os-release\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.940554 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.940556    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-multus-conf-dir\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.940848 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.940587    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-etc-openvswitch\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.940848 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.940612    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7079d352-3e5d-40bd-a225-a20aff5172ac-sys\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.940848 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.940637    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7079d352-3e5d-40bd-a225-a20aff5172ac-var-lib-kubelet\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.940848 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.940661    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ebb4c244-b823-48ea-a747-09b0330ce4e7-multus-daemon-config\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.940848 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.940667    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-etc-openvswitch\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.940848 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.940673    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-multus-conf-dir\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.940848 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.940694    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-os-release\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.940848 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.940683    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7079d352-3e5d-40bd-a225-a20aff5172ac-tmp\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.940848 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.940747    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pmq7\" (UniqueName: \"kubernetes.io/projected/2fdaea84-a3f8-4b68-bf69-ed84f92fcdbb-kube-api-access-5pmq7\") pod \"node-resolver-m6gh9\" (UID: \"2fdaea84-a3f8-4b68-bf69-ed84f92fcdbb\") " pod="openshift-dns/node-resolver-m6gh9"
Jul 13 16:40:15.940848 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.940742    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7079d352-3e5d-40bd-a225-a20aff5172ac-sys\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.940848 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.940777    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7079d352-3e5d-40bd-a225-a20aff5172ac-var-lib-kubelet\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.940848 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.940784    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-host-var-lib-kubelet\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.940848 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.940812    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-host-kubelet\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.940848 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.940856    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-host-kubelet\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.941607 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.940892    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-host-var-lib-kubelet\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.941607 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.940910    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/096914c3-b654-44bc-9051-f695fe22f31d-socket-dir\") pod \"aws-ebs-csi-driver-node-7vzvs\" (UID: \"096914c3-b654-44bc-9051-f695fe22f31d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7vzvs"
Jul 13 16:40:15.941607 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.940940    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/650632b5-fb83-42dd-9274-4be5ff84ffe9-cnibin\") pod \"multus-additional-cni-plugins-6lnrp\" (UID: \"650632b5-fb83-42dd-9274-4be5ff84ffe9\") " pod="openshift-multus/multus-additional-cni-plugins-6lnrp"
Jul 13 16:40:15.941607 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.941005    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/650632b5-fb83-42dd-9274-4be5ff84ffe9-cni-binary-copy\") pod \"multus-additional-cni-plugins-6lnrp\" (UID: \"650632b5-fb83-42dd-9274-4be5ff84ffe9\") " pod="openshift-multus/multus-additional-cni-plugins-6lnrp"
Jul 13 16:40:15.941607 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.941031    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-multus-cni-dir\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.941607 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.941077    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-multus-socket-dir-parent\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.941607 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.941102    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7079d352-3e5d-40bd-a225-a20aff5172ac-etc-systemd\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.941607 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.941125    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/096914c3-b654-44bc-9051-f695fe22f31d-socket-dir\") pod \"aws-ebs-csi-driver-node-7vzvs\" (UID: \"096914c3-b654-44bc-9051-f695fe22f31d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7vzvs"
Jul 13 16:40:15.941607 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.941147    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7079d352-3e5d-40bd-a225-a20aff5172ac-host\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.941607 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.941177    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60216865-6676-407c-bd07-6c87d339c76d-metrics-certs\") pod \"network-metrics-daemon-5hxjw\" (UID: \"60216865-6676-407c-bd07-6c87d339c76d\") " pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:40:15.941607 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.941202    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/92e4c739-d71c-49bf-8d3b-04cfb0ae6bfb-iptables-alerter-script\") pod \"iptables-alerter-f8h6c\" (UID: \"92e4c739-d71c-49bf-8d3b-04cfb0ae6bfb\") " pod="openshift-network-operator/iptables-alerter-f8h6c"
Jul 13 16:40:15.941607 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.941205    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-multus-cni-dir\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.941607 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.941199    2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 13 16:40:15.941607 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.941231    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9f58z\" (UniqueName: \"kubernetes.io/projected/650632b5-fb83-42dd-9274-4be5ff84ffe9-kube-api-access-9f58z\") pod \"multus-additional-cni-plugins-6lnrp\" (UID: \"650632b5-fb83-42dd-9274-4be5ff84ffe9\") " pod="openshift-multus/multus-additional-cni-plugins-6lnrp"
Jul 13 16:40:15.941607 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.941253    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/650632b5-fb83-42dd-9274-4be5ff84ffe9-cnibin\") pod \"multus-additional-cni-plugins-6lnrp\" (UID: \"650632b5-fb83-42dd-9274-4be5ff84ffe9\") " pod="openshift-multus/multus-additional-cni-plugins-6lnrp"
Jul 13 16:40:15.941607 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.941263    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbv92\" (UniqueName: \"kubernetes.io/projected/096914c3-b654-44bc-9051-f695fe22f31d-kube-api-access-jbv92\") pod \"aws-ebs-csi-driver-node-7vzvs\" (UID: \"096914c3-b654-44bc-9051-f695fe22f31d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7vzvs"
Jul 13 16:40:15.941607 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.941317    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7079d352-3e5d-40bd-a225-a20aff5172ac-etc-sysconfig\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.942505 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.941348    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7079d352-3e5d-40bd-a225-a20aff5172ac-run\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.942505 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.941384    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/de1a001e-9d1f-4b04-ad7c-c186a513e940-konnectivity-ca\") pod \"konnectivity-agent-lrq6k\" (UID: \"de1a001e-9d1f-4b04-ad7c-c186a513e940\") " pod="kube-system/konnectivity-agent-lrq6k"
Jul 13 16:40:15.942505 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.941430    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/92e4c739-d71c-49bf-8d3b-04cfb0ae6bfb-host-slash\") pod \"iptables-alerter-f8h6c\" (UID: \"92e4c739-d71c-49bf-8d3b-04cfb0ae6bfb\") " pod="openshift-network-operator/iptables-alerter-f8h6c"
Jul 13 16:40:15.942505 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.941457    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gm4x\" (UniqueName: \"kubernetes.io/projected/92e4c739-d71c-49bf-8d3b-04cfb0ae6bfb-kube-api-access-9gm4x\") pod \"iptables-alerter-f8h6c\" (UID: \"92e4c739-d71c-49bf-8d3b-04cfb0ae6bfb\") " pod="openshift-network-operator/iptables-alerter-f8h6c"
Jul 13 16:40:15.942505 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.941467    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ebb4c244-b823-48ea-a747-09b0330ce4e7-multus-daemon-config\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.942505 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.941484    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/650632b5-fb83-42dd-9274-4be5ff84ffe9-os-release\") pod \"multus-additional-cni-plugins-6lnrp\" (UID: \"650632b5-fb83-42dd-9274-4be5ff84ffe9\") " pod="openshift-multus/multus-additional-cni-plugins-6lnrp"
Jul 13 16:40:15.942505 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.941520    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/650632b5-fb83-42dd-9274-4be5ff84ffe9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6lnrp\" (UID: \"650632b5-fb83-42dd-9274-4be5ff84ffe9\") " pod="openshift-multus/multus-additional-cni-plugins-6lnrp"
Jul 13 16:40:15.942505 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.941545    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-etc-kubernetes\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.942505 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.941567    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-var-lib-openvswitch\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.942505 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.941592    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-host-cni-bin\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.942505 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.941647    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6a33d4fc-2988-4f88-a92c-212d95ec2060-env-overrides\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.942505 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.941650    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/650632b5-fb83-42dd-9274-4be5ff84ffe9-os-release\") pod \"multus-additional-cni-plugins-6lnrp\" (UID: \"650632b5-fb83-42dd-9274-4be5ff84ffe9\") " pod="openshift-multus/multus-additional-cni-plugins-6lnrp"
Jul 13 16:40:15.942505 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.941702    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7079d352-3e5d-40bd-a225-a20aff5172ac-etc-systemd\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.942505 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.941728    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-var-lib-openvswitch\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.942505 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.941736    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7079d352-3e5d-40bd-a225-a20aff5172ac-etc-sysctl-conf\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.942505 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.941773    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-etc-kubernetes\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.942505 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.941787    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7079d352-3e5d-40bd-a225-a20aff5172ac-host\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.943262 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.941910    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7079d352-3e5d-40bd-a225-a20aff5172ac-etc-sysctl-conf\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.943262 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.941940    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7079d352-3e5d-40bd-a225-a20aff5172ac-lib-modules\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.943262 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.941970    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/096914c3-b654-44bc-9051-f695fe22f31d-registration-dir\") pod \"aws-ebs-csi-driver-node-7vzvs\" (UID: \"096914c3-b654-44bc-9051-f695fe22f31d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7vzvs"
Jul 13 16:40:15.943262 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.941995    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-host-run-k8s-cni-cncf-io\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.943262 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.942020    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-host-run-multus-certs\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.943262 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.942046    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.943262 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.942075    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/096914c3-b654-44bc-9051-f695fe22f31d-sys-fs\") pod \"aws-ebs-csi-driver-node-7vzvs\" (UID: \"096914c3-b654-44bc-9051-f695fe22f31d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7vzvs"
Jul 13 16:40:15.943262 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.942099    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7079d352-3e5d-40bd-a225-a20aff5172ac-etc-kubernetes\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.943262 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.942096    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/096914c3-b654-44bc-9051-f695fe22f31d-registration-dir\") pod \"aws-ebs-csi-driver-node-7vzvs\" (UID: \"096914c3-b654-44bc-9051-f695fe22f31d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7vzvs"
Jul 13 16:40:15.943262 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.942118    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/650632b5-fb83-42dd-9274-4be5ff84ffe9-cni-binary-copy\") pod \"multus-additional-cni-plugins-6lnrp\" (UID: \"650632b5-fb83-42dd-9274-4be5ff84ffe9\") " pod="openshift-multus/multus-additional-cni-plugins-6lnrp"
Jul 13 16:40:15.943262 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.942160    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-www66\" (UniqueName: \"kubernetes.io/projected/559d8901-967f-4100-a1df-0600fd3d305a-kube-api-access-www66\") pod \"node-ca-j2pwp\" (UID: \"559d8901-967f-4100-a1df-0600fd3d305a\") " pod="openshift-image-registry/node-ca-j2pwp"
Jul 13 16:40:15.943262 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.942168    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/650632b5-fb83-42dd-9274-4be5ff84ffe9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6lnrp\" (UID: \"650632b5-fb83-42dd-9274-4be5ff84ffe9\") " pod="openshift-multus/multus-additional-cni-plugins-6lnrp"
Jul 13 16:40:15.943262 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.942188    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zppx9\" (UniqueName: \"kubernetes.io/projected/60216865-6676-407c-bd07-6c87d339c76d-kube-api-access-zppx9\") pod \"network-metrics-daemon-5hxjw\" (UID: \"60216865-6676-407c-bd07-6c87d339c76d\") " pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:40:15.943262 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.942192    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7079d352-3e5d-40bd-a225-a20aff5172ac-etc-sysconfig\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.943262 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.942239    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-host-run-k8s-cni-cncf-io\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.943262 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.942257    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7079d352-3e5d-40bd-a225-a20aff5172ac-run\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.943262 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.942260    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7079d352-3e5d-40bd-a225-a20aff5172ac-lib-modules\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.944143 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.942282    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-host-run-multus-certs\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.944143 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.942341    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/650632b5-fb83-42dd-9274-4be5ff84ffe9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6lnrp\" (UID: \"650632b5-fb83-42dd-9274-4be5ff84ffe9\") " pod="openshift-multus/multus-additional-cni-plugins-6lnrp"
Jul 13 16:40:15.944143 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.942395    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-host-var-lib-cni-multus\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.944143 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.942402    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-multus-socket-dir-parent\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.944143 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.942444    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/559d8901-967f-4100-a1df-0600fd3d305a-serviceca\") pod \"node-ca-j2pwp\" (UID: \"559d8901-967f-4100-a1df-0600fd3d305a\") " pod="openshift-image-registry/node-ca-j2pwp"
Jul 13 16:40:15.944143 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.942459    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7079d352-3e5d-40bd-a225-a20aff5172ac-etc-kubernetes\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.944143 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.942471    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/650632b5-fb83-42dd-9274-4be5ff84ffe9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6lnrp\" (UID: \"650632b5-fb83-42dd-9274-4be5ff84ffe9\") " pod="openshift-multus/multus-additional-cni-plugins-6lnrp"
Jul 13 16:40:15.944143 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.942497    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5cdf4449-5e1c-453d-b7f0-ae71276fd373-kubelet-config\") pod \"global-pull-secret-syncer-cx6bw\" (UID: \"5cdf4449-5e1c-453d-b7f0-ae71276fd373\") " pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:40:15.944143 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.942506    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-host-var-lib-cni-multus\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.944143 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.942548    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5cdf4449-5e1c-453d-b7f0-ae71276fd373-original-pull-secret\") pod \"global-pull-secret-syncer-cx6bw\" (UID: \"5cdf4449-5e1c-453d-b7f0-ae71276fd373\") " pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:40:15.944143 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.942295    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-host-cni-bin\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.944143 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.942578    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/096914c3-b654-44bc-9051-f695fe22f31d-sys-fs\") pod \"aws-ebs-csi-driver-node-7vzvs\" (UID: \"096914c3-b654-44bc-9051-f695fe22f31d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7vzvs"
Jul 13 16:40:15.944143 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.943458    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6a33d4fc-2988-4f88-a92c-212d95ec2060-env-overrides\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.944143 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.943767    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7079d352-3e5d-40bd-a225-a20aff5172ac-etc-modprobe-d\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.944143 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:15.943915    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 16:40:15.944143 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.943917    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7079d352-3e5d-40bd-a225-a20aff5172ac-etc-modprobe-d\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.944143 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.943924    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/650632b5-fb83-42dd-9274-4be5ff84ffe9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6lnrp\" (UID: \"650632b5-fb83-42dd-9274-4be5ff84ffe9\") " pod="openshift-multus/multus-additional-cni-plugins-6lnrp"
Jul 13 16:40:15.944143 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.942346    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.945150 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.943971    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7079d352-3e5d-40bd-a225-a20aff5172ac-etc-tuned\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.945150 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:15.943997    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cdf4449-5e1c-453d-b7f0-ae71276fd373-original-pull-secret podName:5cdf4449-5e1c-453d-b7f0-ae71276fd373 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:16.443971864 +0000 UTC m=+2.985352288 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5cdf4449-5e1c-453d-b7f0-ae71276fd373-original-pull-secret") pod "global-pull-secret-syncer-cx6bw" (UID: "5cdf4449-5e1c-453d-b7f0-ae71276fd373") : object "kube-system"/"original-pull-secret" not registered
Jul 13 16:40:15.945150 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.944057    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5cdf4449-5e1c-453d-b7f0-ae71276fd373-kubelet-config\") pod \"global-pull-secret-syncer-cx6bw\" (UID: \"5cdf4449-5e1c-453d-b7f0-ae71276fd373\") " pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:40:15.945150 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.944145    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92qk7\" (UniqueName: \"kubernetes.io/projected/b9ba2c1f-5b34-4948-b18f-661b82e24e08-kube-api-access-92qk7\") pod \"network-check-target-72twn\" (UID: \"b9ba2c1f-5b34-4948-b18f-661b82e24e08\") " pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:40:15.945150 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.944185    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/650632b5-fb83-42dd-9274-4be5ff84ffe9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6lnrp\" (UID: \"650632b5-fb83-42dd-9274-4be5ff84ffe9\") " pod="openshift-multus/multus-additional-cni-plugins-6lnrp"
Jul 13 16:40:15.945150 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.944209    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6a33d4fc-2988-4f88-a92c-212d95ec2060-ovnkube-config\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.945150 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.944270    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2fdaea84-a3f8-4b68-bf69-ed84f92fcdbb-hosts-file\") pod \"node-resolver-m6gh9\" (UID: \"2fdaea84-a3f8-4b68-bf69-ed84f92fcdbb\") " pod="openshift-dns/node-resolver-m6gh9"
Jul 13 16:40:15.945150 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.944376    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fdaea84-a3f8-4b68-bf69-ed84f92fcdbb-tmp-dir\") pod \"node-resolver-m6gh9\" (UID: \"2fdaea84-a3f8-4b68-bf69-ed84f92fcdbb\") " pod="openshift-dns/node-resolver-m6gh9"
Jul 13 16:40:15.945150 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.944436    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-hostroot\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.945150 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.944503    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-systemd-units\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.945150 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.944540    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/650632b5-fb83-42dd-9274-4be5ff84ffe9-system-cni-dir\") pod \"multus-additional-cni-plugins-6lnrp\" (UID: \"650632b5-fb83-42dd-9274-4be5ff84ffe9\") " pod="openshift-multus/multus-additional-cni-plugins-6lnrp"
Jul 13 16:40:15.945150 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.944679    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-system-cni-dir\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.945150 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.944712    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6a33d4fc-2988-4f88-a92c-212d95ec2060-ovnkube-script-lib\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.945150 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.944756    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkrb5\" (UniqueName: \"kubernetes.io/projected/7079d352-3e5d-40bd-a225-a20aff5172ac-kube-api-access-jkrb5\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.945150 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.944787    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxfqp\" (UniqueName: \"kubernetes.io/projected/ebb4c244-b823-48ea-a747-09b0330ce4e7-kube-api-access-sxfqp\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.945150 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.944820    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-log-socket\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.946034 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.944845    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-host-run-ovn-kubernetes\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.946034 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.944876    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/096914c3-b654-44bc-9051-f695fe22f31d-device-dir\") pod \"aws-ebs-csi-driver-node-7vzvs\" (UID: \"096914c3-b654-44bc-9051-f695fe22f31d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7vzvs"
Jul 13 16:40:15.946034 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.944897    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6a33d4fc-2988-4f88-a92c-212d95ec2060-ovnkube-config\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.946034 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.944906    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/096914c3-b654-44bc-9051-f695fe22f31d-etc-selinux\") pod \"aws-ebs-csi-driver-node-7vzvs\" (UID: \"096914c3-b654-44bc-9051-f695fe22f31d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7vzvs"
Jul 13 16:40:15.946034 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.944956    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7079d352-3e5d-40bd-a225-a20aff5172ac-etc-sysctl-d\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.946034 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.944994    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-system-cni-dir\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.946034 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.944993    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/de1a001e-9d1f-4b04-ad7c-c186a513e940-agent-certs\") pod \"konnectivity-agent-lrq6k\" (UID: \"de1a001e-9d1f-4b04-ad7c-c186a513e940\") " pod="kube-system/konnectivity-agent-lrq6k"
Jul 13 16:40:15.946034 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.945054    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/096914c3-b654-44bc-9051-f695fe22f31d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7vzvs\" (UID: \"096914c3-b654-44bc-9051-f695fe22f31d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7vzvs"
Jul 13 16:40:15.946034 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.945104    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ebb4c244-b823-48ea-a747-09b0330ce4e7-cni-binary-copy\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.946034 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.945164    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-host-run-netns\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.946034 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.945213    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-run-systemd\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.946034 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.945227    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/096914c3-b654-44bc-9051-f695fe22f31d-device-dir\") pod \"aws-ebs-csi-driver-node-7vzvs\" (UID: \"096914c3-b654-44bc-9051-f695fe22f31d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7vzvs"
Jul 13 16:40:15.946034 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.945367    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7079d352-3e5d-40bd-a225-a20aff5172ac-etc-sysctl-d\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.946034 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.945434    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-host-run-ovn-kubernetes\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.946034 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.945479    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-run-ovn\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.946034 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.945807    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-log-socket\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.946034 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.945807    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-hostroot\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.946034 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.945891    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-systemd-units\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.949639 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.946040    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/559d8901-967f-4100-a1df-0600fd3d305a-serviceca\") pod \"node-ca-j2pwp\" (UID: \"559d8901-967f-4100-a1df-0600fd3d305a\") " pod="openshift-image-registry/node-ca-j2pwp"
Jul 13 16:40:15.949639 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.946064    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/650632b5-fb83-42dd-9274-4be5ff84ffe9-system-cni-dir\") pod \"multus-additional-cni-plugins-6lnrp\" (UID: \"650632b5-fb83-42dd-9274-4be5ff84ffe9\") " pod="openshift-multus/multus-additional-cni-plugins-6lnrp"
Jul 13 16:40:15.949639 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.946130    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/096914c3-b654-44bc-9051-f695fe22f31d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7vzvs\" (UID: \"096914c3-b654-44bc-9051-f695fe22f31d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7vzvs"
Jul 13 16:40:15.949639 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.946134    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-host-run-netns\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.949639 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.946187    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-run-systemd\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.949639 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.946507    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/096914c3-b654-44bc-9051-f695fe22f31d-etc-selinux\") pod \"aws-ebs-csi-driver-node-7vzvs\" (UID: \"096914c3-b654-44bc-9051-f695fe22f31d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7vzvs"
Jul 13 16:40:15.949639 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.946636    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6a33d4fc-2988-4f88-a92c-212d95ec2060-ovnkube-script-lib\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.949639 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.947699    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7079d352-3e5d-40bd-a225-a20aff5172ac-etc-tuned\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.949639 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.948964    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7079d352-3e5d-40bd-a225-a20aff5172ac-tmp\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.949639 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.945249    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-run-ovn\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.949639 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.949146    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzvj4\" (UniqueName: \"kubernetes.io/projected/6a33d4fc-2988-4f88-a92c-212d95ec2060-kube-api-access-kzvj4\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.949639 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.949201    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/559d8901-967f-4100-a1df-0600fd3d305a-host\") pod \"node-ca-j2pwp\" (UID: \"559d8901-967f-4100-a1df-0600fd3d305a\") " pod="openshift-image-registry/node-ca-j2pwp"
Jul 13 16:40:15.949639 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.949234    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-cnibin\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.949639 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.949286    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5cdf4449-5e1c-453d-b7f0-ae71276fd373-dbus\") pod \"global-pull-secret-syncer-cx6bw\" (UID: \"5cdf4449-5e1c-453d-b7f0-ae71276fd373\") " pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:40:15.949639 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.949314    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-host-run-netns\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.949639 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.949356    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-node-log\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.949639 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.949418    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-node-log\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.949639 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.949461    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-cnibin\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.950598 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.949523    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-host-cni-netd\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.950598 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.949567    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5cdf4449-5e1c-453d-b7f0-ae71276fd373-dbus\") pod \"global-pull-secret-syncer-cx6bw\" (UID: \"5cdf4449-5e1c-453d-b7f0-ae71276fd373\") " pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:40:15.950598 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.949578    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6a33d4fc-2988-4f88-a92c-212d95ec2060-ovn-node-metrics-cert\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.950598 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.949614    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-host-var-lib-cni-bin\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.950598 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.949619    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-host-run-netns\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.950598 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.949668    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-host-slash\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.950598 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.949710    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-run-openvswitch\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.950598 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.949713    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ebb4c244-b823-48ea-a747-09b0330ce4e7-cni-binary-copy\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.950598 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.949748    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-host-cni-netd\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.950598 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.949909    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/559d8901-967f-4100-a1df-0600fd3d305a-host\") pod \"node-ca-j2pwp\" (UID: \"559d8901-967f-4100-a1df-0600fd3d305a\") " pod="openshift-image-registry/node-ca-j2pwp"
Jul 13 16:40:15.950598 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.949970    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-host-slash\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.950598 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.950006    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ebb4c244-b823-48ea-a747-09b0330ce4e7-host-var-lib-cni-bin\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.950598 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.950012    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6a33d4fc-2988-4f88-a92c-212d95ec2060-run-openvswitch\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.957074 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.956652    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6a33d4fc-2988-4f88-a92c-212d95ec2060-ovn-node-metrics-cert\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:15.957074 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.956853    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxfqp\" (UniqueName: \"kubernetes.io/projected/ebb4c244-b823-48ea-a747-09b0330ce4e7-kube-api-access-sxfqp\") pod \"multus-4wq9f\" (UID: \"ebb4c244-b823-48ea-a747-09b0330ce4e7\") " pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:15.957074 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.956912    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbv92\" (UniqueName: \"kubernetes.io/projected/096914c3-b654-44bc-9051-f695fe22f31d-kube-api-access-jbv92\") pod \"aws-ebs-csi-driver-node-7vzvs\" (UID: \"096914c3-b654-44bc-9051-f695fe22f31d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7vzvs"
Jul 13 16:40:15.957437 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.957108    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-www66\" (UniqueName: \"kubernetes.io/projected/559d8901-967f-4100-a1df-0600fd3d305a-kube-api-access-www66\") pod \"node-ca-j2pwp\" (UID: \"559d8901-967f-4100-a1df-0600fd3d305a\") " pod="openshift-image-registry/node-ca-j2pwp"
Jul 13 16:40:15.957544 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.957518    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9f58z\" (UniqueName: \"kubernetes.io/projected/650632b5-fb83-42dd-9274-4be5ff84ffe9-kube-api-access-9f58z\") pod \"multus-additional-cni-plugins-6lnrp\" (UID: \"650632b5-fb83-42dd-9274-4be5ff84ffe9\") " pod="openshift-multus/multus-additional-cni-plugins-6lnrp"
Jul 13 16:40:15.958196 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.958168    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkrb5\" (UniqueName: \"kubernetes.io/projected/7079d352-3e5d-40bd-a225-a20aff5172ac-kube-api-access-jkrb5\") pod \"tuned-ck8t2\" (UID: \"7079d352-3e5d-40bd-a225-a20aff5172ac\") " pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:15.959901 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:15.959875    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzvj4\" (UniqueName: \"kubernetes.io/projected/6a33d4fc-2988-4f88-a92c-212d95ec2060-kube-api-access-kzvj4\") pod \"ovnkube-node-gdqcb\" (UID: \"6a33d4fc-2988-4f88-a92c-212d95ec2060\") " pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:16.050690 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:16.050587    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60216865-6676-407c-bd07-6c87d339c76d-metrics-certs\") pod \"network-metrics-daemon-5hxjw\" (UID: \"60216865-6676-407c-bd07-6c87d339c76d\") " pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:40:16.050690 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:16.050641    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/92e4c739-d71c-49bf-8d3b-04cfb0ae6bfb-iptables-alerter-script\") pod \"iptables-alerter-f8h6c\" (UID: \"92e4c739-d71c-49bf-8d3b-04cfb0ae6bfb\") " pod="openshift-network-operator/iptables-alerter-f8h6c"
Jul 13 16:40:16.050690 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:16.050671    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/de1a001e-9d1f-4b04-ad7c-c186a513e940-konnectivity-ca\") pod \"konnectivity-agent-lrq6k\" (UID: \"de1a001e-9d1f-4b04-ad7c-c186a513e940\") " pod="kube-system/konnectivity-agent-lrq6k"
Jul 13 16:40:16.050979 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:16.050696    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/92e4c739-d71c-49bf-8d3b-04cfb0ae6bfb-host-slash\") pod \"iptables-alerter-f8h6c\" (UID: \"92e4c739-d71c-49bf-8d3b-04cfb0ae6bfb\") " pod="openshift-network-operator/iptables-alerter-f8h6c"
Jul 13 16:40:16.050979 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:16.050731    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gm4x\" (UniqueName: \"kubernetes.io/projected/92e4c739-d71c-49bf-8d3b-04cfb0ae6bfb-kube-api-access-9gm4x\") pod \"iptables-alerter-f8h6c\" (UID: \"92e4c739-d71c-49bf-8d3b-04cfb0ae6bfb\") " pod="openshift-network-operator/iptables-alerter-f8h6c"
Jul 13 16:40:16.050979 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:16.050762    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 16:40:16.050979 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:16.050779    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zppx9\" (UniqueName: \"kubernetes.io/projected/60216865-6676-407c-bd07-6c87d339c76d-kube-api-access-zppx9\") pod \"network-metrics-daemon-5hxjw\" (UID: \"60216865-6676-407c-bd07-6c87d339c76d\") " pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:40:16.050979 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:16.050827    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92qk7\" (UniqueName: \"kubernetes.io/projected/b9ba2c1f-5b34-4948-b18f-661b82e24e08-kube-api-access-92qk7\") pod \"network-check-target-72twn\" (UID: \"b9ba2c1f-5b34-4948-b18f-661b82e24e08\") " pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:40:16.050979 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:16.050843    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60216865-6676-407c-bd07-6c87d339c76d-metrics-certs podName:60216865-6676-407c-bd07-6c87d339c76d nodeName:}" failed. No retries permitted until 2026-07-13 16:40:16.550821402 +0000 UTC m=+3.092201831 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/60216865-6676-407c-bd07-6c87d339c76d-metrics-certs") pod "network-metrics-daemon-5hxjw" (UID: "60216865-6676-407c-bd07-6c87d339c76d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 16:40:16.051344 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:16.050989    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2fdaea84-a3f8-4b68-bf69-ed84f92fcdbb-hosts-file\") pod \"node-resolver-m6gh9\" (UID: \"2fdaea84-a3f8-4b68-bf69-ed84f92fcdbb\") " pod="openshift-dns/node-resolver-m6gh9"
Jul 13 16:40:16.051344 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:16.051027    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fdaea84-a3f8-4b68-bf69-ed84f92fcdbb-tmp-dir\") pod \"node-resolver-m6gh9\" (UID: \"2fdaea84-a3f8-4b68-bf69-ed84f92fcdbb\") " pod="openshift-dns/node-resolver-m6gh9"
Jul 13 16:40:16.051344 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:16.051081    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/de1a001e-9d1f-4b04-ad7c-c186a513e940-agent-certs\") pod \"konnectivity-agent-lrq6k\" (UID: \"de1a001e-9d1f-4b04-ad7c-c186a513e940\") " pod="kube-system/konnectivity-agent-lrq6k"
Jul 13 16:40:16.051344 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:16.051131    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pmq7\" (UniqueName: \"kubernetes.io/projected/2fdaea84-a3f8-4b68-bf69-ed84f92fcdbb-kube-api-access-5pmq7\") pod \"node-resolver-m6gh9\" (UID: \"2fdaea84-a3f8-4b68-bf69-ed84f92fcdbb\") " pod="openshift-dns/node-resolver-m6gh9"
Jul 13 16:40:16.051344 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:16.051205    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2fdaea84-a3f8-4b68-bf69-ed84f92fcdbb-hosts-file\") pod \"node-resolver-m6gh9\" (UID: \"2fdaea84-a3f8-4b68-bf69-ed84f92fcdbb\") " pod="openshift-dns/node-resolver-m6gh9"
Jul 13 16:40:16.051592 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:16.051389    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/92e4c739-d71c-49bf-8d3b-04cfb0ae6bfb-host-slash\") pod \"iptables-alerter-f8h6c\" (UID: \"92e4c739-d71c-49bf-8d3b-04cfb0ae6bfb\") " pod="openshift-network-operator/iptables-alerter-f8h6c"
Jul 13 16:40:16.051723 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:16.051695    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fdaea84-a3f8-4b68-bf69-ed84f92fcdbb-tmp-dir\") pod \"node-resolver-m6gh9\" (UID: \"2fdaea84-a3f8-4b68-bf69-ed84f92fcdbb\") " pod="openshift-dns/node-resolver-m6gh9"
Jul 13 16:40:16.052069 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:16.052011    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/de1a001e-9d1f-4b04-ad7c-c186a513e940-konnectivity-ca\") pod \"konnectivity-agent-lrq6k\" (UID: \"de1a001e-9d1f-4b04-ad7c-c186a513e940\") " pod="kube-system/konnectivity-agent-lrq6k"
Jul 13 16:40:16.052069 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:16.052037    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/92e4c739-d71c-49bf-8d3b-04cfb0ae6bfb-iptables-alerter-script\") pod \"iptables-alerter-f8h6c\" (UID: \"92e4c739-d71c-49bf-8d3b-04cfb0ae6bfb\") " pod="openshift-network-operator/iptables-alerter-f8h6c"
Jul 13 16:40:16.054466 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:16.054439    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/de1a001e-9d1f-4b04-ad7c-c186a513e940-agent-certs\") pod \"konnectivity-agent-lrq6k\" (UID: \"de1a001e-9d1f-4b04-ad7c-c186a513e940\") " pod="kube-system/konnectivity-agent-lrq6k"
Jul 13 16:40:16.057637 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:16.057597    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 16:40:16.057637 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:16.057626    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 16:40:16.057637 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:16.057641    2566 projected.go:196] Error preparing data for projected volume kube-api-access-92qk7 for pod openshift-network-diagnostics/network-check-target-72twn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 16:40:16.057891 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:16.057718    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b9ba2c1f-5b34-4948-b18f-661b82e24e08-kube-api-access-92qk7 podName:b9ba2c1f-5b34-4948-b18f-661b82e24e08 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:16.557696179 +0000 UTC m=+3.099076605 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-92qk7" (UniqueName: "kubernetes.io/projected/b9ba2c1f-5b34-4948-b18f-661b82e24e08-kube-api-access-92qk7") pod "network-check-target-72twn" (UID: "b9ba2c1f-5b34-4948-b18f-661b82e24e08") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 16:40:16.061102 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:16.061074    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zppx9\" (UniqueName: \"kubernetes.io/projected/60216865-6676-407c-bd07-6c87d339c76d-kube-api-access-zppx9\") pod \"network-metrics-daemon-5hxjw\" (UID: \"60216865-6676-407c-bd07-6c87d339c76d\") " pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:40:16.061244 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:16.061124    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pmq7\" (UniqueName: \"kubernetes.io/projected/2fdaea84-a3f8-4b68-bf69-ed84f92fcdbb-kube-api-access-5pmq7\") pod \"node-resolver-m6gh9\" (UID: \"2fdaea84-a3f8-4b68-bf69-ed84f92fcdbb\") " pod="openshift-dns/node-resolver-m6gh9"
Jul 13 16:40:16.061244 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:16.061203    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gm4x\" (UniqueName: \"kubernetes.io/projected/92e4c739-d71c-49bf-8d3b-04cfb0ae6bfb-kube-api-access-9gm4x\") pod \"iptables-alerter-f8h6c\" (UID: \"92e4c739-d71c-49bf-8d3b-04cfb0ae6bfb\") " pod="openshift-network-operator/iptables-alerter-f8h6c"
Jul 13 16:40:16.132693 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:16.132653    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6lnrp"
Jul 13 16:40:16.149502 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:16.149464    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:16.160186 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:16.160149    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7vzvs"
Jul 13 16:40:16.166808 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:16.166777    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ck8t2"
Jul 13 16:40:16.176141 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:16.176096    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-j2pwp"
Jul 13 16:40:16.185059 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:16.184847    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4wq9f"
Jul 13 16:40:16.194211 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:16.194176    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-lrq6k"
Jul 13 16:40:16.201550 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:16.201521    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-m6gh9"
Jul 13 16:40:16.212371 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:16.212328    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-f8h6c"
Jul 13 16:40:16.455226 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:16.455120    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5cdf4449-5e1c-453d-b7f0-ae71276fd373-original-pull-secret\") pod \"global-pull-secret-syncer-cx6bw\" (UID: \"5cdf4449-5e1c-453d-b7f0-ae71276fd373\") " pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:40:16.455408 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:16.455265    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 16:40:16.455408 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:16.455348    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cdf4449-5e1c-453d-b7f0-ae71276fd373-original-pull-secret podName:5cdf4449-5e1c-453d-b7f0-ae71276fd373 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:17.455327583 +0000 UTC m=+3.996707991 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5cdf4449-5e1c-453d-b7f0-ae71276fd373-original-pull-secret") pod "global-pull-secret-syncer-cx6bw" (UID: "5cdf4449-5e1c-453d-b7f0-ae71276fd373") : object "kube-system"/"original-pull-secret" not registered
Jul 13 16:40:16.556293 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:16.556233    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60216865-6676-407c-bd07-6c87d339c76d-metrics-certs\") pod \"network-metrics-daemon-5hxjw\" (UID: \"60216865-6676-407c-bd07-6c87d339c76d\") " pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:40:16.556495 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:16.556411    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 16:40:16.556561 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:16.556503    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60216865-6676-407c-bd07-6c87d339c76d-metrics-certs podName:60216865-6676-407c-bd07-6c87d339c76d nodeName:}" failed. No retries permitted until 2026-07-13 16:40:17.556480818 +0000 UTC m=+4.097861232 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/60216865-6676-407c-bd07-6c87d339c76d-metrics-certs") pod "network-metrics-daemon-5hxjw" (UID: "60216865-6676-407c-bd07-6c87d339c76d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 16:40:16.656978 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:16.656933    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92qk7\" (UniqueName: \"kubernetes.io/projected/b9ba2c1f-5b34-4948-b18f-661b82e24e08-kube-api-access-92qk7\") pod \"network-check-target-72twn\" (UID: \"b9ba2c1f-5b34-4948-b18f-661b82e24e08\") " pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:40:16.657185 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:16.657109    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 16:40:16.657185 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:16.657138    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 16:40:16.657185 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:16.657149    2566 projected.go:196] Error preparing data for projected volume kube-api-access-92qk7 for pod openshift-network-diagnostics/network-check-target-72twn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 16:40:16.657381 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:16.657216    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b9ba2c1f-5b34-4948-b18f-661b82e24e08-kube-api-access-92qk7 podName:b9ba2c1f-5b34-4948-b18f-661b82e24e08 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:17.657195905 +0000 UTC m=+4.198576318 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-92qk7" (UniqueName: "kubernetes.io/projected/b9ba2c1f-5b34-4948-b18f-661b82e24e08-kube-api-access-92qk7") pod "network-check-target-72twn" (UID: "b9ba2c1f-5b34-4948-b18f-661b82e24e08") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 16:40:16.884022 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:16.883915    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 16:35:16 +0000 UTC" deadline="2028-01-23 18:31:47.386319241 +0000 UTC"
Jul 13 16:40:16.884022 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:16.883963    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13417h51m30.502360109s"
Jul 13 16:40:16.974222 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:16.974182    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:40:16.974442 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:16.974328    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72twn" podUID="b9ba2c1f-5b34-4948-b18f-661b82e24e08"
Jul 13 16:40:17.190432 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:17.190381    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod650632b5_fb83_42dd_9274_4be5ff84ffe9.slice/crio-4f3f4e340b8ba70232178bde2e90380ad51e891b8468f1eb63c242b0d144fcb9 WatchSource:0}: Error finding container 4f3f4e340b8ba70232178bde2e90380ad51e891b8468f1eb63c242b0d144fcb9: Status 404 returned error can't find the container with id 4f3f4e340b8ba70232178bde2e90380ad51e891b8468f1eb63c242b0d144fcb9
Jul 13 16:40:17.193627 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:17.193494    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod559d8901_967f_4100_a1df_0600fd3d305a.slice/crio-f5fe40883a59dbdb4ffcd394d1deeb20069b6122c6b26894e5846ad2445a402e WatchSource:0}: Error finding container f5fe40883a59dbdb4ffcd394d1deeb20069b6122c6b26894e5846ad2445a402e: Status 404 returned error can't find the container with id f5fe40883a59dbdb4ffcd394d1deeb20069b6122c6b26894e5846ad2445a402e
Jul 13 16:40:17.194649 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:17.194620    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7079d352_3e5d_40bd_a225_a20aff5172ac.slice/crio-3396dce7c8d62f873b4922a5ac84ccf4bcf3298db4f814bb671678d46d317943 WatchSource:0}: Error finding container 3396dce7c8d62f873b4922a5ac84ccf4bcf3298db4f814bb671678d46d317943: Status 404 returned error can't find the container with id 3396dce7c8d62f873b4922a5ac84ccf4bcf3298db4f814bb671678d46d317943
Jul 13 16:40:17.196334 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:17.196308    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a33d4fc_2988_4f88_a92c_212d95ec2060.slice/crio-90a40bc5c9928c18ec4f9751ce779da0a3a3f9d42b664eb91a920fad5a613e1a WatchSource:0}: Error finding container 90a40bc5c9928c18ec4f9751ce779da0a3a3f9d42b664eb91a920fad5a613e1a: Status 404 returned error can't find the container with id 90a40bc5c9928c18ec4f9751ce779da0a3a3f9d42b664eb91a920fad5a613e1a
Jul 13 16:40:17.197540 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:17.197514    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fdaea84_a3f8_4b68_bf69_ed84f92fcdbb.slice/crio-31ba1cba1fbf09568d2bc3e89b771b35f9f51eddbf32389f767482c2e0852e64 WatchSource:0}: Error finding container 31ba1cba1fbf09568d2bc3e89b771b35f9f51eddbf32389f767482c2e0852e64: Status 404 returned error can't find the container with id 31ba1cba1fbf09568d2bc3e89b771b35f9f51eddbf32389f767482c2e0852e64
Jul 13 16:40:17.199254 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:17.199224    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde1a001e_9d1f_4b04_ad7c_c186a513e940.slice/crio-e3b97abd34dd216d861e93604db54804cbb5d02cd22b06146a7ac21a835e1d8a WatchSource:0}: Error finding container e3b97abd34dd216d861e93604db54804cbb5d02cd22b06146a7ac21a835e1d8a: Status 404 returned error can't find the container with id e3b97abd34dd216d861e93604db54804cbb5d02cd22b06146a7ac21a835e1d8a
Jul 13 16:40:17.201285 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:17.201148    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod096914c3_b654_44bc_9051_f695fe22f31d.slice/crio-3aa6cecded944be8e9ed58509e8d146fad2e265da11ab0fffac252ad0ebfc2e3 WatchSource:0}: Error finding container 3aa6cecded944be8e9ed58509e8d146fad2e265da11ab0fffac252ad0ebfc2e3: Status 404 returned error can't find the container with id 3aa6cecded944be8e9ed58509e8d146fad2e265da11ab0fffac252ad0ebfc2e3
Jul 13 16:40:17.202591 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:40:17.202567    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebb4c244_b823_48ea_a747_09b0330ce4e7.slice/crio-d4c4e0248c3f34403d170734d215d1cb1416878e02cdf657089cee1c123cc7c1 WatchSource:0}: Error finding container d4c4e0248c3f34403d170734d215d1cb1416878e02cdf657089cee1c123cc7c1: Status 404 returned error can't find the container with id d4c4e0248c3f34403d170734d215d1cb1416878e02cdf657089cee1c123cc7c1
Jul 13 16:40:17.463820 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:17.463610    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5cdf4449-5e1c-453d-b7f0-ae71276fd373-original-pull-secret\") pod \"global-pull-secret-syncer-cx6bw\" (UID: \"5cdf4449-5e1c-453d-b7f0-ae71276fd373\") " pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:40:17.463957 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:17.463775    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 16:40:17.463957 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:17.463903    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cdf4449-5e1c-453d-b7f0-ae71276fd373-original-pull-secret podName:5cdf4449-5e1c-453d-b7f0-ae71276fd373 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:19.463887312 +0000 UTC m=+6.005267722 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5cdf4449-5e1c-453d-b7f0-ae71276fd373-original-pull-secret") pod "global-pull-secret-syncer-cx6bw" (UID: "5cdf4449-5e1c-453d-b7f0-ae71276fd373") : object "kube-system"/"original-pull-secret" not registered
Jul 13 16:40:17.564798 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:17.564751    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60216865-6676-407c-bd07-6c87d339c76d-metrics-certs\") pod \"network-metrics-daemon-5hxjw\" (UID: \"60216865-6676-407c-bd07-6c87d339c76d\") " pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:40:17.565005 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:17.564943    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 16:40:17.565005 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:17.565006    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60216865-6676-407c-bd07-6c87d339c76d-metrics-certs podName:60216865-6676-407c-bd07-6c87d339c76d nodeName:}" failed. No retries permitted until 2026-07-13 16:40:19.56499068 +0000 UTC m=+6.106371090 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/60216865-6676-407c-bd07-6c87d339c76d-metrics-certs") pod "network-metrics-daemon-5hxjw" (UID: "60216865-6676-407c-bd07-6c87d339c76d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 16:40:17.665678 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:17.665605    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92qk7\" (UniqueName: \"kubernetes.io/projected/b9ba2c1f-5b34-4948-b18f-661b82e24e08-kube-api-access-92qk7\") pod \"network-check-target-72twn\" (UID: \"b9ba2c1f-5b34-4948-b18f-661b82e24e08\") " pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:40:17.665862 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:17.665792    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 16:40:17.665862 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:17.665814    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 16:40:17.665862 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:17.665827    2566 projected.go:196] Error preparing data for projected volume kube-api-access-92qk7 for pod openshift-network-diagnostics/network-check-target-72twn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 16:40:17.666040 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:17.665899    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b9ba2c1f-5b34-4948-b18f-661b82e24e08-kube-api-access-92qk7 podName:b9ba2c1f-5b34-4948-b18f-661b82e24e08 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:19.665876274 +0000 UTC m=+6.207256703 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-92qk7" (UniqueName: "kubernetes.io/projected/b9ba2c1f-5b34-4948-b18f-661b82e24e08-kube-api-access-92qk7") pod "network-check-target-72twn" (UID: "b9ba2c1f-5b34-4948-b18f-661b82e24e08") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 16:40:17.884527 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:17.884446    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 16:35:16 +0000 UTC" deadline="2028-03-02 00:42:43.252323027 +0000 UTC"
Jul 13 16:40:17.884527 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:17.884516    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14336h2m25.367811764s"
Jul 13 16:40:17.977970 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:17.976599    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:40:17.977970 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:17.976744    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cx6bw" podUID="5cdf4449-5e1c-453d-b7f0-ae71276fd373"
Jul 13 16:40:17.977970 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:17.977032    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:40:17.977970 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:17.977145    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5hxjw" podUID="60216865-6676-407c-bd07-6c87d339c76d"
Jul 13 16:40:17.988773 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:17.987412    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7vzvs" event={"ID":"096914c3-b654-44bc-9051-f695fe22f31d","Type":"ContainerStarted","Data":"3aa6cecded944be8e9ed58509e8d146fad2e265da11ab0fffac252ad0ebfc2e3"}
Jul 13 16:40:17.991220 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:17.991136    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lrq6k" event={"ID":"de1a001e-9d1f-4b04-ad7c-c186a513e940","Type":"ContainerStarted","Data":"e3b97abd34dd216d861e93604db54804cbb5d02cd22b06146a7ac21a835e1d8a"}
Jul 13 16:40:17.995917 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:17.995848    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-m6gh9" event={"ID":"2fdaea84-a3f8-4b68-bf69-ed84f92fcdbb","Type":"ContainerStarted","Data":"31ba1cba1fbf09568d2bc3e89b771b35f9f51eddbf32389f767482c2e0852e64"}
Jul 13 16:40:17.998103 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:17.998024    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ck8t2" event={"ID":"7079d352-3e5d-40bd-a225-a20aff5172ac","Type":"ContainerStarted","Data":"3396dce7c8d62f873b4922a5ac84ccf4bcf3298db4f814bb671678d46d317943"}
Jul 13 16:40:18.001898 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:18.001804    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4wq9f" event={"ID":"ebb4c244-b823-48ea-a747-09b0330ce4e7","Type":"ContainerStarted","Data":"d4c4e0248c3f34403d170734d215d1cb1416878e02cdf657089cee1c123cc7c1"}
Jul 13 16:40:18.008478 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:18.008288    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb" event={"ID":"6a33d4fc-2988-4f88-a92c-212d95ec2060","Type":"ContainerStarted","Data":"90a40bc5c9928c18ec4f9751ce779da0a3a3f9d42b664eb91a920fad5a613e1a"}
Jul 13 16:40:18.010181 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:18.010108    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-j2pwp" event={"ID":"559d8901-967f-4100-a1df-0600fd3d305a","Type":"ContainerStarted","Data":"f5fe40883a59dbdb4ffcd394d1deeb20069b6122c6b26894e5846ad2445a402e"}
Jul 13 16:40:18.013487 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:18.013339    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6lnrp" event={"ID":"650632b5-fb83-42dd-9274-4be5ff84ffe9","Type":"ContainerStarted","Data":"4f3f4e340b8ba70232178bde2e90380ad51e891b8468f1eb63c242b0d144fcb9"}
Jul 13 16:40:18.019163 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:18.019129    2566 generic.go:356] "Generic (PLEG): container finished" podID="0016bcdb5f11dd2bf90f9728e967d7e2" containerID="4b1e8dcdbf97961b3b337ccde764e83ad9231319d5c6b0d16c1fa320510595b4" exitCode=0
Jul 13 16:40:18.019291 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:18.019237    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal" event={"ID":"0016bcdb5f11dd2bf90f9728e967d7e2","Type":"ContainerDied","Data":"4b1e8dcdbf97961b3b337ccde764e83ad9231319d5c6b0d16c1fa320510595b4"}
Jul 13 16:40:18.025142 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:18.025088    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-173.ec2.internal" event={"ID":"f5c60a2d059f0ab2782f7f7ac54db0b9","Type":"ContainerStarted","Data":"a75a562ef1cd875f59b7d450cfac1cba0ebaf3125e1c22dce92daf989609b194"}
Jul 13 16:40:18.034701 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:18.034634    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-f8h6c" event={"ID":"92e4c739-d71c-49bf-8d3b-04cfb0ae6bfb","Type":"ContainerStarted","Data":"03af69ed3f7891e11f8a8288f3642b4f6bfdff6d2be623e59e54a444af6bd569"}
Jul 13 16:40:18.974765 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:18.974725    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:40:18.975414 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:18.974873    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72twn" podUID="b9ba2c1f-5b34-4948-b18f-661b82e24e08"
Jul 13 16:40:19.065066 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:19.064971    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal" event={"ID":"0016bcdb5f11dd2bf90f9728e967d7e2","Type":"ContainerStarted","Data":"919abd5999d9045c372c6ce063e278e13f5063ea0f6d6e7199f83ab6e18db114"}
Jul 13 16:40:19.085122 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:19.084586    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-173.ec2.internal" podStartSLOduration=3.084560629 podStartE2EDuration="3.084560629s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 16:40:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:40:18.056999315 +0000 UTC m=+4.598379749" watchObservedRunningTime="2026-07-13 16:40:19.084560629 +0000 UTC m=+5.625941064"
Jul 13 16:40:19.481007 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:19.480962    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5cdf4449-5e1c-453d-b7f0-ae71276fd373-original-pull-secret\") pod \"global-pull-secret-syncer-cx6bw\" (UID: \"5cdf4449-5e1c-453d-b7f0-ae71276fd373\") " pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:40:19.481364 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:19.481338    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 16:40:19.481460 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:19.481420    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cdf4449-5e1c-453d-b7f0-ae71276fd373-original-pull-secret podName:5cdf4449-5e1c-453d-b7f0-ae71276fd373 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:23.481400107 +0000 UTC m=+10.022780519 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5cdf4449-5e1c-453d-b7f0-ae71276fd373-original-pull-secret") pod "global-pull-secret-syncer-cx6bw" (UID: "5cdf4449-5e1c-453d-b7f0-ae71276fd373") : object "kube-system"/"original-pull-secret" not registered
Jul 13 16:40:19.582368 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:19.582318    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60216865-6676-407c-bd07-6c87d339c76d-metrics-certs\") pod \"network-metrics-daemon-5hxjw\" (UID: \"60216865-6676-407c-bd07-6c87d339c76d\") " pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:40:19.582562 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:19.582545    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 16:40:19.582654 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:19.582617    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60216865-6676-407c-bd07-6c87d339c76d-metrics-certs podName:60216865-6676-407c-bd07-6c87d339c76d nodeName:}" failed. No retries permitted until 2026-07-13 16:40:23.582596769 +0000 UTC m=+10.123977181 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/60216865-6676-407c-bd07-6c87d339c76d-metrics-certs") pod "network-metrics-daemon-5hxjw" (UID: "60216865-6676-407c-bd07-6c87d339c76d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 16:40:19.683728 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:19.683683    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92qk7\" (UniqueName: \"kubernetes.io/projected/b9ba2c1f-5b34-4948-b18f-661b82e24e08-kube-api-access-92qk7\") pod \"network-check-target-72twn\" (UID: \"b9ba2c1f-5b34-4948-b18f-661b82e24e08\") " pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:40:19.683928 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:19.683870    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 16:40:19.683928 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:19.683894    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 16:40:19.683928 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:19.683909    2566 projected.go:196] Error preparing data for projected volume kube-api-access-92qk7 for pod openshift-network-diagnostics/network-check-target-72twn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 16:40:19.684112 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:19.683976    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b9ba2c1f-5b34-4948-b18f-661b82e24e08-kube-api-access-92qk7 podName:b9ba2c1f-5b34-4948-b18f-661b82e24e08 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:23.683955531 +0000 UTC m=+10.225335941 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-92qk7" (UniqueName: "kubernetes.io/projected/b9ba2c1f-5b34-4948-b18f-661b82e24e08-kube-api-access-92qk7") pod "network-check-target-72twn" (UID: "b9ba2c1f-5b34-4948-b18f-661b82e24e08") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 16:40:19.977184 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:19.977045    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:40:19.977705 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:19.977205    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5hxjw" podUID="60216865-6676-407c-bd07-6c87d339c76d"
Jul 13 16:40:19.977705 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:19.977273    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:40:19.977705 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:19.977366    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cx6bw" podUID="5cdf4449-5e1c-453d-b7f0-ae71276fd373"
Jul 13 16:40:20.975747 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:20.975123    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:40:20.975747 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:20.975330    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72twn" podUID="b9ba2c1f-5b34-4948-b18f-661b82e24e08"
Jul 13 16:40:21.974785 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:21.974693    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:40:21.975264 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:21.974846    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cx6bw" podUID="5cdf4449-5e1c-453d-b7f0-ae71276fd373"
Jul 13 16:40:21.975264 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:21.974954    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:40:21.975264 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:21.975056    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5hxjw" podUID="60216865-6676-407c-bd07-6c87d339c76d"
Jul 13 16:40:22.974831 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:22.974789    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:40:22.975329 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:22.974932    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72twn" podUID="b9ba2c1f-5b34-4948-b18f-661b82e24e08"
Jul 13 16:40:23.519355 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:23.519313    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5cdf4449-5e1c-453d-b7f0-ae71276fd373-original-pull-secret\") pod \"global-pull-secret-syncer-cx6bw\" (UID: \"5cdf4449-5e1c-453d-b7f0-ae71276fd373\") " pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:40:23.519572 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:23.519513    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 16:40:23.519645 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:23.519596    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cdf4449-5e1c-453d-b7f0-ae71276fd373-original-pull-secret podName:5cdf4449-5e1c-453d-b7f0-ae71276fd373 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:31.519565161 +0000 UTC m=+18.060945574 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5cdf4449-5e1c-453d-b7f0-ae71276fd373-original-pull-secret") pod "global-pull-secret-syncer-cx6bw" (UID: "5cdf4449-5e1c-453d-b7f0-ae71276fd373") : object "kube-system"/"original-pull-secret" not registered
Jul 13 16:40:23.620315 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:23.620184    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60216865-6676-407c-bd07-6c87d339c76d-metrics-certs\") pod \"network-metrics-daemon-5hxjw\" (UID: \"60216865-6676-407c-bd07-6c87d339c76d\") " pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:40:23.620517 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:23.620364    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 16:40:23.620517 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:23.620442    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60216865-6676-407c-bd07-6c87d339c76d-metrics-certs podName:60216865-6676-407c-bd07-6c87d339c76d nodeName:}" failed. No retries permitted until 2026-07-13 16:40:31.620423015 +0000 UTC m=+18.161803440 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/60216865-6676-407c-bd07-6c87d339c76d-metrics-certs") pod "network-metrics-daemon-5hxjw" (UID: "60216865-6676-407c-bd07-6c87d339c76d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 16:40:23.721164 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:23.721088    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92qk7\" (UniqueName: \"kubernetes.io/projected/b9ba2c1f-5b34-4948-b18f-661b82e24e08-kube-api-access-92qk7\") pod \"network-check-target-72twn\" (UID: \"b9ba2c1f-5b34-4948-b18f-661b82e24e08\") " pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:40:23.721392 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:23.721257    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 16:40:23.721392 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:23.721280    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 16:40:23.721392 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:23.721306    2566 projected.go:196] Error preparing data for projected volume kube-api-access-92qk7 for pod openshift-network-diagnostics/network-check-target-72twn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 16:40:23.721392 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:23.721376    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b9ba2c1f-5b34-4948-b18f-661b82e24e08-kube-api-access-92qk7 podName:b9ba2c1f-5b34-4948-b18f-661b82e24e08 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:31.721355615 +0000 UTC m=+18.262736038 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-92qk7" (UniqueName: "kubernetes.io/projected/b9ba2c1f-5b34-4948-b18f-661b82e24e08-kube-api-access-92qk7") pod "network-check-target-72twn" (UID: "b9ba2c1f-5b34-4948-b18f-661b82e24e08") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 16:40:23.975711 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:23.975563    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:40:23.975711 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:23.975686    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5hxjw" podUID="60216865-6676-407c-bd07-6c87d339c76d"
Jul 13 16:40:23.976235 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:23.976039    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:40:23.976235 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:23.976137    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cx6bw" podUID="5cdf4449-5e1c-453d-b7f0-ae71276fd373"
Jul 13 16:40:24.974273 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:24.974222    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:40:24.974476 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:24.974373    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72twn" podUID="b9ba2c1f-5b34-4948-b18f-661b82e24e08"
Jul 13 16:40:25.974770 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:25.974687    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:40:25.974770 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:25.974717    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:40:25.975324 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:25.974901    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cx6bw" podUID="5cdf4449-5e1c-453d-b7f0-ae71276fd373"
Jul 13 16:40:25.975324 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:25.975003    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5hxjw" podUID="60216865-6676-407c-bd07-6c87d339c76d"
Jul 13 16:40:26.974820 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:26.974779    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:40:26.975325 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:26.974919    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72twn" podUID="b9ba2c1f-5b34-4948-b18f-661b82e24e08"
Jul 13 16:40:27.974973 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:27.974937    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:40:27.975479 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:27.974935    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:40:27.975479 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:27.975083    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cx6bw" podUID="5cdf4449-5e1c-453d-b7f0-ae71276fd373"
Jul 13 16:40:27.975479 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:27.975220    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5hxjw" podUID="60216865-6676-407c-bd07-6c87d339c76d"
Jul 13 16:40:28.975161 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:28.975075    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:40:28.975633 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:28.975206    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72twn" podUID="b9ba2c1f-5b34-4948-b18f-661b82e24e08"
Jul 13 16:40:29.974362 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:29.974328    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:40:29.974362 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:29.974354    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:40:29.974597 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:29.974482    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5hxjw" podUID="60216865-6676-407c-bd07-6c87d339c76d"
Jul 13 16:40:29.974597 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:29.974580    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cx6bw" podUID="5cdf4449-5e1c-453d-b7f0-ae71276fd373"
Jul 13 16:40:30.974399 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:30.974356    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:40:30.974952 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:30.974499    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72twn" podUID="b9ba2c1f-5b34-4948-b18f-661b82e24e08"
Jul 13 16:40:31.583541 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:31.583499    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5cdf4449-5e1c-453d-b7f0-ae71276fd373-original-pull-secret\") pod \"global-pull-secret-syncer-cx6bw\" (UID: \"5cdf4449-5e1c-453d-b7f0-ae71276fd373\") " pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:40:31.583712 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:31.583656    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 16:40:31.583753 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:31.583719    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cdf4449-5e1c-453d-b7f0-ae71276fd373-original-pull-secret podName:5cdf4449-5e1c-453d-b7f0-ae71276fd373 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:47.583704095 +0000 UTC m=+34.125084501 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5cdf4449-5e1c-453d-b7f0-ae71276fd373-original-pull-secret") pod "global-pull-secret-syncer-cx6bw" (UID: "5cdf4449-5e1c-453d-b7f0-ae71276fd373") : object "kube-system"/"original-pull-secret" not registered
Jul 13 16:40:31.684845 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:31.684807    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60216865-6676-407c-bd07-6c87d339c76d-metrics-certs\") pod \"network-metrics-daemon-5hxjw\" (UID: \"60216865-6676-407c-bd07-6c87d339c76d\") " pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:40:31.685040 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:31.684978    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 16:40:31.685103 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:31.685060    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60216865-6676-407c-bd07-6c87d339c76d-metrics-certs podName:60216865-6676-407c-bd07-6c87d339c76d nodeName:}" failed. No retries permitted until 2026-07-13 16:40:47.685038282 +0000 UTC m=+34.226418688 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/60216865-6676-407c-bd07-6c87d339c76d-metrics-certs") pod "network-metrics-daemon-5hxjw" (UID: "60216865-6676-407c-bd07-6c87d339c76d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 16:40:31.785735 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:31.785697    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92qk7\" (UniqueName: \"kubernetes.io/projected/b9ba2c1f-5b34-4948-b18f-661b82e24e08-kube-api-access-92qk7\") pod \"network-check-target-72twn\" (UID: \"b9ba2c1f-5b34-4948-b18f-661b82e24e08\") " pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:40:31.785921 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:31.785848    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 16:40:31.785921 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:31.785867    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 16:40:31.785921 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:31.785879    2566 projected.go:196] Error preparing data for projected volume kube-api-access-92qk7 for pod openshift-network-diagnostics/network-check-target-72twn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 16:40:31.786125 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:31.785942    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b9ba2c1f-5b34-4948-b18f-661b82e24e08-kube-api-access-92qk7 podName:b9ba2c1f-5b34-4948-b18f-661b82e24e08 nodeName:}" failed. No retries permitted until 2026-07-13 16:40:47.785925754 +0000 UTC m=+34.327306180 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-92qk7" (UniqueName: "kubernetes.io/projected/b9ba2c1f-5b34-4948-b18f-661b82e24e08-kube-api-access-92qk7") pod "network-check-target-72twn" (UID: "b9ba2c1f-5b34-4948-b18f-661b82e24e08") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 16:40:31.974171 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:31.974074    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:40:31.974381 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:31.974087    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:40:31.974381 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:31.974199    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cx6bw" podUID="5cdf4449-5e1c-453d-b7f0-ae71276fd373"
Jul 13 16:40:31.974381 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:31.974324    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5hxjw" podUID="60216865-6676-407c-bd07-6c87d339c76d"
Jul 13 16:40:32.974677 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:32.974643    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:40:32.975053 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:32.974757    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72twn" podUID="b9ba2c1f-5b34-4948-b18f-661b82e24e08"
Jul 13 16:40:33.975834 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:33.975788    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:40:33.976312 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:33.975920    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5hxjw" podUID="60216865-6676-407c-bd07-6c87d339c76d"
Jul 13 16:40:33.976312 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:33.975981    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:40:33.976312 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:33.976122    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cx6bw" podUID="5cdf4449-5e1c-453d-b7f0-ae71276fd373"
Jul 13 16:40:34.974960 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:34.974918    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:40:34.975153 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:34.975068    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72twn" podUID="b9ba2c1f-5b34-4948-b18f-661b82e24e08"
Jul 13 16:40:35.975053 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:35.974965    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:40:35.975533 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:35.975125    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5hxjw" podUID="60216865-6676-407c-bd07-6c87d339c76d"
Jul 13 16:40:35.975533 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:35.975185    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:40:35.975533 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:35.975298    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cx6bw" podUID="5cdf4449-5e1c-453d-b7f0-ae71276fd373"
Jul 13 16:40:36.974458 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:36.974289    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:40:36.974568 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:36.974543    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72twn" podUID="b9ba2c1f-5b34-4948-b18f-661b82e24e08"
Jul 13 16:40:37.102947 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:37.102713    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-m6gh9" event={"ID":"2fdaea84-a3f8-4b68-bf69-ed84f92fcdbb","Type":"ContainerStarted","Data":"78c0051d3350044c5be5bfecd637a51cee0ebd41471e0876ba96e82a1de598b0"}
Jul 13 16:40:37.104197 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:37.104173    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ck8t2" event={"ID":"7079d352-3e5d-40bd-a225-a20aff5172ac","Type":"ContainerStarted","Data":"4afe54251e6990de4a1583cc8a5260c80df816d8c25d4f9cb815fb4a28a96af6"}
Jul 13 16:40:37.105701 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:37.105679    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4wq9f" event={"ID":"ebb4c244-b823-48ea-a747-09b0330ce4e7","Type":"ContainerStarted","Data":"190094eb2c4840b4bb76357c02adb56a18449cabcce5df83defc855c55ef5967"}
Jul 13 16:40:37.108006 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:37.107990    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gdqcb_6a33d4fc-2988-4f88-a92c-212d95ec2060/ovn-acl-logging/0.log"
Jul 13 16:40:37.108377 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:37.108356    2566 generic.go:356] "Generic (PLEG): container finished" podID="6a33d4fc-2988-4f88-a92c-212d95ec2060" containerID="9ecc61695d6f552a013996681984a31088dcf66374d66cf59621f7baa28a2d7b" exitCode=1
Jul 13 16:40:37.108458 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:37.108426    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb" event={"ID":"6a33d4fc-2988-4f88-a92c-212d95ec2060","Type":"ContainerStarted","Data":"6c3ae8200ff25ae59592dc48b3dba3f679ec3965474c6dbb44763b2110f62864"}
Jul 13 16:40:37.108512 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:37.108501    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb" event={"ID":"6a33d4fc-2988-4f88-a92c-212d95ec2060","Type":"ContainerStarted","Data":"6d80da521ce12c8f78aff19d45ab46a8d6ea83ae1e5c2aa3542aadfd19f479d0"}
Jul 13 16:40:37.108571 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:37.108556    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb" event={"ID":"6a33d4fc-2988-4f88-a92c-212d95ec2060","Type":"ContainerDied","Data":"9ecc61695d6f552a013996681984a31088dcf66374d66cf59621f7baa28a2d7b"}
Jul 13 16:40:37.108628 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:37.108613    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb" event={"ID":"6a33d4fc-2988-4f88-a92c-212d95ec2060","Type":"ContainerStarted","Data":"d699c306725ef8d190427909c16a23410ed35038e7e5a9cc0e4adf088c1c7b8b"}
Jul 13 16:40:37.111082 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:37.111055    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-j2pwp" event={"ID":"559d8901-967f-4100-a1df-0600fd3d305a","Type":"ContainerStarted","Data":"9d2a02ab1755e19c2e303e6752e0010a2903e276218fdf76d2052ac070bd4f65"}
Jul 13 16:40:37.113718 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:37.113692    2566 generic.go:356] "Generic (PLEG): container finished" podID="650632b5-fb83-42dd-9274-4be5ff84ffe9" containerID="abd5ebeb6128e0399e9788c0449e88136c5cda13d721a3b90c71c9595fc5bb5f" exitCode=0
Jul 13 16:40:37.113832 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:37.113769    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6lnrp" event={"ID":"650632b5-fb83-42dd-9274-4be5ff84ffe9","Type":"ContainerDied","Data":"abd5ebeb6128e0399e9788c0449e88136c5cda13d721a3b90c71c9595fc5bb5f"}
Jul 13 16:40:37.115342 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:37.115288    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7vzvs" event={"ID":"096914c3-b654-44bc-9051-f695fe22f31d","Type":"ContainerStarted","Data":"820e20141334d0ef0438b4990b078c19024b8db35765feecd000cb29f50553b1"}
Jul 13 16:40:37.116990 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:37.116969    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-lrq6k" event={"ID":"de1a001e-9d1f-4b04-ad7c-c186a513e940","Type":"ContainerStarted","Data":"5ddaac40fc382d2a23fb6761c2c4fe86b72b13e3d391d7fd17e56511ccab99f3"}
Jul 13 16:40:37.120604 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:37.120588    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-m6gh9_2fdaea84-a3f8-4b68-bf69-ed84f92fcdbb/dns-node-resolver/0.log"
Jul 13 16:40:37.123695 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:37.123658    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-m6gh9" podStartSLOduration=3.639032662 podStartE2EDuration="23.123647312s" totalImagesPullingTime="19.48461465s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 16:40:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:40:37.123040472 +0000 UTC m=+23.664420899" watchObservedRunningTime="2026-07-13 16:40:37.123647312 +0000 UTC m=+23.665027741"
Jul 13 16:40:37.123773 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:37.123755    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-173.ec2.internal" podStartSLOduration=21.123749262 podStartE2EDuration="21.123749262s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 16:40:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:40:19.084229142 +0000 UTC m=+5.625609575" watchObservedRunningTime="2026-07-13 16:40:37.123749262 +0000 UTC m=+23.665129690"
Jul 13 16:40:37.138378 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:37.138338    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-lrq6k" podStartSLOduration=3.655635389 podStartE2EDuration="23.13832097s" totalImagesPullingTime="19.482685581s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:40:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:40:37.137850901 +0000 UTC m=+23.679231331" watchObservedRunningTime="2026-07-13 16:40:37.13832097 +0000 UTC m=+23.679701399"
Jul 13 16:40:37.157892 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:37.157833    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-ck8t2" podStartSLOduration=3.66775779 podStartE2EDuration="23.157811865s" totalImagesPullingTime="19.490054075s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 16:40:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:40:37.156626194 +0000 UTC m=+23.698006618" watchObservedRunningTime="2026-07-13 16:40:37.157811865 +0000 UTC m=+23.699192297"
Jul 13 16:40:37.172721 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:37.172659    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-j2pwp" podStartSLOduration=3.685598084 podStartE2EDuration="23.172643248s" totalImagesPullingTime="19.487045164s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 16:40:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:40:37.17241588 +0000 UTC m=+23.713796311" watchObservedRunningTime="2026-07-13 16:40:37.172643248 +0000 UTC m=+23.714023677"
Jul 13 16:40:37.217093 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:37.217035    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-4wq9f" podStartSLOduration=3.670889142 podStartE2EDuration="23.217012064s" totalImagesPullingTime="19.546122922s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 16:40:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:40:37.216511505 +0000 UTC m=+23.757891937" watchObservedRunningTime="2026-07-13 16:40:37.217012064 +0000 UTC m=+23.758392496"
Jul 13 16:40:37.975646 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:37.974940    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:40:37.975646 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:37.975087    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5hxjw" podUID="60216865-6676-407c-bd07-6c87d339c76d"
Jul 13 16:40:37.975646 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:37.975292    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:40:37.975646 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:37.975441    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cx6bw" podUID="5cdf4449-5e1c-453d-b7f0-ae71276fd373"
Jul 13 16:40:38.121183 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:38.120956    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-f8h6c" event={"ID":"92e4c739-d71c-49bf-8d3b-04cfb0ae6bfb","Type":"ContainerStarted","Data":"353f26fca2b889dacbf14a6236690358c8368151683a5a7e260f77c59ebbac9b"}
Jul 13 16:40:38.124684 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:38.124659    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gdqcb_6a33d4fc-2988-4f88-a92c-212d95ec2060/ovn-acl-logging/0.log"
Jul 13 16:40:38.125144 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:38.125111    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb" event={"ID":"6a33d4fc-2988-4f88-a92c-212d95ec2060","Type":"ContainerStarted","Data":"fff572fd88ccd94859f2c580bb0324ff4f12d643db14a58239e45dcc24335f59"}
Jul 13 16:40:38.125285 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:38.125208    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb" event={"ID":"6a33d4fc-2988-4f88-a92c-212d95ec2060","Type":"ContainerStarted","Data":"ea0ac090f3bbc6163bd154723eac75ca9115b40f8abc6b3ab7a780010cb96650"}
Jul 13 16:40:38.141330 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:38.141139    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-f8h6c" podStartSLOduration=4.663487314 podStartE2EDuration="24.141119265s" totalImagesPullingTime="19.477631951s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 16:40:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:40:38.141062115 +0000 UTC m=+24.682442546" watchObservedRunningTime="2026-07-13 16:40:38.141119265 +0000 UTC m=+24.682499697"
Jul 13 16:40:38.295749 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:38.295670    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-j2pwp_559d8901-967f-4100-a1df-0600fd3d305a/node-ca/0.log"
Jul 13 16:40:38.862220 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:38.862185    2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 13 16:40:38.901007 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:38.900978    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-lrq6k"
Jul 13 16:40:38.901226 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:38.900946    2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-13T16:40:38.862216121Z","UUID":"10d19a0d-976f-46a2-a210-0c23ccdb15e6","Handler":null,"Name":"","Endpoint":""}
Jul 13 16:40:38.902492 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:38.902473    2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 13 16:40:38.902592 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:38.902514    2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 13 16:40:38.974128 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:38.974096    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:40:38.974324 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:38.974216    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72twn" podUID="b9ba2c1f-5b34-4948-b18f-661b82e24e08"
Jul 13 16:40:39.129410 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:39.129302    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7vzvs" event={"ID":"096914c3-b654-44bc-9051-f695fe22f31d","Type":"ContainerStarted","Data":"dc8530c6ce9f5f322a6fbafa4b1abee8d8d5cf7e9af03dd2138624aa9933b989"}
Jul 13 16:40:39.974384 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:39.974346    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:40:39.974641 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:39.974461    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cx6bw" podUID="5cdf4449-5e1c-453d-b7f0-ae71276fd373"
Jul 13 16:40:39.974641 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:39.974525    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:40:39.974729 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:39.974646    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5hxjw" podUID="60216865-6676-407c-bd07-6c87d339c76d"
Jul 13 16:40:40.134666 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:40.134634    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gdqcb_6a33d4fc-2988-4f88-a92c-212d95ec2060/ovn-acl-logging/0.log"
Jul 13 16:40:40.135091 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:40.134992    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb" event={"ID":"6a33d4fc-2988-4f88-a92c-212d95ec2060","Type":"ContainerStarted","Data":"7647935dd35250bfd47c169dec9285addf8b9aeb636acc59cd6b595d144e0eb7"}
Jul 13 16:40:40.974592 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:40.974559    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:40:40.974838 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:40.974667    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72twn" podUID="b9ba2c1f-5b34-4948-b18f-661b82e24e08"
Jul 13 16:40:41.425657 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:41.425581    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-lrq6k"
Jul 13 16:40:41.426214 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:41.426196    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-lrq6k"
Jul 13 16:40:41.974444 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:41.974137    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:40:41.974444 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:41.974180    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:40:41.974663 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:41.974496    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5hxjw" podUID="60216865-6676-407c-bd07-6c87d339c76d"
Jul 13 16:40:41.974663 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:41.974599    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cx6bw" podUID="5cdf4449-5e1c-453d-b7f0-ae71276fd373"
Jul 13 16:40:42.146137 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:42.146109    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gdqcb_6a33d4fc-2988-4f88-a92c-212d95ec2060/ovn-acl-logging/0.log"
Jul 13 16:40:42.146493 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:42.146471    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb" event={"ID":"6a33d4fc-2988-4f88-a92c-212d95ec2060","Type":"ContainerStarted","Data":"75dd3b98935300fc2af27fdc4de3446af63c2725e048c7db21e666dbe45e1cba"}
Jul 13 16:40:42.146745 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:42.146713    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:42.146938 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:42.146923    2566 scope.go:117] "RemoveContainer" containerID="9ecc61695d6f552a013996681984a31088dcf66374d66cf59621f7baa28a2d7b"
Jul 13 16:40:42.148445 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:42.148418    2566 generic.go:356] "Generic (PLEG): container finished" podID="650632b5-fb83-42dd-9274-4be5ff84ffe9" containerID="ae32605885123aedda3faeef4f6fcf004e1af87c01619dcaedcbaeabf6e962c8" exitCode=0
Jul 13 16:40:42.148559 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:42.148506    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6lnrp" event={"ID":"650632b5-fb83-42dd-9274-4be5ff84ffe9","Type":"ContainerDied","Data":"ae32605885123aedda3faeef4f6fcf004e1af87c01619dcaedcbaeabf6e962c8"}
Jul 13 16:40:42.150865 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:42.150837    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7vzvs" event={"ID":"096914c3-b654-44bc-9051-f695fe22f31d","Type":"ContainerStarted","Data":"b5d115c4f02c7f60e6cc9758de0866979b5a9b3fc65fe3a659bbfc02e3d65187"}
Jul 13 16:40:42.151572 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:42.151556    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-lrq6k"
Jul 13 16:40:42.163530 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:42.163503    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:42.282826 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:42.282729    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7vzvs" podStartSLOduration=12.100355198 podStartE2EDuration="28.282713597s" totalImagesPullingTime="16.182358399s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 16:40:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:40:42.282491058 +0000 UTC m=+28.823871489" watchObservedRunningTime="2026-07-13 16:40:42.282713597 +0000 UTC m=+28.824094023"
Jul 13 16:40:42.974369 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:42.974285    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:40:42.975019 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:42.974389    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72twn" podUID="b9ba2c1f-5b34-4948-b18f-661b82e24e08"
Jul 13 16:40:43.156004 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:43.155977    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gdqcb_6a33d4fc-2988-4f88-a92c-212d95ec2060/ovn-acl-logging/0.log"
Jul 13 16:40:43.156415 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:43.156385    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb" event={"ID":"6a33d4fc-2988-4f88-a92c-212d95ec2060","Type":"ContainerStarted","Data":"3c41d77363596fb890528ea770638ccdce12351cc046c6abfc5785d56cd34e40"}
Jul 13 16:40:43.156526 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:43.156481    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:43.156757 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:43.156738    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:43.172476 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:43.172447    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:40:43.193387 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:43.193332    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb" podStartSLOduration=9.64251869 podStartE2EDuration="29.193318354s" totalImagesPullingTime="19.550799664s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 16:40:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:40:43.193068502 +0000 UTC m=+29.734448922" watchObservedRunningTime="2026-07-13 16:40:43.193318354 +0000 UTC m=+29.734698782"
Jul 13 16:40:43.975302 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:43.975270    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:40:43.975746 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:43.975362    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5hxjw" podUID="60216865-6676-407c-bd07-6c87d339c76d"
Jul 13 16:40:43.975746 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:43.975415    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:40:43.975746 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:43.975516    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cx6bw" podUID="5cdf4449-5e1c-453d-b7f0-ae71276fd373"
Jul 13 16:40:44.162343 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:44.162311    2566 generic.go:356] "Generic (PLEG): container finished" podID="650632b5-fb83-42dd-9274-4be5ff84ffe9" containerID="1ee321c46933c4b5c17ab334b3b072c3e740bdfe73c4c53f81ae0b1d19b76002" exitCode=0
Jul 13 16:40:44.162532 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:44.162372    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6lnrp" event={"ID":"650632b5-fb83-42dd-9274-4be5ff84ffe9","Type":"ContainerDied","Data":"1ee321c46933c4b5c17ab334b3b072c3e740bdfe73c4c53f81ae0b1d19b76002"}
Jul 13 16:40:44.974094 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:44.974056    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:40:44.974294 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:44.974163    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72twn" podUID="b9ba2c1f-5b34-4948-b18f-661b82e24e08"
Jul 13 16:40:45.977340 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:45.977310    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:40:45.977713 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:45.977405    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cx6bw" podUID="5cdf4449-5e1c-453d-b7f0-ae71276fd373"
Jul 13 16:40:45.977713 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:45.977460    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:40:45.977713 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:45.977549    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5hxjw" podUID="60216865-6676-407c-bd07-6c87d339c76d"
Jul 13 16:40:46.169619 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:46.169586    2566 generic.go:356] "Generic (PLEG): container finished" podID="650632b5-fb83-42dd-9274-4be5ff84ffe9" containerID="2b3740e6c0d60fc5fe72f8a33c3b67ca95617e5b1322f4de58d7b59c8d244d7d" exitCode=0
Jul 13 16:40:46.169792 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:46.169638    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6lnrp" event={"ID":"650632b5-fb83-42dd-9274-4be5ff84ffe9","Type":"ContainerDied","Data":"2b3740e6c0d60fc5fe72f8a33c3b67ca95617e5b1322f4de58d7b59c8d244d7d"}
Jul 13 16:40:46.974178 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:46.974139    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:40:46.974387 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:46.974289    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72twn" podUID="b9ba2c1f-5b34-4948-b18f-661b82e24e08"
Jul 13 16:40:47.606592 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:47.606549    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5cdf4449-5e1c-453d-b7f0-ae71276fd373-original-pull-secret\") pod \"global-pull-secret-syncer-cx6bw\" (UID: \"5cdf4449-5e1c-453d-b7f0-ae71276fd373\") " pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:40:47.607063 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:47.606701    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 16:40:47.607063 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:47.606782    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cdf4449-5e1c-453d-b7f0-ae71276fd373-original-pull-secret podName:5cdf4449-5e1c-453d-b7f0-ae71276fd373 nodeName:}" failed. No retries permitted until 2026-07-13 16:41:19.606761414 +0000 UTC m=+66.148141823 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5cdf4449-5e1c-453d-b7f0-ae71276fd373-original-pull-secret") pod "global-pull-secret-syncer-cx6bw" (UID: "5cdf4449-5e1c-453d-b7f0-ae71276fd373") : object "kube-system"/"original-pull-secret" not registered
Jul 13 16:40:47.706954 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:47.706915    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60216865-6676-407c-bd07-6c87d339c76d-metrics-certs\") pod \"network-metrics-daemon-5hxjw\" (UID: \"60216865-6676-407c-bd07-6c87d339c76d\") " pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:40:47.707126 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:47.707101    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 16:40:47.707197 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:47.707184    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60216865-6676-407c-bd07-6c87d339c76d-metrics-certs podName:60216865-6676-407c-bd07-6c87d339c76d nodeName:}" failed. No retries permitted until 2026-07-13 16:41:19.70716239 +0000 UTC m=+66.248542801 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/60216865-6676-407c-bd07-6c87d339c76d-metrics-certs") pod "network-metrics-daemon-5hxjw" (UID: "60216865-6676-407c-bd07-6c87d339c76d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 16:40:47.808419 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:47.808376    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92qk7\" (UniqueName: \"kubernetes.io/projected/b9ba2c1f-5b34-4948-b18f-661b82e24e08-kube-api-access-92qk7\") pod \"network-check-target-72twn\" (UID: \"b9ba2c1f-5b34-4948-b18f-661b82e24e08\") " pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:40:47.808604 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:47.808539    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 16:40:47.808604 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:47.808558    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 16:40:47.808604 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:47.808572    2566 projected.go:196] Error preparing data for projected volume kube-api-access-92qk7 for pod openshift-network-diagnostics/network-check-target-72twn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 16:40:47.808742 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:47.808634    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b9ba2c1f-5b34-4948-b18f-661b82e24e08-kube-api-access-92qk7 podName:b9ba2c1f-5b34-4948-b18f-661b82e24e08 nodeName:}" failed. No retries permitted until 2026-07-13 16:41:19.808615766 +0000 UTC m=+66.349996199 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-92qk7" (UniqueName: "kubernetes.io/projected/b9ba2c1f-5b34-4948-b18f-661b82e24e08-kube-api-access-92qk7") pod "network-check-target-72twn" (UID: "b9ba2c1f-5b34-4948-b18f-661b82e24e08") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 16:40:47.975031 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:47.974937    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:40:47.975031 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:47.974958    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:40:47.975267 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:47.975074    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5hxjw" podUID="60216865-6676-407c-bd07-6c87d339c76d"
Jul 13 16:40:47.975267 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:47.975188    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cx6bw" podUID="5cdf4449-5e1c-453d-b7f0-ae71276fd373"
Jul 13 16:40:48.974352 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:48.974311    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:40:48.974774 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:48.974432    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72twn" podUID="b9ba2c1f-5b34-4948-b18f-661b82e24e08"
Jul 13 16:40:49.978152 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:49.978118    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:40:49.978613 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:49.978223    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cx6bw" podUID="5cdf4449-5e1c-453d-b7f0-ae71276fd373"
Jul 13 16:40:49.978613 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:49.978284    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:40:49.978613 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:49.978382    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5hxjw" podUID="60216865-6676-407c-bd07-6c87d339c76d"
Jul 13 16:40:50.974650 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:50.974615    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:40:50.974834 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:50.974720    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72twn" podUID="b9ba2c1f-5b34-4948-b18f-661b82e24e08"
Jul 13 16:40:51.183845 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:51.183807    2566 generic.go:356] "Generic (PLEG): container finished" podID="650632b5-fb83-42dd-9274-4be5ff84ffe9" containerID="4c5248f5ac88cebaea73ab41bc75e50b1279fce924c52089c5d6e7cef580869d" exitCode=0
Jul 13 16:40:51.184205 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:51.183874    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6lnrp" event={"ID":"650632b5-fb83-42dd-9274-4be5ff84ffe9","Type":"ContainerDied","Data":"4c5248f5ac88cebaea73ab41bc75e50b1279fce924c52089c5d6e7cef580869d"}
Jul 13 16:40:51.974806 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:51.974769    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:40:51.974980 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:51.974812    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:40:51.974980 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:51.974887    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5hxjw" podUID="60216865-6676-407c-bd07-6c87d339c76d"
Jul 13 16:40:51.975083 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:51.974989    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cx6bw" podUID="5cdf4449-5e1c-453d-b7f0-ae71276fd373"
Jul 13 16:40:52.189605 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:52.189574    2566 generic.go:356] "Generic (PLEG): container finished" podID="650632b5-fb83-42dd-9274-4be5ff84ffe9" containerID="b5a8ca14e2c632aeca48e4a23dc53b27801a2d5f0a59e2fc95d36425db5f7045" exitCode=0
Jul 13 16:40:52.189990 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:52.189625    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6lnrp" event={"ID":"650632b5-fb83-42dd-9274-4be5ff84ffe9","Type":"ContainerDied","Data":"b5a8ca14e2c632aeca48e4a23dc53b27801a2d5f0a59e2fc95d36425db5f7045"}
Jul 13 16:40:52.974564 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:52.974373    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:40:52.974718 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:52.974636    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72twn" podUID="b9ba2c1f-5b34-4948-b18f-661b82e24e08"
Jul 13 16:40:53.195660 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:53.195626    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6lnrp" event={"ID":"650632b5-fb83-42dd-9274-4be5ff84ffe9","Type":"ContainerStarted","Data":"b002c68aa037c6cb18b01c17bb3fff7279392b45a574f6b19a1740eb6aa2855a"}
Jul 13 16:40:53.225228 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:53.225118    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-6lnrp" podStartSLOduration=8.715317314 podStartE2EDuration="39.225102022s" totalImagesPullingTime="30.509784708s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 16:40:14 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:40:53.224850626 +0000 UTC m=+39.766231055" watchObservedRunningTime="2026-07-13 16:40:53.225102022 +0000 UTC m=+39.766482450"
Jul 13 16:40:53.975265 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:53.975211    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:40:53.975435 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:53.975344    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5hxjw" podUID="60216865-6676-407c-bd07-6c87d339c76d"
Jul 13 16:40:53.975435 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:53.975379    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:40:53.975511 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:53.975447    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cx6bw" podUID="5cdf4449-5e1c-453d-b7f0-ae71276fd373"
Jul 13 16:40:54.974945 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:54.974907    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:40:54.975321 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:54.975010    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72twn" podUID="b9ba2c1f-5b34-4948-b18f-661b82e24e08"
Jul 13 16:40:55.977230 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:55.977192    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:40:55.977674 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:55.977317    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5hxjw" podUID="60216865-6676-407c-bd07-6c87d339c76d"
Jul 13 16:40:55.977674 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:55.977377    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:40:55.977674 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:55.977492    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cx6bw" podUID="5cdf4449-5e1c-453d-b7f0-ae71276fd373"
Jul 13 16:40:56.974986 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:56.974953    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:40:56.975133 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:56.975069    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72twn" podUID="b9ba2c1f-5b34-4948-b18f-661b82e24e08"
Jul 13 16:40:57.974180 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:57.974139    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:40:57.974629 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:57.974140    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:40:57.974629 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:57.974297    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5hxjw" podUID="60216865-6676-407c-bd07-6c87d339c76d"
Jul 13 16:40:57.974629 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:57.974314    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cx6bw" podUID="5cdf4449-5e1c-453d-b7f0-ae71276fd373"
Jul 13 16:40:58.974235 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:58.974195    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:40:58.974647 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:58.974324    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72twn" podUID="b9ba2c1f-5b34-4948-b18f-661b82e24e08"
Jul 13 16:40:59.974335 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:59.974297    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:40:59.974857 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:59.974432    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cx6bw" podUID="5cdf4449-5e1c-453d-b7f0-ae71276fd373"
Jul 13 16:40:59.974857 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:40:59.974490    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:40:59.974857 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:40:59.974642    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5hxjw" podUID="60216865-6676-407c-bd07-6c87d339c76d"
Jul 13 16:41:00.974370 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:00.974332    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:41:00.974749 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:41:00.974441    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72twn" podUID="b9ba2c1f-5b34-4948-b18f-661b82e24e08"
Jul 13 16:41:01.977218 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:01.977189    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:41:01.977589 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:41:01.977313    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cx6bw" podUID="5cdf4449-5e1c-453d-b7f0-ae71276fd373"
Jul 13 16:41:01.977589 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:01.977383    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:41:01.977589 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:41:01.977505    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5hxjw" podUID="60216865-6676-407c-bd07-6c87d339c76d"
Jul 13 16:41:02.974225 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:02.974202    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:41:02.974363 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:41:02.974345    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72twn" podUID="b9ba2c1f-5b34-4948-b18f-661b82e24e08"
Jul 13 16:41:03.059219 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:03.059177    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-72twn"]
Jul 13 16:41:03.063199 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:03.063134    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cx6bw"]
Jul 13 16:41:03.063414 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:03.063268    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:41:03.067500 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:41:03.063634    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cx6bw" podUID="5cdf4449-5e1c-453d-b7f0-ae71276fd373"
Jul 13 16:41:03.067972 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:03.067781    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5hxjw"]
Jul 13 16:41:03.068064 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:03.068009    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:41:03.068168 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:41:03.068141    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5hxjw" podUID="60216865-6676-407c-bd07-6c87d339c76d"
Jul 13 16:41:03.217233 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:03.217205    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:41:03.217421 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:41:03.217328    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72twn" podUID="b9ba2c1f-5b34-4948-b18f-661b82e24e08"
Jul 13 16:41:04.974748 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:04.974713    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:41:04.975130 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:04.974713    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:41:04.975130 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:41:04.974834    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5hxjw" podUID="60216865-6676-407c-bd07-6c87d339c76d"
Jul 13 16:41:04.975130 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:04.974717    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:41:04.975130 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:41:04.974891    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72twn" podUID="b9ba2c1f-5b34-4948-b18f-661b82e24e08"
Jul 13 16:41:04.975130 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:41:04.974980    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cx6bw" podUID="5cdf4449-5e1c-453d-b7f0-ae71276fd373"
Jul 13 16:41:06.974821 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:06.974781    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:41:06.975286 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:06.974791    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:41:06.975286 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:41:06.974886    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-72twn" podUID="b9ba2c1f-5b34-4948-b18f-661b82e24e08"
Jul 13 16:41:06.975286 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:41:06.974958    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cx6bw" podUID="5cdf4449-5e1c-453d-b7f0-ae71276fd373"
Jul 13 16:41:06.975286 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:06.974791    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:41:06.975286 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:41:06.975031    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5hxjw" podUID="60216865-6676-407c-bd07-6c87d339c76d"
Jul 13 16:41:07.318817 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.318561    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-173.ec2.internal" event="NodeReady"
Jul 13 16:41:07.318954 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.318862    2566 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 13 16:41:07.373425 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.373396    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-pmlmx"]
Jul 13 16:41:07.378195 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.378172    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-kcg7j"]
Jul 13 16:41:07.378330 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.378304    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pmlmx"
Jul 13 16:41:07.381172 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.381145    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mgdqr\""
Jul 13 16:41:07.381391 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.381375    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 13 16:41:07.381639 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.381623    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 13 16:41:07.385017 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.384986    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kcg7j"
Jul 13 16:41:07.387666 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.387645    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 13 16:41:07.387843 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.387829    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 13 16:41:07.388466 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.388445    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-76zsq\""
Jul 13 16:41:07.388889 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.388871    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 13 16:41:07.389761 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.389745    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pmlmx"]
Jul 13 16:41:07.393932 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.393878    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kcg7j"]
Jul 13 16:41:07.449265 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.449224    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d2910d0-a6df-45ed-87ba-991eb6179dde-metrics-tls\") pod \"dns-default-pmlmx\" (UID: \"0d2910d0-a6df-45ed-87ba-991eb6179dde\") " pod="openshift-dns/dns-default-pmlmx"
Jul 13 16:41:07.449451 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.449301    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfhd8\" (UniqueName: \"kubernetes.io/projected/0d2910d0-a6df-45ed-87ba-991eb6179dde-kube-api-access-mfhd8\") pod \"dns-default-pmlmx\" (UID: \"0d2910d0-a6df-45ed-87ba-991eb6179dde\") " pod="openshift-dns/dns-default-pmlmx"
Jul 13 16:41:07.449451 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.449326    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/62d5929c-f868-4354-b47a-4ae2b62e1362-cert\") pod \"ingress-canary-kcg7j\" (UID: \"62d5929c-f868-4354-b47a-4ae2b62e1362\") " pod="openshift-ingress-canary/ingress-canary-kcg7j"
Jul 13 16:41:07.449451 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.449356    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5vxb\" (UniqueName: \"kubernetes.io/projected/62d5929c-f868-4354-b47a-4ae2b62e1362-kube-api-access-n5vxb\") pod \"ingress-canary-kcg7j\" (UID: \"62d5929c-f868-4354-b47a-4ae2b62e1362\") " pod="openshift-ingress-canary/ingress-canary-kcg7j"
Jul 13 16:41:07.449451 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.449374    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0d2910d0-a6df-45ed-87ba-991eb6179dde-config-volume\") pod \"dns-default-pmlmx\" (UID: \"0d2910d0-a6df-45ed-87ba-991eb6179dde\") " pod="openshift-dns/dns-default-pmlmx"
Jul 13 16:41:07.449451 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.449400    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d2910d0-a6df-45ed-87ba-991eb6179dde-tmp-dir\") pod \"dns-default-pmlmx\" (UID: \"0d2910d0-a6df-45ed-87ba-991eb6179dde\") " pod="openshift-dns/dns-default-pmlmx"
Jul 13 16:41:07.478425 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.478392    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-798sv"]
Jul 13 16:41:07.492731 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.492700    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-798sv"]
Jul 13 16:41:07.492868 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.492831    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-798sv"
Jul 13 16:41:07.496084 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.496051    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 13 16:41:07.496084 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.496069    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 13 16:41:07.496275 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.496069    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 13 16:41:07.496275 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.496171    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 13 16:41:07.496403 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.496387    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ntdf5\""
Jul 13 16:41:07.549646 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.549607    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0d2910d0-a6df-45ed-87ba-991eb6179dde-config-volume\") pod \"dns-default-pmlmx\" (UID: \"0d2910d0-a6df-45ed-87ba-991eb6179dde\") " pod="openshift-dns/dns-default-pmlmx"
Jul 13 16:41:07.549831 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.549657    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2f8a3a0f-9ff7-4a01-b0b2-c9edac2e964e-crio-socket\") pod \"insights-runtime-extractor-798sv\" (UID: \"2f8a3a0f-9ff7-4a01-b0b2-c9edac2e964e\") " pod="openshift-insights/insights-runtime-extractor-798sv"
Jul 13 16:41:07.549831 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.549685    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2f8a3a0f-9ff7-4a01-b0b2-c9edac2e964e-data-volume\") pod \"insights-runtime-extractor-798sv\" (UID: \"2f8a3a0f-9ff7-4a01-b0b2-c9edac2e964e\") " pod="openshift-insights/insights-runtime-extractor-798sv"
Jul 13 16:41:07.549831 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.549711    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d2910d0-a6df-45ed-87ba-991eb6179dde-tmp-dir\") pod \"dns-default-pmlmx\" (UID: \"0d2910d0-a6df-45ed-87ba-991eb6179dde\") " pod="openshift-dns/dns-default-pmlmx"
Jul 13 16:41:07.549831 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.549734    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2f8a3a0f-9ff7-4a01-b0b2-c9edac2e964e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-798sv\" (UID: \"2f8a3a0f-9ff7-4a01-b0b2-c9edac2e964e\") " pod="openshift-insights/insights-runtime-extractor-798sv"
Jul 13 16:41:07.549831 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.549756    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d2910d0-a6df-45ed-87ba-991eb6179dde-metrics-tls\") pod \"dns-default-pmlmx\" (UID: \"0d2910d0-a6df-45ed-87ba-991eb6179dde\") " pod="openshift-dns/dns-default-pmlmx"
Jul 13 16:41:07.550063 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.549905    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfhd8\" (UniqueName: \"kubernetes.io/projected/0d2910d0-a6df-45ed-87ba-991eb6179dde-kube-api-access-mfhd8\") pod \"dns-default-pmlmx\" (UID: \"0d2910d0-a6df-45ed-87ba-991eb6179dde\") " pod="openshift-dns/dns-default-pmlmx"
Jul 13 16:41:07.550063 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.549969    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/62d5929c-f868-4354-b47a-4ae2b62e1362-cert\") pod \"ingress-canary-kcg7j\" (UID: \"62d5929c-f868-4354-b47a-4ae2b62e1362\") " pod="openshift-ingress-canary/ingress-canary-kcg7j"
Jul 13 16:41:07.550063 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.550014    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d2910d0-a6df-45ed-87ba-991eb6179dde-tmp-dir\") pod \"dns-default-pmlmx\" (UID: \"0d2910d0-a6df-45ed-87ba-991eb6179dde\") " pod="openshift-dns/dns-default-pmlmx"
Jul 13 16:41:07.550063 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.550023    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdfzf\" (UniqueName: \"kubernetes.io/projected/2f8a3a0f-9ff7-4a01-b0b2-c9edac2e964e-kube-api-access-sdfzf\") pod \"insights-runtime-extractor-798sv\" (UID: \"2f8a3a0f-9ff7-4a01-b0b2-c9edac2e964e\") " pod="openshift-insights/insights-runtime-extractor-798sv"
Jul 13 16:41:07.550063 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.550055    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2f8a3a0f-9ff7-4a01-b0b2-c9edac2e964e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-798sv\" (UID: \"2f8a3a0f-9ff7-4a01-b0b2-c9edac2e964e\") " pod="openshift-insights/insights-runtime-extractor-798sv"
Jul 13 16:41:07.550302 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.550085    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5vxb\" (UniqueName: \"kubernetes.io/projected/62d5929c-f868-4354-b47a-4ae2b62e1362-kube-api-access-n5vxb\") pod \"ingress-canary-kcg7j\" (UID: \"62d5929c-f868-4354-b47a-4ae2b62e1362\") " pod="openshift-ingress-canary/ingress-canary-kcg7j"
Jul 13 16:41:07.550342 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.550297    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0d2910d0-a6df-45ed-87ba-991eb6179dde-config-volume\") pod \"dns-default-pmlmx\" (UID: \"0d2910d0-a6df-45ed-87ba-991eb6179dde\") " pod="openshift-dns/dns-default-pmlmx"
Jul 13 16:41:07.553791 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.553765    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d2910d0-a6df-45ed-87ba-991eb6179dde-metrics-tls\") pod \"dns-default-pmlmx\" (UID: \"0d2910d0-a6df-45ed-87ba-991eb6179dde\") " pod="openshift-dns/dns-default-pmlmx"
Jul 13 16:41:07.554139 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.554124    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/62d5929c-f868-4354-b47a-4ae2b62e1362-cert\") pod \"ingress-canary-kcg7j\" (UID: \"62d5929c-f868-4354-b47a-4ae2b62e1362\") " pod="openshift-ingress-canary/ingress-canary-kcg7j"
Jul 13 16:41:07.565911 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.565881    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfhd8\" (UniqueName: \"kubernetes.io/projected/0d2910d0-a6df-45ed-87ba-991eb6179dde-kube-api-access-mfhd8\") pod \"dns-default-pmlmx\" (UID: \"0d2910d0-a6df-45ed-87ba-991eb6179dde\") " pod="openshift-dns/dns-default-pmlmx"
Jul 13 16:41:07.566016 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.565947    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5vxb\" (UniqueName: \"kubernetes.io/projected/62d5929c-f868-4354-b47a-4ae2b62e1362-kube-api-access-n5vxb\") pod \"ingress-canary-kcg7j\" (UID: \"62d5929c-f868-4354-b47a-4ae2b62e1362\") " pod="openshift-ingress-canary/ingress-canary-kcg7j"
Jul 13 16:41:07.651161 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.651071    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2f8a3a0f-9ff7-4a01-b0b2-c9edac2e964e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-798sv\" (UID: \"2f8a3a0f-9ff7-4a01-b0b2-c9edac2e964e\") " pod="openshift-insights/insights-runtime-extractor-798sv"
Jul 13 16:41:07.651161 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.651140    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdfzf\" (UniqueName: \"kubernetes.io/projected/2f8a3a0f-9ff7-4a01-b0b2-c9edac2e964e-kube-api-access-sdfzf\") pod \"insights-runtime-extractor-798sv\" (UID: \"2f8a3a0f-9ff7-4a01-b0b2-c9edac2e964e\") " pod="openshift-insights/insights-runtime-extractor-798sv"
Jul 13 16:41:07.651161 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.651158    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2f8a3a0f-9ff7-4a01-b0b2-c9edac2e964e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-798sv\" (UID: \"2f8a3a0f-9ff7-4a01-b0b2-c9edac2e964e\") " pod="openshift-insights/insights-runtime-extractor-798sv"
Jul 13 16:41:07.651467 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.651182    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2f8a3a0f-9ff7-4a01-b0b2-c9edac2e964e-crio-socket\") pod \"insights-runtime-extractor-798sv\" (UID: \"2f8a3a0f-9ff7-4a01-b0b2-c9edac2e964e\") " pod="openshift-insights/insights-runtime-extractor-798sv"
Jul 13 16:41:07.651467 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.651199    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2f8a3a0f-9ff7-4a01-b0b2-c9edac2e964e-data-volume\") pod \"insights-runtime-extractor-798sv\" (UID: \"2f8a3a0f-9ff7-4a01-b0b2-c9edac2e964e\") " pod="openshift-insights/insights-runtime-extractor-798sv"
Jul 13 16:41:07.651467 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.651406    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2f8a3a0f-9ff7-4a01-b0b2-c9edac2e964e-crio-socket\") pod \"insights-runtime-extractor-798sv\" (UID: \"2f8a3a0f-9ff7-4a01-b0b2-c9edac2e964e\") " pod="openshift-insights/insights-runtime-extractor-798sv"
Jul 13 16:41:07.651626 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.651553    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2f8a3a0f-9ff7-4a01-b0b2-c9edac2e964e-data-volume\") pod \"insights-runtime-extractor-798sv\" (UID: \"2f8a3a0f-9ff7-4a01-b0b2-c9edac2e964e\") " pod="openshift-insights/insights-runtime-extractor-798sv"
Jul 13 16:41:07.651686 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.651630    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2f8a3a0f-9ff7-4a01-b0b2-c9edac2e964e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-798sv\" (UID: \"2f8a3a0f-9ff7-4a01-b0b2-c9edac2e964e\") " pod="openshift-insights/insights-runtime-extractor-798sv"
Jul 13 16:41:07.653470 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.653446    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2f8a3a0f-9ff7-4a01-b0b2-c9edac2e964e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-798sv\" (UID: \"2f8a3a0f-9ff7-4a01-b0b2-c9edac2e964e\") " pod="openshift-insights/insights-runtime-extractor-798sv"
Jul 13 16:41:07.660298 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.660273    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdfzf\" (UniqueName: \"kubernetes.io/projected/2f8a3a0f-9ff7-4a01-b0b2-c9edac2e964e-kube-api-access-sdfzf\") pod \"insights-runtime-extractor-798sv\" (UID: \"2f8a3a0f-9ff7-4a01-b0b2-c9edac2e964e\") " pod="openshift-insights/insights-runtime-extractor-798sv"
Jul 13 16:41:07.688333 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.688284    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pmlmx"
Jul 13 16:41:07.695375 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.695349    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kcg7j"
Jul 13 16:41:07.774656 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.774625    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-gnnw7"]
Jul 13 16:41:07.798977 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.798950    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-gnnw7"]
Jul 13 16:41:07.799108 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.799038    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-gnnw7"
Jul 13 16:41:07.801732 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.801681    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-798sv"
Jul 13 16:41:07.802152 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.802135    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-cfrbb\""
Jul 13 16:41:07.802278 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.802238    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 13 16:41:07.852521 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.852452    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a38eb5f0-9cea-4996-891d-a86a96d4d88c-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-gnnw7\" (UID: \"a38eb5f0-9cea-4996-891d-a86a96d4d88c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-gnnw7"
Jul 13 16:41:07.916064 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.916035    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pmlmx"]
Jul 13 16:41:07.919076 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.919055    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kcg7j"]
Jul 13 16:41:07.920085 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:41:07.920056    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d2910d0_a6df_45ed_87ba_991eb6179dde.slice/crio-f1d027de6c721faaa809f2aef4b40325384a58df79a9578e5d83dcf447a7338b WatchSource:0}: Error finding container f1d027de6c721faaa809f2aef4b40325384a58df79a9578e5d83dcf447a7338b: Status 404 returned error can't find the container with id f1d027de6c721faaa809f2aef4b40325384a58df79a9578e5d83dcf447a7338b
Jul 13 16:41:07.921508 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:41:07.921485    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62d5929c_f868_4354_b47a_4ae2b62e1362.slice/crio-e3335b5f17cf00741e359826bfcf420667319ea6fce38dc53aa963f37746ac77 WatchSource:0}: Error finding container e3335b5f17cf00741e359826bfcf420667319ea6fce38dc53aa963f37746ac77: Status 404 returned error can't find the container with id e3335b5f17cf00741e359826bfcf420667319ea6fce38dc53aa963f37746ac77
Jul 13 16:41:07.953160 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.953107    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a38eb5f0-9cea-4996-891d-a86a96d4d88c-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-gnnw7\" (UID: \"a38eb5f0-9cea-4996-891d-a86a96d4d88c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-gnnw7"
Jul 13 16:41:07.953893 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.953864    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-798sv"]
Jul 13 16:41:07.956109 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:07.956087    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a38eb5f0-9cea-4996-891d-a86a96d4d88c-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-gnnw7\" (UID: \"a38eb5f0-9cea-4996-891d-a86a96d4d88c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-gnnw7"
Jul 13 16:41:07.956598 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:41:07.956575    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f8a3a0f_9ff7_4a01_b0b2_c9edac2e964e.slice/crio-0f518fda60e0007967cfa067bd01b240cd24120294ad0f70b11fd67ebaa257b8 WatchSource:0}: Error finding container 0f518fda60e0007967cfa067bd01b240cd24120294ad0f70b11fd67ebaa257b8: Status 404 returned error can't find the container with id 0f518fda60e0007967cfa067bd01b240cd24120294ad0f70b11fd67ebaa257b8
Jul 13 16:41:08.107940 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.107904    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-gnnw7"
Jul 13 16:41:08.229112 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.229074    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-798sv" event={"ID":"2f8a3a0f-9ff7-4a01-b0b2-c9edac2e964e","Type":"ContainerStarted","Data":"0f518fda60e0007967cfa067bd01b240cd24120294ad0f70b11fd67ebaa257b8"}
Jul 13 16:41:08.230217 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.230191    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kcg7j" event={"ID":"62d5929c-f868-4354-b47a-4ae2b62e1362","Type":"ContainerStarted","Data":"e3335b5f17cf00741e359826bfcf420667319ea6fce38dc53aa963f37746ac77"}
Jul 13 16:41:08.231295 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.231271    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pmlmx" event={"ID":"0d2910d0-a6df-45ed-87ba-991eb6179dde","Type":"ContainerStarted","Data":"f1d027de6c721faaa809f2aef4b40325384a58df79a9578e5d83dcf447a7338b"}
Jul 13 16:41:08.260930 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.260896    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-gnnw7"]
Jul 13 16:41:08.264318 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:41:08.264293    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda38eb5f0_9cea_4996_891d_a86a96d4d88c.slice/crio-f7780b11389f0aadfc1bf798ef1e6cf490a5d13a2350e27d1fdacd3900df30b5 WatchSource:0}: Error finding container f7780b11389f0aadfc1bf798ef1e6cf490a5d13a2350e27d1fdacd3900df30b5: Status 404 returned error can't find the container with id f7780b11389f0aadfc1bf798ef1e6cf490a5d13a2350e27d1fdacd3900df30b5
Jul 13 16:41:08.299742 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.299711    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6488dc44d4-jjscb"]
Jul 13 16:41:08.335497 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.335463    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6488dc44d4-jjscb"]
Jul 13 16:41:08.335681 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.335597    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6488dc44d4-jjscb"
Jul 13 16:41:08.338864 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.338825    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 13 16:41:08.339025 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.338872    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 13 16:41:08.339025 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.338977    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 13 16:41:08.339147 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.339123    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 13 16:41:08.339147 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.339140    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 13 16:41:08.339275 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.339128    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 13 16:41:08.339700 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.339686    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 13 16:41:08.340317 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.340304    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-8bczz\""
Jul 13 16:41:08.344635 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.344615    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 13 16:41:08.356421 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.356397    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-console-serving-cert\") pod \"console-6488dc44d4-jjscb\" (UID: \"3fa9699a-a5bc-4ebe-9f00-2506acddeb7c\") " pod="openshift-console/console-6488dc44d4-jjscb"
Jul 13 16:41:08.356553 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.356437    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-console-oauth-config\") pod \"console-6488dc44d4-jjscb\" (UID: \"3fa9699a-a5bc-4ebe-9f00-2506acddeb7c\") " pod="openshift-console/console-6488dc44d4-jjscb"
Jul 13 16:41:08.356553 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.356462    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-oauth-serving-cert\") pod \"console-6488dc44d4-jjscb\" (UID: \"3fa9699a-a5bc-4ebe-9f00-2506acddeb7c\") " pod="openshift-console/console-6488dc44d4-jjscb"
Jul 13 16:41:08.356619 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.356549    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-service-ca\") pod \"console-6488dc44d4-jjscb\" (UID: \"3fa9699a-a5bc-4ebe-9f00-2506acddeb7c\") " pod="openshift-console/console-6488dc44d4-jjscb"
Jul 13 16:41:08.356619 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.356573    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbwcz\" (UniqueName: \"kubernetes.io/projected/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-kube-api-access-mbwcz\") pod \"console-6488dc44d4-jjscb\" (UID: \"3fa9699a-a5bc-4ebe-9f00-2506acddeb7c\") " pod="openshift-console/console-6488dc44d4-jjscb"
Jul 13 16:41:08.356619 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.356612    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-console-config\") pod \"console-6488dc44d4-jjscb\" (UID: \"3fa9699a-a5bc-4ebe-9f00-2506acddeb7c\") " pod="openshift-console/console-6488dc44d4-jjscb"
Jul 13 16:41:08.356706 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.356630    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-trusted-ca-bundle\") pod \"console-6488dc44d4-jjscb\" (UID: \"3fa9699a-a5bc-4ebe-9f00-2506acddeb7c\") " pod="openshift-console/console-6488dc44d4-jjscb"
Jul 13 16:41:08.457996 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.457904    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-service-ca\") pod \"console-6488dc44d4-jjscb\" (UID: \"3fa9699a-a5bc-4ebe-9f00-2506acddeb7c\") " pod="openshift-console/console-6488dc44d4-jjscb"
Jul 13 16:41:08.457996 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.457940    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbwcz\" (UniqueName: \"kubernetes.io/projected/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-kube-api-access-mbwcz\") pod \"console-6488dc44d4-jjscb\" (UID: \"3fa9699a-a5bc-4ebe-9f00-2506acddeb7c\") " pod="openshift-console/console-6488dc44d4-jjscb"
Jul 13 16:41:08.458230 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.457991    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-console-config\") pod \"console-6488dc44d4-jjscb\" (UID: \"3fa9699a-a5bc-4ebe-9f00-2506acddeb7c\") " pod="openshift-console/console-6488dc44d4-jjscb"
Jul 13 16:41:08.458230 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.458049    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-trusted-ca-bundle\") pod \"console-6488dc44d4-jjscb\" (UID: \"3fa9699a-a5bc-4ebe-9f00-2506acddeb7c\") " pod="openshift-console/console-6488dc44d4-jjscb"
Jul 13 16:41:08.458230 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.458082    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-console-serving-cert\") pod \"console-6488dc44d4-jjscb\" (UID: \"3fa9699a-a5bc-4ebe-9f00-2506acddeb7c\") " pod="openshift-console/console-6488dc44d4-jjscb"
Jul 13 16:41:08.458230 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.458136    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-console-oauth-config\") pod \"console-6488dc44d4-jjscb\" (UID: \"3fa9699a-a5bc-4ebe-9f00-2506acddeb7c\") " pod="openshift-console/console-6488dc44d4-jjscb"
Jul 13 16:41:08.458230 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.458168    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-oauth-serving-cert\") pod \"console-6488dc44d4-jjscb\" (UID: \"3fa9699a-a5bc-4ebe-9f00-2506acddeb7c\") " pod="openshift-console/console-6488dc44d4-jjscb"
Jul 13 16:41:08.458935 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.458860    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-service-ca\") pod \"console-6488dc44d4-jjscb\" (UID: \"3fa9699a-a5bc-4ebe-9f00-2506acddeb7c\") " pod="openshift-console/console-6488dc44d4-jjscb"
Jul 13 16:41:08.458935 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.458864    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-console-config\") pod \"console-6488dc44d4-jjscb\" (UID: \"3fa9699a-a5bc-4ebe-9f00-2506acddeb7c\") " pod="openshift-console/console-6488dc44d4-jjscb"
Jul 13 16:41:08.459124 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.459014    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-oauth-serving-cert\") pod \"console-6488dc44d4-jjscb\" (UID: \"3fa9699a-a5bc-4ebe-9f00-2506acddeb7c\") " pod="openshift-console/console-6488dc44d4-jjscb"
Jul 13 16:41:08.459280 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.459233    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-trusted-ca-bundle\") pod \"console-6488dc44d4-jjscb\" (UID: \"3fa9699a-a5bc-4ebe-9f00-2506acddeb7c\") " pod="openshift-console/console-6488dc44d4-jjscb"
Jul 13 16:41:08.462296 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.462269    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-console-oauth-config\") pod \"console-6488dc44d4-jjscb\" (UID: \"3fa9699a-a5bc-4ebe-9f00-2506acddeb7c\") " pod="openshift-console/console-6488dc44d4-jjscb"
Jul 13 16:41:08.462490 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.462472    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-console-serving-cert\") pod \"console-6488dc44d4-jjscb\" (UID: \"3fa9699a-a5bc-4ebe-9f00-2506acddeb7c\") " pod="openshift-console/console-6488dc44d4-jjscb"
Jul 13 16:41:08.466962 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.466933    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbwcz\" (UniqueName: \"kubernetes.io/projected/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-kube-api-access-mbwcz\") pod \"console-6488dc44d4-jjscb\" (UID: \"3fa9699a-a5bc-4ebe-9f00-2506acddeb7c\") " pod="openshift-console/console-6488dc44d4-jjscb"
Jul 13 16:41:08.645839 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.645638    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6488dc44d4-jjscb"
Jul 13 16:41:08.819843 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.819783    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6488dc44d4-jjscb"]
Jul 13 16:41:08.826609 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:41:08.824699    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fa9699a_a5bc_4ebe_9f00_2506acddeb7c.slice/crio-8e11f3de2ce9e2b5e5a4325db988afa99c72a8442bc9316269ce3ee7d864cae7 WatchSource:0}: Error finding container 8e11f3de2ce9e2b5e5a4325db988afa99c72a8442bc9316269ce3ee7d864cae7: Status 404 returned error can't find the container with id 8e11f3de2ce9e2b5e5a4325db988afa99c72a8442bc9316269ce3ee7d864cae7
Jul 13 16:41:08.974201 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.974117    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:41:08.974389 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.974119    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:41:08.974389 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.974127    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:41:08.981578 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.978839    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 13 16:41:08.981578 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.978912    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 13 16:41:08.981578 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.979141    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-whvdd\""
Jul 13 16:41:08.981578 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.979366    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-79qpb\""
Jul 13 16:41:08.981578 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.979648    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 13 16:41:08.981578 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:08.979845    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 13 16:41:09.236744 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:09.236621    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6488dc44d4-jjscb" event={"ID":"3fa9699a-a5bc-4ebe-9f00-2506acddeb7c","Type":"ContainerStarted","Data":"8e11f3de2ce9e2b5e5a4325db988afa99c72a8442bc9316269ce3ee7d864cae7"}
Jul 13 16:41:09.238459 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:09.238377    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-gnnw7" event={"ID":"a38eb5f0-9cea-4996-891d-a86a96d4d88c","Type":"ContainerStarted","Data":"f7780b11389f0aadfc1bf798ef1e6cf490a5d13a2350e27d1fdacd3900df30b5"}
Jul 13 16:41:15.182529 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:15.182480    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gdqcb"
Jul 13 16:41:15.259786 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:15.259744    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-gnnw7" event={"ID":"a38eb5f0-9cea-4996-891d-a86a96d4d88c","Type":"ContainerStarted","Data":"fa8eb5869630ff7a49b04d3f23a83f97699902c8bda06bc500c9d6e01bda8c54"}
Jul 13 16:41:15.259965 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:15.259818    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-gnnw7"
Jul 13 16:41:15.261491 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:15.261462    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pmlmx" event={"ID":"0d2910d0-a6df-45ed-87ba-991eb6179dde","Type":"ContainerStarted","Data":"512eaa5e868553744af88dccf7622d12aaf8536dfe6a0cc8a5bc8bf1e62dac4d"}
Jul 13 16:41:15.261623 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:15.261515    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pmlmx" event={"ID":"0d2910d0-a6df-45ed-87ba-991eb6179dde","Type":"ContainerStarted","Data":"5d0543899798138d2e825b89068369653636746989bb64a44587a9c7029fe0a4"}
Jul 13 16:41:15.261623 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:15.261575    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-pmlmx"
Jul 13 16:41:15.263096 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:15.263065    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6488dc44d4-jjscb" event={"ID":"3fa9699a-a5bc-4ebe-9f00-2506acddeb7c","Type":"ContainerStarted","Data":"7e711291f30a3e465c12feecf332f9fd1b5eaad2830431de567f58783449d9e7"}
Jul 13 16:41:15.264795 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:15.264744    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kcg7j" event={"ID":"62d5929c-f868-4354-b47a-4ae2b62e1362","Type":"ContainerStarted","Data":"de23a8f93ec108885290dac57d67ce57f1056286d125cff24d3d2d6a5691513f"}
Jul 13 16:41:15.265930 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:15.265909    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-gnnw7"
Jul 13 16:41:15.294043 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:15.294001    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-gnnw7" podStartSLOduration=1.6739335770000001 podStartE2EDuration="8.293986129s" totalImagesPullingTime="6.620052552s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:41:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:41:15.293732776 +0000 UTC m=+61.835113206" watchObservedRunningTime="2026-07-13 16:41:15.293986129 +0000 UTC m=+61.835366557"
Jul 13 16:41:15.340534 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:15.340476    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6488dc44d4-jjscb" podStartSLOduration=1.281019999 podStartE2EDuration="7.34046069s" totalImagesPullingTime="6.059440691s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:41:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:41:15.336985196 +0000 UTC m=+61.878365639" watchObservedRunningTime="2026-07-13 16:41:15.34046069 +0000 UTC m=+61.881841118"
Jul 13 16:41:15.365551 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:15.365500    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-kcg7j" podStartSLOduration=1.4027180750000001 podStartE2EDuration="8.365486283s" totalImagesPullingTime="6.962768208s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:41:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:41:15.365096937 +0000 UTC m=+61.906477381" watchObservedRunningTime="2026-07-13 16:41:15.365486283 +0000 UTC m=+61.906866711"
Jul 13 16:41:15.401105 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:15.401003    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-pmlmx" podStartSLOduration=1.832379767 podStartE2EDuration="8.400985764s" totalImagesPullingTime="6.568605997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:41:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:41:15.400862709 +0000 UTC m=+61.942243139" watchObservedRunningTime="2026-07-13 16:41:15.400985764 +0000 UTC m=+61.942366192"
Jul 13 16:41:15.842631 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:15.842586    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-zw42s"]
Jul 13 16:41:15.844651 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:15.844625    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-zw42s"
Jul 13 16:41:15.847821 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:15.847794    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-bbmgd\""
Jul 13 16:41:15.847821 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:15.847808    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 13 16:41:15.847962 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:15.847834    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 13 16:41:15.847962 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:15.847809    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 13 16:41:15.847962 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:15.847945    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 13 16:41:15.848087 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:15.848000    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 13 16:41:15.857029 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:15.856997    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-zw42s"]
Jul 13 16:41:15.925154 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:15.925108    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9d54ffd4-e8a2-4745-bfc5-bdcb7abd45b2-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-zw42s\" (UID: \"9d54ffd4-e8a2-4745-bfc5-bdcb7abd45b2\") " pod="openshift-monitoring/prometheus-operator-64785b6675-zw42s"
Jul 13 16:41:15.925378 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:15.925213    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jprxm\" (UniqueName: \"kubernetes.io/projected/9d54ffd4-e8a2-4745-bfc5-bdcb7abd45b2-kube-api-access-jprxm\") pod \"prometheus-operator-64785b6675-zw42s\" (UID: \"9d54ffd4-e8a2-4745-bfc5-bdcb7abd45b2\") " pod="openshift-monitoring/prometheus-operator-64785b6675-zw42s"
Jul 13 16:41:15.925378 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:15.925323    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d54ffd4-e8a2-4745-bfc5-bdcb7abd45b2-metrics-client-ca\") pod \"prometheus-operator-64785b6675-zw42s\" (UID: \"9d54ffd4-e8a2-4745-bfc5-bdcb7abd45b2\") " pod="openshift-monitoring/prometheus-operator-64785b6675-zw42s"
Jul 13 16:41:15.925378 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:15.925368    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9d54ffd4-e8a2-4745-bfc5-bdcb7abd45b2-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-zw42s\" (UID: \"9d54ffd4-e8a2-4745-bfc5-bdcb7abd45b2\") " pod="openshift-monitoring/prometheus-operator-64785b6675-zw42s"
Jul 13 16:41:16.026012 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:16.025988    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jprxm\" (UniqueName: \"kubernetes.io/projected/9d54ffd4-e8a2-4745-bfc5-bdcb7abd45b2-kube-api-access-jprxm\") pod \"prometheus-operator-64785b6675-zw42s\" (UID: \"9d54ffd4-e8a2-4745-bfc5-bdcb7abd45b2\") " pod="openshift-monitoring/prometheus-operator-64785b6675-zw42s"
Jul 13 16:41:16.026105 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:16.026057    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d54ffd4-e8a2-4745-bfc5-bdcb7abd45b2-metrics-client-ca\") pod \"prometheus-operator-64785b6675-zw42s\" (UID: \"9d54ffd4-e8a2-4745-bfc5-bdcb7abd45b2\") " pod="openshift-monitoring/prometheus-operator-64785b6675-zw42s"
Jul 13 16:41:16.026105 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:16.026090    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9d54ffd4-e8a2-4745-bfc5-bdcb7abd45b2-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-zw42s\" (UID: \"9d54ffd4-e8a2-4745-bfc5-bdcb7abd45b2\") " pod="openshift-monitoring/prometheus-operator-64785b6675-zw42s"
Jul 13 16:41:16.026178 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:16.026130    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9d54ffd4-e8a2-4745-bfc5-bdcb7abd45b2-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-zw42s\" (UID: \"9d54ffd4-e8a2-4745-bfc5-bdcb7abd45b2\") " pod="openshift-monitoring/prometheus-operator-64785b6675-zw42s"
Jul 13 16:41:16.026730 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:16.026707    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d54ffd4-e8a2-4745-bfc5-bdcb7abd45b2-metrics-client-ca\") pod \"prometheus-operator-64785b6675-zw42s\" (UID: \"9d54ffd4-e8a2-4745-bfc5-bdcb7abd45b2\") " pod="openshift-monitoring/prometheus-operator-64785b6675-zw42s"
Jul 13 16:41:16.028682 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:16.028663    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9d54ffd4-e8a2-4745-bfc5-bdcb7abd45b2-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-zw42s\" (UID: \"9d54ffd4-e8a2-4745-bfc5-bdcb7abd45b2\") " pod="openshift-monitoring/prometheus-operator-64785b6675-zw42s"
Jul 13 16:41:16.028805 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:16.028772    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9d54ffd4-e8a2-4745-bfc5-bdcb7abd45b2-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-zw42s\" (UID: \"9d54ffd4-e8a2-4745-bfc5-bdcb7abd45b2\") " pod="openshift-monitoring/prometheus-operator-64785b6675-zw42s"
Jul 13 16:41:16.034462 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:16.034435    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jprxm\" (UniqueName: \"kubernetes.io/projected/9d54ffd4-e8a2-4745-bfc5-bdcb7abd45b2-kube-api-access-jprxm\") pod \"prometheus-operator-64785b6675-zw42s\" (UID: \"9d54ffd4-e8a2-4745-bfc5-bdcb7abd45b2\") " pod="openshift-monitoring/prometheus-operator-64785b6675-zw42s"
Jul 13 16:41:16.156398 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:16.156311    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-zw42s"
Jul 13 16:41:16.270323 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:16.270283    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-798sv" event={"ID":"2f8a3a0f-9ff7-4a01-b0b2-c9edac2e964e","Type":"ContainerStarted","Data":"ee14fc5b28809f5383090c93331442b1ee63f3f8f636c76ec431dbf2232cdb00"}
Jul 13 16:41:16.270765 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:16.270356    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-798sv" event={"ID":"2f8a3a0f-9ff7-4a01-b0b2-c9edac2e964e","Type":"ContainerStarted","Data":"1f8d5577edce382d76539659effd16e7d02776cece1d21a5230ed3a1e2f618a2"}
Jul 13 16:41:16.279705 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:16.279676    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-zw42s"]
Jul 13 16:41:16.282345 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:41:16.282319    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d54ffd4_e8a2_4745_bfc5_bdcb7abd45b2.slice/crio-780c444a54c5d65a68e8e22a1b0807d75864ce8d9e3a7a541a25eae5de75178b WatchSource:0}: Error finding container 780c444a54c5d65a68e8e22a1b0807d75864ce8d9e3a7a541a25eae5de75178b: Status 404 returned error can't find the container with id 780c444a54c5d65a68e8e22a1b0807d75864ce8d9e3a7a541a25eae5de75178b
Jul 13 16:41:17.276227 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:17.276129    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-zw42s" event={"ID":"9d54ffd4-e8a2-4745-bfc5-bdcb7abd45b2","Type":"ContainerStarted","Data":"780c444a54c5d65a68e8e22a1b0807d75864ce8d9e3a7a541a25eae5de75178b"}
Jul 13 16:41:18.646521 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:18.646468    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6488dc44d4-jjscb"
Jul 13 16:41:18.646946 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:18.646540    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6488dc44d4-jjscb"
Jul 13 16:41:18.651947 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:18.651690    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6488dc44d4-jjscb"
Jul 13 16:41:19.287233 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:19.287200    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-798sv" event={"ID":"2f8a3a0f-9ff7-4a01-b0b2-c9edac2e964e","Type":"ContainerStarted","Data":"1456f9a162aa9fef8da7a5653d8bdb412b43623e1503a5749d66c578d9c7aabd"}
Jul 13 16:41:19.288897 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:19.288868    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-zw42s" event={"ID":"9d54ffd4-e8a2-4745-bfc5-bdcb7abd45b2","Type":"ContainerStarted","Data":"b267b2bb057fa4b3e8ad2fdfaf1283a86a2e80b7076a255834340d49facd56cf"}
Jul 13 16:41:19.289027 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:19.288914    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-zw42s" event={"ID":"9d54ffd4-e8a2-4745-bfc5-bdcb7abd45b2","Type":"ContainerStarted","Data":"3bb845b275e1e65145a0eb9cd68ef51e21100f37bf7ee96976514d722ecb72e5"}
Jul 13 16:41:19.292751 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:19.292730    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6488dc44d4-jjscb"
Jul 13 16:41:19.307302 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:19.307238    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-798sv" podStartSLOduration=1.744732157 podStartE2EDuration="12.307222125s" totalImagesPullingTime="10.562489968s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 16:41:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:41:19.306610491 +0000 UTC m=+65.847990919" watchObservedRunningTime="2026-07-13 16:41:19.307222125 +0000 UTC m=+65.848602553"
Jul 13 16:41:19.343005 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:19.342944    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-zw42s" podStartSLOduration=1.997960046 podStartE2EDuration="4.342926638s" totalImagesPullingTime="2.344966592s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:41:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:41:19.342094879 +0000 UTC m=+65.883475310" watchObservedRunningTime="2026-07-13 16:41:19.342926638 +0000 UTC m=+65.884307067"
Jul 13 16:41:19.658305 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:19.658172    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5cdf4449-5e1c-453d-b7f0-ae71276fd373-original-pull-secret\") pod \"global-pull-secret-syncer-cx6bw\" (UID: \"5cdf4449-5e1c-453d-b7f0-ae71276fd373\") " pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:41:19.661274 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:19.661239    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 13 16:41:19.671139 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:19.671108    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5cdf4449-5e1c-453d-b7f0-ae71276fd373-original-pull-secret\") pod \"global-pull-secret-syncer-cx6bw\" (UID: \"5cdf4449-5e1c-453d-b7f0-ae71276fd373\") " pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:41:19.759476 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:19.759416    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60216865-6676-407c-bd07-6c87d339c76d-metrics-certs\") pod \"network-metrics-daemon-5hxjw\" (UID: \"60216865-6676-407c-bd07-6c87d339c76d\") " pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:41:19.762401 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:19.762375    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 13 16:41:19.772021 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:19.771983    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60216865-6676-407c-bd07-6c87d339c76d-metrics-certs\") pod \"network-metrics-daemon-5hxjw\" (UID: \"60216865-6676-407c-bd07-6c87d339c76d\") " pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:41:19.797062 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:19.797026    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-79qpb\""
Jul 13 16:41:19.804397 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:19.804371    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5hxjw"
Jul 13 16:41:19.804545 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:19.804411    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cx6bw"
Jul 13 16:41:19.860915 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:19.859930    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92qk7\" (UniqueName: \"kubernetes.io/projected/b9ba2c1f-5b34-4948-b18f-661b82e24e08-kube-api-access-92qk7\") pod \"network-check-target-72twn\" (UID: \"b9ba2c1f-5b34-4948-b18f-661b82e24e08\") " pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:41:19.863183 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:19.863153    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 13 16:41:19.874004 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:19.873404    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 13 16:41:19.883581 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:19.883553    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92qk7\" (UniqueName: \"kubernetes.io/projected/b9ba2c1f-5b34-4948-b18f-661b82e24e08-kube-api-access-92qk7\") pod \"network-check-target-72twn\" (UID: \"b9ba2c1f-5b34-4948-b18f-661b82e24e08\") " pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:41:19.942503 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:19.942387    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5hxjw"]
Jul 13 16:41:19.945042 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:41:19.945014    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60216865_6676_407c_bd07_6c87d339c76d.slice/crio-707c573623d0db416e8099fa943e20a890e866eee36d43edddbfcb2bdbbf8d32 WatchSource:0}: Error finding container 707c573623d0db416e8099fa943e20a890e866eee36d43edddbfcb2bdbbf8d32: Status 404 returned error can't find the container with id 707c573623d0db416e8099fa943e20a890e866eee36d43edddbfcb2bdbbf8d32
Jul 13 16:41:19.957904 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:19.957880    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cx6bw"]
Jul 13 16:41:19.960447 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:41:19.960417    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cdf4449_5e1c_453d_b7f0_ae71276fd373.slice/crio-87adab827533bdf645e992f98acee1d84b560b7c86a8acd308a0a1526371f6f1 WatchSource:0}: Error finding container 87adab827533bdf645e992f98acee1d84b560b7c86a8acd308a0a1526371f6f1: Status 404 returned error can't find the container with id 87adab827533bdf645e992f98acee1d84b560b7c86a8acd308a0a1526371f6f1
Jul 13 16:41:20.121357 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:20.121330    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-whvdd\""
Jul 13 16:41:20.128025 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:20.127988    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:41:20.252737 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:20.252660    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-72twn"]
Jul 13 16:41:20.256424 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:41:20.256390    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9ba2c1f_5b34_4948_b18f_661b82e24e08.slice/crio-42e7b3f23baf1833b44743844aac5988582718bd59ff6e28bff9fd124a75121d WatchSource:0}: Error finding container 42e7b3f23baf1833b44743844aac5988582718bd59ff6e28bff9fd124a75121d: Status 404 returned error can't find the container with id 42e7b3f23baf1833b44743844aac5988582718bd59ff6e28bff9fd124a75121d
Jul 13 16:41:20.293657 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:20.293619    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cx6bw" event={"ID":"5cdf4449-5e1c-453d-b7f0-ae71276fd373","Type":"ContainerStarted","Data":"87adab827533bdf645e992f98acee1d84b560b7c86a8acd308a0a1526371f6f1"}
Jul 13 16:41:20.294700 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:20.294668    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-72twn" event={"ID":"b9ba2c1f-5b34-4948-b18f-661b82e24e08","Type":"ContainerStarted","Data":"42e7b3f23baf1833b44743844aac5988582718bd59ff6e28bff9fd124a75121d"}
Jul 13 16:41:20.295827 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:20.295803    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5hxjw" event={"ID":"60216865-6676-407c-bd07-6c87d339c76d","Type":"ContainerStarted","Data":"707c573623d0db416e8099fa943e20a890e866eee36d43edddbfcb2bdbbf8d32"}
Jul 13 16:41:21.195290 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.195238    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-mmkvq"]
Jul 13 16:41:21.219454 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.219400    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5m2pj"]
Jul 13 16:41:21.219769 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.219738    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mmkvq"
Jul 13 16:41:21.224196 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.223943    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-mjn6l\""
Jul 13 16:41:21.224196 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.223960    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 13 16:41:21.225266 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.225038    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 13 16:41:21.235318 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.235238    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-mmkvq"]
Jul 13 16:41:21.235318 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.235320    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-m6wbp"]
Jul 13 16:41:21.235538 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.235367    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5m2pj"
Jul 13 16:41:21.240921 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.240871    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 13 16:41:21.241092 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.240984    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bgr75\""
Jul 13 16:41:21.241092 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.241036    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 13 16:41:21.241203 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.241105    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 13 16:41:21.257119 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.257058    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-m6wbp"]
Jul 13 16:41:21.257294 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.257203    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-m6wbp"
Jul 13 16:41:21.260785 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.260759    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 13 16:41:21.260934 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.260819    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 13 16:41:21.260934 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.260771    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-6lhxb\""
Jul 13 16:41:21.261271 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.261083    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 13 16:41:21.271499 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.271465    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/79f7dffb-624b-4874-be76-01f1a6d59a2d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-mmkvq\" (UID: \"79f7dffb-624b-4874-be76-01f1a6d59a2d\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mmkvq"
Jul 13 16:41:21.271650 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.271516    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/79f7dffb-624b-4874-be76-01f1a6d59a2d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-mmkvq\" (UID: \"79f7dffb-624b-4874-be76-01f1a6d59a2d\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mmkvq"
Jul 13 16:41:21.271650 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.271547    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bwrc\" (UniqueName: \"kubernetes.io/projected/79f7dffb-624b-4874-be76-01f1a6d59a2d-kube-api-access-5bwrc\") pod \"openshift-state-metrics-84bd68b896-mmkvq\" (UID: \"79f7dffb-624b-4874-be76-01f1a6d59a2d\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mmkvq"
Jul 13 16:41:21.271650 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.271584    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/79f7dffb-624b-4874-be76-01f1a6d59a2d-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-mmkvq\" (UID: \"79f7dffb-624b-4874-be76-01f1a6d59a2d\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mmkvq"
Jul 13 16:41:21.372229 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.372181    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/79f7dffb-624b-4874-be76-01f1a6d59a2d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-mmkvq\" (UID: \"79f7dffb-624b-4874-be76-01f1a6d59a2d\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mmkvq"
Jul 13 16:41:21.372419 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.372263    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zrnk\" (UniqueName: \"kubernetes.io/projected/d757b55d-630e-4ecb-8380-1925f6da02d5-kube-api-access-4zrnk\") pod \"kube-state-metrics-785778ff74-m6wbp\" (UID: \"d757b55d-630e-4ecb-8380-1925f6da02d5\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-m6wbp"
Jul 13 16:41:21.372419 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.372291    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899-sys\") pod \"node-exporter-5m2pj\" (UID: \"4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899\") " pod="openshift-monitoring/node-exporter-5m2pj"
Jul 13 16:41:21.372419 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.372320    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bwrc\" (UniqueName: \"kubernetes.io/projected/79f7dffb-624b-4874-be76-01f1a6d59a2d-kube-api-access-5bwrc\") pod \"openshift-state-metrics-84bd68b896-mmkvq\" (UID: \"79f7dffb-624b-4874-be76-01f1a6d59a2d\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mmkvq"
Jul 13 16:41:21.372419 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.372345    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d757b55d-630e-4ecb-8380-1925f6da02d5-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-m6wbp\" (UID: \"d757b55d-630e-4ecb-8380-1925f6da02d5\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-m6wbp"
Jul 13 16:41:21.372419 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.372371    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d757b55d-630e-4ecb-8380-1925f6da02d5-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-m6wbp\" (UID: \"d757b55d-630e-4ecb-8380-1925f6da02d5\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-m6wbp"
Jul 13 16:41:21.372746 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.372430    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899-node-exporter-tls\") pod \"node-exporter-5m2pj\" (UID: \"4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899\") " pod="openshift-monitoring/node-exporter-5m2pj"
Jul 13 16:41:21.372746 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.372474    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899-metrics-client-ca\") pod \"node-exporter-5m2pj\" (UID: \"4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899\") " pod="openshift-monitoring/node-exporter-5m2pj"
Jul 13 16:41:21.372746 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.372543    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/79f7dffb-624b-4874-be76-01f1a6d59a2d-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-mmkvq\" (UID: \"79f7dffb-624b-4874-be76-01f1a6d59a2d\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mmkvq"
Jul 13 16:41:21.372746 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.372591    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5m2pj\" (UID: \"4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899\") " pod="openshift-monitoring/node-exporter-5m2pj"
Jul 13 16:41:21.372746 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.372623    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d757b55d-630e-4ecb-8380-1925f6da02d5-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-m6wbp\" (UID: \"d757b55d-630e-4ecb-8380-1925f6da02d5\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-m6wbp"
Jul 13 16:41:21.372746 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.372677    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d757b55d-630e-4ecb-8380-1925f6da02d5-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-m6wbp\" (UID: \"d757b55d-630e-4ecb-8380-1925f6da02d5\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-m6wbp"
Jul 13 16:41:21.372746 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.372719    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899-node-exporter-accelerators-collector-config\") pod \"node-exporter-5m2pj\" (UID: \"4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899\") " pod="openshift-monitoring/node-exporter-5m2pj"
Jul 13 16:41:21.373124 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.372761    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899-node-exporter-textfile\") pod \"node-exporter-5m2pj\" (UID: \"4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899\") " pod="openshift-monitoring/node-exporter-5m2pj"
Jul 13 16:41:21.373124 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.372788    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d757b55d-630e-4ecb-8380-1925f6da02d5-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-m6wbp\" (UID: \"d757b55d-630e-4ecb-8380-1925f6da02d5\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-m6wbp"
Jul 13 16:41:21.373124 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.372813    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899-root\") pod \"node-exporter-5m2pj\" (UID: \"4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899\") " pod="openshift-monitoring/node-exporter-5m2pj"
Jul 13 16:41:21.373124 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.372835    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899-node-exporter-wtmp\") pod \"node-exporter-5m2pj\" (UID: \"4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899\") " pod="openshift-monitoring/node-exporter-5m2pj"
Jul 13 16:41:21.373124 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.372882    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/79f7dffb-624b-4874-be76-01f1a6d59a2d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-mmkvq\" (UID: \"79f7dffb-624b-4874-be76-01f1a6d59a2d\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mmkvq"
Jul 13 16:41:21.373124 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.372911    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tz7q\" (UniqueName: \"kubernetes.io/projected/4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899-kube-api-access-4tz7q\") pod \"node-exporter-5m2pj\" (UID: \"4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899\") " pod="openshift-monitoring/node-exporter-5m2pj"
Jul 13 16:41:21.373124 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:41:21.372976    2566 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 13 16:41:21.373124 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:41:21.373053    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79f7dffb-624b-4874-be76-01f1a6d59a2d-openshift-state-metrics-tls podName:79f7dffb-624b-4874-be76-01f1a6d59a2d nodeName:}" failed. No retries permitted until 2026-07-13 16:41:21.873031971 +0000 UTC m=+68.414412393 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/79f7dffb-624b-4874-be76-01f1a6d59a2d-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-mmkvq" (UID: "79f7dffb-624b-4874-be76-01f1a6d59a2d") : secret "openshift-state-metrics-tls" not found
Jul 13 16:41:21.373581 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.373331    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/79f7dffb-624b-4874-be76-01f1a6d59a2d-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-mmkvq\" (UID: \"79f7dffb-624b-4874-be76-01f1a6d59a2d\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mmkvq"
Jul 13 16:41:21.375413 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.375380    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/79f7dffb-624b-4874-be76-01f1a6d59a2d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-mmkvq\" (UID: \"79f7dffb-624b-4874-be76-01f1a6d59a2d\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mmkvq"
Jul 13 16:41:21.382182 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.382127    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bwrc\" (UniqueName: \"kubernetes.io/projected/79f7dffb-624b-4874-be76-01f1a6d59a2d-kube-api-access-5bwrc\") pod \"openshift-state-metrics-84bd68b896-mmkvq\" (UID: \"79f7dffb-624b-4874-be76-01f1a6d59a2d\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mmkvq"
Jul 13 16:41:21.473728 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.473631    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tz7q\" (UniqueName: \"kubernetes.io/projected/4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899-kube-api-access-4tz7q\") pod \"node-exporter-5m2pj\" (UID: \"4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899\") " pod="openshift-monitoring/node-exporter-5m2pj"
Jul 13 16:41:21.473728 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.473698    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zrnk\" (UniqueName: \"kubernetes.io/projected/d757b55d-630e-4ecb-8380-1925f6da02d5-kube-api-access-4zrnk\") pod \"kube-state-metrics-785778ff74-m6wbp\" (UID: \"d757b55d-630e-4ecb-8380-1925f6da02d5\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-m6wbp"
Jul 13 16:41:21.473728 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.473722    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899-sys\") pod \"node-exporter-5m2pj\" (UID: \"4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899\") " pod="openshift-monitoring/node-exporter-5m2pj"
Jul 13 16:41:21.473982 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.473750    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d757b55d-630e-4ecb-8380-1925f6da02d5-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-m6wbp\" (UID: \"d757b55d-630e-4ecb-8380-1925f6da02d5\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-m6wbp"
Jul 13 16:41:21.473982 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.473774    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d757b55d-630e-4ecb-8380-1925f6da02d5-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-m6wbp\" (UID: \"d757b55d-630e-4ecb-8380-1925f6da02d5\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-m6wbp"
Jul 13 16:41:21.473982 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.473802    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899-node-exporter-tls\") pod \"node-exporter-5m2pj\" (UID: \"4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899\") " pod="openshift-monitoring/node-exporter-5m2pj"
Jul 13 16:41:21.473982 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.473825    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899-metrics-client-ca\") pod \"node-exporter-5m2pj\" (UID: \"4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899\") " pod="openshift-monitoring/node-exporter-5m2pj"
Jul 13 16:41:21.473982 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.473861    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5m2pj\" (UID: \"4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899\") " pod="openshift-monitoring/node-exporter-5m2pj"
Jul 13 16:41:21.473982 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.473892    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d757b55d-630e-4ecb-8380-1925f6da02d5-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-m6wbp\" (UID: \"d757b55d-630e-4ecb-8380-1925f6da02d5\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-m6wbp"
Jul 13 16:41:21.473982 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.473924    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d757b55d-630e-4ecb-8380-1925f6da02d5-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-m6wbp\" (UID: \"d757b55d-630e-4ecb-8380-1925f6da02d5\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-m6wbp"
Jul 13 16:41:21.473982 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.473966    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899-node-exporter-accelerators-collector-config\") pod \"node-exporter-5m2pj\" (UID: \"4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899\") " pod="openshift-monitoring/node-exporter-5m2pj"
Jul 13 16:41:21.474360 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.474007    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899-node-exporter-textfile\") pod \"node-exporter-5m2pj\" (UID: \"4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899\") " pod="openshift-monitoring/node-exporter-5m2pj"
Jul 13 16:41:21.474360 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.474033    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d757b55d-630e-4ecb-8380-1925f6da02d5-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-m6wbp\" (UID: \"d757b55d-630e-4ecb-8380-1925f6da02d5\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-m6wbp"
Jul 13 16:41:21.474360 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.474056    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899-root\") pod \"node-exporter-5m2pj\" (UID: \"4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899\") " pod="openshift-monitoring/node-exporter-5m2pj"
Jul 13 16:41:21.474360 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.474078    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899-node-exporter-wtmp\") pod \"node-exporter-5m2pj\" (UID: \"4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899\") " pod="openshift-monitoring/node-exporter-5m2pj"
Jul 13 16:41:21.474360 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.474273    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899-node-exporter-wtmp\") pod \"node-exporter-5m2pj\" (UID: \"4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899\") " pod="openshift-monitoring/node-exporter-5m2pj"
Jul 13 16:41:21.474976 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.474945    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899-metrics-client-ca\") pod \"node-exporter-5m2pj\" (UID: \"4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899\") " pod="openshift-monitoring/node-exporter-5m2pj"
Jul 13 16:41:21.475790 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.475197    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899-node-exporter-textfile\") pod \"node-exporter-5m2pj\" (UID: \"4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899\") " pod="openshift-monitoring/node-exporter-5m2pj"
Jul 13 16:41:21.475790 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.474951    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d757b55d-630e-4ecb-8380-1925f6da02d5-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-m6wbp\" (UID: \"d757b55d-630e-4ecb-8380-1925f6da02d5\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-m6wbp"
Jul 13 16:41:21.475790 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.475221    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d757b55d-630e-4ecb-8380-1925f6da02d5-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-m6wbp\" (UID: \"d757b55d-630e-4ecb-8380-1925f6da02d5\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-m6wbp"
Jul 13 16:41:21.475790 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.475283    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899-root\") pod \"node-exporter-5m2pj\" (UID: \"4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899\") " pod="openshift-monitoring/node-exporter-5m2pj"
Jul 13 16:41:21.475790 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.475330    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899-sys\") pod \"node-exporter-5m2pj\" (UID: \"4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899\") " pod="openshift-monitoring/node-exporter-5m2pj"
Jul 13 16:41:21.475790 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.475751    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899-node-exporter-accelerators-collector-config\") pod \"node-exporter-5m2pj\" (UID: \"4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899\") " pod="openshift-monitoring/node-exporter-5m2pj"
Jul 13 16:41:21.476290 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.476211    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d757b55d-630e-4ecb-8380-1925f6da02d5-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-m6wbp\" (UID: \"d757b55d-630e-4ecb-8380-1925f6da02d5\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-m6wbp"
Jul 13 16:41:21.477779 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.477736    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d757b55d-630e-4ecb-8380-1925f6da02d5-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-m6wbp\" (UID: \"d757b55d-630e-4ecb-8380-1925f6da02d5\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-m6wbp"
Jul 13 16:41:21.478709 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.478561    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899-node-exporter-tls\") pod \"node-exporter-5m2pj\" (UID: \"4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899\") " pod="openshift-monitoring/node-exporter-5m2pj"
Jul 13 16:41:21.478975 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.478927    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5m2pj\" (UID: \"4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899\") " pod="openshift-monitoring/node-exporter-5m2pj"
Jul 13 16:41:21.480297 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.480268    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d757b55d-630e-4ecb-8380-1925f6da02d5-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-m6wbp\" (UID: \"d757b55d-630e-4ecb-8380-1925f6da02d5\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-m6wbp"
Jul 13 16:41:21.483081 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.483011    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tz7q\" (UniqueName: \"kubernetes.io/projected/4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899-kube-api-access-4tz7q\") pod \"node-exporter-5m2pj\" (UID: \"4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899\") " pod="openshift-monitoring/node-exporter-5m2pj"
Jul 13 16:41:21.483081 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.483027    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zrnk\" (UniqueName: \"kubernetes.io/projected/d757b55d-630e-4ecb-8380-1925f6da02d5-kube-api-access-4zrnk\") pod \"kube-state-metrics-785778ff74-m6wbp\" (UID: \"d757b55d-630e-4ecb-8380-1925f6da02d5\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-m6wbp"
Jul 13 16:41:21.547256 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.547203    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5m2pj"
Jul 13 16:41:21.570402 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.570365    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-m6wbp"
Jul 13 16:41:21.588615 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:41:21.588581    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4249a8a0_d7f7_4ac1_8653_8f8dc7ceb899.slice/crio-45c3d3d3887dfebe81267d9275aa1bdda12e5025f0d3c0ad0a074a76a1f39531 WatchSource:0}: Error finding container 45c3d3d3887dfebe81267d9275aa1bdda12e5025f0d3c0ad0a074a76a1f39531: Status 404 returned error can't find the container with id 45c3d3d3887dfebe81267d9275aa1bdda12e5025f0d3c0ad0a074a76a1f39531
Jul 13 16:41:21.772215 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.772028    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-m6wbp"]
Jul 13 16:41:21.776306 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:41:21.776275    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd757b55d_630e_4ecb_8380_1925f6da02d5.slice/crio-911f10bcc151fae4376d3b762400f088e7dce19719563a2669267e88ce2ada6c WatchSource:0}: Error finding container 911f10bcc151fae4376d3b762400f088e7dce19719563a2669267e88ce2ada6c: Status 404 returned error can't find the container with id 911f10bcc151fae4376d3b762400f088e7dce19719563a2669267e88ce2ada6c
Jul 13 16:41:21.879838 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.879800    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/79f7dffb-624b-4874-be76-01f1a6d59a2d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-mmkvq\" (UID: \"79f7dffb-624b-4874-be76-01f1a6d59a2d\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mmkvq"
Jul 13 16:41:21.882658 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:21.882618    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/79f7dffb-624b-4874-be76-01f1a6d59a2d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-mmkvq\" (UID: \"79f7dffb-624b-4874-be76-01f1a6d59a2d\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mmkvq"
Jul 13 16:41:22.134048 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.133936    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mmkvq"
Jul 13 16:41:22.303758 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.303721    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 13 16:41:22.330602 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.329532    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 13 16:41:22.330602 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.329604    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-m6wbp" event={"ID":"d757b55d-630e-4ecb-8380-1925f6da02d5","Type":"ContainerStarted","Data":"911f10bcc151fae4376d3b762400f088e7dce19719563a2669267e88ce2ada6c"}
Jul 13 16:41:22.330602 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.329649    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5m2pj" event={"ID":"4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899","Type":"ContainerStarted","Data":"45c3d3d3887dfebe81267d9275aa1bdda12e5025f0d3c0ad0a074a76a1f39531"}
Jul 13 16:41:22.330602 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.329792    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:22.332938 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.332778    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 13 16:41:22.333807 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.333226    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 13 16:41:22.335781 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.334277    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 13 16:41:22.335781 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.334541    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 13 16:41:22.335781 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.335006    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 13 16:41:22.335781 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.335134    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 13 16:41:22.335781 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.335218    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 13 16:41:22.335781 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.335435    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-pc8dh\""
Jul 13 16:41:22.335781 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.335501    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 13 16:41:22.335781 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.335435    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 13 16:41:22.385105 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.384868    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76kzv\" (UniqueName: \"kubernetes.io/projected/6a578cc2-bbf1-4f21-86fa-980c61491d7b-kube-api-access-76kzv\") pod \"alertmanager-main-0\" (UID: \"6a578cc2-bbf1-4f21-86fa-980c61491d7b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:22.385105 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.384948    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6a578cc2-bbf1-4f21-86fa-980c61491d7b-config-out\") pod \"alertmanager-main-0\" (UID: \"6a578cc2-bbf1-4f21-86fa-980c61491d7b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:22.385105 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.384981    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6a578cc2-bbf1-4f21-86fa-980c61491d7b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6a578cc2-bbf1-4f21-86fa-980c61491d7b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:22.385105 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.385004    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6a578cc2-bbf1-4f21-86fa-980c61491d7b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6a578cc2-bbf1-4f21-86fa-980c61491d7b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:22.385105 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.385035    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6a578cc2-bbf1-4f21-86fa-980c61491d7b-config-volume\") pod \"alertmanager-main-0\" (UID: \"6a578cc2-bbf1-4f21-86fa-980c61491d7b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:22.385105 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.385059    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6a578cc2-bbf1-4f21-86fa-980c61491d7b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6a578cc2-bbf1-4f21-86fa-980c61491d7b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:22.385105 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.385096    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6a578cc2-bbf1-4f21-86fa-980c61491d7b-web-config\") pod \"alertmanager-main-0\" (UID: \"6a578cc2-bbf1-4f21-86fa-980c61491d7b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:22.385581 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.385144    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6a578cc2-bbf1-4f21-86fa-980c61491d7b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6a578cc2-bbf1-4f21-86fa-980c61491d7b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:22.385581 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.385167    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6a578cc2-bbf1-4f21-86fa-980c61491d7b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6a578cc2-bbf1-4f21-86fa-980c61491d7b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:22.385581 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.385189    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6a578cc2-bbf1-4f21-86fa-980c61491d7b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6a578cc2-bbf1-4f21-86fa-980c61491d7b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:22.385581 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.385213    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6a578cc2-bbf1-4f21-86fa-980c61491d7b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6a578cc2-bbf1-4f21-86fa-980c61491d7b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:22.385581 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.385241    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6a578cc2-bbf1-4f21-86fa-980c61491d7b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6a578cc2-bbf1-4f21-86fa-980c61491d7b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:22.385581 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.385295    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a578cc2-bbf1-4f21-86fa-980c61491d7b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6a578cc2-bbf1-4f21-86fa-980c61491d7b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:22.486736 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.486697    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6a578cc2-bbf1-4f21-86fa-980c61491d7b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6a578cc2-bbf1-4f21-86fa-980c61491d7b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:22.486935 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.486742    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6a578cc2-bbf1-4f21-86fa-980c61491d7b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6a578cc2-bbf1-4f21-86fa-980c61491d7b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:22.486935 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.486773    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6a578cc2-bbf1-4f21-86fa-980c61491d7b-config-volume\") pod \"alertmanager-main-0\" (UID: \"6a578cc2-bbf1-4f21-86fa-980c61491d7b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:22.486935 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.486797    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6a578cc2-bbf1-4f21-86fa-980c61491d7b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6a578cc2-bbf1-4f21-86fa-980c61491d7b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:22.486935 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.486833    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6a578cc2-bbf1-4f21-86fa-980c61491d7b-web-config\") pod \"alertmanager-main-0\" (UID: \"6a578cc2-bbf1-4f21-86fa-980c61491d7b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:22.486935 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.486873    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6a578cc2-bbf1-4f21-86fa-980c61491d7b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6a578cc2-bbf1-4f21-86fa-980c61491d7b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:22.486935 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.486892    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6a578cc2-bbf1-4f21-86fa-980c61491d7b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6a578cc2-bbf1-4f21-86fa-980c61491d7b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:22.486935 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.486917    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6a578cc2-bbf1-4f21-86fa-980c61491d7b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6a578cc2-bbf1-4f21-86fa-980c61491d7b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:22.487284 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.486950    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6a578cc2-bbf1-4f21-86fa-980c61491d7b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6a578cc2-bbf1-4f21-86fa-980c61491d7b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:22.487284 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.486978    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6a578cc2-bbf1-4f21-86fa-980c61491d7b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6a578cc2-bbf1-4f21-86fa-980c61491d7b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:22.487284 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.487000    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a578cc2-bbf1-4f21-86fa-980c61491d7b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6a578cc2-bbf1-4f21-86fa-980c61491d7b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:22.487284 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.487021    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76kzv\" (UniqueName: \"kubernetes.io/projected/6a578cc2-bbf1-4f21-86fa-980c61491d7b-kube-api-access-76kzv\") pod \"alertmanager-main-0\" (UID: \"6a578cc2-bbf1-4f21-86fa-980c61491d7b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:22.487284 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.487054    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6a578cc2-bbf1-4f21-86fa-980c61491d7b-config-out\") pod \"alertmanager-main-0\" (UID: \"6a578cc2-bbf1-4f21-86fa-980c61491d7b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:22.487543 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:41:22.487316    2566 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 13 16:41:22.487543 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:41:22.487407    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a578cc2-bbf1-4f21-86fa-980c61491d7b-secret-alertmanager-main-tls podName:6a578cc2-bbf1-4f21-86fa-980c61491d7b nodeName:}" failed. No retries permitted until 2026-07-13 16:41:22.987383843 +0000 UTC m=+69.528764264 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/6a578cc2-bbf1-4f21-86fa-980c61491d7b-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "6a578cc2-bbf1-4f21-86fa-980c61491d7b") : secret "alertmanager-main-tls" not found
Jul 13 16:41:22.488714 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.488635    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a578cc2-bbf1-4f21-86fa-980c61491d7b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6a578cc2-bbf1-4f21-86fa-980c61491d7b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:22.488869 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.488756    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6a578cc2-bbf1-4f21-86fa-980c61491d7b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6a578cc2-bbf1-4f21-86fa-980c61491d7b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:22.488869 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.488811    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6a578cc2-bbf1-4f21-86fa-980c61491d7b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6a578cc2-bbf1-4f21-86fa-980c61491d7b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:22.490418 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.490368    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6a578cc2-bbf1-4f21-86fa-980c61491d7b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6a578cc2-bbf1-4f21-86fa-980c61491d7b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:22.490583 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.490534    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6a578cc2-bbf1-4f21-86fa-980c61491d7b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6a578cc2-bbf1-4f21-86fa-980c61491d7b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:22.491074 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.490915    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6a578cc2-bbf1-4f21-86fa-980c61491d7b-config-out\") pod \"alertmanager-main-0\" (UID: \"6a578cc2-bbf1-4f21-86fa-980c61491d7b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:22.491074 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.491016    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6a578cc2-bbf1-4f21-86fa-980c61491d7b-web-config\") pod \"alertmanager-main-0\" (UID: \"6a578cc2-bbf1-4f21-86fa-980c61491d7b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:22.491074 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.491036    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6a578cc2-bbf1-4f21-86fa-980c61491d7b-config-volume\") pod \"alertmanager-main-0\" (UID: \"6a578cc2-bbf1-4f21-86fa-980c61491d7b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:22.491074 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.491055    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6a578cc2-bbf1-4f21-86fa-980c61491d7b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6a578cc2-bbf1-4f21-86fa-980c61491d7b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:22.491526 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.491507    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6a578cc2-bbf1-4f21-86fa-980c61491d7b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6a578cc2-bbf1-4f21-86fa-980c61491d7b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:22.491685 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.491661    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6a578cc2-bbf1-4f21-86fa-980c61491d7b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6a578cc2-bbf1-4f21-86fa-980c61491d7b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:22.496189 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.496165    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76kzv\" (UniqueName: \"kubernetes.io/projected/6a578cc2-bbf1-4f21-86fa-980c61491d7b-kube-api-access-76kzv\") pod \"alertmanager-main-0\" (UID: \"6a578cc2-bbf1-4f21-86fa-980c61491d7b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:22.991673 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.991640    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6a578cc2-bbf1-4f21-86fa-980c61491d7b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6a578cc2-bbf1-4f21-86fa-980c61491d7b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:22.995200 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:22.995166    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6a578cc2-bbf1-4f21-86fa-980c61491d7b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6a578cc2-bbf1-4f21-86fa-980c61491d7b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:23.247658 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:23.247566    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 13 16:41:23.306786 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:23.306746    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-686f4d8bc6-d6fv8"]
Jul 13 16:41:23.309520 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:23.309477    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-686f4d8bc6-d6fv8"
Jul 13 16:41:23.312562 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:23.312533    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 13 16:41:23.312562 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:23.312549    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 13 16:41:23.312765 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:23.312610    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 13 16:41:23.312883 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:23.312865    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-2u57u0chdtpj\""
Jul 13 16:41:23.312883 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:23.312875    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 13 16:41:23.313031 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:23.312883    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 13 16:41:23.313031 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:23.312894    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-w2c4h\""
Jul 13 16:41:23.321803 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:23.321777    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-686f4d8bc6-d6fv8"]
Jul 13 16:41:23.395330 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:23.395280    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c4959268-4527-4c5f-aae4-6d47b89c3410-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-686f4d8bc6-d6fv8\" (UID: \"c4959268-4527-4c5f-aae4-6d47b89c3410\") " pod="openshift-monitoring/thanos-querier-686f4d8bc6-d6fv8"
Jul 13 16:41:23.395531 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:23.395335    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c4959268-4527-4c5f-aae4-6d47b89c3410-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-686f4d8bc6-d6fv8\" (UID: \"c4959268-4527-4c5f-aae4-6d47b89c3410\") " pod="openshift-monitoring/thanos-querier-686f4d8bc6-d6fv8"
Jul 13 16:41:23.395531 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:23.395381    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c4959268-4527-4c5f-aae4-6d47b89c3410-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-686f4d8bc6-d6fv8\" (UID: \"c4959268-4527-4c5f-aae4-6d47b89c3410\") " pod="openshift-monitoring/thanos-querier-686f4d8bc6-d6fv8"
Jul 13 16:41:23.395531 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:23.395469    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c4959268-4527-4c5f-aae4-6d47b89c3410-secret-grpc-tls\") pod \"thanos-querier-686f4d8bc6-d6fv8\" (UID: \"c4959268-4527-4c5f-aae4-6d47b89c3410\") " pod="openshift-monitoring/thanos-querier-686f4d8bc6-d6fv8"
Jul 13 16:41:23.395699 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:23.395541    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c4959268-4527-4c5f-aae4-6d47b89c3410-secret-thanos-querier-tls\") pod \"thanos-querier-686f4d8bc6-d6fv8\" (UID: \"c4959268-4527-4c5f-aae4-6d47b89c3410\") " pod="openshift-monitoring/thanos-querier-686f4d8bc6-d6fv8"
Jul 13 16:41:23.395699 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:23.395584    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7pdz\" (UniqueName: \"kubernetes.io/projected/c4959268-4527-4c5f-aae4-6d47b89c3410-kube-api-access-x7pdz\") pod \"thanos-querier-686f4d8bc6-d6fv8\" (UID: \"c4959268-4527-4c5f-aae4-6d47b89c3410\") " pod="openshift-monitoring/thanos-querier-686f4d8bc6-d6fv8"
Jul 13 16:41:23.395699 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:23.395636    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c4959268-4527-4c5f-aae4-6d47b89c3410-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-686f4d8bc6-d6fv8\" (UID: \"c4959268-4527-4c5f-aae4-6d47b89c3410\") " pod="openshift-monitoring/thanos-querier-686f4d8bc6-d6fv8"
Jul 13 16:41:23.395828 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:23.395701    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c4959268-4527-4c5f-aae4-6d47b89c3410-metrics-client-ca\") pod \"thanos-querier-686f4d8bc6-d6fv8\" (UID: \"c4959268-4527-4c5f-aae4-6d47b89c3410\") " pod="openshift-monitoring/thanos-querier-686f4d8bc6-d6fv8"
Jul 13 16:41:23.496580 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:23.496540    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c4959268-4527-4c5f-aae4-6d47b89c3410-metrics-client-ca\") pod \"thanos-querier-686f4d8bc6-d6fv8\" (UID: \"c4959268-4527-4c5f-aae4-6d47b89c3410\") " pod="openshift-monitoring/thanos-querier-686f4d8bc6-d6fv8"
Jul 13 16:41:23.496580 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:23.496586    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c4959268-4527-4c5f-aae4-6d47b89c3410-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-686f4d8bc6-d6fv8\" (UID: \"c4959268-4527-4c5f-aae4-6d47b89c3410\") " pod="openshift-monitoring/thanos-querier-686f4d8bc6-d6fv8"
Jul 13 16:41:23.496828 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:23.496614    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c4959268-4527-4c5f-aae4-6d47b89c3410-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-686f4d8bc6-d6fv8\" (UID: \"c4959268-4527-4c5f-aae4-6d47b89c3410\") " pod="openshift-monitoring/thanos-querier-686f4d8bc6-d6fv8"
Jul 13 16:41:23.496828 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:23.496653    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c4959268-4527-4c5f-aae4-6d47b89c3410-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-686f4d8bc6-d6fv8\" (UID: \"c4959268-4527-4c5f-aae4-6d47b89c3410\") " pod="openshift-monitoring/thanos-querier-686f4d8bc6-d6fv8"
Jul 13 16:41:23.496828 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:23.496714    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c4959268-4527-4c5f-aae4-6d47b89c3410-secret-grpc-tls\") pod \"thanos-querier-686f4d8bc6-d6fv8\" (UID: \"c4959268-4527-4c5f-aae4-6d47b89c3410\") " pod="openshift-monitoring/thanos-querier-686f4d8bc6-d6fv8"
Jul 13 16:41:23.496828 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:23.496739    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c4959268-4527-4c5f-aae4-6d47b89c3410-secret-thanos-querier-tls\") pod \"thanos-querier-686f4d8bc6-d6fv8\" (UID: \"c4959268-4527-4c5f-aae4-6d47b89c3410\") " pod="openshift-monitoring/thanos-querier-686f4d8bc6-d6fv8"
Jul 13 16:41:23.496828 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:23.496763    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7pdz\" (UniqueName: \"kubernetes.io/projected/c4959268-4527-4c5f-aae4-6d47b89c3410-kube-api-access-x7pdz\") pod \"thanos-querier-686f4d8bc6-d6fv8\" (UID: \"c4959268-4527-4c5f-aae4-6d47b89c3410\") " pod="openshift-monitoring/thanos-querier-686f4d8bc6-d6fv8"
Jul 13 16:41:23.496828 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:23.496807    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c4959268-4527-4c5f-aae4-6d47b89c3410-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-686f4d8bc6-d6fv8\" (UID: \"c4959268-4527-4c5f-aae4-6d47b89c3410\") " pod="openshift-monitoring/thanos-querier-686f4d8bc6-d6fv8"
Jul 13 16:41:23.497494 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:23.497441    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c4959268-4527-4c5f-aae4-6d47b89c3410-metrics-client-ca\") pod \"thanos-querier-686f4d8bc6-d6fv8\" (UID: \"c4959268-4527-4c5f-aae4-6d47b89c3410\") " pod="openshift-monitoring/thanos-querier-686f4d8bc6-d6fv8"
Jul 13 16:41:23.500000 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:23.499853    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c4959268-4527-4c5f-aae4-6d47b89c3410-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-686f4d8bc6-d6fv8\" (UID: \"c4959268-4527-4c5f-aae4-6d47b89c3410\") " pod="openshift-monitoring/thanos-querier-686f4d8bc6-d6fv8"
Jul 13 16:41:23.504873 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:23.500660    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c4959268-4527-4c5f-aae4-6d47b89c3410-secret-grpc-tls\") pod \"thanos-querier-686f4d8bc6-d6fv8\" (UID: \"c4959268-4527-4c5f-aae4-6d47b89c3410\") " pod="openshift-monitoring/thanos-querier-686f4d8bc6-d6fv8"
Jul 13 16:41:23.504873 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:23.500744    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c4959268-4527-4c5f-aae4-6d47b89c3410-secret-thanos-querier-tls\") pod \"thanos-querier-686f4d8bc6-d6fv8\" (UID: \"c4959268-4527-4c5f-aae4-6d47b89c3410\") " pod="openshift-monitoring/thanos-querier-686f4d8bc6-d6fv8"
Jul 13 16:41:23.504873 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:23.500812    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c4959268-4527-4c5f-aae4-6d47b89c3410-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-686f4d8bc6-d6fv8\" (UID: \"c4959268-4527-4c5f-aae4-6d47b89c3410\") " pod="openshift-monitoring/thanos-querier-686f4d8bc6-d6fv8"
Jul 13 16:41:23.504873 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:23.500858    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c4959268-4527-4c5f-aae4-6d47b89c3410-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-686f4d8bc6-d6fv8\" (UID: \"c4959268-4527-4c5f-aae4-6d47b89c3410\") " pod="openshift-monitoring/thanos-querier-686f4d8bc6-d6fv8"
Jul 13 16:41:23.504873 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:23.500798    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c4959268-4527-4c5f-aae4-6d47b89c3410-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-686f4d8bc6-d6fv8\" (UID: \"c4959268-4527-4c5f-aae4-6d47b89c3410\") " pod="openshift-monitoring/thanos-querier-686f4d8bc6-d6fv8"
Jul 13 16:41:23.506464 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:23.506439    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7pdz\" (UniqueName: \"kubernetes.io/projected/c4959268-4527-4c5f-aae4-6d47b89c3410-kube-api-access-x7pdz\") pod \"thanos-querier-686f4d8bc6-d6fv8\" (UID: \"c4959268-4527-4c5f-aae4-6d47b89c3410\") " pod="openshift-monitoring/thanos-querier-686f4d8bc6-d6fv8"
Jul 13 16:41:23.621556 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:23.621515    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-686f4d8bc6-d6fv8"
Jul 13 16:41:25.272294 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:25.272265    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-pmlmx"
Jul 13 16:41:25.436593 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:25.436439    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-mmkvq"]
Jul 13 16:41:25.511677 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:25.511643    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5b48f74765-gbsc7"]
Jul 13 16:41:25.515990 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:25.515968    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5b48f74765-gbsc7"
Jul 13 16:41:25.519385 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:25.519137    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-2tfh9\""
Jul 13 16:41:25.519385 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:25.519164    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 13 16:41:25.519385 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:25.519200    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 13 16:41:25.519385 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:25.519223    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 13 16:41:25.519385 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:25.519166    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 13 16:41:25.519764 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:25.519467    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-67ilcq5p62skm\""
Jul 13 16:41:25.527490 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:25.527467    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5b48f74765-gbsc7"]
Jul 13 16:41:25.613697 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:25.613660    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prts9\" (UniqueName: \"kubernetes.io/projected/c86f5354-b9d8-4f7e-b02c-6603f3b217d1-kube-api-access-prts9\") pod \"metrics-server-5b48f74765-gbsc7\" (UID: \"c86f5354-b9d8-4f7e-b02c-6603f3b217d1\") " pod="openshift-monitoring/metrics-server-5b48f74765-gbsc7"
Jul 13 16:41:25.613905 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:25.613738    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c86f5354-b9d8-4f7e-b02c-6603f3b217d1-secret-metrics-server-client-certs\") pod \"metrics-server-5b48f74765-gbsc7\" (UID: \"c86f5354-b9d8-4f7e-b02c-6603f3b217d1\") " pod="openshift-monitoring/metrics-server-5b48f74765-gbsc7"
Jul 13 16:41:25.613905 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:25.613826    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c86f5354-b9d8-4f7e-b02c-6603f3b217d1-metrics-server-audit-profiles\") pod \"metrics-server-5b48f74765-gbsc7\" (UID: \"c86f5354-b9d8-4f7e-b02c-6603f3b217d1\") " pod="openshift-monitoring/metrics-server-5b48f74765-gbsc7"
Jul 13 16:41:25.613905 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:25.613868    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c86f5354-b9d8-4f7e-b02c-6603f3b217d1-audit-log\") pod \"metrics-server-5b48f74765-gbsc7\" (UID: \"c86f5354-b9d8-4f7e-b02c-6603f3b217d1\") " pod="openshift-monitoring/metrics-server-5b48f74765-gbsc7"
Jul 13 16:41:25.613905 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:25.613899    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c86f5354-b9d8-4f7e-b02c-6603f3b217d1-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5b48f74765-gbsc7\" (UID: \"c86f5354-b9d8-4f7e-b02c-6603f3b217d1\") " pod="openshift-monitoring/metrics-server-5b48f74765-gbsc7"
Jul 13 16:41:25.614096 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:25.613933    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c86f5354-b9d8-4f7e-b02c-6603f3b217d1-secret-metrics-server-tls\") pod \"metrics-server-5b48f74765-gbsc7\" (UID: \"c86f5354-b9d8-4f7e-b02c-6603f3b217d1\") " pod="openshift-monitoring/metrics-server-5b48f74765-gbsc7"
Jul 13 16:41:25.614096 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:25.614035    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c86f5354-b9d8-4f7e-b02c-6603f3b217d1-client-ca-bundle\") pod \"metrics-server-5b48f74765-gbsc7\" (UID: \"c86f5354-b9d8-4f7e-b02c-6603f3b217d1\") " pod="openshift-monitoring/metrics-server-5b48f74765-gbsc7"
Jul 13 16:41:25.644101 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:41:25.644058    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79f7dffb_624b_4874_be76_01f1a6d59a2d.slice/crio-9836195ce6ead5cfd9755fa76b401280743631c48edfd9a4a365552c4b468400 WatchSource:0}: Error finding container 9836195ce6ead5cfd9755fa76b401280743631c48edfd9a4a365552c4b468400: Status 404 returned error can't find the container with id 9836195ce6ead5cfd9755fa76b401280743631c48edfd9a4a365552c4b468400
Jul 13 16:41:25.716677 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:25.715552    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c86f5354-b9d8-4f7e-b02c-6603f3b217d1-secret-metrics-server-client-certs\") pod \"metrics-server-5b48f74765-gbsc7\" (UID: \"c86f5354-b9d8-4f7e-b02c-6603f3b217d1\") " pod="openshift-monitoring/metrics-server-5b48f74765-gbsc7"
Jul 13 16:41:25.716677 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:25.715616    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c86f5354-b9d8-4f7e-b02c-6603f3b217d1-metrics-server-audit-profiles\") pod \"metrics-server-5b48f74765-gbsc7\" (UID: \"c86f5354-b9d8-4f7e-b02c-6603f3b217d1\") " pod="openshift-monitoring/metrics-server-5b48f74765-gbsc7"
Jul 13 16:41:25.716677 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:25.715652    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c86f5354-b9d8-4f7e-b02c-6603f3b217d1-audit-log\") pod \"metrics-server-5b48f74765-gbsc7\" (UID: \"c86f5354-b9d8-4f7e-b02c-6603f3b217d1\") " pod="openshift-monitoring/metrics-server-5b48f74765-gbsc7"
Jul 13 16:41:25.716677 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:25.715681    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c86f5354-b9d8-4f7e-b02c-6603f3b217d1-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5b48f74765-gbsc7\" (UID: \"c86f5354-b9d8-4f7e-b02c-6603f3b217d1\") " pod="openshift-monitoring/metrics-server-5b48f74765-gbsc7"
Jul 13 16:41:25.716677 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:25.715711    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c86f5354-b9d8-4f7e-b02c-6603f3b217d1-secret-metrics-server-tls\") pod \"metrics-server-5b48f74765-gbsc7\" (UID: \"c86f5354-b9d8-4f7e-b02c-6603f3b217d1\") " pod="openshift-monitoring/metrics-server-5b48f74765-gbsc7"
Jul 13 16:41:25.716677 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:25.715769    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c86f5354-b9d8-4f7e-b02c-6603f3b217d1-client-ca-bundle\") pod \"metrics-server-5b48f74765-gbsc7\" (UID: \"c86f5354-b9d8-4f7e-b02c-6603f3b217d1\") " pod="openshift-monitoring/metrics-server-5b48f74765-gbsc7"
Jul 13 16:41:25.716677 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:25.715841    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prts9\" (UniqueName: \"kubernetes.io/projected/c86f5354-b9d8-4f7e-b02c-6603f3b217d1-kube-api-access-prts9\") pod \"metrics-server-5b48f74765-gbsc7\" (UID: \"c86f5354-b9d8-4f7e-b02c-6603f3b217d1\") " pod="openshift-monitoring/metrics-server-5b48f74765-gbsc7"
Jul 13 16:41:25.718614 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:25.717933    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c86f5354-b9d8-4f7e-b02c-6603f3b217d1-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5b48f74765-gbsc7\" (UID: \"c86f5354-b9d8-4f7e-b02c-6603f3b217d1\") " pod="openshift-monitoring/metrics-server-5b48f74765-gbsc7"
Jul 13 16:41:25.720362 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:25.719633    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/c86f5354-b9d8-4f7e-b02c-6603f3b217d1-metrics-server-audit-profiles\") pod \"metrics-server-5b48f74765-gbsc7\" (UID: \"c86f5354-b9d8-4f7e-b02c-6603f3b217d1\") " pod="openshift-monitoring/metrics-server-5b48f74765-gbsc7"
Jul 13 16:41:25.720362 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:25.719908    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/c86f5354-b9d8-4f7e-b02c-6603f3b217d1-audit-log\") pod \"metrics-server-5b48f74765-gbsc7\" (UID: \"c86f5354-b9d8-4f7e-b02c-6603f3b217d1\") " pod="openshift-monitoring/metrics-server-5b48f74765-gbsc7"
Jul 13 16:41:25.723883 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:25.723387    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/c86f5354-b9d8-4f7e-b02c-6603f3b217d1-client-ca-bundle\") pod \"metrics-server-5b48f74765-gbsc7\" (UID: \"c86f5354-b9d8-4f7e-b02c-6603f3b217d1\") " pod="openshift-monitoring/metrics-server-5b48f74765-gbsc7"
Jul 13 16:41:25.723883 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:25.723829    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/c86f5354-b9d8-4f7e-b02c-6603f3b217d1-secret-metrics-server-client-certs\") pod \"metrics-server-5b48f74765-gbsc7\" (UID: \"c86f5354-b9d8-4f7e-b02c-6603f3b217d1\") " pod="openshift-monitoring/metrics-server-5b48f74765-gbsc7"
Jul 13 16:41:25.725723 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:25.725680    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/c86f5354-b9d8-4f7e-b02c-6603f3b217d1-secret-metrics-server-tls\") pod \"metrics-server-5b48f74765-gbsc7\" (UID: \"c86f5354-b9d8-4f7e-b02c-6603f3b217d1\") " pod="openshift-monitoring/metrics-server-5b48f74765-gbsc7"
Jul 13 16:41:25.732309 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:25.732234    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prts9\" (UniqueName: \"kubernetes.io/projected/c86f5354-b9d8-4f7e-b02c-6603f3b217d1-kube-api-access-prts9\") pod \"metrics-server-5b48f74765-gbsc7\" (UID: \"c86f5354-b9d8-4f7e-b02c-6603f3b217d1\") " pod="openshift-monitoring/metrics-server-5b48f74765-gbsc7"
Jul 13 16:41:25.812926 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:25.810835    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-686f4d8bc6-d6fv8"]
Jul 13 16:41:25.829398 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:41:25.827856    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4959268_4527_4c5f_aae4_6d47b89c3410.slice/crio-4aeeb1714a3009902387578445506c281ba43336cc2b96daf889341cda5cd15d WatchSource:0}: Error finding container 4aeeb1714a3009902387578445506c281ba43336cc2b96daf889341cda5cd15d: Status 404 returned error can't find the container with id 4aeeb1714a3009902387578445506c281ba43336cc2b96daf889341cda5cd15d
Jul 13 16:41:25.829398 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:25.828188    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5b48f74765-gbsc7"
Jul 13 16:41:25.854802 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:25.854568    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 13 16:41:26.014346 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.012629    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-84984d445-qnmhv"]
Jul 13 16:41:26.018620 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.017826    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84984d445-qnmhv"
Jul 13 16:41:26.035323 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.032367    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84984d445-qnmhv"]
Jul 13 16:41:26.037999 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.037955    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-frs55"]
Jul 13 16:41:26.045982 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.044516    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-frs55"
Jul 13 16:41:26.047753 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.047709    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 13 16:41:26.048019 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.047709    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-wfmn4\""
Jul 13 16:41:26.058146 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.057828    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-frs55"]
Jul 13 16:41:26.079003 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.078731    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5b48f74765-gbsc7"]
Jul 13 16:41:26.090172 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:41:26.089108    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc86f5354_b9d8_4f7e_b02c_6603f3b217d1.slice/crio-a652f30d74cf4f0ece595745f477b80454245342af81dd68285590d1d3c5a94f WatchSource:0}: Error finding container a652f30d74cf4f0ece595745f477b80454245342af81dd68285590d1d3c5a94f: Status 404 returned error can't find the container with id a652f30d74cf4f0ece595745f477b80454245342af81dd68285590d1d3c5a94f
Jul 13 16:41:26.118640 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.118524    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjj8m\" (UniqueName: \"kubernetes.io/projected/02eaec2b-07c4-499e-9f74-8bd44257577f-kube-api-access-zjj8m\") pod \"console-84984d445-qnmhv\" (UID: \"02eaec2b-07c4-499e-9f74-8bd44257577f\") " pod="openshift-console/console-84984d445-qnmhv"
Jul 13 16:41:26.118640 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.118588    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/02eaec2b-07c4-499e-9f74-8bd44257577f-console-config\") pod \"console-84984d445-qnmhv\" (UID: \"02eaec2b-07c4-499e-9f74-8bd44257577f\") " pod="openshift-console/console-84984d445-qnmhv"
Jul 13 16:41:26.118640 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.118623    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/02eaec2b-07c4-499e-9f74-8bd44257577f-trusted-ca-bundle\") pod \"console-84984d445-qnmhv\" (UID: \"02eaec2b-07c4-499e-9f74-8bd44257577f\") " pod="openshift-console/console-84984d445-qnmhv"
Jul 13 16:41:26.118640 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.118655    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1dbd2873-477a-48e1-aff4-f5ad6342fd9b-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-frs55\" (UID: \"1dbd2873-477a-48e1-aff4-f5ad6342fd9b\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-frs55"
Jul 13 16:41:26.118984 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.118705    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/02eaec2b-07c4-499e-9f74-8bd44257577f-console-oauth-config\") pod \"console-84984d445-qnmhv\" (UID: \"02eaec2b-07c4-499e-9f74-8bd44257577f\") " pod="openshift-console/console-84984d445-qnmhv"
Jul 13 16:41:26.118984 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.118770    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/02eaec2b-07c4-499e-9f74-8bd44257577f-console-serving-cert\") pod \"console-84984d445-qnmhv\" (UID: \"02eaec2b-07c4-499e-9f74-8bd44257577f\") " pod="openshift-console/console-84984d445-qnmhv"
Jul 13 16:41:26.118984 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.118826    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/02eaec2b-07c4-499e-9f74-8bd44257577f-service-ca\") pod \"console-84984d445-qnmhv\" (UID: \"02eaec2b-07c4-499e-9f74-8bd44257577f\") " pod="openshift-console/console-84984d445-qnmhv"
Jul 13 16:41:26.118984 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.118849    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/02eaec2b-07c4-499e-9f74-8bd44257577f-oauth-serving-cert\") pod \"console-84984d445-qnmhv\" (UID: \"02eaec2b-07c4-499e-9f74-8bd44257577f\") " pod="openshift-console/console-84984d445-qnmhv"
Jul 13 16:41:26.118984 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.118881    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6jhb\" (UniqueName: \"kubernetes.io/projected/1dbd2873-477a-48e1-aff4-f5ad6342fd9b-kube-api-access-h6jhb\") pod \"monitoring-plugin-5f68df7f58-frs55\" (UID: \"1dbd2873-477a-48e1-aff4-f5ad6342fd9b\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-frs55"
Jul 13 16:41:26.220351 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.220318    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjj8m\" (UniqueName: \"kubernetes.io/projected/02eaec2b-07c4-499e-9f74-8bd44257577f-kube-api-access-zjj8m\") pod \"console-84984d445-qnmhv\" (UID: \"02eaec2b-07c4-499e-9f74-8bd44257577f\") " pod="openshift-console/console-84984d445-qnmhv"
Jul 13 16:41:26.220536 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.220377    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/02eaec2b-07c4-499e-9f74-8bd44257577f-console-config\") pod \"console-84984d445-qnmhv\" (UID: \"02eaec2b-07c4-499e-9f74-8bd44257577f\") " pod="openshift-console/console-84984d445-qnmhv"
Jul 13 16:41:26.220536 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.220401    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/02eaec2b-07c4-499e-9f74-8bd44257577f-trusted-ca-bundle\") pod \"console-84984d445-qnmhv\" (UID: \"02eaec2b-07c4-499e-9f74-8bd44257577f\") " pod="openshift-console/console-84984d445-qnmhv"
Jul 13 16:41:26.220536 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.220429    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1dbd2873-477a-48e1-aff4-f5ad6342fd9b-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-frs55\" (UID: \"1dbd2873-477a-48e1-aff4-f5ad6342fd9b\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-frs55"
Jul 13 16:41:26.220536 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.220456    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/02eaec2b-07c4-499e-9f74-8bd44257577f-console-oauth-config\") pod \"console-84984d445-qnmhv\" (UID: \"02eaec2b-07c4-499e-9f74-8bd44257577f\") " pod="openshift-console/console-84984d445-qnmhv"
Jul 13 16:41:26.220536 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.220494    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/02eaec2b-07c4-499e-9f74-8bd44257577f-console-serving-cert\") pod \"console-84984d445-qnmhv\" (UID: \"02eaec2b-07c4-499e-9f74-8bd44257577f\") " pod="openshift-console/console-84984d445-qnmhv"
Jul 13 16:41:26.220777 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.220550    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/02eaec2b-07c4-499e-9f74-8bd44257577f-service-ca\") pod \"console-84984d445-qnmhv\" (UID: \"02eaec2b-07c4-499e-9f74-8bd44257577f\") " pod="openshift-console/console-84984d445-qnmhv"
Jul 13 16:41:26.220777 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.220572    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/02eaec2b-07c4-499e-9f74-8bd44257577f-oauth-serving-cert\") pod \"console-84984d445-qnmhv\" (UID: \"02eaec2b-07c4-499e-9f74-8bd44257577f\") " pod="openshift-console/console-84984d445-qnmhv"
Jul 13 16:41:26.220777 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.220612    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6jhb\" (UniqueName: \"kubernetes.io/projected/1dbd2873-477a-48e1-aff4-f5ad6342fd9b-kube-api-access-h6jhb\") pod \"monitoring-plugin-5f68df7f58-frs55\" (UID: \"1dbd2873-477a-48e1-aff4-f5ad6342fd9b\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-frs55"
Jul 13 16:41:26.221134 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.221101    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/02eaec2b-07c4-499e-9f74-8bd44257577f-console-config\") pod \"console-84984d445-qnmhv\" (UID: \"02eaec2b-07c4-499e-9f74-8bd44257577f\") " pod="openshift-console/console-84984d445-qnmhv"
Jul 13 16:41:26.221697 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.221670    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/02eaec2b-07c4-499e-9f74-8bd44257577f-service-ca\") pod \"console-84984d445-qnmhv\" (UID: \"02eaec2b-07c4-499e-9f74-8bd44257577f\") " pod="openshift-console/console-84984d445-qnmhv"
Jul 13 16:41:26.221863 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.221825    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/02eaec2b-07c4-499e-9f74-8bd44257577f-oauth-serving-cert\") pod \"console-84984d445-qnmhv\" (UID: \"02eaec2b-07c4-499e-9f74-8bd44257577f\") " pod="openshift-console/console-84984d445-qnmhv"
Jul 13 16:41:26.221979 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.221944    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/02eaec2b-07c4-499e-9f74-8bd44257577f-trusted-ca-bundle\") pod \"console-84984d445-qnmhv\" (UID: \"02eaec2b-07c4-499e-9f74-8bd44257577f\") " pod="openshift-console/console-84984d445-qnmhv"
Jul 13 16:41:26.223662 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.223633    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/1dbd2873-477a-48e1-aff4-f5ad6342fd9b-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-frs55\" (UID: \"1dbd2873-477a-48e1-aff4-f5ad6342fd9b\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-frs55"
Jul 13 16:41:26.223880 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.223860    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/02eaec2b-07c4-499e-9f74-8bd44257577f-console-serving-cert\") pod \"console-84984d445-qnmhv\" (UID: \"02eaec2b-07c4-499e-9f74-8bd44257577f\") " pod="openshift-console/console-84984d445-qnmhv"
Jul 13 16:41:26.224441 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.224417    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/02eaec2b-07c4-499e-9f74-8bd44257577f-console-oauth-config\") pod \"console-84984d445-qnmhv\" (UID: \"02eaec2b-07c4-499e-9f74-8bd44257577f\") " pod="openshift-console/console-84984d445-qnmhv"
Jul 13 16:41:26.233090 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.233036    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6jhb\" (UniqueName: \"kubernetes.io/projected/1dbd2873-477a-48e1-aff4-f5ad6342fd9b-kube-api-access-h6jhb\") pod \"monitoring-plugin-5f68df7f58-frs55\" (UID: \"1dbd2873-477a-48e1-aff4-f5ad6342fd9b\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-frs55"
Jul 13 16:41:26.233090 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.233036    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjj8m\" (UniqueName: \"kubernetes.io/projected/02eaec2b-07c4-499e-9f74-8bd44257577f-kube-api-access-zjj8m\") pod \"console-84984d445-qnmhv\" (UID: \"02eaec2b-07c4-499e-9f74-8bd44257577f\") " pod="openshift-console/console-84984d445-qnmhv"
Jul 13 16:41:26.339603 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.339052    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5hxjw" event={"ID":"60216865-6676-407c-bd07-6c87d339c76d","Type":"ContainerStarted","Data":"2ea81ccf620c108442cf8f65e218d558c1dc2745b08fe6fa5828b9247a561298"}
Jul 13 16:41:26.339603 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.339116    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5hxjw" event={"ID":"60216865-6676-407c-bd07-6c87d339c76d","Type":"ContainerStarted","Data":"5f14c550b6f17191bf88f044b63fae20375e66b60cc262ab769010fec597fb8d"}
Jul 13 16:41:26.341758 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.341647    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-686f4d8bc6-d6fv8" event={"ID":"c4959268-4527-4c5f-aae4-6d47b89c3410","Type":"ContainerStarted","Data":"4aeeb1714a3009902387578445506c281ba43336cc2b96daf889341cda5cd15d"}
Jul 13 16:41:26.345017 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.344981    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mmkvq" event={"ID":"79f7dffb-624b-4874-be76-01f1a6d59a2d","Type":"ContainerStarted","Data":"7570cbb2ecea6acbcc1d4e27ee286c12ef3919b4e1e537d6dcc7c85d2901bba1"}
Jul 13 16:41:26.345147 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.345045    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mmkvq" event={"ID":"79f7dffb-624b-4874-be76-01f1a6d59a2d","Type":"ContainerStarted","Data":"87d03a1955cbea03dd79d48e60f5495ef4c3a2f408806fb99231937b1e0c94ba"}
Jul 13 16:41:26.345147 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.345073    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mmkvq" event={"ID":"79f7dffb-624b-4874-be76-01f1a6d59a2d","Type":"ContainerStarted","Data":"9836195ce6ead5cfd9755fa76b401280743631c48edfd9a4a365552c4b468400"}
Jul 13 16:41:26.347518 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.347490    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cx6bw" event={"ID":"5cdf4449-5e1c-453d-b7f0-ae71276fd373","Type":"ContainerStarted","Data":"68c6a21b58716b349106a04d8ea9af36f2cb7a6957c60c8069bd06ffc1a4d8c5"}
Jul 13 16:41:26.350789 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.350670    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-m6wbp" event={"ID":"d757b55d-630e-4ecb-8380-1925f6da02d5","Type":"ContainerStarted","Data":"e451f0f38ad5b8a20b06e891831ef474943419e3b9fca4fe023d8735724bc61b"}
Jul 13 16:41:26.350789 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.350727    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-m6wbp" event={"ID":"d757b55d-630e-4ecb-8380-1925f6da02d5","Type":"ContainerStarted","Data":"9e3e14279d8595900650a50faf8dc86ca15a9813b1c0c18881ec8ca7a0e0a523"}
Jul 13 16:41:26.350789 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.350756    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-m6wbp" event={"ID":"d757b55d-630e-4ecb-8380-1925f6da02d5","Type":"ContainerStarted","Data":"ede6a763292c1ba8d020e8b04b03f214910233c792572514cc38d5a02ad8ada9"}
Jul 13 16:41:26.354017 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.353259    2566 generic.go:356] "Generic (PLEG): container finished" podID="4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899" containerID="2f8e57b3f15b131f462badfb52ff85dccdc30c0a231954e1079617738050f3c0" exitCode=0
Jul 13 16:41:26.354017 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.353514    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5m2pj" event={"ID":"4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899","Type":"ContainerDied","Data":"2f8e57b3f15b131f462badfb52ff85dccdc30c0a231954e1079617738050f3c0"}
Jul 13 16:41:26.358466 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.356196    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-72twn" event={"ID":"b9ba2c1f-5b34-4948-b18f-661b82e24e08","Type":"ContainerStarted","Data":"d0ee43ad0d1fcadf267b91642a756d5e6a8ca0cccd558e77249e3fbc5d5487e1"}
Jul 13 16:41:26.358466 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.356892    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:41:26.358466 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.357809    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-5hxjw" podStartSLOduration=67.001948363 podStartE2EDuration="1m12.357790818s" totalImagesPullingTime="5.355842455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:40:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:41:26.356874772 +0000 UTC m=+72.898255203" watchObservedRunningTime="2026-07-13 16:41:26.357790818 +0000 UTC m=+72.899171248"
Jul 13 16:41:26.359211 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.359164    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5b48f74765-gbsc7" event={"ID":"c86f5354-b9d8-4f7e-b02c-6603f3b217d1","Type":"ContainerStarted","Data":"a652f30d74cf4f0ece595745f477b80454245342af81dd68285590d1d3c5a94f"}
Jul 13 16:41:26.361811 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.361783    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6a578cc2-bbf1-4f21-86fa-980c61491d7b","Type":"ContainerStarted","Data":"a5837c54f490a94cfe1f2b51934b639a2546c5b4035c256629c76936c0bd856b"}
Jul 13 16:41:26.374391 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.374327    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-72twn" podStartSLOduration=66.986966722 podStartE2EDuration="1m12.374311825s" totalImagesPullingTime="5.387345103s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:40:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:41:26.37317566 +0000 UTC m=+72.914556090" watchObservedRunningTime="2026-07-13 16:41:26.374311825 +0000 UTC m=+72.915692253"
Jul 13 16:41:26.383932 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.383891    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84984d445-qnmhv"
Jul 13 16:41:26.398537 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.398505    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-frs55"
Jul 13 16:41:26.400723 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.400592    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-m6wbp" podStartSLOduration=1.4610239680000001 podStartE2EDuration="5.400572086s" totalImagesPullingTime="3.939548118s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:41:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:41:26.399178022 +0000 UTC m=+72.940558452" watchObservedRunningTime="2026-07-13 16:41:26.400572086 +0000 UTC m=+72.941952515"
Jul 13 16:41:26.578076 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.577999    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-cx6bw" podStartSLOduration=66.894227633 podStartE2EDuration="1m12.577976126s" totalImagesPullingTime="5.683748493s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 16:40:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:41:26.444664749 +0000 UTC m=+72.986045182" watchObservedRunningTime="2026-07-13 16:41:26.577976126 +0000 UTC m=+73.119356558"
Jul 13 16:41:26.578332 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.578239    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84984d445-qnmhv"]
Jul 13 16:41:26.583163 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:41:26.583126    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02eaec2b_07c4_499e_9f74_8bd44257577f.slice/crio-25ae49e6899967086173b52ab63ac1caec4a9c5e6bb4255711e825d542b3a1a5 WatchSource:0}: Error finding container 25ae49e6899967086173b52ab63ac1caec4a9c5e6bb4255711e825d542b3a1a5: Status 404 returned error can't find the container with id 25ae49e6899967086173b52ab63ac1caec4a9c5e6bb4255711e825d542b3a1a5
Jul 13 16:41:26.602338 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:26.602258    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-frs55"]
Jul 13 16:41:26.607593 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:41:26.607157    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dbd2873_477a_48e1_aff4_f5ad6342fd9b.slice/crio-27c7013db12ac176fb7462123f22d82673a762618851cbf92f2b2d282529de29 WatchSource:0}: Error finding container 27c7013db12ac176fb7462123f22d82673a762618851cbf92f2b2d282529de29: Status 404 returned error can't find the container with id 27c7013db12ac176fb7462123f22d82673a762618851cbf92f2b2d282529de29
Jul 13 16:41:27.371189 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.371082    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mmkvq" event={"ID":"79f7dffb-624b-4874-be76-01f1a6d59a2d","Type":"ContainerStarted","Data":"87a372117bf58d10517c97a13059fb2a1425cafa72543cb98e5fe57e090f217d"}
Jul 13 16:41:27.377473 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.377308    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5m2pj" event={"ID":"4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899","Type":"ContainerStarted","Data":"7d7cabc6cc1effed999eb7f88e80e68d0a06e6f0261402dc9c44e7cfeac854f3"}
Jul 13 16:41:27.377473 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.377375    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5m2pj" event={"ID":"4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899","Type":"ContainerStarted","Data":"2509a2e82875745345644f9f224351bda6040d0fed5b170b1935b90b21452010"}
Jul 13 16:41:27.379054 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.379020    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-frs55" event={"ID":"1dbd2873-477a-48e1-aff4-f5ad6342fd9b","Type":"ContainerStarted","Data":"27c7013db12ac176fb7462123f22d82673a762618851cbf92f2b2d282529de29"}
Jul 13 16:41:27.381502 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.381328    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84984d445-qnmhv" event={"ID":"02eaec2b-07c4-499e-9f74-8bd44257577f","Type":"ContainerStarted","Data":"07451e28a88dbe027a5a5369e5c85b44aaf27b575aab7b1497090fdd5876246c"}
Jul 13 16:41:27.381502 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.381377    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84984d445-qnmhv" event={"ID":"02eaec2b-07c4-499e-9f74-8bd44257577f","Type":"ContainerStarted","Data":"25ae49e6899967086173b52ab63ac1caec4a9c5e6bb4255711e825d542b3a1a5"}
Jul 13 16:41:27.390806 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.390732    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mmkvq" podStartSLOduration=5.177369905 podStartE2EDuration="6.390715706s" totalImagesPullingTime="1.213345801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:41:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:41:27.389372378 +0000 UTC m=+73.930752821" watchObservedRunningTime="2026-07-13 16:41:27.390715706 +0000 UTC m=+73.932096135"
Jul 13 16:41:27.436033 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.435971    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-84984d445-qnmhv" podStartSLOduration=2.435950365 podStartE2EDuration="2.435950365s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:41:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:41:27.410473919 +0000 UTC m=+73.951854350" watchObservedRunningTime="2026-07-13 16:41:27.435950365 +0000 UTC m=+73.977330795"
Jul 13 16:41:27.437340 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.437241    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5m2pj" podStartSLOduration=2.205515103 podStartE2EDuration="6.437227422s" totalImagesPullingTime="4.054900854s" totalInitContainerRuntime="176.811465ms" isStatefulPod=true podCreationTimestamp="2026-07-13 16:41:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:41:27.43534251 +0000 UTC m=+73.976722940" watchObservedRunningTime="2026-07-13 16:41:27.437227422 +0000 UTC m=+73.978607852"
Jul 13 16:41:27.504540 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.502194    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 13 16:41:27.507118 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.507082    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.511326 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.511296    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 13 16:41:27.511979 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.511600    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 13 16:41:27.511979 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.511677    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 13 16:41:27.511979 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.511839    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-nt2xr\""
Jul 13 16:41:27.511979 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.511884    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 13 16:41:27.512480 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.512333    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 13 16:41:27.513565 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.512626    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 13 16:41:27.513565 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.512845    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 13 16:41:27.513565 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.513068    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 13 16:41:27.513565 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.513225    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 13 16:41:27.513565 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.513422    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 13 16:41:27.513565 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.513465    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-2iffemc6sifi4\""
Jul 13 16:41:27.517145 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.517083    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 13 16:41:27.521980 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.521088    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 13 16:41:27.521980 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.521617    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 13 16:41:27.523149 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.523030    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 13 16:41:27.524844 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.524663    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 13 16:41:27.636018 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.635930    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47e1ddb3-5111-448e-a9fd-93903f35e886-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.636018 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.635994    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/47e1ddb3-5111-448e-a9fd-93903f35e886-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.636274 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.636021    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/47e1ddb3-5111-448e-a9fd-93903f35e886-web-config\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.636274 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.636073    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/47e1ddb3-5111-448e-a9fd-93903f35e886-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.636274 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.636113    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/47e1ddb3-5111-448e-a9fd-93903f35e886-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.636274 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.636192    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/47e1ddb3-5111-448e-a9fd-93903f35e886-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.636274 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.636225    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/47e1ddb3-5111-448e-a9fd-93903f35e886-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.636274 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.636274    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/47e1ddb3-5111-448e-a9fd-93903f35e886-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.636568 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.636301    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/47e1ddb3-5111-448e-a9fd-93903f35e886-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.636568 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.636325    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/47e1ddb3-5111-448e-a9fd-93903f35e886-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.636568 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.636348    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/47e1ddb3-5111-448e-a9fd-93903f35e886-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.636568 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.636488    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/47e1ddb3-5111-448e-a9fd-93903f35e886-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.636773 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.636580    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/47e1ddb3-5111-448e-a9fd-93903f35e886-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.636773 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.636661    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/47e1ddb3-5111-448e-a9fd-93903f35e886-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.636773 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.636725    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/47e1ddb3-5111-448e-a9fd-93903f35e886-config-out\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.636953 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.636808    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/47e1ddb3-5111-448e-a9fd-93903f35e886-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.636953 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.636843    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84tfk\" (UniqueName: \"kubernetes.io/projected/47e1ddb3-5111-448e-a9fd-93903f35e886-kube-api-access-84tfk\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.636953 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.636892    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47e1ddb3-5111-448e-a9fd-93903f35e886-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.636953 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.636919    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47e1ddb3-5111-448e-a9fd-93903f35e886-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.637150 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.636974    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/47e1ddb3-5111-448e-a9fd-93903f35e886-config\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.738537 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.738504    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/47e1ddb3-5111-448e-a9fd-93903f35e886-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.738537 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.738542    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/47e1ddb3-5111-448e-a9fd-93903f35e886-web-config\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.738808 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.738580    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/47e1ddb3-5111-448e-a9fd-93903f35e886-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.738808 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.738614    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/47e1ddb3-5111-448e-a9fd-93903f35e886-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.738808 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.738682    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/47e1ddb3-5111-448e-a9fd-93903f35e886-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.738947 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.738827    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/47e1ddb3-5111-448e-a9fd-93903f35e886-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.738947 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.738873    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/47e1ddb3-5111-448e-a9fd-93903f35e886-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.738947 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.738904    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/47e1ddb3-5111-448e-a9fd-93903f35e886-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.738947 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.738928    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/47e1ddb3-5111-448e-a9fd-93903f35e886-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.739128 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.738951    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/47e1ddb3-5111-448e-a9fd-93903f35e886-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.739128 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.738985    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/47e1ddb3-5111-448e-a9fd-93903f35e886-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.739128 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.739009    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/47e1ddb3-5111-448e-a9fd-93903f35e886-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.739128 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.739040    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/47e1ddb3-5111-448e-a9fd-93903f35e886-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.739128 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.739068    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/47e1ddb3-5111-448e-a9fd-93903f35e886-config-out\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.739128 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.739090    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/47e1ddb3-5111-448e-a9fd-93903f35e886-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.739128 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.739116    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84tfk\" (UniqueName: \"kubernetes.io/projected/47e1ddb3-5111-448e-a9fd-93903f35e886-kube-api-access-84tfk\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.739527 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.739179    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47e1ddb3-5111-448e-a9fd-93903f35e886-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.739527 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.739202    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47e1ddb3-5111-448e-a9fd-93903f35e886-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.739527 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.739230    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/47e1ddb3-5111-448e-a9fd-93903f35e886-config\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.739527 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.739288    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47e1ddb3-5111-448e-a9fd-93903f35e886-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.740377 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.740349    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47e1ddb3-5111-448e-a9fd-93903f35e886-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.740488 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.740402    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47e1ddb3-5111-448e-a9fd-93903f35e886-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.740953 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.740904    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/47e1ddb3-5111-448e-a9fd-93903f35e886-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.741041 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.740996    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/47e1ddb3-5111-448e-a9fd-93903f35e886-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.743978 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.742027    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/47e1ddb3-5111-448e-a9fd-93903f35e886-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.743978 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.742181    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/47e1ddb3-5111-448e-a9fd-93903f35e886-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.743978 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.742749    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/47e1ddb3-5111-448e-a9fd-93903f35e886-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.743978 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.743639    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/47e1ddb3-5111-448e-a9fd-93903f35e886-web-config\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.745697 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.745327    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/47e1ddb3-5111-448e-a9fd-93903f35e886-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.745896 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.745863    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/47e1ddb3-5111-448e-a9fd-93903f35e886-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.746325 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.746199    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/47e1ddb3-5111-448e-a9fd-93903f35e886-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.748085 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.748057    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/47e1ddb3-5111-448e-a9fd-93903f35e886-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.748995 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.748589    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/47e1ddb3-5111-448e-a9fd-93903f35e886-config-out\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.748995 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.748929    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/47e1ddb3-5111-448e-a9fd-93903f35e886-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.749475 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.749452    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/47e1ddb3-5111-448e-a9fd-93903f35e886-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.750073 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.750047    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/47e1ddb3-5111-448e-a9fd-93903f35e886-config\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.750148 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.750053    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84tfk\" (UniqueName: \"kubernetes.io/projected/47e1ddb3-5111-448e-a9fd-93903f35e886-kube-api-access-84tfk\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.750148 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.750104    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/47e1ddb3-5111-448e-a9fd-93903f35e886-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.750276 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.750148    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/47e1ddb3-5111-448e-a9fd-93903f35e886-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.750601 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.750575    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/47e1ddb3-5111-448e-a9fd-93903f35e886-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"47e1ddb3-5111-448e-a9fd-93903f35e886\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:27.824512 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:27.824465    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:28.975600 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:28.975570    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 13 16:41:29.105641 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:41:29.105542    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47e1ddb3_5111_448e_a9fd_93903f35e886.slice/crio-e786a1d2a02d8f2ae300e02791fd3b76395527dcd432d26fd7d4c223fc93b1fe WatchSource:0}: Error finding container e786a1d2a02d8f2ae300e02791fd3b76395527dcd432d26fd7d4c223fc93b1fe: Status 404 returned error can't find the container with id e786a1d2a02d8f2ae300e02791fd3b76395527dcd432d26fd7d4c223fc93b1fe
Jul 13 16:41:29.393188 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:29.393159    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"47e1ddb3-5111-448e-a9fd-93903f35e886","Type":"ContainerStarted","Data":"e786a1d2a02d8f2ae300e02791fd3b76395527dcd432d26fd7d4c223fc93b1fe"}
Jul 13 16:41:29.395599 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:29.395567    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-686f4d8bc6-d6fv8" event={"ID":"c4959268-4527-4c5f-aae4-6d47b89c3410","Type":"ContainerStarted","Data":"d864ff6c5e0bcaf17f9db6aa74b098e5568bd1f48a09624b52a890c7aaa6c7a9"}
Jul 13 16:41:29.395696 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:29.395649    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-686f4d8bc6-d6fv8" event={"ID":"c4959268-4527-4c5f-aae4-6d47b89c3410","Type":"ContainerStarted","Data":"9791d1127548610113ad1a503b7fe7bb2da0be5bf0dee5279363987ca7dae127"}
Jul 13 16:41:29.395696 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:29.395688    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-686f4d8bc6-d6fv8" event={"ID":"c4959268-4527-4c5f-aae4-6d47b89c3410","Type":"ContainerStarted","Data":"0ab3aa1fb6aaa43efe5adbd1f3cc53c53a33ba3cfeceecbe664106ebb49864af"}
Jul 13 16:41:29.397226 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:29.397198    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5b48f74765-gbsc7" event={"ID":"c86f5354-b9d8-4f7e-b02c-6603f3b217d1","Type":"ContainerStarted","Data":"6ef39ec96d13f7d6927e03d2d4b47c98943292fe8d95d021526b23c9e4087097"}
Jul 13 16:41:29.398756 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:29.398724    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-frs55" event={"ID":"1dbd2873-477a-48e1-aff4-f5ad6342fd9b","Type":"ContainerStarted","Data":"af3c35df411200f7a6946fef0d74950a1b2dc9ecbc312cb07e476723779be721"}
Jul 13 16:41:29.398897 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:29.398849    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-frs55"
Jul 13 16:41:29.400377 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:29.400351    2566 generic.go:356] "Generic (PLEG): container finished" podID="6a578cc2-bbf1-4f21-86fa-980c61491d7b" containerID="9351606ee34bf14adae76f708984618beafe7d7754a846d1ab45afbb3d1f6cdb" exitCode=0
Jul 13 16:41:29.400658 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:29.400392    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6a578cc2-bbf1-4f21-86fa-980c61491d7b","Type":"ContainerDied","Data":"9351606ee34bf14adae76f708984618beafe7d7754a846d1ab45afbb3d1f6cdb"}
Jul 13 16:41:29.405557 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:29.405533    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-frs55"
Jul 13 16:41:29.428327 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:29.428264    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5b48f74765-gbsc7" podStartSLOduration=1.413968734 podStartE2EDuration="4.428223357s" totalImagesPullingTime="3.014254623s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:41:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:41:29.425335947 +0000 UTC m=+75.966716378" watchObservedRunningTime="2026-07-13 16:41:29.428223357 +0000 UTC m=+75.969603791"
Jul 13 16:41:30.406547 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:30.406457    2566 generic.go:356] "Generic (PLEG): container finished" podID="47e1ddb3-5111-448e-a9fd-93903f35e886" containerID="60fd37bd17e5c8bb5734b25365315178ee2eb7aadaa1fb1edd8449593d9e00aa" exitCode=0
Jul 13 16:41:30.406990 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:30.406538    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"47e1ddb3-5111-448e-a9fd-93903f35e886","Type":"ContainerDied","Data":"60fd37bd17e5c8bb5734b25365315178ee2eb7aadaa1fb1edd8449593d9e00aa"}
Jul 13 16:41:30.410150 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:30.410111    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-686f4d8bc6-d6fv8" event={"ID":"c4959268-4527-4c5f-aae4-6d47b89c3410","Type":"ContainerStarted","Data":"07029b30d5f89089a02d79cfc98cda9a2f1785e0dddfd2a0a0f20f4d35c96028"}
Jul 13 16:41:30.410204 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:30.410183    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-686f4d8bc6-d6fv8" event={"ID":"c4959268-4527-4c5f-aae4-6d47b89c3410","Type":"ContainerStarted","Data":"e0636c3e59f132f3256b582660ac2d791d9b13ce4a8267295bff3b9a489d83b0"}
Jul 13 16:41:30.410270 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:30.410226    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-686f4d8bc6-d6fv8" event={"ID":"c4959268-4527-4c5f-aae4-6d47b89c3410","Type":"ContainerStarted","Data":"3c3b6207dc4b7f79f2b7fa77baa5fc3080a3b3d206384768d4268f994c081de4"}
Jul 13 16:41:30.445486 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:30.445424    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-frs55" podStartSLOduration=1.951233607 podStartE2EDuration="4.445401106s" totalImagesPullingTime="2.494167499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:41:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:41:29.493992975 +0000 UTC m=+76.035373407" watchObservedRunningTime="2026-07-13 16:41:30.445401106 +0000 UTC m=+76.986781536"
Jul 13 16:41:30.480525 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:30.480467    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-686f4d8bc6-d6fv8" podStartSLOduration=3.495026466 podStartE2EDuration="7.480450327s" totalImagesPullingTime="3.985423861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:41:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:41:30.478415913 +0000 UTC m=+77.019796346" watchObservedRunningTime="2026-07-13 16:41:30.480450327 +0000 UTC m=+77.021830755"
Jul 13 16:41:31.418548 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:31.418508    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6a578cc2-bbf1-4f21-86fa-980c61491d7b","Type":"ContainerStarted","Data":"f0f5c6815b9fe5ca20366ba8dda06f9503bcc9bd41adf161f6d6be86f4c70013"}
Jul 13 16:41:31.418997 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:31.418617    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6a578cc2-bbf1-4f21-86fa-980c61491d7b","Type":"ContainerStarted","Data":"fae6bdca8b4561692b5169bb3c2f7d7dac0cd6b9c8ff0ecac7acd7374c925c3f"}
Jul 13 16:41:31.418997 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:31.418653    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6a578cc2-bbf1-4f21-86fa-980c61491d7b","Type":"ContainerStarted","Data":"e3491a93a76ffaab588baf25760b0af87ac0ef8fb9664c766345e31486f9f352"}
Jul 13 16:41:31.418997 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:31.418682    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6a578cc2-bbf1-4f21-86fa-980c61491d7b","Type":"ContainerStarted","Data":"d8a06ce0c1dbf266af1289475744c44f56adbe82a8a02cfa53b784c29813f51d"}
Jul 13 16:41:31.418997 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:31.418707    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6a578cc2-bbf1-4f21-86fa-980c61491d7b","Type":"ContainerStarted","Data":"52a341829bfae0b424e386d908e53835475847d43a6fc3ad8d1308df9b6d7331"}
Jul 13 16:41:31.418997 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:31.418737    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6a578cc2-bbf1-4f21-86fa-980c61491d7b","Type":"ContainerStarted","Data":"1a3a2107ae0aa62086c3a58fc3b68be08e547db652fab8ce192edbb6c3999e94"}
Jul 13 16:41:31.419290 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:31.419181    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-686f4d8bc6-d6fv8"
Jul 13 16:41:31.454047 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:31.453992    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.832816962 podStartE2EDuration="9.45397427s" totalImagesPullingTime="4.596245558s" totalInitContainerRuntime="24.91175ms" isStatefulPod=false podCreationTimestamp="2026-07-13 16:41:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:41:31.453340088 +0000 UTC m=+77.994720517" watchObservedRunningTime="2026-07-13 16:41:31.45397427 +0000 UTC m=+77.995354699"
Jul 13 16:41:34.436326 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:34.436287    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"47e1ddb3-5111-448e-a9fd-93903f35e886","Type":"ContainerStarted","Data":"37bb76fc8b823df4ec86c46a7614155ce8e7eba68e7465dde22722a56a561fd8"}
Jul 13 16:41:34.436711 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:34.436360    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"47e1ddb3-5111-448e-a9fd-93903f35e886","Type":"ContainerStarted","Data":"25d2c84a41dd52849b353fb24e9c2eaaae4c61e44f9e8def58807a863b11676e"}
Jul 13 16:41:34.436711 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:34.436388    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"47e1ddb3-5111-448e-a9fd-93903f35e886","Type":"ContainerStarted","Data":"02c3d08cc7b057d5d6bbc69df61c67fad8cd82d49317873429fa53017b89342a"}
Jul 13 16:41:34.436711 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:34.436412    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"47e1ddb3-5111-448e-a9fd-93903f35e886","Type":"ContainerStarted","Data":"1574619ad27a4bc8e9659417ed183185c6bb07274cb4018ebbb68271775a0a49"}
Jul 13 16:41:34.436711 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:34.436436    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"47e1ddb3-5111-448e-a9fd-93903f35e886","Type":"ContainerStarted","Data":"cbb8fb755f8d8dcf54fe9d73be266beced78366ce70891f6dd8dbc52f1a49f2c"}
Jul 13 16:41:34.436711 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:34.436460    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"47e1ddb3-5111-448e-a9fd-93903f35e886","Type":"ContainerStarted","Data":"a55bca7514072cf78679b6d9589f8cd93e06e1c550d64a992afbab8fd4f28c65"}
Jul 13 16:41:34.474516 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:34.474466    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.013668369 podStartE2EDuration="7.474448118s" totalImagesPullingTime="3.436585832s" totalInitContainerRuntime="24.193917ms" isStatefulPod=false podCreationTimestamp="2026-07-13 16:41:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:41:34.472010059 +0000 UTC m=+81.013390511" watchObservedRunningTime="2026-07-13 16:41:34.474448118 +0000 UTC m=+81.015828547"
Jul 13 16:41:36.384473 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:36.384350    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-84984d445-qnmhv"
Jul 13 16:41:36.384908 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:36.384540    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-84984d445-qnmhv"
Jul 13 16:41:36.389219 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:36.389196    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-84984d445-qnmhv"
Jul 13 16:41:36.448995 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:36.448968    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-84984d445-qnmhv"
Jul 13 16:41:36.515502 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:36.515468    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6488dc44d4-jjscb"]
Jul 13 16:41:37.430228 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:37.430200    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-686f4d8bc6-d6fv8"
Jul 13 16:41:37.825934 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:37.825426    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:41:45.829210 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:45.829173    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5b48f74765-gbsc7"
Jul 13 16:41:45.829678 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:45.829284    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5b48f74765-gbsc7"
Jul 13 16:41:47.248876 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:47.248846    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kcg7j_62d5929c-f868-4354-b47a-4ae2b62e1362/serve-healthcheck-canary/0.log"
Jul 13 16:41:58.389383 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:41:58.389339    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-72twn"
Jul 13 16:42:01.538619 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:01.538574    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6488dc44d4-jjscb" podUID="3fa9699a-a5bc-4ebe-9f00-2506acddeb7c" containerName="console" containerID="cri-o://7e711291f30a3e465c12feecf332f9fd1b5eaad2830431de567f58783449d9e7" gracePeriod=15
Jul 13 16:42:02.345947 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:02.345911    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6488dc44d4-jjscb"
Jul 13 16:42:02.464211 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:02.464113    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbwcz\" (UniqueName: \"kubernetes.io/projected/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-kube-api-access-mbwcz\") pod \"3fa9699a-a5bc-4ebe-9f00-2506acddeb7c\" (UID: \"3fa9699a-a5bc-4ebe-9f00-2506acddeb7c\") "
Jul 13 16:42:02.464211 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:02.464157    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-trusted-ca-bundle\") pod \"3fa9699a-a5bc-4ebe-9f00-2506acddeb7c\" (UID: \"3fa9699a-a5bc-4ebe-9f00-2506acddeb7c\") "
Jul 13 16:42:02.464211 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:02.464197    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-console-serving-cert\") pod \"3fa9699a-a5bc-4ebe-9f00-2506acddeb7c\" (UID: \"3fa9699a-a5bc-4ebe-9f00-2506acddeb7c\") "
Jul 13 16:42:02.464523 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:02.464233    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-console-config\") pod \"3fa9699a-a5bc-4ebe-9f00-2506acddeb7c\" (UID: \"3fa9699a-a5bc-4ebe-9f00-2506acddeb7c\") "
Jul 13 16:42:02.464523 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:02.464278    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-oauth-serving-cert\") pod \"3fa9699a-a5bc-4ebe-9f00-2506acddeb7c\" (UID: \"3fa9699a-a5bc-4ebe-9f00-2506acddeb7c\") "
Jul 13 16:42:02.464523 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:02.464341    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-console-oauth-config\") pod \"3fa9699a-a5bc-4ebe-9f00-2506acddeb7c\" (UID: \"3fa9699a-a5bc-4ebe-9f00-2506acddeb7c\") "
Jul 13 16:42:02.464523 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:02.464371    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-service-ca\") pod \"3fa9699a-a5bc-4ebe-9f00-2506acddeb7c\" (UID: \"3fa9699a-a5bc-4ebe-9f00-2506acddeb7c\") "
Jul 13 16:42:02.464727 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:02.464686    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-console-config" (OuterVolumeSpecName: "console-config") pod "3fa9699a-a5bc-4ebe-9f00-2506acddeb7c" (UID: "3fa9699a-a5bc-4ebe-9f00-2506acddeb7c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 16:42:02.464816 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:02.464779    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3fa9699a-a5bc-4ebe-9f00-2506acddeb7c" (UID: "3fa9699a-a5bc-4ebe-9f00-2506acddeb7c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 16:42:02.464866 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:02.464813    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3fa9699a-a5bc-4ebe-9f00-2506acddeb7c" (UID: "3fa9699a-a5bc-4ebe-9f00-2506acddeb7c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 16:42:02.464935 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:02.464911    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-service-ca" (OuterVolumeSpecName: "service-ca") pod "3fa9699a-a5bc-4ebe-9f00-2506acddeb7c" (UID: "3fa9699a-a5bc-4ebe-9f00-2506acddeb7c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 16:42:02.466694 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:02.466674    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-kube-api-access-mbwcz" (OuterVolumeSpecName: "kube-api-access-mbwcz") pod "3fa9699a-a5bc-4ebe-9f00-2506acddeb7c" (UID: "3fa9699a-a5bc-4ebe-9f00-2506acddeb7c"). InnerVolumeSpecName "kube-api-access-mbwcz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 16:42:02.466769 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:02.466676    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3fa9699a-a5bc-4ebe-9f00-2506acddeb7c" (UID: "3fa9699a-a5bc-4ebe-9f00-2506acddeb7c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 16:42:02.466807 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:02.466777    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3fa9699a-a5bc-4ebe-9f00-2506acddeb7c" (UID: "3fa9699a-a5bc-4ebe-9f00-2506acddeb7c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 16:42:02.560057 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:02.560023    2566 generic.go:356] "Generic (PLEG): container finished" podID="3fa9699a-a5bc-4ebe-9f00-2506acddeb7c" containerID="7e711291f30a3e465c12feecf332f9fd1b5eaad2830431de567f58783449d9e7" exitCode=0
Jul 13 16:42:02.560520 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:02.560064    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6488dc44d4-jjscb" event={"ID":"3fa9699a-a5bc-4ebe-9f00-2506acddeb7c","Type":"ContainerDied","Data":"7e711291f30a3e465c12feecf332f9fd1b5eaad2830431de567f58783449d9e7"}
Jul 13 16:42:02.560520 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:02.560071    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6488dc44d4-jjscb"
Jul 13 16:42:02.560520 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:02.560105    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6488dc44d4-jjscb" event={"ID":"3fa9699a-a5bc-4ebe-9f00-2506acddeb7c","Type":"ContainerDied","Data":"8e11f3de2ce9e2b5e5a4325db988afa99c72a8442bc9316269ce3ee7d864cae7"}
Jul 13 16:42:02.560520 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:02.560125    2566 scope.go:117] "RemoveContainer" containerID="7e711291f30a3e465c12feecf332f9fd1b5eaad2830431de567f58783449d9e7"
Jul 13 16:42:02.564983 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:02.564957    2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-console-serving-cert\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 16:42:02.564983 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:02.564985    2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-console-config\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 16:42:02.565175 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:02.564999    2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-oauth-serving-cert\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 16:42:02.565175 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:02.565012    2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-console-oauth-config\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 16:42:02.565175 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:02.565023    2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 16:42:02.565175 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:02.565035    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbwcz\" (UniqueName: \"kubernetes.io/projected/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-kube-api-access-mbwcz\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 16:42:02.565175 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:02.565051    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c-trusted-ca-bundle\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 16:42:02.569385 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:02.569361    2566 scope.go:117] "RemoveContainer" containerID="7e711291f30a3e465c12feecf332f9fd1b5eaad2830431de567f58783449d9e7"
Jul 13 16:42:02.569740 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:42:02.569716    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e711291f30a3e465c12feecf332f9fd1b5eaad2830431de567f58783449d9e7\": container with ID starting with 7e711291f30a3e465c12feecf332f9fd1b5eaad2830431de567f58783449d9e7 not found: ID does not exist" containerID="7e711291f30a3e465c12feecf332f9fd1b5eaad2830431de567f58783449d9e7"
Jul 13 16:42:02.569840 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:02.569749    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e711291f30a3e465c12feecf332f9fd1b5eaad2830431de567f58783449d9e7"} err="failed to get container status \"7e711291f30a3e465c12feecf332f9fd1b5eaad2830431de567f58783449d9e7\": rpc error: code = NotFound desc = could not find container \"7e711291f30a3e465c12feecf332f9fd1b5eaad2830431de567f58783449d9e7\": container with ID starting with 7e711291f30a3e465c12feecf332f9fd1b5eaad2830431de567f58783449d9e7 not found: ID does not exist"
Jul 13 16:42:02.582298 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:02.582241    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6488dc44d4-jjscb"]
Jul 13 16:42:02.586166 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:02.586134    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6488dc44d4-jjscb"]
Jul 13 16:42:03.982902 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:03.982862    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fa9699a-a5bc-4ebe-9f00-2506acddeb7c" path="/var/lib/kubelet/pods/3fa9699a-a5bc-4ebe-9f00-2506acddeb7c/volumes"
Jul 13 16:42:05.834921 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:05.834892    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5b48f74765-gbsc7"
Jul 13 16:42:05.838874 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:05.838847    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5b48f74765-gbsc7"
Jul 13 16:42:27.825137 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:27.825099    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:42:27.844488 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:27.844458    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:42:28.684334 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:28.684307    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 16:42:45.614703 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:45.614659    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5b75c7698b-445mg"]
Jul 13 16:42:45.615190 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:45.615171    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fa9699a-a5bc-4ebe-9f00-2506acddeb7c" containerName="console"
Jul 13 16:42:45.615237 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:45.615194    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fa9699a-a5bc-4ebe-9f00-2506acddeb7c" containerName="console"
Jul 13 16:42:45.615316 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:45.615284    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fa9699a-a5bc-4ebe-9f00-2506acddeb7c" containerName="console"
Jul 13 16:42:45.617669 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:45.617655    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5b75c7698b-445mg"
Jul 13 16:42:45.620595 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:45.620568    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 13 16:42:45.620756 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:45.620574    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 13 16:42:45.620834 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:45.620763    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 13 16:42:45.621063 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:45.621038    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 13 16:42:45.621212 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:45.621112    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 13 16:42:45.621297 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:45.621239    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-2hwvl\""
Jul 13 16:42:45.627598 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:45.627577    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 13 16:42:45.638056 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:45.638035    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5b75c7698b-445mg"]
Jul 13 16:42:45.762109 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:45.762077    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a68dd61e-fe5e-46c6-9a48-745740af28b6-serving-certs-ca-bundle\") pod \"telemeter-client-5b75c7698b-445mg\" (UID: \"a68dd61e-fe5e-46c6-9a48-745740af28b6\") " pod="openshift-monitoring/telemeter-client-5b75c7698b-445mg"
Jul 13 16:42:45.762109 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:45.762112    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a68dd61e-fe5e-46c6-9a48-745740af28b6-federate-client-tls\") pod \"telemeter-client-5b75c7698b-445mg\" (UID: \"a68dd61e-fe5e-46c6-9a48-745740af28b6\") " pod="openshift-monitoring/telemeter-client-5b75c7698b-445mg"
Jul 13 16:42:45.762367 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:45.762147    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a68dd61e-fe5e-46c6-9a48-745740af28b6-telemeter-client-tls\") pod \"telemeter-client-5b75c7698b-445mg\" (UID: \"a68dd61e-fe5e-46c6-9a48-745740af28b6\") " pod="openshift-monitoring/telemeter-client-5b75c7698b-445mg"
Jul 13 16:42:45.762367 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:45.762168    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a68dd61e-fe5e-46c6-9a48-745740af28b6-metrics-client-ca\") pod \"telemeter-client-5b75c7698b-445mg\" (UID: \"a68dd61e-fe5e-46c6-9a48-745740af28b6\") " pod="openshift-monitoring/telemeter-client-5b75c7698b-445mg"
Jul 13 16:42:45.762367 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:45.762220    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ln9fj\" (UniqueName: \"kubernetes.io/projected/a68dd61e-fe5e-46c6-9a48-745740af28b6-kube-api-access-ln9fj\") pod \"telemeter-client-5b75c7698b-445mg\" (UID: \"a68dd61e-fe5e-46c6-9a48-745740af28b6\") " pod="openshift-monitoring/telemeter-client-5b75c7698b-445mg"
Jul 13 16:42:45.762367 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:45.762235    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a68dd61e-fe5e-46c6-9a48-745740af28b6-secret-telemeter-client\") pod \"telemeter-client-5b75c7698b-445mg\" (UID: \"a68dd61e-fe5e-46c6-9a48-745740af28b6\") " pod="openshift-monitoring/telemeter-client-5b75c7698b-445mg"
Jul 13 16:42:45.762367 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:45.762298    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a68dd61e-fe5e-46c6-9a48-745740af28b6-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5b75c7698b-445mg\" (UID: \"a68dd61e-fe5e-46c6-9a48-745740af28b6\") " pod="openshift-monitoring/telemeter-client-5b75c7698b-445mg"
Jul 13 16:42:45.762367 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:45.762323    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a68dd61e-fe5e-46c6-9a48-745740af28b6-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5b75c7698b-445mg\" (UID: \"a68dd61e-fe5e-46c6-9a48-745740af28b6\") " pod="openshift-monitoring/telemeter-client-5b75c7698b-445mg"
Jul 13 16:42:45.863103 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:45.863074    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a68dd61e-fe5e-46c6-9a48-745740af28b6-serving-certs-ca-bundle\") pod \"telemeter-client-5b75c7698b-445mg\" (UID: \"a68dd61e-fe5e-46c6-9a48-745740af28b6\") " pod="openshift-monitoring/telemeter-client-5b75c7698b-445mg"
Jul 13 16:42:45.863103 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:45.863107    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a68dd61e-fe5e-46c6-9a48-745740af28b6-federate-client-tls\") pod \"telemeter-client-5b75c7698b-445mg\" (UID: \"a68dd61e-fe5e-46c6-9a48-745740af28b6\") " pod="openshift-monitoring/telemeter-client-5b75c7698b-445mg"
Jul 13 16:42:45.863328 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:45.863153    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a68dd61e-fe5e-46c6-9a48-745740af28b6-telemeter-client-tls\") pod \"telemeter-client-5b75c7698b-445mg\" (UID: \"a68dd61e-fe5e-46c6-9a48-745740af28b6\") " pod="openshift-monitoring/telemeter-client-5b75c7698b-445mg"
Jul 13 16:42:45.863328 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:45.863180    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a68dd61e-fe5e-46c6-9a48-745740af28b6-metrics-client-ca\") pod \"telemeter-client-5b75c7698b-445mg\" (UID: \"a68dd61e-fe5e-46c6-9a48-745740af28b6\") " pod="openshift-monitoring/telemeter-client-5b75c7698b-445mg"
Jul 13 16:42:45.863328 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:45.863216    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ln9fj\" (UniqueName: \"kubernetes.io/projected/a68dd61e-fe5e-46c6-9a48-745740af28b6-kube-api-access-ln9fj\") pod \"telemeter-client-5b75c7698b-445mg\" (UID: \"a68dd61e-fe5e-46c6-9a48-745740af28b6\") " pod="openshift-monitoring/telemeter-client-5b75c7698b-445mg"
Jul 13 16:42:45.863487 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:45.863387    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a68dd61e-fe5e-46c6-9a48-745740af28b6-secret-telemeter-client\") pod \"telemeter-client-5b75c7698b-445mg\" (UID: \"a68dd61e-fe5e-46c6-9a48-745740af28b6\") " pod="openshift-monitoring/telemeter-client-5b75c7698b-445mg"
Jul 13 16:42:45.863487 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:45.863475    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a68dd61e-fe5e-46c6-9a48-745740af28b6-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5b75c7698b-445mg\" (UID: \"a68dd61e-fe5e-46c6-9a48-745740af28b6\") " pod="openshift-monitoring/telemeter-client-5b75c7698b-445mg"
Jul 13 16:42:45.863614 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:45.863510    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a68dd61e-fe5e-46c6-9a48-745740af28b6-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5b75c7698b-445mg\" (UID: \"a68dd61e-fe5e-46c6-9a48-745740af28b6\") " pod="openshift-monitoring/telemeter-client-5b75c7698b-445mg"
Jul 13 16:42:45.863909 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:45.863886    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a68dd61e-fe5e-46c6-9a48-745740af28b6-serving-certs-ca-bundle\") pod \"telemeter-client-5b75c7698b-445mg\" (UID: \"a68dd61e-fe5e-46c6-9a48-745740af28b6\") " pod="openshift-monitoring/telemeter-client-5b75c7698b-445mg"
Jul 13 16:42:45.864014 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:45.863986    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a68dd61e-fe5e-46c6-9a48-745740af28b6-metrics-client-ca\") pod \"telemeter-client-5b75c7698b-445mg\" (UID: \"a68dd61e-fe5e-46c6-9a48-745740af28b6\") " pod="openshift-monitoring/telemeter-client-5b75c7698b-445mg"
Jul 13 16:42:45.864264 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:45.864229    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a68dd61e-fe5e-46c6-9a48-745740af28b6-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5b75c7698b-445mg\" (UID: \"a68dd61e-fe5e-46c6-9a48-745740af28b6\") " pod="openshift-monitoring/telemeter-client-5b75c7698b-445mg"
Jul 13 16:42:45.865790 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:45.865729    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/a68dd61e-fe5e-46c6-9a48-745740af28b6-telemeter-client-tls\") pod \"telemeter-client-5b75c7698b-445mg\" (UID: \"a68dd61e-fe5e-46c6-9a48-745740af28b6\") " pod="openshift-monitoring/telemeter-client-5b75c7698b-445mg"
Jul 13 16:42:45.865951 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:45.865932    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/a68dd61e-fe5e-46c6-9a48-745740af28b6-federate-client-tls\") pod \"telemeter-client-5b75c7698b-445mg\" (UID: \"a68dd61e-fe5e-46c6-9a48-745740af28b6\") " pod="openshift-monitoring/telemeter-client-5b75c7698b-445mg"
Jul 13 16:42:45.866160 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:45.866142    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a68dd61e-fe5e-46c6-9a48-745740af28b6-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5b75c7698b-445mg\" (UID: \"a68dd61e-fe5e-46c6-9a48-745740af28b6\") " pod="openshift-monitoring/telemeter-client-5b75c7698b-445mg"
Jul 13 16:42:45.866206 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:45.866188    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/a68dd61e-fe5e-46c6-9a48-745740af28b6-secret-telemeter-client\") pod \"telemeter-client-5b75c7698b-445mg\" (UID: \"a68dd61e-fe5e-46c6-9a48-745740af28b6\") " pod="openshift-monitoring/telemeter-client-5b75c7698b-445mg"
Jul 13 16:42:45.872427 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:45.872404    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ln9fj\" (UniqueName: \"kubernetes.io/projected/a68dd61e-fe5e-46c6-9a48-745740af28b6-kube-api-access-ln9fj\") pod \"telemeter-client-5b75c7698b-445mg\" (UID: \"a68dd61e-fe5e-46c6-9a48-745740af28b6\") " pod="openshift-monitoring/telemeter-client-5b75c7698b-445mg"
Jul 13 16:42:45.928239 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:45.928207    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5b75c7698b-445mg"
Jul 13 16:42:46.082695 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:46.082667    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5b75c7698b-445mg"]
Jul 13 16:42:46.084160 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:42:46.084135    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda68dd61e_fe5e_46c6_9a48_745740af28b6.slice/crio-2b2eca54a203d51572fa3da06a775dce816f11dfe66e901f062088d6d7d6355d WatchSource:0}: Error finding container 2b2eca54a203d51572fa3da06a775dce816f11dfe66e901f062088d6d7d6355d: Status 404 returned error can't find the container with id 2b2eca54a203d51572fa3da06a775dce816f11dfe66e901f062088d6d7d6355d
Jul 13 16:42:46.742272 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:46.742218    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5b75c7698b-445mg" event={"ID":"a68dd61e-fe5e-46c6-9a48-745740af28b6","Type":"ContainerStarted","Data":"2b2eca54a203d51572fa3da06a775dce816f11dfe66e901f062088d6d7d6355d"}
Jul 13 16:42:48.751378 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:48.751340    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5b75c7698b-445mg" event={"ID":"a68dd61e-fe5e-46c6-9a48-745740af28b6","Type":"ContainerStarted","Data":"4a759bd58a87189131eddbe21af390b0e561d58c25e4610bea7ff2cb1aa0fe67"}
Jul 13 16:42:48.751751 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:48.751407    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5b75c7698b-445mg" event={"ID":"a68dd61e-fe5e-46c6-9a48-745740af28b6","Type":"ContainerStarted","Data":"c3f9e61d1d99fce7e528de2c852c11b7ee1ee6dae9e8c9036903328ae323be5b"}
Jul 13 16:42:48.751751 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:48.751428    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5b75c7698b-445mg" event={"ID":"a68dd61e-fe5e-46c6-9a48-745740af28b6","Type":"ContainerStarted","Data":"1da6bf8edfc0355990d8601f64a039e8299290d3a825222db01696ebdf98e9ec"}
Jul 13 16:42:48.774856 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:42:48.774772    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5b75c7698b-445mg" podStartSLOduration=2.124748194 podStartE2EDuration="3.774754714s" totalImagesPullingTime="1.65000652s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:42:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:42:48.773113654 +0000 UTC m=+155.314494083" watchObservedRunningTime="2026-07-13 16:42:48.774754714 +0000 UTC m=+155.316135144"
Jul 13 16:43:07.710954 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:43:07.710869    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84984d445-qnmhv"]
Jul 13 16:43:32.731461 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:43:32.731406    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-84984d445-qnmhv" podUID="02eaec2b-07c4-499e-9f74-8bd44257577f" containerName="console" containerID="cri-o://07451e28a88dbe027a5a5369e5c85b44aaf27b575aab7b1497090fdd5876246c" gracePeriod=15
Jul 13 16:43:32.930968 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:43:32.930932    2566 generic.go:356] "Generic (PLEG): container finished" podID="02eaec2b-07c4-499e-9f74-8bd44257577f" containerID="07451e28a88dbe027a5a5369e5c85b44aaf27b575aab7b1497090fdd5876246c" exitCode=0
Jul 13 16:43:32.931103 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:43:32.931001    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84984d445-qnmhv" event={"ID":"02eaec2b-07c4-499e-9f74-8bd44257577f","Type":"ContainerDied","Data":"07451e28a88dbe027a5a5369e5c85b44aaf27b575aab7b1497090fdd5876246c"}
Jul 13 16:43:32.977377 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:43:32.977352    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84984d445-qnmhv"
Jul 13 16:43:33.060835 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:43:33.060749    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/02eaec2b-07c4-499e-9f74-8bd44257577f-trusted-ca-bundle\") pod \"02eaec2b-07c4-499e-9f74-8bd44257577f\" (UID: \"02eaec2b-07c4-499e-9f74-8bd44257577f\") "
Jul 13 16:43:33.060835 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:43:33.060795    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/02eaec2b-07c4-499e-9f74-8bd44257577f-console-serving-cert\") pod \"02eaec2b-07c4-499e-9f74-8bd44257577f\" (UID: \"02eaec2b-07c4-499e-9f74-8bd44257577f\") "
Jul 13 16:43:33.061057 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:43:33.060885    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/02eaec2b-07c4-499e-9f74-8bd44257577f-oauth-serving-cert\") pod \"02eaec2b-07c4-499e-9f74-8bd44257577f\" (UID: \"02eaec2b-07c4-499e-9f74-8bd44257577f\") "
Jul 13 16:43:33.061057 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:43:33.060975    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/02eaec2b-07c4-499e-9f74-8bd44257577f-console-oauth-config\") pod \"02eaec2b-07c4-499e-9f74-8bd44257577f\" (UID: \"02eaec2b-07c4-499e-9f74-8bd44257577f\") "
Jul 13 16:43:33.061167 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:43:33.061111    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/02eaec2b-07c4-499e-9f74-8bd44257577f-console-config\") pod \"02eaec2b-07c4-499e-9f74-8bd44257577f\" (UID: \"02eaec2b-07c4-499e-9f74-8bd44257577f\") "
Jul 13 16:43:33.061167 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:43:33.061143    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zjj8m\" (UniqueName: \"kubernetes.io/projected/02eaec2b-07c4-499e-9f74-8bd44257577f-kube-api-access-zjj8m\") pod \"02eaec2b-07c4-499e-9f74-8bd44257577f\" (UID: \"02eaec2b-07c4-499e-9f74-8bd44257577f\") "
Jul 13 16:43:33.061287 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:43:33.061179    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/02eaec2b-07c4-499e-9f74-8bd44257577f-service-ca\") pod \"02eaec2b-07c4-499e-9f74-8bd44257577f\" (UID: \"02eaec2b-07c4-499e-9f74-8bd44257577f\") "
Jul 13 16:43:33.061287 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:43:33.061233    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02eaec2b-07c4-499e-9f74-8bd44257577f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "02eaec2b-07c4-499e-9f74-8bd44257577f" (UID: "02eaec2b-07c4-499e-9f74-8bd44257577f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 16:43:33.061287 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:43:33.061279    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02eaec2b-07c4-499e-9f74-8bd44257577f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "02eaec2b-07c4-499e-9f74-8bd44257577f" (UID: "02eaec2b-07c4-499e-9f74-8bd44257577f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 16:43:33.061517 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:43:33.061495    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/02eaec2b-07c4-499e-9f74-8bd44257577f-trusted-ca-bundle\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 16:43:33.061589 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:43:33.061525    2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/02eaec2b-07c4-499e-9f74-8bd44257577f-oauth-serving-cert\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 16:43:33.061589 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:43:33.061491    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02eaec2b-07c4-499e-9f74-8bd44257577f-console-config" (OuterVolumeSpecName: "console-config") pod "02eaec2b-07c4-499e-9f74-8bd44257577f" (UID: "02eaec2b-07c4-499e-9f74-8bd44257577f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 16:43:33.061589 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:43:33.061539    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02eaec2b-07c4-499e-9f74-8bd44257577f-service-ca" (OuterVolumeSpecName: "service-ca") pod "02eaec2b-07c4-499e-9f74-8bd44257577f" (UID: "02eaec2b-07c4-499e-9f74-8bd44257577f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 16:43:33.062979 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:43:33.062958    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02eaec2b-07c4-499e-9f74-8bd44257577f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "02eaec2b-07c4-499e-9f74-8bd44257577f" (UID: "02eaec2b-07c4-499e-9f74-8bd44257577f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 16:43:33.063106 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:43:33.063088    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02eaec2b-07c4-499e-9f74-8bd44257577f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "02eaec2b-07c4-499e-9f74-8bd44257577f" (UID: "02eaec2b-07c4-499e-9f74-8bd44257577f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 16:43:33.063176 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:43:33.063153    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02eaec2b-07c4-499e-9f74-8bd44257577f-kube-api-access-zjj8m" (OuterVolumeSpecName: "kube-api-access-zjj8m") pod "02eaec2b-07c4-499e-9f74-8bd44257577f" (UID: "02eaec2b-07c4-499e-9f74-8bd44257577f"). InnerVolumeSpecName "kube-api-access-zjj8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 16:43:33.162982 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:43:33.162949    2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/02eaec2b-07c4-499e-9f74-8bd44257577f-console-oauth-config\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 16:43:33.162982 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:43:33.162982    2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/02eaec2b-07c4-499e-9f74-8bd44257577f-console-config\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 16:43:33.163156 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:43:33.162998    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjj8m\" (UniqueName: \"kubernetes.io/projected/02eaec2b-07c4-499e-9f74-8bd44257577f-kube-api-access-zjj8m\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 16:43:33.163156 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:43:33.163013    2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/02eaec2b-07c4-499e-9f74-8bd44257577f-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 16:43:33.163156 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:43:33.163025    2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/02eaec2b-07c4-499e-9f74-8bd44257577f-console-serving-cert\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 16:43:33.935668 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:43:33.935642    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84984d445-qnmhv"
Jul 13 16:43:33.935668 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:43:33.935662    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84984d445-qnmhv" event={"ID":"02eaec2b-07c4-499e-9f74-8bd44257577f","Type":"ContainerDied","Data":"25ae49e6899967086173b52ab63ac1caec4a9c5e6bb4255711e825d542b3a1a5"}
Jul 13 16:43:33.936203 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:43:33.935728    2566 scope.go:117] "RemoveContainer" containerID="07451e28a88dbe027a5a5369e5c85b44aaf27b575aab7b1497090fdd5876246c"
Jul 13 16:43:33.959830 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:43:33.959777    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-84984d445-qnmhv"]
Jul 13 16:43:33.961697 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:43:33.961673    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-84984d445-qnmhv"]
Jul 13 16:43:33.982408 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:43:33.982371    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02eaec2b-07c4-499e-9f74-8bd44257577f" path="/var/lib/kubelet/pods/02eaec2b-07c4-499e-9f74-8bd44257577f/volumes"
Jul 13 16:44:53.804206 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:44:53.804172    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-rm8sp"]
Jul 13 16:44:53.804693 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:44:53.804545    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02eaec2b-07c4-499e-9f74-8bd44257577f" containerName="console"
Jul 13 16:44:53.804693 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:44:53.804557    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="02eaec2b-07c4-499e-9f74-8bd44257577f" containerName="console"
Jul 13 16:44:53.804693 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:44:53.804606    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="02eaec2b-07c4-499e-9f74-8bd44257577f" containerName="console"
Jul 13 16:44:53.806400 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:44:53.806373    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-rm8sp"
Jul 13 16:44:53.809455 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:44:53.809434    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-r4rdd\""
Jul 13 16:44:53.809590 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:44:53.809575    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 13 16:44:53.810618 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:44:53.810605    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 13 16:44:53.819145 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:44:53.819127    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-rm8sp"]
Jul 13 16:44:53.834302 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:44:53.834280    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-288bl\" (UniqueName: \"kubernetes.io/projected/5ae4d001-a98e-4e75-a822-186c4334000b-kube-api-access-288bl\") pod \"cert-manager-webhook-6486ffd997-rm8sp\" (UID: \"5ae4d001-a98e-4e75-a822-186c4334000b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rm8sp"
Jul 13 16:44:53.834459 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:44:53.834409    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5ae4d001-a98e-4e75-a822-186c4334000b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-rm8sp\" (UID: \"5ae4d001-a98e-4e75-a822-186c4334000b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rm8sp"
Jul 13 16:44:53.935222 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:44:53.935191    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5ae4d001-a98e-4e75-a822-186c4334000b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-rm8sp\" (UID: \"5ae4d001-a98e-4e75-a822-186c4334000b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rm8sp"
Jul 13 16:44:53.935419 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:44:53.935237    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-288bl\" (UniqueName: \"kubernetes.io/projected/5ae4d001-a98e-4e75-a822-186c4334000b-kube-api-access-288bl\") pod \"cert-manager-webhook-6486ffd997-rm8sp\" (UID: \"5ae4d001-a98e-4e75-a822-186c4334000b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rm8sp"
Jul 13 16:44:53.944058 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:44:53.944027    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-288bl\" (UniqueName: \"kubernetes.io/projected/5ae4d001-a98e-4e75-a822-186c4334000b-kube-api-access-288bl\") pod \"cert-manager-webhook-6486ffd997-rm8sp\" (UID: \"5ae4d001-a98e-4e75-a822-186c4334000b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rm8sp"
Jul 13 16:44:53.944176 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:44:53.944027    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5ae4d001-a98e-4e75-a822-186c4334000b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-rm8sp\" (UID: \"5ae4d001-a98e-4e75-a822-186c4334000b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rm8sp"
Jul 13 16:44:54.128240 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:44:54.128164    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-rm8sp"
Jul 13 16:44:54.251485 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:44:54.251436    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-rm8sp"]
Jul 13 16:44:54.253767 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:44:54.253739    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5ae4d001_a98e_4e75_a822_186c4334000b.slice/crio-4a11cdc2ed0661c8af140f725058a262e23be6d38f10a661f163789442c7510e WatchSource:0}: Error finding container 4a11cdc2ed0661c8af140f725058a262e23be6d38f10a661f163789442c7510e: Status 404 returned error can't find the container with id 4a11cdc2ed0661c8af140f725058a262e23be6d38f10a661f163789442c7510e
Jul 13 16:44:55.246763 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:44:55.246723    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-rm8sp" event={"ID":"5ae4d001-a98e-4e75-a822-186c4334000b","Type":"ContainerStarted","Data":"4a11cdc2ed0661c8af140f725058a262e23be6d38f10a661f163789442c7510e"}
Jul 13 16:44:57.224919 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:44:57.224896    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-cxwmq"]
Jul 13 16:44:57.227387 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:44:57.227371    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-cxwmq"
Jul 13 16:44:57.230615 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:44:57.230599    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-7v8lt\""
Jul 13 16:44:57.238788 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:44:57.238767    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-cxwmq"]
Jul 13 16:44:57.263079 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:44:57.263056    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgl77\" (UniqueName: \"kubernetes.io/projected/9f0afb01-2449-42af-9b0b-54295d5cd228-kube-api-access-mgl77\") pod \"cert-manager-55b4d954d5-cxwmq\" (UID: \"9f0afb01-2449-42af-9b0b-54295d5cd228\") " pod="cert-manager/cert-manager-55b4d954d5-cxwmq"
Jul 13 16:44:57.263172 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:44:57.263142    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9f0afb01-2449-42af-9b0b-54295d5cd228-bound-sa-token\") pod \"cert-manager-55b4d954d5-cxwmq\" (UID: \"9f0afb01-2449-42af-9b0b-54295d5cd228\") " pod="cert-manager/cert-manager-55b4d954d5-cxwmq"
Jul 13 16:44:57.364355 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:44:57.364312    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgl77\" (UniqueName: \"kubernetes.io/projected/9f0afb01-2449-42af-9b0b-54295d5cd228-kube-api-access-mgl77\") pod \"cert-manager-55b4d954d5-cxwmq\" (UID: \"9f0afb01-2449-42af-9b0b-54295d5cd228\") " pod="cert-manager/cert-manager-55b4d954d5-cxwmq"
Jul 13 16:44:57.364547 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:44:57.364373    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9f0afb01-2449-42af-9b0b-54295d5cd228-bound-sa-token\") pod \"cert-manager-55b4d954d5-cxwmq\" (UID: \"9f0afb01-2449-42af-9b0b-54295d5cd228\") " pod="cert-manager/cert-manager-55b4d954d5-cxwmq"
Jul 13 16:44:57.373027 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:44:57.372988    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9f0afb01-2449-42af-9b0b-54295d5cd228-bound-sa-token\") pod \"cert-manager-55b4d954d5-cxwmq\" (UID: \"9f0afb01-2449-42af-9b0b-54295d5cd228\") " pod="cert-manager/cert-manager-55b4d954d5-cxwmq"
Jul 13 16:44:57.373136 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:44:57.373048    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgl77\" (UniqueName: \"kubernetes.io/projected/9f0afb01-2449-42af-9b0b-54295d5cd228-kube-api-access-mgl77\") pod \"cert-manager-55b4d954d5-cxwmq\" (UID: \"9f0afb01-2449-42af-9b0b-54295d5cd228\") " pod="cert-manager/cert-manager-55b4d954d5-cxwmq"
Jul 13 16:44:57.546949 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:44:57.546868    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-cxwmq"
Jul 13 16:44:57.668583 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:44:57.668558    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-cxwmq"]
Jul 13 16:44:57.670238 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:44:57.670207    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9f0afb01_2449_42af_9b0b_54295d5cd228.slice/crio-5525647c508e16fc4b8fc53fb3db401926784b21d8e82ba4253b9f546124a621 WatchSource:0}: Error finding container 5525647c508e16fc4b8fc53fb3db401926784b21d8e82ba4253b9f546124a621: Status 404 returned error can't find the container with id 5525647c508e16fc4b8fc53fb3db401926784b21d8e82ba4253b9f546124a621
Jul 13 16:44:58.261825 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:44:58.261790    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-rm8sp" event={"ID":"5ae4d001-a98e-4e75-a822-186c4334000b","Type":"ContainerStarted","Data":"a000fefe316f3f905698da62e526be72012d2273225fd0ae3b4a62901ee87983"}
Jul 13 16:44:58.262214 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:44:58.261933    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-rm8sp"
Jul 13 16:44:58.263152 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:44:58.263133    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-cxwmq" event={"ID":"9f0afb01-2449-42af-9b0b-54295d5cd228","Type":"ContainerStarted","Data":"fd9af08de5be04412bedb7b86c8e9db820bd42c31af231375191b0c8707a4e86"}
Jul 13 16:44:58.263264 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:44:58.263164    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-cxwmq" event={"ID":"9f0afb01-2449-42af-9b0b-54295d5cd228","Type":"ContainerStarted","Data":"5525647c508e16fc4b8fc53fb3db401926784b21d8e82ba4253b9f546124a621"}
Jul 13 16:44:58.285927 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:44:58.285886    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-rm8sp" podStartSLOduration=2.336591388 podStartE2EDuration="5.285873634s" totalImagesPullingTime="2.949282246s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:44:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:44:58.283735709 +0000 UTC m=+284.825116138" watchObservedRunningTime="2026-07-13 16:44:58.285873634 +0000 UTC m=+284.827254063"
Jul 13 16:44:58.303728 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:44:58.303689    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-cxwmq" podStartSLOduration=1.303676976 podStartE2EDuration="1.303676976s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:44:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:44:58.302678465 +0000 UTC m=+284.844058895" watchObservedRunningTime="2026-07-13 16:44:58.303676976 +0000 UTC m=+284.845057405"
Jul 13 16:45:04.269843 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:04.269813    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-rm8sp"
Jul 13 16:45:13.846999 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:13.846973    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gdqcb_6a33d4fc-2988-4f88-a92c-212d95ec2060/ovn-acl-logging/0.log"
Jul 13 16:45:13.851054 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:13.851030    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gdqcb_6a33d4fc-2988-4f88-a92c-212d95ec2060/ovn-acl-logging/0.log"
Jul 13 16:45:13.852854 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:13.852833    2566 kubelet.go:1648] "Image garbage collection succeeded"
Jul 13 16:45:16.421360 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:16.421320    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-78cc96c4cb-llcvt"]
Jul 13 16:45:16.424183 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:16.424167    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-78cc96c4cb-llcvt"
Jul 13 16:45:16.427472 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:16.427449    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 13 16:45:16.427585 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:16.427449    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 13 16:45:16.428840 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:16.428820    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 13 16:45:16.428950 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:16.428880    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 13 16:45:16.428950 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:16.428945    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-8lrhf\""
Jul 13 16:45:16.429031 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:16.428991    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 13 16:45:16.439909 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:16.439886    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-78cc96c4cb-llcvt"]
Jul 13 16:45:16.520650 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:16.520622    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9e74dc84-65ad-41c1-a938-66ceb207e631-metrics-cert\") pod \"lws-controller-manager-78cc96c4cb-llcvt\" (UID: \"9e74dc84-65ad-41c1-a938-66ceb207e631\") " pod="openshift-lws-operator/lws-controller-manager-78cc96c4cb-llcvt"
Jul 13 16:45:16.520831 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:16.520664    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqvb6\" (UniqueName: \"kubernetes.io/projected/9e74dc84-65ad-41c1-a938-66ceb207e631-kube-api-access-tqvb6\") pod \"lws-controller-manager-78cc96c4cb-llcvt\" (UID: \"9e74dc84-65ad-41c1-a938-66ceb207e631\") " pod="openshift-lws-operator/lws-controller-manager-78cc96c4cb-llcvt"
Jul 13 16:45:16.520831 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:16.520750    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9e74dc84-65ad-41c1-a938-66ceb207e631-cert\") pod \"lws-controller-manager-78cc96c4cb-llcvt\" (UID: \"9e74dc84-65ad-41c1-a938-66ceb207e631\") " pod="openshift-lws-operator/lws-controller-manager-78cc96c4cb-llcvt"
Jul 13 16:45:16.520946 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:16.520837    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9e74dc84-65ad-41c1-a938-66ceb207e631-manager-config\") pod \"lws-controller-manager-78cc96c4cb-llcvt\" (UID: \"9e74dc84-65ad-41c1-a938-66ceb207e631\") " pod="openshift-lws-operator/lws-controller-manager-78cc96c4cb-llcvt"
Jul 13 16:45:16.621613 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:16.621578    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9e74dc84-65ad-41c1-a938-66ceb207e631-cert\") pod \"lws-controller-manager-78cc96c4cb-llcvt\" (UID: \"9e74dc84-65ad-41c1-a938-66ceb207e631\") " pod="openshift-lws-operator/lws-controller-manager-78cc96c4cb-llcvt"
Jul 13 16:45:16.621779 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:16.621629    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9e74dc84-65ad-41c1-a938-66ceb207e631-manager-config\") pod \"lws-controller-manager-78cc96c4cb-llcvt\" (UID: \"9e74dc84-65ad-41c1-a938-66ceb207e631\") " pod="openshift-lws-operator/lws-controller-manager-78cc96c4cb-llcvt"
Jul 13 16:45:16.621779 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:16.621662    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9e74dc84-65ad-41c1-a938-66ceb207e631-metrics-cert\") pod \"lws-controller-manager-78cc96c4cb-llcvt\" (UID: \"9e74dc84-65ad-41c1-a938-66ceb207e631\") " pod="openshift-lws-operator/lws-controller-manager-78cc96c4cb-llcvt"
Jul 13 16:45:16.621779 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:16.621687    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqvb6\" (UniqueName: \"kubernetes.io/projected/9e74dc84-65ad-41c1-a938-66ceb207e631-kube-api-access-tqvb6\") pod \"lws-controller-manager-78cc96c4cb-llcvt\" (UID: \"9e74dc84-65ad-41c1-a938-66ceb207e631\") " pod="openshift-lws-operator/lws-controller-manager-78cc96c4cb-llcvt"
Jul 13 16:45:16.622444 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:16.622419    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9e74dc84-65ad-41c1-a938-66ceb207e631-manager-config\") pod \"lws-controller-manager-78cc96c4cb-llcvt\" (UID: \"9e74dc84-65ad-41c1-a938-66ceb207e631\") " pod="openshift-lws-operator/lws-controller-manager-78cc96c4cb-llcvt"
Jul 13 16:45:16.624144 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:16.624124    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9e74dc84-65ad-41c1-a938-66ceb207e631-cert\") pod \"lws-controller-manager-78cc96c4cb-llcvt\" (UID: \"9e74dc84-65ad-41c1-a938-66ceb207e631\") " pod="openshift-lws-operator/lws-controller-manager-78cc96c4cb-llcvt"
Jul 13 16:45:16.624220 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:16.624183    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9e74dc84-65ad-41c1-a938-66ceb207e631-metrics-cert\") pod \"lws-controller-manager-78cc96c4cb-llcvt\" (UID: \"9e74dc84-65ad-41c1-a938-66ceb207e631\") " pod="openshift-lws-operator/lws-controller-manager-78cc96c4cb-llcvt"
Jul 13 16:45:16.635277 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:16.635229    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqvb6\" (UniqueName: \"kubernetes.io/projected/9e74dc84-65ad-41c1-a938-66ceb207e631-kube-api-access-tqvb6\") pod \"lws-controller-manager-78cc96c4cb-llcvt\" (UID: \"9e74dc84-65ad-41c1-a938-66ceb207e631\") " pod="openshift-lws-operator/lws-controller-manager-78cc96c4cb-llcvt"
Jul 13 16:45:16.733477 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:16.733411    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-78cc96c4cb-llcvt"
Jul 13 16:45:16.863457 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:16.863433    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-78cc96c4cb-llcvt"]
Jul 13 16:45:16.865139 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:45:16.865116    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e74dc84_65ad_41c1_a938_66ceb207e631.slice/crio-550bcc7949dacd561c4d98f6367ff4a6e7bddd4dd6b5af895372523519e4a5d9 WatchSource:0}: Error finding container 550bcc7949dacd561c4d98f6367ff4a6e7bddd4dd6b5af895372523519e4a5d9: Status 404 returned error can't find the container with id 550bcc7949dacd561c4d98f6367ff4a6e7bddd4dd6b5af895372523519e4a5d9
Jul 13 16:45:16.867037 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:16.867022    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 16:45:17.345968 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:17.345933    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-78cc96c4cb-llcvt" event={"ID":"9e74dc84-65ad-41c1-a938-66ceb207e631","Type":"ContainerStarted","Data":"550bcc7949dacd561c4d98f6367ff4a6e7bddd4dd6b5af895372523519e4a5d9"}
Jul 13 16:45:21.365103 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:21.365069    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-78cc96c4cb-llcvt" event={"ID":"9e74dc84-65ad-41c1-a938-66ceb207e631","Type":"ContainerStarted","Data":"3320ebc225dbf5c8080c98f6bc138e466fa75788361cfd666485362b5595cf57"}
Jul 13 16:45:21.365560 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:21.365164    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-78cc96c4cb-llcvt"
Jul 13 16:45:21.383665 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:21.383615    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-78cc96c4cb-llcvt" podStartSLOduration=1.411156375 podStartE2EDuration="5.383600522s" totalImagesPullingTime="3.972444147s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:45:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:45:21.381919615 +0000 UTC m=+307.923300056" watchObservedRunningTime="2026-07-13 16:45:21.383600522 +0000 UTC m=+307.924980950"
Jul 13 16:45:24.919986 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:24.919954    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5857456dfc-fdgtp"]
Jul 13 16:45:24.922475 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:24.922455    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-fdgtp"
Jul 13 16:45:24.930861 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:24.930843    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 13 16:45:24.931276 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:24.931239    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 13 16:45:24.931545 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:24.931530    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 13 16:45:24.931615 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:24.931594    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 13 16:45:24.932002 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:24.931988    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-h2scv\""
Jul 13 16:45:24.958805 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:24.958775    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5857456dfc-fdgtp"]
Jul 13 16:45:24.993775 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:24.993753    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sjcn\" (UniqueName: \"kubernetes.io/projected/e33cc5db-422b-42db-805c-cb9a18300419-kube-api-access-7sjcn\") pod \"opendatahub-operator-controller-manager-5857456dfc-fdgtp\" (UID: \"e33cc5db-422b-42db-805c-cb9a18300419\") " pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-fdgtp"
Jul 13 16:45:24.993873 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:24.993782    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e33cc5db-422b-42db-805c-cb9a18300419-tmp\") pod \"opendatahub-operator-controller-manager-5857456dfc-fdgtp\" (UID: \"e33cc5db-422b-42db-805c-cb9a18300419\") " pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-fdgtp"
Jul 13 16:45:24.993873 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:24.993801    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e33cc5db-422b-42db-805c-cb9a18300419-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5857456dfc-fdgtp\" (UID: \"e33cc5db-422b-42db-805c-cb9a18300419\") " pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-fdgtp"
Jul 13 16:45:24.993873 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:24.993839    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e33cc5db-422b-42db-805c-cb9a18300419-webhook-cert\") pod \"opendatahub-operator-controller-manager-5857456dfc-fdgtp\" (UID: \"e33cc5db-422b-42db-805c-cb9a18300419\") " pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-fdgtp"
Jul 13 16:45:24.993873 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:24.993871    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e33cc5db-422b-42db-805c-cb9a18300419-manifests\") pod \"opendatahub-operator-controller-manager-5857456dfc-fdgtp\" (UID: \"e33cc5db-422b-42db-805c-cb9a18300419\") " pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-fdgtp"
Jul 13 16:45:25.094912 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:25.094889    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sjcn\" (UniqueName: \"kubernetes.io/projected/e33cc5db-422b-42db-805c-cb9a18300419-kube-api-access-7sjcn\") pod \"opendatahub-operator-controller-manager-5857456dfc-fdgtp\" (UID: \"e33cc5db-422b-42db-805c-cb9a18300419\") " pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-fdgtp"
Jul 13 16:45:25.095040 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:25.094931    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e33cc5db-422b-42db-805c-cb9a18300419-tmp\") pod \"opendatahub-operator-controller-manager-5857456dfc-fdgtp\" (UID: \"e33cc5db-422b-42db-805c-cb9a18300419\") " pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-fdgtp"
Jul 13 16:45:25.095123 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:25.095102    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e33cc5db-422b-42db-805c-cb9a18300419-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5857456dfc-fdgtp\" (UID: \"e33cc5db-422b-42db-805c-cb9a18300419\") " pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-fdgtp"
Jul 13 16:45:25.095218 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:25.095196    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e33cc5db-422b-42db-805c-cb9a18300419-webhook-cert\") pod \"opendatahub-operator-controller-manager-5857456dfc-fdgtp\" (UID: \"e33cc5db-422b-42db-805c-cb9a18300419\") " pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-fdgtp"
Jul 13 16:45:25.095296 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:25.095271    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e33cc5db-422b-42db-805c-cb9a18300419-manifests\") pod \"opendatahub-operator-controller-manager-5857456dfc-fdgtp\" (UID: \"e33cc5db-422b-42db-805c-cb9a18300419\") " pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-fdgtp"
Jul 13 16:45:25.095616 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:25.095595    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e33cc5db-422b-42db-805c-cb9a18300419-manifests\") pod \"opendatahub-operator-controller-manager-5857456dfc-fdgtp\" (UID: \"e33cc5db-422b-42db-805c-cb9a18300419\") " pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-fdgtp"
Jul 13 16:45:25.097616 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:25.097596    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e33cc5db-422b-42db-805c-cb9a18300419-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5857456dfc-fdgtp\" (UID: \"e33cc5db-422b-42db-805c-cb9a18300419\") " pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-fdgtp"
Jul 13 16:45:25.097696 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:25.097670    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e33cc5db-422b-42db-805c-cb9a18300419-tmp\") pod \"opendatahub-operator-controller-manager-5857456dfc-fdgtp\" (UID: \"e33cc5db-422b-42db-805c-cb9a18300419\") " pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-fdgtp"
Jul 13 16:45:25.097784 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:25.097767    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e33cc5db-422b-42db-805c-cb9a18300419-webhook-cert\") pod \"opendatahub-operator-controller-manager-5857456dfc-fdgtp\" (UID: \"e33cc5db-422b-42db-805c-cb9a18300419\") " pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-fdgtp"
Jul 13 16:45:25.111506 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:25.111483    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sjcn\" (UniqueName: \"kubernetes.io/projected/e33cc5db-422b-42db-805c-cb9a18300419-kube-api-access-7sjcn\") pod \"opendatahub-operator-controller-manager-5857456dfc-fdgtp\" (UID: \"e33cc5db-422b-42db-805c-cb9a18300419\") " pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-fdgtp"
Jul 13 16:45:25.232677 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:25.232588    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-fdgtp"
Jul 13 16:45:25.364900 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:25.364876    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5857456dfc-fdgtp"]
Jul 13 16:45:25.367228 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:45:25.367203    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode33cc5db_422b_42db_805c_cb9a18300419.slice/crio-c7949b4c843e3eec6f484c53ba98a7efe13cb8c879e39984e24922cf8586fe67 WatchSource:0}: Error finding container c7949b4c843e3eec6f484c53ba98a7efe13cb8c879e39984e24922cf8586fe67: Status 404 returned error can't find the container with id c7949b4c843e3eec6f484c53ba98a7efe13cb8c879e39984e24922cf8586fe67
Jul 13 16:45:25.382029 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:25.382006    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-fdgtp" event={"ID":"e33cc5db-422b-42db-805c-cb9a18300419","Type":"ContainerStarted","Data":"c7949b4c843e3eec6f484c53ba98a7efe13cb8c879e39984e24922cf8586fe67"}
Jul 13 16:45:29.404681 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:29.404649    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-fdgtp" event={"ID":"e33cc5db-422b-42db-805c-cb9a18300419","Type":"ContainerStarted","Data":"f5766349a0fbd57f62316cddee34d77ca47a7888b17eeca45aa7d5de0c7f063b"}
Jul 13 16:45:30.410634 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:30.410595    2566 generic.go:356] "Generic (PLEG): container finished" podID="e33cc5db-422b-42db-805c-cb9a18300419" containerID="f5766349a0fbd57f62316cddee34d77ca47a7888b17eeca45aa7d5de0c7f063b" exitCode=0
Jul 13 16:45:30.410988 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:30.410682    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-fdgtp" event={"ID":"e33cc5db-422b-42db-805c-cb9a18300419","Type":"ContainerDied","Data":"f5766349a0fbd57f62316cddee34d77ca47a7888b17eeca45aa7d5de0c7f063b"}
Jul 13 16:45:31.417443 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:31.417407    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-fdgtp" event={"ID":"e33cc5db-422b-42db-805c-cb9a18300419","Type":"ContainerStarted","Data":"d9f56a2bafd71103f04bd09d956d68283f6839888ca2c57fea75672ad9e99e30"}
Jul 13 16:45:31.417801 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:31.417541    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-fdgtp"
Jul 13 16:45:31.440954 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:31.440910    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-fdgtp" podStartSLOduration=2.916317668 podStartE2EDuration="7.440895371s" totalImagesPullingTime="3.415858195s" totalInitContainerRuntime="1.108719508s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:45:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:45:31.438556428 +0000 UTC m=+317.979936858" watchObservedRunningTime="2026-07-13 16:45:31.440895371 +0000 UTC m=+317.982275800"
Jul 13 16:45:32.371430 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:32.371387    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-78cc96c4cb-llcvt"
Jul 13 16:45:42.424573 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:42.424545    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5857456dfc-fdgtp"
Jul 13 16:45:50.040502 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:50.040467    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-8b9cf5f88-7jl4m"]
Jul 13 16:45:50.048466 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:50.048444    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-8b9cf5f88-7jl4m"
Jul 13 16:45:50.051332 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:50.051299    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 13 16:45:50.051465 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:50.051311    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-gt2bf\""
Jul 13 16:45:50.051465 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:50.051362    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 13 16:45:50.051465 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:50.051386    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 13 16:45:50.051465 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:50.051425    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 13 16:45:50.054971 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:50.054949    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-8b9cf5f88-7jl4m"]
Jul 13 16:45:50.117414 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:50.117381    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96b28493-8066-4d87-a7d7-fd7667387c48-tls-certs\") pod \"kube-auth-proxy-8b9cf5f88-7jl4m\" (UID: \"96b28493-8066-4d87-a7d7-fd7667387c48\") " pod="openshift-ingress/kube-auth-proxy-8b9cf5f88-7jl4m"
Jul 13 16:45:50.117570 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:50.117450    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/96b28493-8066-4d87-a7d7-fd7667387c48-tmp\") pod \"kube-auth-proxy-8b9cf5f88-7jl4m\" (UID: \"96b28493-8066-4d87-a7d7-fd7667387c48\") " pod="openshift-ingress/kube-auth-proxy-8b9cf5f88-7jl4m"
Jul 13 16:45:50.117570 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:50.117485    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8247s\" (UniqueName: \"kubernetes.io/projected/96b28493-8066-4d87-a7d7-fd7667387c48-kube-api-access-8247s\") pod \"kube-auth-proxy-8b9cf5f88-7jl4m\" (UID: \"96b28493-8066-4d87-a7d7-fd7667387c48\") " pod="openshift-ingress/kube-auth-proxy-8b9cf5f88-7jl4m"
Jul 13 16:45:50.218737 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:50.218697    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/96b28493-8066-4d87-a7d7-fd7667387c48-tmp\") pod \"kube-auth-proxy-8b9cf5f88-7jl4m\" (UID: \"96b28493-8066-4d87-a7d7-fd7667387c48\") " pod="openshift-ingress/kube-auth-proxy-8b9cf5f88-7jl4m"
Jul 13 16:45:50.218737 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:50.218742    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8247s\" (UniqueName: \"kubernetes.io/projected/96b28493-8066-4d87-a7d7-fd7667387c48-kube-api-access-8247s\") pod \"kube-auth-proxy-8b9cf5f88-7jl4m\" (UID: \"96b28493-8066-4d87-a7d7-fd7667387c48\") " pod="openshift-ingress/kube-auth-proxy-8b9cf5f88-7jl4m"
Jul 13 16:45:50.218932 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:50.218799    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96b28493-8066-4d87-a7d7-fd7667387c48-tls-certs\") pod \"kube-auth-proxy-8b9cf5f88-7jl4m\" (UID: \"96b28493-8066-4d87-a7d7-fd7667387c48\") " pod="openshift-ingress/kube-auth-proxy-8b9cf5f88-7jl4m"
Jul 13 16:45:50.218932 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:45:50.218902    2566 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 13 16:45:50.218994 ip-10-0-128-173 kubenswrapper[2566]: E0713 16:45:50.218967    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96b28493-8066-4d87-a7d7-fd7667387c48-tls-certs podName:96b28493-8066-4d87-a7d7-fd7667387c48 nodeName:}" failed. No retries permitted until 2026-07-13 16:45:50.718947567 +0000 UTC m=+337.260327991 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/96b28493-8066-4d87-a7d7-fd7667387c48-tls-certs") pod "kube-auth-proxy-8b9cf5f88-7jl4m" (UID: "96b28493-8066-4d87-a7d7-fd7667387c48") : secret "kube-auth-proxy-tls" not found
Jul 13 16:45:50.220944 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:50.220923    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/96b28493-8066-4d87-a7d7-fd7667387c48-tmp\") pod \"kube-auth-proxy-8b9cf5f88-7jl4m\" (UID: \"96b28493-8066-4d87-a7d7-fd7667387c48\") " pod="openshift-ingress/kube-auth-proxy-8b9cf5f88-7jl4m"
Jul 13 16:45:50.227839 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:50.227817    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8247s\" (UniqueName: \"kubernetes.io/projected/96b28493-8066-4d87-a7d7-fd7667387c48-kube-api-access-8247s\") pod \"kube-auth-proxy-8b9cf5f88-7jl4m\" (UID: \"96b28493-8066-4d87-a7d7-fd7667387c48\") " pod="openshift-ingress/kube-auth-proxy-8b9cf5f88-7jl4m"
Jul 13 16:45:50.723087 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:50.723046    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96b28493-8066-4d87-a7d7-fd7667387c48-tls-certs\") pod \"kube-auth-proxy-8b9cf5f88-7jl4m\" (UID: \"96b28493-8066-4d87-a7d7-fd7667387c48\") " pod="openshift-ingress/kube-auth-proxy-8b9cf5f88-7jl4m"
Jul 13 16:45:50.725510 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:50.725487    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96b28493-8066-4d87-a7d7-fd7667387c48-tls-certs\") pod \"kube-auth-proxy-8b9cf5f88-7jl4m\" (UID: \"96b28493-8066-4d87-a7d7-fd7667387c48\") " pod="openshift-ingress/kube-auth-proxy-8b9cf5f88-7jl4m"
Jul 13 16:45:50.959239 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:50.959205    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-8b9cf5f88-7jl4m"
Jul 13 16:45:51.096637 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:51.096612    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-8b9cf5f88-7jl4m"]
Jul 13 16:45:51.099452 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:45:51.099424    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96b28493_8066_4d87_a7d7_fd7667387c48.slice/crio-d11e90ed5d08d685c199c2ec324eaa5c3db630df1668675ef28d3e0917064f74 WatchSource:0}: Error finding container d11e90ed5d08d685c199c2ec324eaa5c3db630df1668675ef28d3e0917064f74: Status 404 returned error can't find the container with id d11e90ed5d08d685c199c2ec324eaa5c3db630df1668675ef28d3e0917064f74
Jul 13 16:45:51.513047 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:51.513015    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-8b9cf5f88-7jl4m" event={"ID":"96b28493-8066-4d87-a7d7-fd7667387c48","Type":"ContainerStarted","Data":"d11e90ed5d08d685c199c2ec324eaa5c3db630df1668675ef28d3e0917064f74"}
Jul 13 16:45:56.541238 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:56.541206    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-8b9cf5f88-7jl4m" event={"ID":"96b28493-8066-4d87-a7d7-fd7667387c48","Type":"ContainerStarted","Data":"e8bfd71031a39b1d1be666563438668da2a55366e98731c350fbbf74c9f36402"}
Jul 13 16:45:56.561476 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:45:56.561426    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-8b9cf5f88-7jl4m" podStartSLOduration=1.797807495 podStartE2EDuration="6.561410211s" totalImagesPullingTime="4.763602716s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:45:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:45:56.559708772 +0000 UTC m=+343.101089204" watchObservedRunningTime="2026-07-13 16:45:56.561410211 +0000 UTC m=+343.102790639"
Jul 13 16:47:33.394063 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:33.394025    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-gvm4k"]
Jul 13 16:47:33.397393 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:33.397376    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gvm4k"
Jul 13 16:47:33.400528 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:33.400504    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 13 16:47:33.400624 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:33.400540    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 13 16:47:33.401705 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:33.401686    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-q7rkl\""
Jul 13 16:47:33.401800 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:33.401716    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 13 16:47:33.401800 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:33.401777    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 13 16:47:33.408690 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:33.408669    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-gvm4k"]
Jul 13 16:47:33.446292 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:33.446223    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e80eca0-f8d1-4e84-8298-c5c501729a23-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-gvm4k\" (UID: \"2e80eca0-f8d1-4e84-8298-c5c501729a23\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gvm4k"
Jul 13 16:47:33.446292 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:33.446296    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dvbv\" (UniqueName: \"kubernetes.io/projected/2e80eca0-f8d1-4e84-8298-c5c501729a23-kube-api-access-7dvbv\") pod \"kuadrant-console-plugin-66b5774985-gvm4k\" (UID: \"2e80eca0-f8d1-4e84-8298-c5c501729a23\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gvm4k"
Jul 13 16:47:33.446492 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:33.446326    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2e80eca0-f8d1-4e84-8298-c5c501729a23-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-gvm4k\" (UID: \"2e80eca0-f8d1-4e84-8298-c5c501729a23\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gvm4k"
Jul 13 16:47:33.547471 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:33.547444    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e80eca0-f8d1-4e84-8298-c5c501729a23-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-gvm4k\" (UID: \"2e80eca0-f8d1-4e84-8298-c5c501729a23\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gvm4k"
Jul 13 16:47:33.547471 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:33.547473    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dvbv\" (UniqueName: \"kubernetes.io/projected/2e80eca0-f8d1-4e84-8298-c5c501729a23-kube-api-access-7dvbv\") pod \"kuadrant-console-plugin-66b5774985-gvm4k\" (UID: \"2e80eca0-f8d1-4e84-8298-c5c501729a23\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gvm4k"
Jul 13 16:47:33.547670 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:33.547494    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2e80eca0-f8d1-4e84-8298-c5c501729a23-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-gvm4k\" (UID: \"2e80eca0-f8d1-4e84-8298-c5c501729a23\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gvm4k"
Jul 13 16:47:33.548122 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:33.548099    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2e80eca0-f8d1-4e84-8298-c5c501729a23-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-gvm4k\" (UID: \"2e80eca0-f8d1-4e84-8298-c5c501729a23\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gvm4k"
Jul 13 16:47:33.549785 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:33.549764    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e80eca0-f8d1-4e84-8298-c5c501729a23-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-gvm4k\" (UID: \"2e80eca0-f8d1-4e84-8298-c5c501729a23\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gvm4k"
Jul 13 16:47:33.556982 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:33.556959    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dvbv\" (UniqueName: \"kubernetes.io/projected/2e80eca0-f8d1-4e84-8298-c5c501729a23-kube-api-access-7dvbv\") pod \"kuadrant-console-plugin-66b5774985-gvm4k\" (UID: \"2e80eca0-f8d1-4e84-8298-c5c501729a23\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gvm4k"
Jul 13 16:47:33.706561 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:33.706525    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gvm4k"
Jul 13 16:47:33.847424 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:33.847400    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-gvm4k"]
Jul 13 16:47:33.849173 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:47:33.849146    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2e80eca0_f8d1_4e84_8298_c5c501729a23.slice/crio-5edfbb841f7955440bba51eceb52264e39fc3c9e133912a4b6720d43185c9954 WatchSource:0}: Error finding container 5edfbb841f7955440bba51eceb52264e39fc3c9e133912a4b6720d43185c9954: Status 404 returned error can't find the container with id 5edfbb841f7955440bba51eceb52264e39fc3c9e133912a4b6720d43185c9954
Jul 13 16:47:33.989542 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:33.989466    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gvm4k" event={"ID":"2e80eca0-f8d1-4e84-8298-c5c501729a23","Type":"ContainerStarted","Data":"5edfbb841f7955440bba51eceb52264e39fc3c9e133912a4b6720d43185c9954"}
Jul 13 16:47:53.230284 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:53.230241    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b7d9d866f-2n8rn"]
Jul 13 16:47:53.317343 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:53.317304    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b7d9d866f-2n8rn"]
Jul 13 16:47:53.317529 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:53.317435    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b7d9d866f-2n8rn"
Jul 13 16:47:53.320765 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:53.320738    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 13 16:47:53.320889 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:53.320800    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 13 16:47:53.322308 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:53.322100    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 13 16:47:53.322308 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:53.322100    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 13 16:47:53.322308 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:53.322164    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-8bczz\""
Jul 13 16:47:53.322308 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:53.322232    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 13 16:47:53.322308 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:53.322283    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 13 16:47:53.322591 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:53.322330    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 13 16:47:53.327271 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:53.327227    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 13 16:47:53.442507 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:53.442466    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4de289db-da0f-47fa-ac8e-24268efe09c8-console-serving-cert\") pod \"console-6b7d9d866f-2n8rn\" (UID: \"4de289db-da0f-47fa-ac8e-24268efe09c8\") " pod="openshift-console/console-6b7d9d866f-2n8rn"
Jul 13 16:47:53.442507 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:53.442509    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzb6s\" (UniqueName: \"kubernetes.io/projected/4de289db-da0f-47fa-ac8e-24268efe09c8-kube-api-access-dzb6s\") pod \"console-6b7d9d866f-2n8rn\" (UID: \"4de289db-da0f-47fa-ac8e-24268efe09c8\") " pod="openshift-console/console-6b7d9d866f-2n8rn"
Jul 13 16:47:53.442730 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:53.442536    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4de289db-da0f-47fa-ac8e-24268efe09c8-console-oauth-config\") pod \"console-6b7d9d866f-2n8rn\" (UID: \"4de289db-da0f-47fa-ac8e-24268efe09c8\") " pod="openshift-console/console-6b7d9d866f-2n8rn"
Jul 13 16:47:53.442730 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:53.442595    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4de289db-da0f-47fa-ac8e-24268efe09c8-service-ca\") pod \"console-6b7d9d866f-2n8rn\" (UID: \"4de289db-da0f-47fa-ac8e-24268efe09c8\") " pod="openshift-console/console-6b7d9d866f-2n8rn"
Jul 13 16:47:53.442730 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:53.442633    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4de289db-da0f-47fa-ac8e-24268efe09c8-trusted-ca-bundle\") pod \"console-6b7d9d866f-2n8rn\" (UID: \"4de289db-da0f-47fa-ac8e-24268efe09c8\") " pod="openshift-console/console-6b7d9d866f-2n8rn"
Jul 13 16:47:53.442730 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:53.442682    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4de289db-da0f-47fa-ac8e-24268efe09c8-console-config\") pod \"console-6b7d9d866f-2n8rn\" (UID: \"4de289db-da0f-47fa-ac8e-24268efe09c8\") " pod="openshift-console/console-6b7d9d866f-2n8rn"
Jul 13 16:47:53.442730 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:53.442726    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4de289db-da0f-47fa-ac8e-24268efe09c8-oauth-serving-cert\") pod \"console-6b7d9d866f-2n8rn\" (UID: \"4de289db-da0f-47fa-ac8e-24268efe09c8\") " pod="openshift-console/console-6b7d9d866f-2n8rn"
Jul 13 16:47:53.544826 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:53.544264    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4de289db-da0f-47fa-ac8e-24268efe09c8-console-serving-cert\") pod \"console-6b7d9d866f-2n8rn\" (UID: \"4de289db-da0f-47fa-ac8e-24268efe09c8\") " pod="openshift-console/console-6b7d9d866f-2n8rn"
Jul 13 16:47:53.544826 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:53.544341    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzb6s\" (UniqueName: \"kubernetes.io/projected/4de289db-da0f-47fa-ac8e-24268efe09c8-kube-api-access-dzb6s\") pod \"console-6b7d9d866f-2n8rn\" (UID: \"4de289db-da0f-47fa-ac8e-24268efe09c8\") " pod="openshift-console/console-6b7d9d866f-2n8rn"
Jul 13 16:47:53.544826 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:53.544380    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4de289db-da0f-47fa-ac8e-24268efe09c8-console-oauth-config\") pod \"console-6b7d9d866f-2n8rn\" (UID: \"4de289db-da0f-47fa-ac8e-24268efe09c8\") " pod="openshift-console/console-6b7d9d866f-2n8rn"
Jul 13 16:47:53.544826 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:53.544430    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4de289db-da0f-47fa-ac8e-24268efe09c8-service-ca\") pod \"console-6b7d9d866f-2n8rn\" (UID: \"4de289db-da0f-47fa-ac8e-24268efe09c8\") " pod="openshift-console/console-6b7d9d866f-2n8rn"
Jul 13 16:47:53.544826 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:53.544458    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4de289db-da0f-47fa-ac8e-24268efe09c8-trusted-ca-bundle\") pod \"console-6b7d9d866f-2n8rn\" (UID: \"4de289db-da0f-47fa-ac8e-24268efe09c8\") " pod="openshift-console/console-6b7d9d866f-2n8rn"
Jul 13 16:47:53.544826 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:53.544512    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4de289db-da0f-47fa-ac8e-24268efe09c8-console-config\") pod \"console-6b7d9d866f-2n8rn\" (UID: \"4de289db-da0f-47fa-ac8e-24268efe09c8\") " pod="openshift-console/console-6b7d9d866f-2n8rn"
Jul 13 16:47:53.544826 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:53.544567    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4de289db-da0f-47fa-ac8e-24268efe09c8-oauth-serving-cert\") pod \"console-6b7d9d866f-2n8rn\" (UID: \"4de289db-da0f-47fa-ac8e-24268efe09c8\") " pod="openshift-console/console-6b7d9d866f-2n8rn"
Jul 13 16:47:53.545589 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:53.545556    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4de289db-da0f-47fa-ac8e-24268efe09c8-oauth-serving-cert\") pod \"console-6b7d9d866f-2n8rn\" (UID: \"4de289db-da0f-47fa-ac8e-24268efe09c8\") " pod="openshift-console/console-6b7d9d866f-2n8rn"
Jul 13 16:47:53.546216 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:53.546168    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4de289db-da0f-47fa-ac8e-24268efe09c8-console-config\") pod \"console-6b7d9d866f-2n8rn\" (UID: \"4de289db-da0f-47fa-ac8e-24268efe09c8\") " pod="openshift-console/console-6b7d9d866f-2n8rn"
Jul 13 16:47:53.546349 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:53.546331    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4de289db-da0f-47fa-ac8e-24268efe09c8-service-ca\") pod \"console-6b7d9d866f-2n8rn\" (UID: \"4de289db-da0f-47fa-ac8e-24268efe09c8\") " pod="openshift-console/console-6b7d9d866f-2n8rn"
Jul 13 16:47:53.546831 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:53.546798    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4de289db-da0f-47fa-ac8e-24268efe09c8-trusted-ca-bundle\") pod \"console-6b7d9d866f-2n8rn\" (UID: \"4de289db-da0f-47fa-ac8e-24268efe09c8\") " pod="openshift-console/console-6b7d9d866f-2n8rn"
Jul 13 16:47:53.548138 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:53.548115    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4de289db-da0f-47fa-ac8e-24268efe09c8-console-oauth-config\") pod \"console-6b7d9d866f-2n8rn\" (UID: \"4de289db-da0f-47fa-ac8e-24268efe09c8\") " pod="openshift-console/console-6b7d9d866f-2n8rn"
Jul 13 16:47:53.548268 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:53.548146    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4de289db-da0f-47fa-ac8e-24268efe09c8-console-serving-cert\") pod \"console-6b7d9d866f-2n8rn\" (UID: \"4de289db-da0f-47fa-ac8e-24268efe09c8\") " pod="openshift-console/console-6b7d9d866f-2n8rn"
Jul 13 16:47:53.556470 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:53.556450    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzb6s\" (UniqueName: \"kubernetes.io/projected/4de289db-da0f-47fa-ac8e-24268efe09c8-kube-api-access-dzb6s\") pod \"console-6b7d9d866f-2n8rn\" (UID: \"4de289db-da0f-47fa-ac8e-24268efe09c8\") " pod="openshift-console/console-6b7d9d866f-2n8rn"
Jul 13 16:47:53.630986 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:53.630953    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b7d9d866f-2n8rn"
Jul 13 16:47:56.187592 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:56.187568    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b7d9d866f-2n8rn"]
Jul 13 16:47:56.188387 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:47:56.188364    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4de289db_da0f_47fa_ac8e_24268efe09c8.slice/crio-7592787045a0683c5f2c66c29ab22fd926fdf076e5fa5491e3acb99c05d8c987 WatchSource:0}: Error finding container 7592787045a0683c5f2c66c29ab22fd926fdf076e5fa5491e3acb99c05d8c987: Status 404 returned error can't find the container with id 7592787045a0683c5f2c66c29ab22fd926fdf076e5fa5491e3acb99c05d8c987
Jul 13 16:47:57.120272 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:57.120206    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b7d9d866f-2n8rn" event={"ID":"4de289db-da0f-47fa-ac8e-24268efe09c8","Type":"ContainerStarted","Data":"ecd33fedf41a4da364c1169337e31760635fadcaa016e033b72533b1d15a5b95"}
Jul 13 16:47:57.120453 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:57.120290    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b7d9d866f-2n8rn" event={"ID":"4de289db-da0f-47fa-ac8e-24268efe09c8","Type":"ContainerStarted","Data":"7592787045a0683c5f2c66c29ab22fd926fdf076e5fa5491e3acb99c05d8c987"}
Jul 13 16:47:57.121627 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:57.121605    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gvm4k" event={"ID":"2e80eca0-f8d1-4e84-8298-c5c501729a23","Type":"ContainerStarted","Data":"bf949d96b2e05f4df629903d423e6057158a254580079ef1efb1519f30ade5e7"}
Jul 13 16:47:57.145656 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:57.145606    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b7d9d866f-2n8rn" podStartSLOduration=4.145591392 podStartE2EDuration="4.145591392s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:47:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:47:57.144136273 +0000 UTC m=+463.685516702" watchObservedRunningTime="2026-07-13 16:47:57.145591392 +0000 UTC m=+463.686971815"
Jul 13 16:47:57.163820 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:47:57.163772    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gvm4k" podStartSLOduration=1.91434032 podStartE2EDuration="24.163755489s" totalImagesPullingTime="22.249415169s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:47:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:47:57.162385195 +0000 UTC m=+463.703765623" watchObservedRunningTime="2026-07-13 16:47:57.163755489 +0000 UTC m=+463.705135917"
Jul 13 16:48:03.631146 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:48:03.631115    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b7d9d866f-2n8rn"
Jul 13 16:48:03.631733 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:48:03.631280    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b7d9d866f-2n8rn"
Jul 13 16:48:03.636225 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:48:03.636203    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b7d9d866f-2n8rn"
Jul 13 16:48:04.156716 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:48:04.156688    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b7d9d866f-2n8rn"
Jul 13 16:48:15.684003 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:48:15.683966    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 16:48:15.699897 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:48:15.699874    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"
Jul 13 16:48:15.703017 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:48:15.702986    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 13 16:48:15.704306 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:48:15.704281    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 16:48:15.723656 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:48:15.723632    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 16:48:15.743981 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:48:15.743936    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/814ec693-bc16-4ee1-a9a0-787b82cd9f6a-config-file\") pod \"limitador-limitador-54d6d988d5-pfdvl\" (UID: \"814ec693-bc16-4ee1-a9a0-787b82cd9f6a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"
Jul 13 16:48:15.744121 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:48:15.744099    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdjx8\" (UniqueName: \"kubernetes.io/projected/814ec693-bc16-4ee1-a9a0-787b82cd9f6a-kube-api-access-gdjx8\") pod \"limitador-limitador-54d6d988d5-pfdvl\" (UID: \"814ec693-bc16-4ee1-a9a0-787b82cd9f6a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"
Jul 13 16:48:15.845044 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:48:15.845008    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdjx8\" (UniqueName: \"kubernetes.io/projected/814ec693-bc16-4ee1-a9a0-787b82cd9f6a-kube-api-access-gdjx8\") pod \"limitador-limitador-54d6d988d5-pfdvl\" (UID: \"814ec693-bc16-4ee1-a9a0-787b82cd9f6a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"
Jul 13 16:48:15.845233 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:48:15.845053    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/814ec693-bc16-4ee1-a9a0-787b82cd9f6a-config-file\") pod \"limitador-limitador-54d6d988d5-pfdvl\" (UID: \"814ec693-bc16-4ee1-a9a0-787b82cd9f6a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"
Jul 13 16:48:15.845740 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:48:15.845713    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/814ec693-bc16-4ee1-a9a0-787b82cd9f6a-config-file\") pod \"limitador-limitador-54d6d988d5-pfdvl\" (UID: \"814ec693-bc16-4ee1-a9a0-787b82cd9f6a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"
Jul 13 16:48:15.854455 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:48:15.854429    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdjx8\" (UniqueName: \"kubernetes.io/projected/814ec693-bc16-4ee1-a9a0-787b82cd9f6a-kube-api-access-gdjx8\") pod \"limitador-limitador-54d6d988d5-pfdvl\" (UID: \"814ec693-bc16-4ee1-a9a0-787b82cd9f6a\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"
Jul 13 16:48:16.010802 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:48:16.010712    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"
Jul 13 16:48:16.154752 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:48:16.154727    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 16:48:16.156776 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:48:16.156751    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod814ec693_bc16_4ee1_a9a0_787b82cd9f6a.slice/crio-e0b383ce802ade978ce1193a4e0d395dbbad44e91065d7d1a4ffc2d5b5708c72 WatchSource:0}: Error finding container e0b383ce802ade978ce1193a4e0d395dbbad44e91065d7d1a4ffc2d5b5708c72: Status 404 returned error can't find the container with id e0b383ce802ade978ce1193a4e0d395dbbad44e91065d7d1a4ffc2d5b5708c72
Jul 13 16:48:16.208712 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:48:16.208680    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-pfdvl" event={"ID":"814ec693-bc16-4ee1-a9a0-787b82cd9f6a","Type":"ContainerStarted","Data":"e0b383ce802ade978ce1193a4e0d395dbbad44e91065d7d1a4ffc2d5b5708c72"}
Jul 13 16:48:20.231881 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:48:20.231842    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-pfdvl" event={"ID":"814ec693-bc16-4ee1-a9a0-787b82cd9f6a","Type":"ContainerStarted","Data":"bf10107527980292678a6c28f66c75195edf6b2bdebe8abd81b99fe4628466bb"}
Jul 13 16:48:20.232411 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:48:20.231990    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"
Jul 13 16:48:20.251027 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:48:20.250972    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-pfdvl" podStartSLOduration=1.864342366 podStartE2EDuration="5.250956944s" totalImagesPullingTime="3.386614578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:48:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:48:20.249979223 +0000 UTC m=+486.791359652" watchObservedRunningTime="2026-07-13 16:48:20.250956944 +0000 UTC m=+486.792337431"
Jul 13 16:48:31.238062 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:48:31.238031    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"
Jul 13 16:49:35.482474 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:49:35.482440    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-jpzhd"]
Jul 13 16:49:35.485656 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:49:35.485639    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-jpzhd"
Jul 13 16:49:35.489344 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:49:35.489323    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-x5cpm\""
Jul 13 16:49:35.489701 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:49:35.489683    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 13 16:49:35.503323 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:49:35.503303    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-jpzhd"]
Jul 13 16:49:35.521375 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:49:35.521348    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 16:49:35.560212 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:49:35.560177    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mscw7\" (UniqueName: \"kubernetes.io/projected/2430d611-a99a-4205-b431-1565dc09982f-kube-api-access-mscw7\") pod \"payload-pre-processing-74584bb9f8-jpzhd\" (UID: \"2430d611-a99a-4205-b431-1565dc09982f\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-jpzhd"
Jul 13 16:49:35.560385 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:49:35.560318    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2430d611-a99a-4205-b431-1565dc09982f-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-jpzhd\" (UID: \"2430d611-a99a-4205-b431-1565dc09982f\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-jpzhd"
Jul 13 16:49:35.661270 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:49:35.661213    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mscw7\" (UniqueName: \"kubernetes.io/projected/2430d611-a99a-4205-b431-1565dc09982f-kube-api-access-mscw7\") pod \"payload-pre-processing-74584bb9f8-jpzhd\" (UID: \"2430d611-a99a-4205-b431-1565dc09982f\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-jpzhd"
Jul 13 16:49:35.661452 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:49:35.661335    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2430d611-a99a-4205-b431-1565dc09982f-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-jpzhd\" (UID: \"2430d611-a99a-4205-b431-1565dc09982f\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-jpzhd"
Jul 13 16:49:35.662030 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:49:35.662010    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2430d611-a99a-4205-b431-1565dc09982f-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-jpzhd\" (UID: \"2430d611-a99a-4205-b431-1565dc09982f\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-jpzhd"
Jul 13 16:49:35.670110 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:49:35.670080    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mscw7\" (UniqueName: \"kubernetes.io/projected/2430d611-a99a-4205-b431-1565dc09982f-kube-api-access-mscw7\") pod \"payload-pre-processing-74584bb9f8-jpzhd\" (UID: \"2430d611-a99a-4205-b431-1565dc09982f\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-jpzhd"
Jul 13 16:49:35.796053 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:49:35.795968    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-jpzhd"
Jul 13 16:49:35.925064 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:49:35.925034    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-jpzhd"]
Jul 13 16:49:35.926291 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:49:35.926257    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2430d611_a99a_4205_b431_1565dc09982f.slice/crio-eedc37610a971f7a959b19e8daa39ec2274300dfa8591792bdad129c16239fdb WatchSource:0}: Error finding container eedc37610a971f7a959b19e8daa39ec2274300dfa8591792bdad129c16239fdb: Status 404 returned error can't find the container with id eedc37610a971f7a959b19e8daa39ec2274300dfa8591792bdad129c16239fdb
Jul 13 16:49:36.590923 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:49:36.590883    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-jpzhd" event={"ID":"2430d611-a99a-4205-b431-1565dc09982f","Type":"ContainerStarted","Data":"eedc37610a971f7a959b19e8daa39ec2274300dfa8591792bdad129c16239fdb"}
Jul 13 16:49:37.596740 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:49:37.596664    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-jpzhd" event={"ID":"2430d611-a99a-4205-b431-1565dc09982f","Type":"ContainerStarted","Data":"a24704980bfae8964c24cd99b52247fe19cc990642afe9d7e3815f9025714c92"}
Jul 13 16:49:37.597070 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:49:37.596771    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-jpzhd"
Jul 13 16:49:37.619718 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:49:37.619668    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-74584bb9f8-jpzhd" podStartSLOduration=1.314704119 podStartE2EDuration="2.619653284s" totalImagesPullingTime="1.304949165s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:49:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:49:37.617947672 +0000 UTC m=+564.159328101" watchObservedRunningTime="2026-07-13 16:49:37.619653284 +0000 UTC m=+564.161033712"
Jul 13 16:49:48.603439 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:49:48.603377    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-74584bb9f8-jpzhd" podUID="2430d611-a99a-4205-b431-1565dc09982f" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.29:9004: connect: connection refused"
Jul 13 16:49:58.602422 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:49:58.602394    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-jpzhd"
Jul 13 16:50:00.291316 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:50:00.291282    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 16:50:12.865856 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:50:12.865814    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-zzv62"]
Jul 13 16:50:12.870380 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:50:12.870356    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zzv62"
Jul 13 16:50:12.877116 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:50:12.877093    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 13 16:50:12.877116 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:50:12.877109    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 13 16:50:12.877292 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:50:12.877093    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 13 16:50:12.877292 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:50:12.877141    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-xms2v\""
Jul 13 16:50:12.885299 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:50:12.885277    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-zzv62"]
Jul 13 16:50:13.004813 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:50:13.004771    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d596154-f0e8-4473-b049-790311b438a8-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zzv62\" (UID: \"6d596154-f0e8-4473-b049-790311b438a8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zzv62"
Jul 13 16:50:13.004999 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:50:13.004842    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d596154-f0e8-4473-b049-790311b438a8-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zzv62\" (UID: \"6d596154-f0e8-4473-b049-790311b438a8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zzv62"
Jul 13 16:50:13.004999 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:50:13.004867    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d596154-f0e8-4473-b049-790311b438a8-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zzv62\" (UID: \"6d596154-f0e8-4473-b049-790311b438a8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zzv62"
Jul 13 16:50:13.004999 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:50:13.004896    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6r8l\" (UniqueName: \"kubernetes.io/projected/6d596154-f0e8-4473-b049-790311b438a8-kube-api-access-g6r8l\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zzv62\" (UID: \"6d596154-f0e8-4473-b049-790311b438a8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zzv62"
Jul 13 16:50:13.004999 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:50:13.004917    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d596154-f0e8-4473-b049-790311b438a8-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zzv62\" (UID: \"6d596154-f0e8-4473-b049-790311b438a8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zzv62"
Jul 13 16:50:13.004999 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:50:13.004955    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d596154-f0e8-4473-b049-790311b438a8-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zzv62\" (UID: \"6d596154-f0e8-4473-b049-790311b438a8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zzv62"
Jul 13 16:50:13.105608 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:50:13.105567    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d596154-f0e8-4473-b049-790311b438a8-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zzv62\" (UID: \"6d596154-f0e8-4473-b049-790311b438a8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zzv62"
Jul 13 16:50:13.105808 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:50:13.105639    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d596154-f0e8-4473-b049-790311b438a8-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zzv62\" (UID: \"6d596154-f0e8-4473-b049-790311b438a8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zzv62"
Jul 13 16:50:13.105808 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:50:13.105671    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d596154-f0e8-4473-b049-790311b438a8-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zzv62\" (UID: \"6d596154-f0e8-4473-b049-790311b438a8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zzv62"
Jul 13 16:50:13.105808 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:50:13.105701    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6r8l\" (UniqueName: \"kubernetes.io/projected/6d596154-f0e8-4473-b049-790311b438a8-kube-api-access-g6r8l\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zzv62\" (UID: \"6d596154-f0e8-4473-b049-790311b438a8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zzv62"
Jul 13 16:50:13.105978 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:50:13.105840    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d596154-f0e8-4473-b049-790311b438a8-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zzv62\" (UID: \"6d596154-f0e8-4473-b049-790311b438a8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zzv62"
Jul 13 16:50:13.105978 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:50:13.105878    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d596154-f0e8-4473-b049-790311b438a8-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zzv62\" (UID: \"6d596154-f0e8-4473-b049-790311b438a8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zzv62"
Jul 13 16:50:13.106080 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:50:13.106055    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d596154-f0e8-4473-b049-790311b438a8-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zzv62\" (UID: \"6d596154-f0e8-4473-b049-790311b438a8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zzv62"
Jul 13 16:50:13.106121 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:50:13.106093    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d596154-f0e8-4473-b049-790311b438a8-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zzv62\" (UID: \"6d596154-f0e8-4473-b049-790311b438a8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zzv62"
Jul 13 16:50:13.106185 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:50:13.106168    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d596154-f0e8-4473-b049-790311b438a8-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zzv62\" (UID: \"6d596154-f0e8-4473-b049-790311b438a8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zzv62"
Jul 13 16:50:13.108032 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:50:13.108008    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d596154-f0e8-4473-b049-790311b438a8-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zzv62\" (UID: \"6d596154-f0e8-4473-b049-790311b438a8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zzv62"
Jul 13 16:50:13.108158 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:50:13.108142    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d596154-f0e8-4473-b049-790311b438a8-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zzv62\" (UID: \"6d596154-f0e8-4473-b049-790311b438a8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zzv62"
Jul 13 16:50:13.116356 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:50:13.116330    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6r8l\" (UniqueName: \"kubernetes.io/projected/6d596154-f0e8-4473-b049-790311b438a8-kube-api-access-g6r8l\") pod \"e2e-distinct-simulated-kserve-59975d69f9-zzv62\" (UID: \"6d596154-f0e8-4473-b049-790311b438a8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zzv62"
Jul 13 16:50:13.179874 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:50:13.179842    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zzv62"
Jul 13 16:50:13.322305 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:50:13.322276    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-zzv62"]
Jul 13 16:50:13.325037 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:50:13.325008    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d596154_f0e8_4473_b049_790311b438a8.slice/crio-70473b088aa7f82d47f4c3774ce9c14511f619cad1e00f83d0bcf409f71b72cf WatchSource:0}: Error finding container 70473b088aa7f82d47f4c3774ce9c14511f619cad1e00f83d0bcf409f71b72cf: Status 404 returned error can't find the container with id 70473b088aa7f82d47f4c3774ce9c14511f619cad1e00f83d0bcf409f71b72cf
Jul 13 16:50:13.712714 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:50:13.712673    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 16:50:13.768542 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:50:13.768503    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zzv62" event={"ID":"6d596154-f0e8-4473-b049-790311b438a8","Type":"ContainerStarted","Data":"70473b088aa7f82d47f4c3774ce9c14511f619cad1e00f83d0bcf409f71b72cf"}
Jul 13 16:50:13.950804 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:50:13.950770    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gdqcb_6a33d4fc-2988-4f88-a92c-212d95ec2060/ovn-acl-logging/0.log"
Jul 13 16:50:13.952182 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:50:13.952153    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gdqcb_6a33d4fc-2988-4f88-a92c-212d95ec2060/ovn-acl-logging/0.log"
Jul 13 16:50:14.774841 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:50:14.774806    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zzv62" event={"ID":"6d596154-f0e8-4473-b049-790311b438a8","Type":"ContainerStarted","Data":"0b105122f740d65acda0e5f6025b15b9023aa9b60c3a7161570e0d894d9e1c3e"}
Jul 13 16:50:14.795853 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:50:14.795773    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zzv62" podStartSLOduration=1.772072718 podStartE2EDuration="2.795756421s" totalImagesPullingTime="1.023683703s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:50:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:50:14.795027016 +0000 UTC m=+601.336407446" watchObservedRunningTime="2026-07-13 16:50:14.795756421 +0000 UTC m=+601.337136850"
Jul 13 16:50:21.598905 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:50:21.598870    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 16:50:23.180669 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:50:23.180637    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zzv62"
Jul 13 16:50:23.181025 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:50:23.180695    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zzv62"
Jul 13 16:50:23.193122 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:50:23.193097    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zzv62"
Jul 13 16:50:23.830369 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:50:23.830343    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-zzv62"
Jul 13 16:50:32.804351 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:50:32.804313    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 16:51:01.609619 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:51:01.609588    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-2mkqj"]
Jul 13 16:51:01.614728 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:51:01.614709    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2mkqj"
Jul 13 16:51:01.621376 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:51:01.621073    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 13 16:51:01.627017 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:51:01.626996    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-2mkqj"]
Jul 13 16:51:01.729876 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:51:01.729838    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24a2fe54-5509-4c12-a86e-d5e77351f94b-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2mkqj\" (UID: \"24a2fe54-5509-4c12-a86e-d5e77351f94b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2mkqj"
Jul 13 16:51:01.729876 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:51:01.729874    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnprv\" (UniqueName: \"kubernetes.io/projected/24a2fe54-5509-4c12-a86e-d5e77351f94b-kube-api-access-cnprv\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2mkqj\" (UID: \"24a2fe54-5509-4c12-a86e-d5e77351f94b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2mkqj"
Jul 13 16:51:01.730085 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:51:01.729998    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24a2fe54-5509-4c12-a86e-d5e77351f94b-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2mkqj\" (UID: \"24a2fe54-5509-4c12-a86e-d5e77351f94b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2mkqj"
Jul 13 16:51:01.730085 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:51:01.730030    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24a2fe54-5509-4c12-a86e-d5e77351f94b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2mkqj\" (UID: \"24a2fe54-5509-4c12-a86e-d5e77351f94b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2mkqj"
Jul 13 16:51:01.730085 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:51:01.730057    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24a2fe54-5509-4c12-a86e-d5e77351f94b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2mkqj\" (UID: \"24a2fe54-5509-4c12-a86e-d5e77351f94b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2mkqj"
Jul 13 16:51:01.730085 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:51:01.730079    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24a2fe54-5509-4c12-a86e-d5e77351f94b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2mkqj\" (UID: \"24a2fe54-5509-4c12-a86e-d5e77351f94b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2mkqj"
Jul 13 16:51:01.831128 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:51:01.831094    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24a2fe54-5509-4c12-a86e-d5e77351f94b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2mkqj\" (UID: \"24a2fe54-5509-4c12-a86e-d5e77351f94b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2mkqj"
Jul 13 16:51:01.831128 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:51:01.831128    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24a2fe54-5509-4c12-a86e-d5e77351f94b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2mkqj\" (UID: \"24a2fe54-5509-4c12-a86e-d5e77351f94b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2mkqj"
Jul 13 16:51:01.831398 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:51:01.831176    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24a2fe54-5509-4c12-a86e-d5e77351f94b-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2mkqj\" (UID: \"24a2fe54-5509-4c12-a86e-d5e77351f94b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2mkqj"
Jul 13 16:51:01.831398 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:51:01.831194    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnprv\" (UniqueName: \"kubernetes.io/projected/24a2fe54-5509-4c12-a86e-d5e77351f94b-kube-api-access-cnprv\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2mkqj\" (UID: \"24a2fe54-5509-4c12-a86e-d5e77351f94b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2mkqj"
Jul 13 16:51:01.831398 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:51:01.831278    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24a2fe54-5509-4c12-a86e-d5e77351f94b-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2mkqj\" (UID: \"24a2fe54-5509-4c12-a86e-d5e77351f94b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2mkqj"
Jul 13 16:51:01.831398 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:51:01.831310    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24a2fe54-5509-4c12-a86e-d5e77351f94b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2mkqj\" (UID: \"24a2fe54-5509-4c12-a86e-d5e77351f94b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2mkqj"
Jul 13 16:51:01.831686 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:51:01.831666    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24a2fe54-5509-4c12-a86e-d5e77351f94b-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2mkqj\" (UID: \"24a2fe54-5509-4c12-a86e-d5e77351f94b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2mkqj"
Jul 13 16:51:01.831747 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:51:01.831723    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24a2fe54-5509-4c12-a86e-d5e77351f94b-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2mkqj\" (UID: \"24a2fe54-5509-4c12-a86e-d5e77351f94b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2mkqj"
Jul 13 16:51:01.831747 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:51:01.831737    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24a2fe54-5509-4c12-a86e-d5e77351f94b-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2mkqj\" (UID: \"24a2fe54-5509-4c12-a86e-d5e77351f94b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2mkqj"
Jul 13 16:51:01.833461 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:51:01.833442    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24a2fe54-5509-4c12-a86e-d5e77351f94b-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2mkqj\" (UID: \"24a2fe54-5509-4c12-a86e-d5e77351f94b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2mkqj"
Jul 13 16:51:01.833624 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:51:01.833605    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24a2fe54-5509-4c12-a86e-d5e77351f94b-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2mkqj\" (UID: \"24a2fe54-5509-4c12-a86e-d5e77351f94b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2mkqj"
Jul 13 16:51:01.843993 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:51:01.843969    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnprv\" (UniqueName: \"kubernetes.io/projected/24a2fe54-5509-4c12-a86e-d5e77351f94b-kube-api-access-cnprv\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2mkqj\" (UID: \"24a2fe54-5509-4c12-a86e-d5e77351f94b\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2mkqj"
Jul 13 16:51:01.924544 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:51:01.924510    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2mkqj"
Jul 13 16:51:02.064182 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:51:02.064153    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-2mkqj"]
Jul 13 16:51:02.065742 ip-10-0-128-173 kubenswrapper[2566]: W0713 16:51:02.065720    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24a2fe54_5509_4c12_a86e_d5e77351f94b.slice/crio-e675ba6d46ef51dd4ba1c80e9819bc4490069a37b5c1f4a3d0bacc2d10f921a0 WatchSource:0}: Error finding container e675ba6d46ef51dd4ba1c80e9819bc4490069a37b5c1f4a3d0bacc2d10f921a0: Status 404 returned error can't find the container with id e675ba6d46ef51dd4ba1c80e9819bc4490069a37b5c1f4a3d0bacc2d10f921a0
Jul 13 16:51:02.066971 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:51:02.066956    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 16:51:03.002090 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:51:03.002049    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2mkqj" event={"ID":"24a2fe54-5509-4c12-a86e-d5e77351f94b","Type":"ContainerStarted","Data":"a334c3b94d0fc95e32094fd412c8d30bc33fcf126589e20e96116572d8ce9606"}
Jul 13 16:51:03.002483 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:51:03.002109    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2mkqj" event={"ID":"24a2fe54-5509-4c12-a86e-d5e77351f94b","Type":"ContainerStarted","Data":"e675ba6d46ef51dd4ba1c80e9819bc4490069a37b5c1f4a3d0bacc2d10f921a0"}
Jul 13 16:51:03.023995 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:51:03.023947    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2mkqj" podStartSLOduration=1.831683053 podStartE2EDuration="2.023933851s" totalImagesPullingTime="192.250798ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 16:51:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 16:51:03.02241617 +0000 UTC m=+649.563796598" watchObservedRunningTime="2026-07-13 16:51:03.023933851 +0000 UTC m=+649.565314280"
Jul 13 16:51:03.598870 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:51:03.598835    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 16:51:11.924666 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:51:11.924617    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2mkqj"
Jul 13 16:51:11.925051 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:51:11.924699    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2mkqj"
Jul 13 16:51:11.936910 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:51:11.936884    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2mkqj"
Jul 13 16:51:12.055004 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:51:12.054973    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2mkqj"
Jul 13 16:51:19.217288 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:51:19.217239    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 16:53:00.693115 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:53:00.693079    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 16:53:10.496524 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:53:10.496485    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 16:53:21.592405 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:53:21.592369    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 16:53:29.193585 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:53:29.193550    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 16:53:40.095960 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:53:40.095667    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 16:54:09.799520 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:54:09.799484    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 16:54:21.891888 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:54:21.891851    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 16:54:57.303608 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:54:57.303568    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 16:55:07.289818 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:55:07.289740    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 16:55:13.992064 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:55:13.992040    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gdqcb_6a33d4fc-2988-4f88-a92c-212d95ec2060/ovn-acl-logging/0.log"
Jul 13 16:55:13.994375 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:55:13.994353    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gdqcb_6a33d4fc-2988-4f88-a92c-212d95ec2060/ovn-acl-logging/0.log"
Jul 13 16:55:16.590488 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:55:16.590452    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 16:55:22.688396 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:55:22.688361    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 16:55:33.702327 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:55:33.702290    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 16:55:45.703605 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:55:45.703562    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 16:55:56.789803 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:55:56.789767    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 16:56:06.593682 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:56:06.593643    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 16:56:28.289327 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:56:28.289289    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 16:56:38.598532 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:56:38.598448    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 16:57:01.697302 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:57:01.697266    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 16:57:12.796113 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:57:12.796080    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 16:57:27.697770 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:57:27.697734    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 16:57:37.104019 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:57:37.103987    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 16:57:47.890377 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:57:47.890343    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 16:57:57.395043 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:57:57.395007    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 16:58:06.999548 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:58:06.999472    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 16:59:57.113123 ip-10-0-128-173 kubenswrapper[2566]: I0713 16:59:57.113083    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:00:07.009334 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:00:07.009297    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:00:14.021971 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:00:14.021937    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gdqcb_6a33d4fc-2988-4f88-a92c-212d95ec2060/ovn-acl-logging/0.log"
Jul 13 17:00:14.025273 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:00:14.025233    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gdqcb_6a33d4fc-2988-4f88-a92c-212d95ec2060/ovn-acl-logging/0.log"
Jul 13 17:00:20.902532 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:00:20.902498    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:00:32.888373 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:00:32.888335    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:00:44.895308 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:00:44.895272    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:00:54.491478 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:00:54.491440    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:01:03.813475 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:01:03.813434    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:01:14.798763 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:01:14.798680    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:01:26.799114 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:01:26.799076    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:01:34.897499 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:01:34.897461    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:01:45.686322 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:01:45.686284    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:02:07.590494 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:02:07.590455    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:02:16.800346 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:02:16.800309    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:02:26.693594 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:02:26.693559    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:02:39.194588 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:02:39.194508    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:02:55.386991 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:02:55.386956    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:03:09.600517 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:03:09.600477    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:03:19.095484 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:03:19.095448    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:03:32.588160 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:03:32.588130    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:03:41.821311 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:03:41.821277    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:03:55.292772 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:03:55.292737    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:04:04.715722 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:04:04.715686    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:04:15.087691 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:04:15.087611    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:04:25.090139 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:04:25.090105    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:04:38.295625 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:04:38.295587    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:04:45.991555 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:04:45.991523    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:04:58.893475 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:04:58.893440    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:05:09.804664 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:05:09.804627    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:05:14.060690 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:05:14.060665    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gdqcb_6a33d4fc-2988-4f88-a92c-212d95ec2060/ovn-acl-logging/0.log"
Jul 13 17:05:14.064416 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:05:14.064395    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gdqcb_6a33d4fc-2988-4f88-a92c-212d95ec2060/ovn-acl-logging/0.log"
Jul 13 17:05:23.106388 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:05:23.106354    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:05:36.396218 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:05:36.396136    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:05:45.601157 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:05:45.601130    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:05:55.493518 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:05:55.493482    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:06:02.985070 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:02.985028    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a5ed8a38-kmcdm"]
Jul 13 17:06:02.988626 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:02.988602    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a5ed8a38-kmcdm"
Jul 13 17:06:02.991979 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:02.991949    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 17:06:02.991979 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:02.991965    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fkks4\""
Jul 13 17:06:02.999989 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:02.999959    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a5ed8a38-kmcdm"]
Jul 13 17:06:03.131367 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:03.131316    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1caba73-cc08-4f09-b6ff-0a345bddf06d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a5ed8a38-kmcdm\" (UID: \"d1caba73-cc08-4f09-b6ff-0a345bddf06d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a5ed8a38-kmcdm"
Jul 13 17:06:03.232642 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:03.232598    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1caba73-cc08-4f09-b6ff-0a345bddf06d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a5ed8a38-kmcdm\" (UID: \"d1caba73-cc08-4f09-b6ff-0a345bddf06d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a5ed8a38-kmcdm"
Jul 13 17:06:03.233235 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:03.233210    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1caba73-cc08-4f09-b6ff-0a345bddf06d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-a5ed8a38-kmcdm\" (UID: \"d1caba73-cc08-4f09-b6ff-0a345bddf06d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a5ed8a38-kmcdm"
Jul 13 17:06:03.300437 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:03.300347    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a5ed8a38-kmcdm"
Jul 13 17:06:03.446789 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:03.446755    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a5ed8a38-kmcdm"]
Jul 13 17:06:03.448780 ip-10-0-128-173 kubenswrapper[2566]: W0713 17:06:03.448755    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1caba73_cc08_4f09_b6ff_0a345bddf06d.slice/crio-c7995ec86ee024d915a2c28cafc6a0bc197ba99b458155277d15100171ca29e3 WatchSource:0}: Error finding container c7995ec86ee024d915a2c28cafc6a0bc197ba99b458155277d15100171ca29e3: Status 404 returned error can't find the container with id c7995ec86ee024d915a2c28cafc6a0bc197ba99b458155277d15100171ca29e3
Jul 13 17:06:03.450638 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:03.450617    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 17:06:04.312241 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:04.312204    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a5ed8a38-kmcdm" event={"ID":"d1caba73-cc08-4f09-b6ff-0a345bddf06d","Type":"ContainerStarted","Data":"c7995ec86ee024d915a2c28cafc6a0bc197ba99b458155277d15100171ca29e3"}
Jul 13 17:06:06.324001 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:06.323911    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a5ed8a38-kmcdm" event={"ID":"d1caba73-cc08-4f09-b6ff-0a345bddf06d","Type":"ContainerStarted","Data":"48563d21dc3f94509619d2efe70c318fec08763a224fb2e91846f0b0e0f45413"}
Jul 13 17:06:06.340550 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:06.340502    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a5ed8a38-kmcdm" podStartSLOduration=2.012151887 podStartE2EDuration="4.340486062s" totalImagesPullingTime="2.328334175s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:06:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:06:06.340020828 +0000 UTC m=+1552.881401260" watchObservedRunningTime="2026-07-13 17:06:06.340486062 +0000 UTC m=+1552.881866495"
Jul 13 17:06:07.337689 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:07.337654    2566 generic.go:356] "Generic (PLEG): container finished" podID="d1caba73-cc08-4f09-b6ff-0a345bddf06d" containerID="48563d21dc3f94509619d2efe70c318fec08763a224fb2e91846f0b0e0f45413" exitCode=7
Jul 13 17:06:07.338063 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:07.337742    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a5ed8a38-kmcdm" event={"ID":"d1caba73-cc08-4f09-b6ff-0a345bddf06d","Type":"ContainerDied","Data":"48563d21dc3f94509619d2efe70c318fec08763a224fb2e91846f0b0e0f45413"}
Jul 13 17:06:07.338063 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:07.337892    2566 scope.go:117] "RemoveContainer" containerID="48563d21dc3f94509619d2efe70c318fec08763a224fb2e91846f0b0e0f45413"
Jul 13 17:06:08.299366 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:08.299334    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:06:08.344680 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:08.344611    2566 generic.go:356] "Generic (PLEG): container finished" podID="d1caba73-cc08-4f09-b6ff-0a345bddf06d" containerID="a94542bb5b359f710ab5981704c66f5a3fba620eb6c586f41eae9fbe634f04ba" exitCode=0
Jul 13 17:06:08.344680 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:08.344660    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a5ed8a38-kmcdm" event={"ID":"d1caba73-cc08-4f09-b6ff-0a345bddf06d","Type":"ContainerDied","Data":"a94542bb5b359f710ab5981704c66f5a3fba620eb6c586f41eae9fbe634f04ba"}
Jul 13 17:06:08.345105 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:08.344708    2566 scope.go:117] "RemoveContainer" containerID="48563d21dc3f94509619d2efe70c318fec08763a224fb2e91846f0b0e0f45413"
Jul 13 17:06:09.483709 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:09.483686    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a5ed8a38-kmcdm"
Jul 13 17:06:09.587978 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:09.587940    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1caba73-cc08-4f09-b6ff-0a345bddf06d-maas-api-service-ca\") pod \"d1caba73-cc08-4f09-b6ff-0a345bddf06d\" (UID: \"d1caba73-cc08-4f09-b6ff-0a345bddf06d\") "
Jul 13 17:06:09.588345 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:09.588323    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1caba73-cc08-4f09-b6ff-0a345bddf06d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d1caba73-cc08-4f09-b6ff-0a345bddf06d" (UID: "d1caba73-cc08-4f09-b6ff-0a345bddf06d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 17:06:09.689425 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:09.689394    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1caba73-cc08-4f09-b6ff-0a345bddf06d-maas-api-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:06:10.359554 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:10.359480    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a5ed8a38-kmcdm"
Jul 13 17:06:10.359554 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:10.359518    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-a5ed8a38-kmcdm" event={"ID":"d1caba73-cc08-4f09-b6ff-0a345bddf06d","Type":"ContainerDied","Data":"c7995ec86ee024d915a2c28cafc6a0bc197ba99b458155277d15100171ca29e3"}
Jul 13 17:06:10.359719 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:10.359560    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c7995ec86ee024d915a2c28cafc6a0bc197ba99b458155277d15100171ca29e3"
Jul 13 17:06:18.513186 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:18.513153    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:06:32.112670 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:32.112635    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:06:35.356092 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:35.356062    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51bf4736-zdnb8"]
Jul 13 17:06:35.356479 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:35.356456    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1caba73-cc08-4f09-b6ff-0a345bddf06d" containerName="revoke-keys"
Jul 13 17:06:35.356479 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:35.356467    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1caba73-cc08-4f09-b6ff-0a345bddf06d" containerName="revoke-keys"
Jul 13 17:06:35.356479 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:35.356479    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1caba73-cc08-4f09-b6ff-0a345bddf06d" containerName="revoke-keys"
Jul 13 17:06:35.356576 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:35.356485    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1caba73-cc08-4f09-b6ff-0a345bddf06d" containerName="revoke-keys"
Jul 13 17:06:35.356576 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:35.356560    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1caba73-cc08-4f09-b6ff-0a345bddf06d" containerName="revoke-keys"
Jul 13 17:06:35.356576 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:35.356567    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1caba73-cc08-4f09-b6ff-0a345bddf06d" containerName="revoke-keys"
Jul 13 17:06:35.359836 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:35.359819    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51bf4736-zdnb8"
Jul 13 17:06:35.362626 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:35.362607    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fkks4\""
Jul 13 17:06:35.362749 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:35.362644    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 17:06:35.367151 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:35.367128    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51bf4736-zdnb8"]
Jul 13 17:06:35.418137 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:35.418107    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21e717a5-8a62-48d1-990d-2e7eb8d89c9e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-51bf4736-zdnb8\" (UID: \"21e717a5-8a62-48d1-990d-2e7eb8d89c9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51bf4736-zdnb8"
Jul 13 17:06:35.519202 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:35.519169    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21e717a5-8a62-48d1-990d-2e7eb8d89c9e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-51bf4736-zdnb8\" (UID: \"21e717a5-8a62-48d1-990d-2e7eb8d89c9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51bf4736-zdnb8"
Jul 13 17:06:35.519802 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:35.519780    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21e717a5-8a62-48d1-990d-2e7eb8d89c9e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-51bf4736-zdnb8\" (UID: \"21e717a5-8a62-48d1-990d-2e7eb8d89c9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51bf4736-zdnb8"
Jul 13 17:06:35.671146 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:35.671116    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51bf4736-zdnb8"
Jul 13 17:06:35.805892 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:35.805864    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51bf4736-zdnb8"]
Jul 13 17:06:35.807989 ip-10-0-128-173 kubenswrapper[2566]: W0713 17:06:35.807963    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21e717a5_8a62_48d1_990d_2e7eb8d89c9e.slice/crio-8a91ea22a0e854268c1f0e838663acef727d935952be93c3c574feee582c15c6 WatchSource:0}: Error finding container 8a91ea22a0e854268c1f0e838663acef727d935952be93c3c574feee582c15c6: Status 404 returned error can't find the container with id 8a91ea22a0e854268c1f0e838663acef727d935952be93c3c574feee582c15c6
Jul 13 17:06:36.492816 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:36.492787    2566 generic.go:356] "Generic (PLEG): container finished" podID="21e717a5-8a62-48d1-990d-2e7eb8d89c9e" containerID="6dbcf16ddaafe1f2669a13c436bb2c9a6510485825369b4066977599d7f3c935" exitCode=7
Jul 13 17:06:36.493177 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:36.492842    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51bf4736-zdnb8" event={"ID":"21e717a5-8a62-48d1-990d-2e7eb8d89c9e","Type":"ContainerDied","Data":"6dbcf16ddaafe1f2669a13c436bb2c9a6510485825369b4066977599d7f3c935"}
Jul 13 17:06:36.493177 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:36.492904    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51bf4736-zdnb8" event={"ID":"21e717a5-8a62-48d1-990d-2e7eb8d89c9e","Type":"ContainerStarted","Data":"8a91ea22a0e854268c1f0e838663acef727d935952be93c3c574feee582c15c6"}
Jul 13 17:06:36.493177 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:36.493119    2566 scope.go:117] "RemoveContainer" containerID="6dbcf16ddaafe1f2669a13c436bb2c9a6510485825369b4066977599d7f3c935"
Jul 13 17:06:37.500536 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:37.500452    2566 generic.go:356] "Generic (PLEG): container finished" podID="21e717a5-8a62-48d1-990d-2e7eb8d89c9e" containerID="e8634375ce961015229759dac9f765c07bd5231d9856a8a0d7d6af781f88e7b8" exitCode=7
Jul 13 17:06:37.500536 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:37.500511    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51bf4736-zdnb8" event={"ID":"21e717a5-8a62-48d1-990d-2e7eb8d89c9e","Type":"ContainerDied","Data":"e8634375ce961015229759dac9f765c07bd5231d9856a8a0d7d6af781f88e7b8"}
Jul 13 17:06:37.501003 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:37.500565    2566 scope.go:117] "RemoveContainer" containerID="6dbcf16ddaafe1f2669a13c436bb2c9a6510485825369b4066977599d7f3c935"
Jul 13 17:06:37.501003 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:37.500805    2566 scope.go:117] "RemoveContainer" containerID="e8634375ce961015229759dac9f765c07bd5231d9856a8a0d7d6af781f88e7b8"
Jul 13 17:06:37.501100 ip-10-0-128-173 kubenswrapper[2566]: E0713 17:06:37.501024    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-51bf4736-zdnb8_odh-ai-gateway-infra(21e717a5-8a62-48d1-990d-2e7eb8d89c9e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51bf4736-zdnb8" podUID="21e717a5-8a62-48d1-990d-2e7eb8d89c9e"
Jul 13 17:06:38.507809 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:38.507776    2566 scope.go:117] "RemoveContainer" containerID="e8634375ce961015229759dac9f765c07bd5231d9856a8a0d7d6af781f88e7b8"
Jul 13 17:06:38.508141 ip-10-0-128-173 kubenswrapper[2566]: E0713 17:06:38.507977    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-51bf4736-zdnb8_odh-ai-gateway-infra(21e717a5-8a62-48d1-990d-2e7eb8d89c9e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51bf4736-zdnb8" podUID="21e717a5-8a62-48d1-990d-2e7eb8d89c9e"
Jul 13 17:06:52.981131 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:52.975389    2566 scope.go:117] "RemoveContainer" containerID="e8634375ce961015229759dac9f765c07bd5231d9856a8a0d7d6af781f88e7b8"
Jul 13 17:06:53.583758 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:53.583674    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51bf4736-zdnb8" event={"ID":"21e717a5-8a62-48d1-990d-2e7eb8d89c9e","Type":"ContainerStarted","Data":"7170d8a758c9a5ca7bfdd07f59c327fc2a718928bf79f4c2e8f184618581ecea"}
Jul 13 17:06:53.602470 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:53.602417    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51bf4736-zdnb8" podStartSLOduration=18.602401051 podStartE2EDuration="18.602401051s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:06:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:06:53.601657959 +0000 UTC m=+1600.143038388" watchObservedRunningTime="2026-07-13 17:06:53.602401051 +0000 UTC m=+1600.143781479"
Jul 13 17:06:54.591977 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:54.591939    2566 generic.go:356] "Generic (PLEG): container finished" podID="21e717a5-8a62-48d1-990d-2e7eb8d89c9e" containerID="7170d8a758c9a5ca7bfdd07f59c327fc2a718928bf79f4c2e8f184618581ecea" exitCode=0
Jul 13 17:06:54.592399 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:54.592011    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51bf4736-zdnb8" event={"ID":"21e717a5-8a62-48d1-990d-2e7eb8d89c9e","Type":"ContainerDied","Data":"7170d8a758c9a5ca7bfdd07f59c327fc2a718928bf79f4c2e8f184618581ecea"}
Jul 13 17:06:54.592399 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:54.592073    2566 scope.go:117] "RemoveContainer" containerID="e8634375ce961015229759dac9f765c07bd5231d9856a8a0d7d6af781f88e7b8"
Jul 13 17:06:54.612536 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:54.612512    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51bf4736-zdnb8"]
Jul 13 17:06:55.720319 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:55.720298    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51bf4736-zdnb8"
Jul 13 17:06:55.802559 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:55.802519    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21e717a5-8a62-48d1-990d-2e7eb8d89c9e-maas-api-service-ca\") pod \"21e717a5-8a62-48d1-990d-2e7eb8d89c9e\" (UID: \"21e717a5-8a62-48d1-990d-2e7eb8d89c9e\") "
Jul 13 17:06:55.802845 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:55.802823    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21e717a5-8a62-48d1-990d-2e7eb8d89c9e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "21e717a5-8a62-48d1-990d-2e7eb8d89c9e" (UID: "21e717a5-8a62-48d1-990d-2e7eb8d89c9e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 17:06:55.904045 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:55.903943    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21e717a5-8a62-48d1-990d-2e7eb8d89c9e-maas-api-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:06:56.604820 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:56.604789    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51bf4736-zdnb8"
Jul 13 17:06:56.604820 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:56.604813    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51bf4736-zdnb8" event={"ID":"21e717a5-8a62-48d1-990d-2e7eb8d89c9e","Type":"ContainerDied","Data":"8a91ea22a0e854268c1f0e838663acef727d935952be93c3c574feee582c15c6"}
Jul 13 17:06:56.605033 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:56.604864    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8a91ea22a0e854268c1f0e838663acef727d935952be93c3c574feee582c15c6"
Jul 13 17:06:56.640856 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:56.640822    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51bf4736-zdnb8"]
Jul 13 17:06:56.646591 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:56.646566    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51bf4736-zdnb8"]
Jul 13 17:06:57.981419 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:06:57.981387    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21e717a5-8a62-48d1-990d-2e7eb8d89c9e" path="/var/lib/kubelet/pods/21e717a5-8a62-48d1-990d-2e7eb8d89c9e/volumes"
Jul 13 17:07:35.409749 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:35.409714    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51bf4736-znh2n"]
Jul 13 17:07:35.410294 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:35.410112    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21e717a5-8a62-48d1-990d-2e7eb8d89c9e" containerName="revoke-keys"
Jul 13 17:07:35.410294 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:35.410122    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="21e717a5-8a62-48d1-990d-2e7eb8d89c9e" containerName="revoke-keys"
Jul 13 17:07:35.410294 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:35.410131    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21e717a5-8a62-48d1-990d-2e7eb8d89c9e" containerName="revoke-keys"
Jul 13 17:07:35.410294 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:35.410137    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="21e717a5-8a62-48d1-990d-2e7eb8d89c9e" containerName="revoke-keys"
Jul 13 17:07:35.410294 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:35.410207    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="21e717a5-8a62-48d1-990d-2e7eb8d89c9e" containerName="revoke-keys"
Jul 13 17:07:35.410294 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:35.410217    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="21e717a5-8a62-48d1-990d-2e7eb8d89c9e" containerName="revoke-keys"
Jul 13 17:07:35.410294 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:35.410223    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="21e717a5-8a62-48d1-990d-2e7eb8d89c9e" containerName="revoke-keys"
Jul 13 17:07:35.413503 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:35.413487    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51bf4736-znh2n"
Jul 13 17:07:35.416691 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:35.416668    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 17:07:35.416804 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:35.416668    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fkks4\""
Jul 13 17:07:35.421671 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:35.421647    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51bf4736-znh2n"]
Jul 13 17:07:35.548460 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:35.548426    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/68afc24e-ba04-48e4-b379-83e8e7b49498-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-51bf4736-znh2n\" (UID: \"68afc24e-ba04-48e4-b379-83e8e7b49498\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51bf4736-znh2n"
Jul 13 17:07:35.649183 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:35.649151    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/68afc24e-ba04-48e4-b379-83e8e7b49498-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-51bf4736-znh2n\" (UID: \"68afc24e-ba04-48e4-b379-83e8e7b49498\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51bf4736-znh2n"
Jul 13 17:07:35.649791 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:35.649771    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/68afc24e-ba04-48e4-b379-83e8e7b49498-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-51bf4736-znh2n\" (UID: \"68afc24e-ba04-48e4-b379-83e8e7b49498\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51bf4736-znh2n"
Jul 13 17:07:35.724600 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:35.724567    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51bf4736-znh2n"
Jul 13 17:07:35.855279 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:35.855240    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51bf4736-znh2n"]
Jul 13 17:07:35.857219 ip-10-0-128-173 kubenswrapper[2566]: W0713 17:07:35.857191    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68afc24e_ba04_48e4_b379_83e8e7b49498.slice/crio-ae9f8925fcf37446cbc00d66b4682184c5a2f6db58ddd35dfb5db1e9b95fd9ce WatchSource:0}: Error finding container ae9f8925fcf37446cbc00d66b4682184c5a2f6db58ddd35dfb5db1e9b95fd9ce: Status 404 returned error can't find the container with id ae9f8925fcf37446cbc00d66b4682184c5a2f6db58ddd35dfb5db1e9b95fd9ce
Jul 13 17:07:36.801880 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:36.801793    2566 generic.go:356] "Generic (PLEG): container finished" podID="68afc24e-ba04-48e4-b379-83e8e7b49498" containerID="e18ad42889bd842618c9922d1681302134d9fd699b6bf6b2ffa0fee6a478382c" exitCode=0
Jul 13 17:07:36.801880 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:36.801829    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51bf4736-znh2n" event={"ID":"68afc24e-ba04-48e4-b379-83e8e7b49498","Type":"ContainerDied","Data":"e18ad42889bd842618c9922d1681302134d9fd699b6bf6b2ffa0fee6a478382c"}
Jul 13 17:07:36.801880 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:36.801866    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51bf4736-znh2n" event={"ID":"68afc24e-ba04-48e4-b379-83e8e7b49498","Type":"ContainerStarted","Data":"ae9f8925fcf37446cbc00d66b4682184c5a2f6db58ddd35dfb5db1e9b95fd9ce"}
Jul 13 17:07:37.948082 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:37.948053    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51bf4736-znh2n"
Jul 13 17:07:38.069063 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:38.068996    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/68afc24e-ba04-48e4-b379-83e8e7b49498-maas-api-service-ca\") pod \"68afc24e-ba04-48e4-b379-83e8e7b49498\" (UID: \"68afc24e-ba04-48e4-b379-83e8e7b49498\") "
Jul 13 17:07:38.069406 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:38.069382    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68afc24e-ba04-48e4-b379-83e8e7b49498-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "68afc24e-ba04-48e4-b379-83e8e7b49498" (UID: "68afc24e-ba04-48e4-b379-83e8e7b49498"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 17:07:38.169912 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:38.169886    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/68afc24e-ba04-48e4-b379-83e8e7b49498-maas-api-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:07:38.500718 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:38.500682    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:07:38.814318 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:38.814223    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51bf4736-znh2n"
Jul 13 17:07:38.814318 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:38.814268    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-51bf4736-znh2n" event={"ID":"68afc24e-ba04-48e4-b379-83e8e7b49498","Type":"ContainerDied","Data":"ae9f8925fcf37446cbc00d66b4682184c5a2f6db58ddd35dfb5db1e9b95fd9ce"}
Jul 13 17:07:38.814318 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:38.814317    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ae9f8925fcf37446cbc00d66b4682184c5a2f6db58ddd35dfb5db1e9b95fd9ce"
Jul 13 17:07:51.988073 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:51.988039    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:07:54.175404 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:54.175366    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-97e65980-bx5n8"]
Jul 13 17:07:54.175813 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:54.175796    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21e717a5-8a62-48d1-990d-2e7eb8d89c9e" containerName="revoke-keys"
Jul 13 17:07:54.175813 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:54.175808    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="21e717a5-8a62-48d1-990d-2e7eb8d89c9e" containerName="revoke-keys"
Jul 13 17:07:54.175899 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:54.175823    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68afc24e-ba04-48e4-b379-83e8e7b49498" containerName="revoke-keys"
Jul 13 17:07:54.175899 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:54.175829    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="68afc24e-ba04-48e4-b379-83e8e7b49498" containerName="revoke-keys"
Jul 13 17:07:54.175899 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:54.175890    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="68afc24e-ba04-48e4-b379-83e8e7b49498" containerName="revoke-keys"
Jul 13 17:07:54.178887 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:54.178871    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-97e65980-bx5n8"
Jul 13 17:07:54.181930 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:54.181911    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 17:07:54.182033 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:54.181989    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fkks4\""
Jul 13 17:07:54.186797 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:54.186434    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-97e65980-bx5n8"]
Jul 13 17:07:54.321379 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:54.321340    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd6e5402-33e7-45d0-847f-231117b037cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-97e65980-bx5n8\" (UID: \"fd6e5402-33e7-45d0-847f-231117b037cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-97e65980-bx5n8"
Jul 13 17:07:54.422050 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:54.422017    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd6e5402-33e7-45d0-847f-231117b037cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-97e65980-bx5n8\" (UID: \"fd6e5402-33e7-45d0-847f-231117b037cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-97e65980-bx5n8"
Jul 13 17:07:54.422642 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:54.422621    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd6e5402-33e7-45d0-847f-231117b037cc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-97e65980-bx5n8\" (UID: \"fd6e5402-33e7-45d0-847f-231117b037cc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-97e65980-bx5n8"
Jul 13 17:07:54.490651 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:54.490583    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-97e65980-bx5n8"
Jul 13 17:07:54.617454 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:54.617429    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-97e65980-bx5n8"]
Jul 13 17:07:54.618802 ip-10-0-128-173 kubenswrapper[2566]: W0713 17:07:54.618776    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd6e5402_33e7_45d0_847f_231117b037cc.slice/crio-c587f0d4553c4def123deb66a60bae557112e1af85d4859c6d35653a9aa9657d WatchSource:0}: Error finding container c587f0d4553c4def123deb66a60bae557112e1af85d4859c6d35653a9aa9657d: Status 404 returned error can't find the container with id c587f0d4553c4def123deb66a60bae557112e1af85d4859c6d35653a9aa9657d
Jul 13 17:07:54.896388 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:54.896361    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-97e65980-bx5n8" event={"ID":"fd6e5402-33e7-45d0-847f-231117b037cc","Type":"ContainerStarted","Data":"f21da9f9064adbf046aeea62d8de46396b5dafaa7147072371dbd04a1e252770"}
Jul 13 17:07:54.896487 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:54.896408    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-97e65980-bx5n8" event={"ID":"fd6e5402-33e7-45d0-847f-231117b037cc","Type":"ContainerStarted","Data":"c587f0d4553c4def123deb66a60bae557112e1af85d4859c6d35653a9aa9657d"}
Jul 13 17:07:54.913446 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:54.913405    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-97e65980-bx5n8" podStartSLOduration=0.913391682 podStartE2EDuration="913.391682ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:07:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:07:54.911833999 +0000 UTC m=+1661.453214429" watchObservedRunningTime="2026-07-13 17:07:54.913391682 +0000 UTC m=+1661.454772110"
Jul 13 17:07:55.902697 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:55.902665    2566 generic.go:356] "Generic (PLEG): container finished" podID="fd6e5402-33e7-45d0-847f-231117b037cc" containerID="f21da9f9064adbf046aeea62d8de46396b5dafaa7147072371dbd04a1e252770" exitCode=7
Jul 13 17:07:55.903046 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:55.902749    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-97e65980-bx5n8" event={"ID":"fd6e5402-33e7-45d0-847f-231117b037cc","Type":"ContainerDied","Data":"f21da9f9064adbf046aeea62d8de46396b5dafaa7147072371dbd04a1e252770"}
Jul 13 17:07:55.903046 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:55.902875    2566 scope.go:117] "RemoveContainer" containerID="f21da9f9064adbf046aeea62d8de46396b5dafaa7147072371dbd04a1e252770"
Jul 13 17:07:56.909694 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:56.909611    2566 generic.go:356] "Generic (PLEG): container finished" podID="fd6e5402-33e7-45d0-847f-231117b037cc" containerID="a079598585c48dd514034a15885d5688888731dd8a8b279b245813de8b8367f4" exitCode=0
Jul 13 17:07:56.910049 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:56.909696    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-97e65980-bx5n8" event={"ID":"fd6e5402-33e7-45d0-847f-231117b037cc","Type":"ContainerDied","Data":"a079598585c48dd514034a15885d5688888731dd8a8b279b245813de8b8367f4"}
Jul 13 17:07:56.910049 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:56.909749    2566 scope.go:117] "RemoveContainer" containerID="f21da9f9064adbf046aeea62d8de46396b5dafaa7147072371dbd04a1e252770"
Jul 13 17:07:58.044111 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:58.044088    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-97e65980-bx5n8"
Jul 13 17:07:58.158197 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:58.158169    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd6e5402-33e7-45d0-847f-231117b037cc-maas-api-service-ca\") pod \"fd6e5402-33e7-45d0-847f-231117b037cc\" (UID: \"fd6e5402-33e7-45d0-847f-231117b037cc\") "
Jul 13 17:07:58.158500 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:58.158479    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd6e5402-33e7-45d0-847f-231117b037cc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fd6e5402-33e7-45d0-847f-231117b037cc" (UID: "fd6e5402-33e7-45d0-847f-231117b037cc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 17:07:58.258859 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:58.258784    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd6e5402-33e7-45d0-847f-231117b037cc-maas-api-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:07:58.921393 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:58.921365    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-97e65980-bx5n8"
Jul 13 17:07:58.921579 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:58.921407    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-97e65980-bx5n8" event={"ID":"fd6e5402-33e7-45d0-847f-231117b037cc","Type":"ContainerDied","Data":"c587f0d4553c4def123deb66a60bae557112e1af85d4859c6d35653a9aa9657d"}
Jul 13 17:07:58.921579 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:07:58.921448    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c587f0d4553c4def123deb66a60bae557112e1af85d4859c6d35653a9aa9657d"
Jul 13 17:08:09.897981 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:08:09.897947    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:08:24.892538 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:08:24.892502    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:08:26.338328 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:08:26.338290    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1890c174-pw89z"]
Jul 13 17:08:26.338717 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:08:26.338673    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd6e5402-33e7-45d0-847f-231117b037cc" containerName="revoke-keys"
Jul 13 17:08:26.338717 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:08:26.338683    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd6e5402-33e7-45d0-847f-231117b037cc" containerName="revoke-keys"
Jul 13 17:08:26.338717 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:08:26.338693    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd6e5402-33e7-45d0-847f-231117b037cc" containerName="revoke-keys"
Jul 13 17:08:26.338717 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:08:26.338698    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd6e5402-33e7-45d0-847f-231117b037cc" containerName="revoke-keys"
Jul 13 17:08:26.338908 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:08:26.338773    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd6e5402-33e7-45d0-847f-231117b037cc" containerName="revoke-keys"
Jul 13 17:08:26.341897 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:08:26.341881    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1890c174-pw89z"
Jul 13 17:08:26.344619 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:08:26.344596    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 17:08:26.344743 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:08:26.344640    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fkks4\""
Jul 13 17:08:26.348673 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:08:26.348648    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1890c174-pw89z"]
Jul 13 17:08:26.408097 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:08:26.408065    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfd75a32-fa82-4625-965d-88b0eea8f78a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-1890c174-pw89z\" (UID: \"cfd75a32-fa82-4625-965d-88b0eea8f78a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1890c174-pw89z"
Jul 13 17:08:26.508580 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:08:26.508545    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfd75a32-fa82-4625-965d-88b0eea8f78a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-1890c174-pw89z\" (UID: \"cfd75a32-fa82-4625-965d-88b0eea8f78a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1890c174-pw89z"
Jul 13 17:08:26.509174 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:08:26.509154    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfd75a32-fa82-4625-965d-88b0eea8f78a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-1890c174-pw89z\" (UID: \"cfd75a32-fa82-4625-965d-88b0eea8f78a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1890c174-pw89z"
Jul 13 17:08:26.653871 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:08:26.653788    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1890c174-pw89z"
Jul 13 17:08:26.786257 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:08:26.786220    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1890c174-pw89z"]
Jul 13 17:08:26.787914 ip-10-0-128-173 kubenswrapper[2566]: W0713 17:08:26.787879    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfd75a32_fa82_4625_965d_88b0eea8f78a.slice/crio-129133e4b120a432b098cc1def24de9d34a24b45940a283378d5b3087b62cb0e WatchSource:0}: Error finding container 129133e4b120a432b098cc1def24de9d34a24b45940a283378d5b3087b62cb0e: Status 404 returned error can't find the container with id 129133e4b120a432b098cc1def24de9d34a24b45940a283378d5b3087b62cb0e
Jul 13 17:08:27.054576 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:08:27.054549    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1890c174-pw89z" event={"ID":"cfd75a32-fa82-4625-965d-88b0eea8f78a","Type":"ContainerStarted","Data":"129133e4b120a432b098cc1def24de9d34a24b45940a283378d5b3087b62cb0e"}
Jul 13 17:08:28.061376 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:08:28.061340    2566 generic.go:356] "Generic (PLEG): container finished" podID="cfd75a32-fa82-4625-965d-88b0eea8f78a" containerID="eb245781f9e7325ef6fd56f8e2b8b88ae2e3547af83fd301a311f8ad3c8feb4e" exitCode=7
Jul 13 17:08:28.061738 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:08:28.061424    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1890c174-pw89z" event={"ID":"cfd75a32-fa82-4625-965d-88b0eea8f78a","Type":"ContainerDied","Data":"eb245781f9e7325ef6fd56f8e2b8b88ae2e3547af83fd301a311f8ad3c8feb4e"}
Jul 13 17:08:28.061738 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:08:28.061564    2566 scope.go:117] "RemoveContainer" containerID="eb245781f9e7325ef6fd56f8e2b8b88ae2e3547af83fd301a311f8ad3c8feb4e"
Jul 13 17:08:29.069065 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:08:29.068968    2566 generic.go:356] "Generic (PLEG): container finished" podID="cfd75a32-fa82-4625-965d-88b0eea8f78a" containerID="b14c1505c4d808a72225d35c2edd8d7ddfb0f2b7c687c184283ba70b7b4e9757" exitCode=7
Jul 13 17:08:29.069538 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:08:29.069048    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1890c174-pw89z" event={"ID":"cfd75a32-fa82-4625-965d-88b0eea8f78a","Type":"ContainerDied","Data":"b14c1505c4d808a72225d35c2edd8d7ddfb0f2b7c687c184283ba70b7b4e9757"}
Jul 13 17:08:29.069538 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:08:29.069124    2566 scope.go:117] "RemoveContainer" containerID="eb245781f9e7325ef6fd56f8e2b8b88ae2e3547af83fd301a311f8ad3c8feb4e"
Jul 13 17:08:29.069538 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:08:29.069348    2566 scope.go:117] "RemoveContainer" containerID="b14c1505c4d808a72225d35c2edd8d7ddfb0f2b7c687c184283ba70b7b4e9757"
Jul 13 17:08:29.069684 ip-10-0-128-173 kubenswrapper[2566]: E0713 17:08:29.069606    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-1890c174-pw89z_odh-ai-gateway-infra(cfd75a32-fa82-4625-965d-88b0eea8f78a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1890c174-pw89z" podUID="cfd75a32-fa82-4625-965d-88b0eea8f78a"
Jul 13 17:08:30.075537 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:08:30.075500    2566 scope.go:117] "RemoveContainer" containerID="b14c1505c4d808a72225d35c2edd8d7ddfb0f2b7c687c184283ba70b7b4e9757"
Jul 13 17:08:30.075897 ip-10-0-128-173 kubenswrapper[2566]: E0713 17:08:30.075718    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-1890c174-pw89z_odh-ai-gateway-infra(cfd75a32-fa82-4625-965d-88b0eea8f78a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1890c174-pw89z" podUID="cfd75a32-fa82-4625-965d-88b0eea8f78a"
Jul 13 17:08:41.981119 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:08:41.981085    2566 scope.go:117] "RemoveContainer" containerID="b14c1505c4d808a72225d35c2edd8d7ddfb0f2b7c687c184283ba70b7b4e9757"
Jul 13 17:08:43.146201 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:08:43.146167    2566 generic.go:356] "Generic (PLEG): container finished" podID="cfd75a32-fa82-4625-965d-88b0eea8f78a" containerID="6d5c3977b4829b7d143f33052a3460d4306d9a4be8941bf88e9fa91491d57764" exitCode=0
Jul 13 17:08:43.146654 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:08:43.146238    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1890c174-pw89z" event={"ID":"cfd75a32-fa82-4625-965d-88b0eea8f78a","Type":"ContainerDied","Data":"6d5c3977b4829b7d143f33052a3460d4306d9a4be8941bf88e9fa91491d57764"}
Jul 13 17:08:43.146654 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:08:43.146320    2566 scope.go:117] "RemoveContainer" containerID="b14c1505c4d808a72225d35c2edd8d7ddfb0f2b7c687c184283ba70b7b4e9757"
Jul 13 17:08:44.175741 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:08:44.175710    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1890c174-pw89z"]
Jul 13 17:08:44.294968 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:08:44.294942    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1890c174-pw89z"
Jul 13 17:08:44.470191 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:08:44.470160    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfd75a32-fa82-4625-965d-88b0eea8f78a-maas-api-service-ca\") pod \"cfd75a32-fa82-4625-965d-88b0eea8f78a\" (UID: \"cfd75a32-fa82-4625-965d-88b0eea8f78a\") "
Jul 13 17:08:44.470561 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:08:44.470526    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cfd75a32-fa82-4625-965d-88b0eea8f78a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cfd75a32-fa82-4625-965d-88b0eea8f78a" (UID: "cfd75a32-fa82-4625-965d-88b0eea8f78a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 17:08:44.571213 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:08:44.571181    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cfd75a32-fa82-4625-965d-88b0eea8f78a-maas-api-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:08:45.160171 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:08:45.160136    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1890c174-pw89z"
Jul 13 17:08:45.160171 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:08:45.160164    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1890c174-pw89z" event={"ID":"cfd75a32-fa82-4625-965d-88b0eea8f78a","Type":"ContainerDied","Data":"129133e4b120a432b098cc1def24de9d34a24b45940a283378d5b3087b62cb0e"}
Jul 13 17:08:45.160389 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:08:45.160206    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="129133e4b120a432b098cc1def24de9d34a24b45940a283378d5b3087b62cb0e"
Jul 13 17:08:45.192712 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:08:45.192687    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1890c174-pw89z"]
Jul 13 17:08:45.194597 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:08:45.194577    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1890c174-pw89z"]
Jul 13 17:08:45.980739 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:08:45.980705    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfd75a32-fa82-4625-965d-88b0eea8f78a" path="/var/lib/kubelet/pods/cfd75a32-fa82-4625-965d-88b0eea8f78a/volumes"
Jul 13 17:09:16.388275 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:09:16.388225    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1890c174-rsbdc"]
Jul 13 17:09:16.388729 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:09:16.388634    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfd75a32-fa82-4625-965d-88b0eea8f78a" containerName="revoke-keys"
Jul 13 17:09:16.388729 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:09:16.388646    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfd75a32-fa82-4625-965d-88b0eea8f78a" containerName="revoke-keys"
Jul 13 17:09:16.388729 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:09:16.388655    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfd75a32-fa82-4625-965d-88b0eea8f78a" containerName="revoke-keys"
Jul 13 17:09:16.388729 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:09:16.388661    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfd75a32-fa82-4625-965d-88b0eea8f78a" containerName="revoke-keys"
Jul 13 17:09:16.388729 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:09:16.388668    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfd75a32-fa82-4625-965d-88b0eea8f78a" containerName="revoke-keys"
Jul 13 17:09:16.388729 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:09:16.388673    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfd75a32-fa82-4625-965d-88b0eea8f78a" containerName="revoke-keys"
Jul 13 17:09:16.388915 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:09:16.388738    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd6e5402-33e7-45d0-847f-231117b037cc" containerName="revoke-keys"
Jul 13 17:09:16.388915 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:09:16.388745    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfd75a32-fa82-4625-965d-88b0eea8f78a" containerName="revoke-keys"
Jul 13 17:09:16.388915 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:09:16.388752    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfd75a32-fa82-4625-965d-88b0eea8f78a" containerName="revoke-keys"
Jul 13 17:09:16.388915 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:09:16.388759    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="cfd75a32-fa82-4625-965d-88b0eea8f78a" containerName="revoke-keys"
Jul 13 17:09:16.391454 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:09:16.391437    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1890c174-rsbdc"
Jul 13 17:09:16.394278 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:09:16.394240    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 17:09:16.394405 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:09:16.394305    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fkks4\""
Jul 13 17:09:16.399272 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:09:16.399165    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1890c174-rsbdc"]
Jul 13 17:09:16.474772 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:09:16.474738    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74d76adb-3cb6-42b0-9882-b2268517180d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-1890c174-rsbdc\" (UID: \"74d76adb-3cb6-42b0-9882-b2268517180d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1890c174-rsbdc"
Jul 13 17:09:16.576144 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:09:16.576112    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74d76adb-3cb6-42b0-9882-b2268517180d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-1890c174-rsbdc\" (UID: \"74d76adb-3cb6-42b0-9882-b2268517180d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1890c174-rsbdc"
Jul 13 17:09:16.576720 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:09:16.576701    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74d76adb-3cb6-42b0-9882-b2268517180d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-1890c174-rsbdc\" (UID: \"74d76adb-3cb6-42b0-9882-b2268517180d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1890c174-rsbdc"
Jul 13 17:09:16.703694 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:09:16.703660    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1890c174-rsbdc"
Jul 13 17:09:16.829776 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:09:16.829753    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1890c174-rsbdc"]
Jul 13 17:09:16.831693 ip-10-0-128-173 kubenswrapper[2566]: W0713 17:09:16.831659    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74d76adb_3cb6_42b0_9882_b2268517180d.slice/crio-d124c1cb74a81282d741e5aa72acbc48c52dc8cfec0268fe4aba1d9547cb898f WatchSource:0}: Error finding container d124c1cb74a81282d741e5aa72acbc48c52dc8cfec0268fe4aba1d9547cb898f: Status 404 returned error can't find the container with id d124c1cb74a81282d741e5aa72acbc48c52dc8cfec0268fe4aba1d9547cb898f
Jul 13 17:09:17.315762 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:09:17.315728    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1890c174-rsbdc" event={"ID":"74d76adb-3cb6-42b0-9882-b2268517180d","Type":"ContainerStarted","Data":"6e3a3f831905eae71c023994bb4d079ae188093c9cf2a4235d7af9328057523a"}
Jul 13 17:09:17.315950 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:09:17.315775    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1890c174-rsbdc" event={"ID":"74d76adb-3cb6-42b0-9882-b2268517180d","Type":"ContainerStarted","Data":"d124c1cb74a81282d741e5aa72acbc48c52dc8cfec0268fe4aba1d9547cb898f"}
Jul 13 17:09:17.335625 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:09:17.335527    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1890c174-rsbdc" podStartSLOduration=1.335513125 podStartE2EDuration="1.335513125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:09:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:09:17.332535435 +0000 UTC m=+1743.873915865" watchObservedRunningTime="2026-07-13 17:09:17.335513125 +0000 UTC m=+1743.876893595"
Jul 13 17:09:18.321970 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:09:18.321937    2566 generic.go:356] "Generic (PLEG): container finished" podID="74d76adb-3cb6-42b0-9882-b2268517180d" containerID="6e3a3f831905eae71c023994bb4d079ae188093c9cf2a4235d7af9328057523a" exitCode=0
Jul 13 17:09:18.322348 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:09:18.322005    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1890c174-rsbdc" event={"ID":"74d76adb-3cb6-42b0-9882-b2268517180d","Type":"ContainerDied","Data":"6e3a3f831905eae71c023994bb4d079ae188093c9cf2a4235d7af9328057523a"}
Jul 13 17:09:19.460055 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:09:19.460028    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1890c174-rsbdc"
Jul 13 17:09:19.608272 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:09:19.608171    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74d76adb-3cb6-42b0-9882-b2268517180d-maas-api-service-ca\") pod \"74d76adb-3cb6-42b0-9882-b2268517180d\" (UID: \"74d76adb-3cb6-42b0-9882-b2268517180d\") "
Jul 13 17:09:19.608516 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:09:19.608494    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74d76adb-3cb6-42b0-9882-b2268517180d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "74d76adb-3cb6-42b0-9882-b2268517180d" (UID: "74d76adb-3cb6-42b0-9882-b2268517180d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 17:09:19.709392 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:09:19.709358    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/74d76adb-3cb6-42b0-9882-b2268517180d-maas-api-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:09:20.333815 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:09:20.333731    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1890c174-rsbdc" event={"ID":"74d76adb-3cb6-42b0-9882-b2268517180d","Type":"ContainerDied","Data":"d124c1cb74a81282d741e5aa72acbc48c52dc8cfec0268fe4aba1d9547cb898f"}
Jul 13 17:09:20.333815 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:09:20.333780    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d124c1cb74a81282d741e5aa72acbc48c52dc8cfec0268fe4aba1d9547cb898f"
Jul 13 17:09:20.333815 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:09:20.333742    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-1890c174-rsbdc"
Jul 13 17:09:29.695850 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:09:29.695814    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:09:44.795767 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:09:44.795736    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:09:56.397220 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:09:56.397179    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:10:14.092695 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:10:14.092663    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gdqcb_6a33d4fc-2988-4f88-a92c-212d95ec2060/ovn-acl-logging/0.log"
Jul 13 17:10:14.097881 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:10:14.097861    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gdqcb_6a33d4fc-2988-4f88-a92c-212d95ec2060/ovn-acl-logging/0.log"
Jul 13 17:11:04.812553 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:11:04.812513    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:11:12.993477 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:11:12.993444    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:11:21.790032 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:11:21.789995    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:11:33.395535 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:11:33.395500    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:11:34.469693 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:11:34.469660    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45a43c1d-6wkg5"]
Jul 13 17:11:34.470092 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:11:34.470078    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74d76adb-3cb6-42b0-9882-b2268517180d" containerName="revoke-keys"
Jul 13 17:11:34.470092 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:11:34.470093    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="74d76adb-3cb6-42b0-9882-b2268517180d" containerName="revoke-keys"
Jul 13 17:11:34.470164 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:11:34.470158    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="74d76adb-3cb6-42b0-9882-b2268517180d" containerName="revoke-keys"
Jul 13 17:11:34.473136 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:11:34.473119    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45a43c1d-6wkg5"
Jul 13 17:11:34.476534 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:11:34.476511    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fkks4\""
Jul 13 17:11:34.476663 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:11:34.476596    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 17:11:34.483501 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:11:34.483473    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45a43c1d-6wkg5"]
Jul 13 17:11:34.575113 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:11:34.575080    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/76ef5af9-5cb6-410b-ba12-7f0f58362ab0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-45a43c1d-6wkg5\" (UID: \"76ef5af9-5cb6-410b-ba12-7f0f58362ab0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45a43c1d-6wkg5"
Jul 13 17:11:34.676439 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:11:34.676399    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/76ef5af9-5cb6-410b-ba12-7f0f58362ab0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-45a43c1d-6wkg5\" (UID: \"76ef5af9-5cb6-410b-ba12-7f0f58362ab0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45a43c1d-6wkg5"
Jul 13 17:11:34.677033 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:11:34.677013    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/76ef5af9-5cb6-410b-ba12-7f0f58362ab0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-45a43c1d-6wkg5\" (UID: \"76ef5af9-5cb6-410b-ba12-7f0f58362ab0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45a43c1d-6wkg5"
Jul 13 17:11:34.784287 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:11:34.784174    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45a43c1d-6wkg5"
Jul 13 17:11:34.933283 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:11:34.933231    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45a43c1d-6wkg5"]
Jul 13 17:11:34.935361 ip-10-0-128-173 kubenswrapper[2566]: W0713 17:11:34.935336    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76ef5af9_5cb6_410b_ba12_7f0f58362ab0.slice/crio-752e73f4b80e18b4b9c691876a3331b9723f09480a55f99b501ef19765e74d02 WatchSource:0}: Error finding container 752e73f4b80e18b4b9c691876a3331b9723f09480a55f99b501ef19765e74d02: Status 404 returned error can't find the container with id 752e73f4b80e18b4b9c691876a3331b9723f09480a55f99b501ef19765e74d02
Jul 13 17:11:35.024961 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:11:35.024930    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45a43c1d-6wkg5" event={"ID":"76ef5af9-5cb6-410b-ba12-7f0f58362ab0","Type":"ContainerStarted","Data":"752e73f4b80e18b4b9c691876a3331b9723f09480a55f99b501ef19765e74d02"}
Jul 13 17:11:36.030994 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:11:36.030904    2566 generic.go:356] "Generic (PLEG): container finished" podID="76ef5af9-5cb6-410b-ba12-7f0f58362ab0" containerID="213f3c3f460357d74bd8d816bf617a486a31f0aec701a1b8a2ec132ecb6710f6" exitCode=0
Jul 13 17:11:36.031432 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:11:36.030991    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45a43c1d-6wkg5" event={"ID":"76ef5af9-5cb6-410b-ba12-7f0f58362ab0","Type":"ContainerDied","Data":"213f3c3f460357d74bd8d816bf617a486a31f0aec701a1b8a2ec132ecb6710f6"}
Jul 13 17:11:37.171510 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:11:37.171480    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45a43c1d-6wkg5"
Jul 13 17:11:37.199580 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:11:37.199547    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/76ef5af9-5cb6-410b-ba12-7f0f58362ab0-maas-api-service-ca\") pod \"76ef5af9-5cb6-410b-ba12-7f0f58362ab0\" (UID: \"76ef5af9-5cb6-410b-ba12-7f0f58362ab0\") "
Jul 13 17:11:37.199893 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:11:37.199871    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76ef5af9-5cb6-410b-ba12-7f0f58362ab0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "76ef5af9-5cb6-410b-ba12-7f0f58362ab0" (UID: "76ef5af9-5cb6-410b-ba12-7f0f58362ab0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 17:11:37.199974 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:11:37.199956    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/76ef5af9-5cb6-410b-ba12-7f0f58362ab0-maas-api-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:11:38.042674 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:11:38.042641    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45a43c1d-6wkg5"
Jul 13 17:11:38.042674 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:11:38.042645    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-45a43c1d-6wkg5" event={"ID":"76ef5af9-5cb6-410b-ba12-7f0f58362ab0","Type":"ContainerDied","Data":"752e73f4b80e18b4b9c691876a3331b9723f09480a55f99b501ef19765e74d02"}
Jul 13 17:11:38.042880 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:11:38.042692    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="752e73f4b80e18b4b9c691876a3331b9723f09480a55f99b501ef19765e74d02"
Jul 13 17:11:47.894102 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:11:47.894068    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:12:31.491497 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:12:31.491450    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:12:54.997778 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:12:54.997741    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:13:14.065048 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:13:14.064979    2566 scope.go:117] "RemoveContainer" containerID="7170d8a758c9a5ca7bfdd07f59c327fc2a718928bf79f4c2e8f184618581ecea"
Jul 13 17:13:15.801055 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:13:15.801020    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:13:19.075169 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:13:19.075133    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-54358301-x6hrn"]
Jul 13 17:13:19.075577 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:13:19.075563    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76ef5af9-5cb6-410b-ba12-7f0f58362ab0" containerName="revoke-keys"
Jul 13 17:13:19.075577 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:13:19.075578    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="76ef5af9-5cb6-410b-ba12-7f0f58362ab0" containerName="revoke-keys"
Jul 13 17:13:19.075660 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:13:19.075651    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="76ef5af9-5cb6-410b-ba12-7f0f58362ab0" containerName="revoke-keys"
Jul 13 17:13:19.078707 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:13:19.078686    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-54358301-x6hrn"
Jul 13 17:13:19.081541 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:13:19.081513    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fkks4\""
Jul 13 17:13:19.081661 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:13:19.081602    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 17:13:19.086497 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:13:19.086192    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-54358301-x6hrn"]
Jul 13 17:13:19.252404 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:13:19.252371    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e716680e-2c26-48af-93ee-de99c913f649-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-54358301-x6hrn\" (UID: \"e716680e-2c26-48af-93ee-de99c913f649\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-54358301-x6hrn"
Jul 13 17:13:19.353752 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:13:19.353652    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e716680e-2c26-48af-93ee-de99c913f649-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-54358301-x6hrn\" (UID: \"e716680e-2c26-48af-93ee-de99c913f649\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-54358301-x6hrn"
Jul 13 17:13:19.354337 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:13:19.354312    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e716680e-2c26-48af-93ee-de99c913f649-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-54358301-x6hrn\" (UID: \"e716680e-2c26-48af-93ee-de99c913f649\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-54358301-x6hrn"
Jul 13 17:13:19.391045 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:13:19.391020    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-54358301-x6hrn"
Jul 13 17:13:19.516384 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:13:19.516361    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-54358301-x6hrn"]
Jul 13 17:13:19.517730 ip-10-0-128-173 kubenswrapper[2566]: W0713 17:13:19.517709    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode716680e_2c26_48af_93ee_de99c913f649.slice/crio-eb5d6cf1b9f18643de10d71557c7e502b926c2f9da9daf3026dc8894d86ac122 WatchSource:0}: Error finding container eb5d6cf1b9f18643de10d71557c7e502b926c2f9da9daf3026dc8894d86ac122: Status 404 returned error can't find the container with id eb5d6cf1b9f18643de10d71557c7e502b926c2f9da9daf3026dc8894d86ac122
Jul 13 17:13:19.566078 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:13:19.566053    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-54358301-x6hrn" event={"ID":"e716680e-2c26-48af-93ee-de99c913f649","Type":"ContainerStarted","Data":"eb5d6cf1b9f18643de10d71557c7e502b926c2f9da9daf3026dc8894d86ac122"}
Jul 13 17:13:20.572573 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:13:20.572540    2566 generic.go:356] "Generic (PLEG): container finished" podID="e716680e-2c26-48af-93ee-de99c913f649" containerID="30cf4012649ec6f4dd783bf3716c879ce2ff321f391765d74f2b462b717a2212" exitCode=0
Jul 13 17:13:20.572947 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:13:20.572597    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-54358301-x6hrn" event={"ID":"e716680e-2c26-48af-93ee-de99c913f649","Type":"ContainerDied","Data":"30cf4012649ec6f4dd783bf3716c879ce2ff321f391765d74f2b462b717a2212"}
Jul 13 17:13:21.705940 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:13:21.705912    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-54358301-x6hrn"
Jul 13 17:13:21.880172 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:13:21.880087    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e716680e-2c26-48af-93ee-de99c913f649-maas-api-service-ca\") pod \"e716680e-2c26-48af-93ee-de99c913f649\" (UID: \"e716680e-2c26-48af-93ee-de99c913f649\") "
Jul 13 17:13:21.880486 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:13:21.880463    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e716680e-2c26-48af-93ee-de99c913f649-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e716680e-2c26-48af-93ee-de99c913f649" (UID: "e716680e-2c26-48af-93ee-de99c913f649"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 17:13:21.981030 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:13:21.981004    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e716680e-2c26-48af-93ee-de99c913f649-maas-api-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:13:22.584314 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:13:22.584277    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-54358301-x6hrn" event={"ID":"e716680e-2c26-48af-93ee-de99c913f649","Type":"ContainerDied","Data":"eb5d6cf1b9f18643de10d71557c7e502b926c2f9da9daf3026dc8894d86ac122"}
Jul 13 17:13:22.584314 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:13:22.584299    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-54358301-x6hrn"
Jul 13 17:13:22.584515 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:13:22.584327    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eb5d6cf1b9f18643de10d71557c7e502b926c2f9da9daf3026dc8894d86ac122"
Jul 13 17:13:33.610917 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:13:33.610883    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:14:50.410952 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:14:50.410913    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:15:04.898702 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:15:04.898664    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:15:14.092640 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:15:14.092605    2566 scope.go:117] "RemoveContainer" containerID="6d5c3977b4829b7d143f33052a3460d4306d9a4be8941bf88e9fa91491d57764"
Jul 13 17:15:14.131823 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:15:14.131789    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gdqcb_6a33d4fc-2988-4f88-a92c-212d95ec2060/ovn-acl-logging/0.log"
Jul 13 17:15:14.134769 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:15:14.134743    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gdqcb_6a33d4fc-2988-4f88-a92c-212d95ec2060/ovn-acl-logging/0.log"
Jul 13 17:15:48.397813 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:15:48.397773    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:15:59.503231 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:15:59.503194    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:16:13.097285 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:13.097062    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-874fa5/test-model-874fa5-kserve-db7d768c5-qdzzw"]
Jul 13 17:16:13.099691 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:13.097769    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e716680e-2c26-48af-93ee-de99c913f649" containerName="revoke-keys"
Jul 13 17:16:13.099691 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:13.097785    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e716680e-2c26-48af-93ee-de99c913f649" containerName="revoke-keys"
Jul 13 17:16:13.099691 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:13.097882    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e716680e-2c26-48af-93ee-de99c913f649" containerName="revoke-keys"
Jul 13 17:16:13.100923 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:13.100901    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-874fa5/test-model-874fa5-kserve-db7d768c5-qdzzw"
Jul 13 17:16:13.105564 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:13.105544    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-874fa5\"/\"test-model-874fa5-kserve-self-signed-certs\""
Jul 13 17:16:13.105564 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:13.105564    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-874fa5\"/\"kube-root-ca.crt\""
Jul 13 17:16:13.105740 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:13.105545    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-874fa5\"/\"default-dockercfg-xfg6m\""
Jul 13 17:16:13.105740 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:13.105544    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-874fa5\"/\"openshift-service-ca.crt\""
Jul 13 17:16:13.109923 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:13.109904    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-874fa5/test-model-874fa5-kserve-db7d768c5-qdzzw"]
Jul 13 17:16:13.205351 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:13.205318    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4669b8b6-22fa-4955-be56-1b222d8ed615-tls-certs\") pod \"test-model-874fa5-kserve-db7d768c5-qdzzw\" (UID: \"4669b8b6-22fa-4955-be56-1b222d8ed615\") " pod="ai-tenant-e2e-inf-a-874fa5/test-model-874fa5-kserve-db7d768c5-qdzzw"
Jul 13 17:16:13.205530 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:13.205368    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjlfd\" (UniqueName: \"kubernetes.io/projected/4669b8b6-22fa-4955-be56-1b222d8ed615-kube-api-access-wjlfd\") pod \"test-model-874fa5-kserve-db7d768c5-qdzzw\" (UID: \"4669b8b6-22fa-4955-be56-1b222d8ed615\") " pod="ai-tenant-e2e-inf-a-874fa5/test-model-874fa5-kserve-db7d768c5-qdzzw"
Jul 13 17:16:13.205530 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:13.205430    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4669b8b6-22fa-4955-be56-1b222d8ed615-tmp-dir\") pod \"test-model-874fa5-kserve-db7d768c5-qdzzw\" (UID: \"4669b8b6-22fa-4955-be56-1b222d8ed615\") " pod="ai-tenant-e2e-inf-a-874fa5/test-model-874fa5-kserve-db7d768c5-qdzzw"
Jul 13 17:16:13.205530 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:13.205473    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4669b8b6-22fa-4955-be56-1b222d8ed615-dshm\") pod \"test-model-874fa5-kserve-db7d768c5-qdzzw\" (UID: \"4669b8b6-22fa-4955-be56-1b222d8ed615\") " pod="ai-tenant-e2e-inf-a-874fa5/test-model-874fa5-kserve-db7d768c5-qdzzw"
Jul 13 17:16:13.205530 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:13.205490    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4669b8b6-22fa-4955-be56-1b222d8ed615-home\") pod \"test-model-874fa5-kserve-db7d768c5-qdzzw\" (UID: \"4669b8b6-22fa-4955-be56-1b222d8ed615\") " pod="ai-tenant-e2e-inf-a-874fa5/test-model-874fa5-kserve-db7d768c5-qdzzw"
Jul 13 17:16:13.205530 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:13.205510    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4669b8b6-22fa-4955-be56-1b222d8ed615-model-cache\") pod \"test-model-874fa5-kserve-db7d768c5-qdzzw\" (UID: \"4669b8b6-22fa-4955-be56-1b222d8ed615\") " pod="ai-tenant-e2e-inf-a-874fa5/test-model-874fa5-kserve-db7d768c5-qdzzw"
Jul 13 17:16:13.306335 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:13.306307    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4669b8b6-22fa-4955-be56-1b222d8ed615-tmp-dir\") pod \"test-model-874fa5-kserve-db7d768c5-qdzzw\" (UID: \"4669b8b6-22fa-4955-be56-1b222d8ed615\") " pod="ai-tenant-e2e-inf-a-874fa5/test-model-874fa5-kserve-db7d768c5-qdzzw"
Jul 13 17:16:13.306511 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:13.306360    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4669b8b6-22fa-4955-be56-1b222d8ed615-dshm\") pod \"test-model-874fa5-kserve-db7d768c5-qdzzw\" (UID: \"4669b8b6-22fa-4955-be56-1b222d8ed615\") " pod="ai-tenant-e2e-inf-a-874fa5/test-model-874fa5-kserve-db7d768c5-qdzzw"
Jul 13 17:16:13.306511 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:13.306388    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4669b8b6-22fa-4955-be56-1b222d8ed615-home\") pod \"test-model-874fa5-kserve-db7d768c5-qdzzw\" (UID: \"4669b8b6-22fa-4955-be56-1b222d8ed615\") " pod="ai-tenant-e2e-inf-a-874fa5/test-model-874fa5-kserve-db7d768c5-qdzzw"
Jul 13 17:16:13.306511 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:13.306415    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4669b8b6-22fa-4955-be56-1b222d8ed615-model-cache\") pod \"test-model-874fa5-kserve-db7d768c5-qdzzw\" (UID: \"4669b8b6-22fa-4955-be56-1b222d8ed615\") " pod="ai-tenant-e2e-inf-a-874fa5/test-model-874fa5-kserve-db7d768c5-qdzzw"
Jul 13 17:16:13.306511 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:13.306468    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4669b8b6-22fa-4955-be56-1b222d8ed615-tls-certs\") pod \"test-model-874fa5-kserve-db7d768c5-qdzzw\" (UID: \"4669b8b6-22fa-4955-be56-1b222d8ed615\") " pod="ai-tenant-e2e-inf-a-874fa5/test-model-874fa5-kserve-db7d768c5-qdzzw"
Jul 13 17:16:13.306726 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:13.306514    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjlfd\" (UniqueName: \"kubernetes.io/projected/4669b8b6-22fa-4955-be56-1b222d8ed615-kube-api-access-wjlfd\") pod \"test-model-874fa5-kserve-db7d768c5-qdzzw\" (UID: \"4669b8b6-22fa-4955-be56-1b222d8ed615\") " pod="ai-tenant-e2e-inf-a-874fa5/test-model-874fa5-kserve-db7d768c5-qdzzw"
Jul 13 17:16:13.306726 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:13.306713    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4669b8b6-22fa-4955-be56-1b222d8ed615-tmp-dir\") pod \"test-model-874fa5-kserve-db7d768c5-qdzzw\" (UID: \"4669b8b6-22fa-4955-be56-1b222d8ed615\") " pod="ai-tenant-e2e-inf-a-874fa5/test-model-874fa5-kserve-db7d768c5-qdzzw"
Jul 13 17:16:13.306848 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:13.306830    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4669b8b6-22fa-4955-be56-1b222d8ed615-home\") pod \"test-model-874fa5-kserve-db7d768c5-qdzzw\" (UID: \"4669b8b6-22fa-4955-be56-1b222d8ed615\") " pod="ai-tenant-e2e-inf-a-874fa5/test-model-874fa5-kserve-db7d768c5-qdzzw"
Jul 13 17:16:13.306950 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:13.306928    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4669b8b6-22fa-4955-be56-1b222d8ed615-model-cache\") pod \"test-model-874fa5-kserve-db7d768c5-qdzzw\" (UID: \"4669b8b6-22fa-4955-be56-1b222d8ed615\") " pod="ai-tenant-e2e-inf-a-874fa5/test-model-874fa5-kserve-db7d768c5-qdzzw"
Jul 13 17:16:13.308659 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:13.308632    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4669b8b6-22fa-4955-be56-1b222d8ed615-dshm\") pod \"test-model-874fa5-kserve-db7d768c5-qdzzw\" (UID: \"4669b8b6-22fa-4955-be56-1b222d8ed615\") " pod="ai-tenant-e2e-inf-a-874fa5/test-model-874fa5-kserve-db7d768c5-qdzzw"
Jul 13 17:16:13.308945 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:13.308926    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4669b8b6-22fa-4955-be56-1b222d8ed615-tls-certs\") pod \"test-model-874fa5-kserve-db7d768c5-qdzzw\" (UID: \"4669b8b6-22fa-4955-be56-1b222d8ed615\") " pod="ai-tenant-e2e-inf-a-874fa5/test-model-874fa5-kserve-db7d768c5-qdzzw"
Jul 13 17:16:13.315379 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:13.315357    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjlfd\" (UniqueName: \"kubernetes.io/projected/4669b8b6-22fa-4955-be56-1b222d8ed615-kube-api-access-wjlfd\") pod \"test-model-874fa5-kserve-db7d768c5-qdzzw\" (UID: \"4669b8b6-22fa-4955-be56-1b222d8ed615\") " pod="ai-tenant-e2e-inf-a-874fa5/test-model-874fa5-kserve-db7d768c5-qdzzw"
Jul 13 17:16:13.412511 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:13.412404    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-874fa5/test-model-874fa5-kserve-db7d768c5-qdzzw"
Jul 13 17:16:13.547575 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:13.547547    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-874fa5/test-model-874fa5-kserve-db7d768c5-qdzzw"]
Jul 13 17:16:13.549085 ip-10-0-128-173 kubenswrapper[2566]: W0713 17:16:13.549055    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4669b8b6_22fa_4955_be56_1b222d8ed615.slice/crio-3efd58582b744715dc63d32929f1b53afb261abeec7c51c8f0e0cf427b3001bc WatchSource:0}: Error finding container 3efd58582b744715dc63d32929f1b53afb261abeec7c51c8f0e0cf427b3001bc: Status 404 returned error can't find the container with id 3efd58582b744715dc63d32929f1b53afb261abeec7c51c8f0e0cf427b3001bc
Jul 13 17:16:14.453114 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:14.453080    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-874fa5/test-model-874fa5-kserve-db7d768c5-qdzzw" event={"ID":"4669b8b6-22fa-4955-be56-1b222d8ed615","Type":"ContainerStarted","Data":"7f3813aecac8a3def725591082cc5fb990e146af7e16f589ef5b3f7c111999e4"}
Jul 13 17:16:14.453114 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:14.453120    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-874fa5/test-model-874fa5-kserve-db7d768c5-qdzzw" event={"ID":"4669b8b6-22fa-4955-be56-1b222d8ed615","Type":"ContainerStarted","Data":"3efd58582b744715dc63d32929f1b53afb261abeec7c51c8f0e0cf427b3001bc"}
Jul 13 17:16:14.472454 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:14.472401    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-874fa5/test-model-874fa5-kserve-db7d768c5-qdzzw" podStartSLOduration=1.4723866829999999 podStartE2EDuration="1.472386683s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:16:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:16:14.47049509 +0000 UTC m=+2161.011875522" watchObservedRunningTime="2026-07-13 17:16:14.472386683 +0000 UTC m=+2161.013767112"
Jul 13 17:16:16.649999 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:16.649964    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:16:23.413507 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:23.413469    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-874fa5/test-model-874fa5-kserve-db7d768c5-qdzzw"
Jul 13 17:16:23.413928 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:23.413523    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-874fa5/test-model-874fa5-kserve-db7d768c5-qdzzw"
Jul 13 17:16:23.426074 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:23.426045    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-874fa5/test-model-874fa5-kserve-db7d768c5-qdzzw"
Jul 13 17:16:23.508941 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:23.508912    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-874fa5/test-model-874fa5-kserve-db7d768c5-qdzzw"
Jul 13 17:16:28.318820 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:28.318788    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:16:43.194697 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:43.194661    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-0c5523/test-model-0c5523-kserve-677d88d89c-wqjpc"]
Jul 13 17:16:43.198355 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:43.198338    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0c5523/test-model-0c5523-kserve-677d88d89c-wqjpc"
Jul 13 17:16:43.202478 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:43.202451    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-0c5523\"/\"test-model-0c5523-kserve-self-signed-certs\""
Jul 13 17:16:43.202478 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:43.202464    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-0c5523\"/\"openshift-service-ca.crt\""
Jul 13 17:16:43.202634 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:43.202499    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-0c5523\"/\"kube-root-ca.crt\""
Jul 13 17:16:43.202634 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:43.202519    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-0c5523\"/\"default-dockercfg-q95v6\""
Jul 13 17:16:43.207752 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:43.207732    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-0c5523/test-model-0c5523-kserve-677d88d89c-wqjpc"]
Jul 13 17:16:43.282894 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:43.282866    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77bdd7d8-7371-4293-a83d-893893480931-model-cache\") pod \"test-model-0c5523-kserve-677d88d89c-wqjpc\" (UID: \"77bdd7d8-7371-4293-a83d-893893480931\") " pod="ai-tenant-e2e-inf-b-0c5523/test-model-0c5523-kserve-677d88d89c-wqjpc"
Jul 13 17:16:43.283069 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:43.282902    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvbnp\" (UniqueName: \"kubernetes.io/projected/77bdd7d8-7371-4293-a83d-893893480931-kube-api-access-pvbnp\") pod \"test-model-0c5523-kserve-677d88d89c-wqjpc\" (UID: \"77bdd7d8-7371-4293-a83d-893893480931\") " pod="ai-tenant-e2e-inf-b-0c5523/test-model-0c5523-kserve-677d88d89c-wqjpc"
Jul 13 17:16:43.283069 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:43.282936    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77bdd7d8-7371-4293-a83d-893893480931-home\") pod \"test-model-0c5523-kserve-677d88d89c-wqjpc\" (UID: \"77bdd7d8-7371-4293-a83d-893893480931\") " pod="ai-tenant-e2e-inf-b-0c5523/test-model-0c5523-kserve-677d88d89c-wqjpc"
Jul 13 17:16:43.283069 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:43.283017    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77bdd7d8-7371-4293-a83d-893893480931-tls-certs\") pod \"test-model-0c5523-kserve-677d88d89c-wqjpc\" (UID: \"77bdd7d8-7371-4293-a83d-893893480931\") " pod="ai-tenant-e2e-inf-b-0c5523/test-model-0c5523-kserve-677d88d89c-wqjpc"
Jul 13 17:16:43.283069 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:43.283062    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77bdd7d8-7371-4293-a83d-893893480931-dshm\") pod \"test-model-0c5523-kserve-677d88d89c-wqjpc\" (UID: \"77bdd7d8-7371-4293-a83d-893893480931\") " pod="ai-tenant-e2e-inf-b-0c5523/test-model-0c5523-kserve-677d88d89c-wqjpc"
Jul 13 17:16:43.283288 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:43.283119    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77bdd7d8-7371-4293-a83d-893893480931-tmp-dir\") pod \"test-model-0c5523-kserve-677d88d89c-wqjpc\" (UID: \"77bdd7d8-7371-4293-a83d-893893480931\") " pod="ai-tenant-e2e-inf-b-0c5523/test-model-0c5523-kserve-677d88d89c-wqjpc"
Jul 13 17:16:43.384480 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:43.384443    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77bdd7d8-7371-4293-a83d-893893480931-home\") pod \"test-model-0c5523-kserve-677d88d89c-wqjpc\" (UID: \"77bdd7d8-7371-4293-a83d-893893480931\") " pod="ai-tenant-e2e-inf-b-0c5523/test-model-0c5523-kserve-677d88d89c-wqjpc"
Jul 13 17:16:43.384670 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:43.384486    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77bdd7d8-7371-4293-a83d-893893480931-tls-certs\") pod \"test-model-0c5523-kserve-677d88d89c-wqjpc\" (UID: \"77bdd7d8-7371-4293-a83d-893893480931\") " pod="ai-tenant-e2e-inf-b-0c5523/test-model-0c5523-kserve-677d88d89c-wqjpc"
Jul 13 17:16:43.384670 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:43.384527    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77bdd7d8-7371-4293-a83d-893893480931-dshm\") pod \"test-model-0c5523-kserve-677d88d89c-wqjpc\" (UID: \"77bdd7d8-7371-4293-a83d-893893480931\") " pod="ai-tenant-e2e-inf-b-0c5523/test-model-0c5523-kserve-677d88d89c-wqjpc"
Jul 13 17:16:43.384670 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:43.384557    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77bdd7d8-7371-4293-a83d-893893480931-tmp-dir\") pod \"test-model-0c5523-kserve-677d88d89c-wqjpc\" (UID: \"77bdd7d8-7371-4293-a83d-893893480931\") " pod="ai-tenant-e2e-inf-b-0c5523/test-model-0c5523-kserve-677d88d89c-wqjpc"
Jul 13 17:16:43.384837 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:43.384713    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77bdd7d8-7371-4293-a83d-893893480931-model-cache\") pod \"test-model-0c5523-kserve-677d88d89c-wqjpc\" (UID: \"77bdd7d8-7371-4293-a83d-893893480931\") " pod="ai-tenant-e2e-inf-b-0c5523/test-model-0c5523-kserve-677d88d89c-wqjpc"
Jul 13 17:16:43.384837 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:43.384772    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvbnp\" (UniqueName: \"kubernetes.io/projected/77bdd7d8-7371-4293-a83d-893893480931-kube-api-access-pvbnp\") pod \"test-model-0c5523-kserve-677d88d89c-wqjpc\" (UID: \"77bdd7d8-7371-4293-a83d-893893480931\") " pod="ai-tenant-e2e-inf-b-0c5523/test-model-0c5523-kserve-677d88d89c-wqjpc"
Jul 13 17:16:43.384946 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:43.384888    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77bdd7d8-7371-4293-a83d-893893480931-home\") pod \"test-model-0c5523-kserve-677d88d89c-wqjpc\" (UID: \"77bdd7d8-7371-4293-a83d-893893480931\") " pod="ai-tenant-e2e-inf-b-0c5523/test-model-0c5523-kserve-677d88d89c-wqjpc"
Jul 13 17:16:43.385007 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:43.384988    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77bdd7d8-7371-4293-a83d-893893480931-tmp-dir\") pod \"test-model-0c5523-kserve-677d88d89c-wqjpc\" (UID: \"77bdd7d8-7371-4293-a83d-893893480931\") " pod="ai-tenant-e2e-inf-b-0c5523/test-model-0c5523-kserve-677d88d89c-wqjpc"
Jul 13 17:16:43.385066 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:43.385044    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77bdd7d8-7371-4293-a83d-893893480931-model-cache\") pod \"test-model-0c5523-kserve-677d88d89c-wqjpc\" (UID: \"77bdd7d8-7371-4293-a83d-893893480931\") " pod="ai-tenant-e2e-inf-b-0c5523/test-model-0c5523-kserve-677d88d89c-wqjpc"
Jul 13 17:16:43.386820 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:43.386792    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77bdd7d8-7371-4293-a83d-893893480931-dshm\") pod \"test-model-0c5523-kserve-677d88d89c-wqjpc\" (UID: \"77bdd7d8-7371-4293-a83d-893893480931\") " pod="ai-tenant-e2e-inf-b-0c5523/test-model-0c5523-kserve-677d88d89c-wqjpc"
Jul 13 17:16:43.387019 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:43.387000    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77bdd7d8-7371-4293-a83d-893893480931-tls-certs\") pod \"test-model-0c5523-kserve-677d88d89c-wqjpc\" (UID: \"77bdd7d8-7371-4293-a83d-893893480931\") " pod="ai-tenant-e2e-inf-b-0c5523/test-model-0c5523-kserve-677d88d89c-wqjpc"
Jul 13 17:16:43.397412 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:43.397391    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvbnp\" (UniqueName: \"kubernetes.io/projected/77bdd7d8-7371-4293-a83d-893893480931-kube-api-access-pvbnp\") pod \"test-model-0c5523-kserve-677d88d89c-wqjpc\" (UID: \"77bdd7d8-7371-4293-a83d-893893480931\") " pod="ai-tenant-e2e-inf-b-0c5523/test-model-0c5523-kserve-677d88d89c-wqjpc"
Jul 13 17:16:43.508858 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:43.508761    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0c5523/test-model-0c5523-kserve-677d88d89c-wqjpc"
Jul 13 17:16:43.646275 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:43.646238    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-0c5523/test-model-0c5523-kserve-677d88d89c-wqjpc"]
Jul 13 17:16:44.610041 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:44.610002    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0c5523/test-model-0c5523-kserve-677d88d89c-wqjpc" event={"ID":"77bdd7d8-7371-4293-a83d-893893480931","Type":"ContainerStarted","Data":"55bab94aeaeecf7af9a04c5e2240ba9f7c13d150f9d151ec6875c8566b2fc137"}
Jul 13 17:16:44.610508 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:44.610048    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0c5523/test-model-0c5523-kserve-677d88d89c-wqjpc" event={"ID":"77bdd7d8-7371-4293-a83d-893893480931","Type":"ContainerStarted","Data":"4d0676560e23858f4fe7709d0ae1d4a77214eb20c1aa1754c733f9808114516c"}
Jul 13 17:16:47.695368 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:47.695308    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-0c5523/test-model-0c5523-kserve-677d88d89c-wqjpc" podStartSLOduration=4.6952917450000005 podStartE2EDuration="4.695291745s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:16:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:16:44.634510632 +0000 UTC m=+2191.175891061" watchObservedRunningTime="2026-07-13 17:16:47.695291745 +0000 UTC m=+2194.236672173"
Jul 13 17:16:47.695773 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:47.695619    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:16:53.509359 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:53.509322    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-0c5523/test-model-0c5523-kserve-677d88d89c-wqjpc"
Jul 13 17:16:53.509807 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:53.509373    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-0c5523/test-model-0c5523-kserve-677d88d89c-wqjpc"
Jul 13 17:16:53.521713 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:53.521688    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-0c5523/test-model-0c5523-kserve-677d88d89c-wqjpc"
Jul 13 17:16:53.667661 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:16:53.667632    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-0c5523/test-model-0c5523-kserve-677d88d89c-wqjpc"
Jul 13 17:17:10.146735 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:10.146700    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-874fa5/test-model-874fa5-kserve-db7d768c5-qdzzw"]
Jul 13 17:17:11.080982 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:11.080953    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-0c5523/test-model-0c5523-kserve-677d88d89c-wqjpc"]
Jul 13 17:17:15.197661 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:15.197629    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:17:25.167147 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:25.167068    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-874fa5/test-model-874fa5-kserve-db7d768c5-qdzzw" podUID="4669b8b6-22fa-4955-be56-1b222d8ed615" containerName="main" containerID="cri-o://7f3813aecac8a3def725591082cc5fb990e146af7e16f589ef5b3f7c111999e4" gracePeriod=45
Jul 13 17:17:25.418825 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:25.418770    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-874fa5/test-model-874fa5-kserve-db7d768c5-qdzzw"
Jul 13 17:17:25.499041 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:25.499012    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4669b8b6-22fa-4955-be56-1b222d8ed615-dshm\") pod \"4669b8b6-22fa-4955-be56-1b222d8ed615\" (UID: \"4669b8b6-22fa-4955-be56-1b222d8ed615\") "
Jul 13 17:17:25.499189 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:25.499053    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4669b8b6-22fa-4955-be56-1b222d8ed615-model-cache\") pod \"4669b8b6-22fa-4955-be56-1b222d8ed615\" (UID: \"4669b8b6-22fa-4955-be56-1b222d8ed615\") "
Jul 13 17:17:25.499189 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:25.499105    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4669b8b6-22fa-4955-be56-1b222d8ed615-tls-certs\") pod \"4669b8b6-22fa-4955-be56-1b222d8ed615\" (UID: \"4669b8b6-22fa-4955-be56-1b222d8ed615\") "
Jul 13 17:17:25.499189 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:25.499159    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4669b8b6-22fa-4955-be56-1b222d8ed615-home\") pod \"4669b8b6-22fa-4955-be56-1b222d8ed615\" (UID: \"4669b8b6-22fa-4955-be56-1b222d8ed615\") "
Jul 13 17:17:25.499189 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:25.499184    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wjlfd\" (UniqueName: \"kubernetes.io/projected/4669b8b6-22fa-4955-be56-1b222d8ed615-kube-api-access-wjlfd\") pod \"4669b8b6-22fa-4955-be56-1b222d8ed615\" (UID: \"4669b8b6-22fa-4955-be56-1b222d8ed615\") "
Jul 13 17:17:25.499411 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:25.499200    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4669b8b6-22fa-4955-be56-1b222d8ed615-tmp-dir\") pod \"4669b8b6-22fa-4955-be56-1b222d8ed615\" (UID: \"4669b8b6-22fa-4955-be56-1b222d8ed615\") "
Jul 13 17:17:25.499466 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:25.499433    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4669b8b6-22fa-4955-be56-1b222d8ed615-model-cache" (OuterVolumeSpecName: "model-cache") pod "4669b8b6-22fa-4955-be56-1b222d8ed615" (UID: "4669b8b6-22fa-4955-be56-1b222d8ed615"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 17:17:25.499466 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:25.499449    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4669b8b6-22fa-4955-be56-1b222d8ed615-home" (OuterVolumeSpecName: "home") pod "4669b8b6-22fa-4955-be56-1b222d8ed615" (UID: "4669b8b6-22fa-4955-be56-1b222d8ed615"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 17:17:25.499578 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:25.499567    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4669b8b6-22fa-4955-be56-1b222d8ed615-model-cache\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:17:25.499705 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:25.499585    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4669b8b6-22fa-4955-be56-1b222d8ed615-home\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:17:25.499705 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:25.499608    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4669b8b6-22fa-4955-be56-1b222d8ed615-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4669b8b6-22fa-4955-be56-1b222d8ed615" (UID: "4669b8b6-22fa-4955-be56-1b222d8ed615"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 17:17:25.501331 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:25.501306    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4669b8b6-22fa-4955-be56-1b222d8ed615-dshm" (OuterVolumeSpecName: "dshm") pod "4669b8b6-22fa-4955-be56-1b222d8ed615" (UID: "4669b8b6-22fa-4955-be56-1b222d8ed615"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 17:17:25.501458 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:25.501363    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4669b8b6-22fa-4955-be56-1b222d8ed615-kube-api-access-wjlfd" (OuterVolumeSpecName: "kube-api-access-wjlfd") pod "4669b8b6-22fa-4955-be56-1b222d8ed615" (UID: "4669b8b6-22fa-4955-be56-1b222d8ed615"). InnerVolumeSpecName "kube-api-access-wjlfd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 17:17:25.501458 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:25.501419    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4669b8b6-22fa-4955-be56-1b222d8ed615-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4669b8b6-22fa-4955-be56-1b222d8ed615" (UID: "4669b8b6-22fa-4955-be56-1b222d8ed615"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 17:17:25.600790 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:25.600756    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4669b8b6-22fa-4955-be56-1b222d8ed615-dshm\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:17:25.600790 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:25.600788    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4669b8b6-22fa-4955-be56-1b222d8ed615-tls-certs\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:17:25.600997 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:25.600803    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wjlfd\" (UniqueName: \"kubernetes.io/projected/4669b8b6-22fa-4955-be56-1b222d8ed615-kube-api-access-wjlfd\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:17:25.600997 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:25.600815    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4669b8b6-22fa-4955-be56-1b222d8ed615-tmp-dir\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:17:25.826732 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:25.826643    2566 generic.go:356] "Generic (PLEG): container finished" podID="4669b8b6-22fa-4955-be56-1b222d8ed615" containerID="7f3813aecac8a3def725591082cc5fb990e146af7e16f589ef5b3f7c111999e4" exitCode=0
Jul 13 17:17:25.826732 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:25.826683    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-874fa5/test-model-874fa5-kserve-db7d768c5-qdzzw"
Jul 13 17:17:25.826732 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:25.826722    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-874fa5/test-model-874fa5-kserve-db7d768c5-qdzzw" event={"ID":"4669b8b6-22fa-4955-be56-1b222d8ed615","Type":"ContainerDied","Data":"7f3813aecac8a3def725591082cc5fb990e146af7e16f589ef5b3f7c111999e4"}
Jul 13 17:17:25.826988 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:25.826761    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-874fa5/test-model-874fa5-kserve-db7d768c5-qdzzw" event={"ID":"4669b8b6-22fa-4955-be56-1b222d8ed615","Type":"ContainerDied","Data":"3efd58582b744715dc63d32929f1b53afb261abeec7c51c8f0e0cf427b3001bc"}
Jul 13 17:17:25.826988 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:25.826778    2566 scope.go:117] "RemoveContainer" containerID="7f3813aecac8a3def725591082cc5fb990e146af7e16f589ef5b3f7c111999e4"
Jul 13 17:17:25.836549 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:25.836534    2566 scope.go:117] "RemoveContainer" containerID="7f3813aecac8a3def725591082cc5fb990e146af7e16f589ef5b3f7c111999e4"
Jul 13 17:17:25.836811 ip-10-0-128-173 kubenswrapper[2566]: E0713 17:17:25.836793    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f3813aecac8a3def725591082cc5fb990e146af7e16f589ef5b3f7c111999e4\": container with ID starting with 7f3813aecac8a3def725591082cc5fb990e146af7e16f589ef5b3f7c111999e4 not found: ID does not exist" containerID="7f3813aecac8a3def725591082cc5fb990e146af7e16f589ef5b3f7c111999e4"
Jul 13 17:17:25.836861 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:25.836819    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f3813aecac8a3def725591082cc5fb990e146af7e16f589ef5b3f7c111999e4"} err="failed to get container status \"7f3813aecac8a3def725591082cc5fb990e146af7e16f589ef5b3f7c111999e4\": rpc error: code = NotFound desc = could not find container \"7f3813aecac8a3def725591082cc5fb990e146af7e16f589ef5b3f7c111999e4\": container with ID starting with 7f3813aecac8a3def725591082cc5fb990e146af7e16f589ef5b3f7c111999e4 not found: ID does not exist"
Jul 13 17:17:25.848424 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:25.848402    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-874fa5/test-model-874fa5-kserve-db7d768c5-qdzzw"]
Jul 13 17:17:25.852023 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:25.852000    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-874fa5/test-model-874fa5-kserve-db7d768c5-qdzzw"]
Jul 13 17:17:25.980336 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:25.980302    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4669b8b6-22fa-4955-be56-1b222d8ed615" path="/var/lib/kubelet/pods/4669b8b6-22fa-4955-be56-1b222d8ed615/volumes"
Jul 13 17:17:26.101516 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:26.101431    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-0c5523/test-model-0c5523-kserve-677d88d89c-wqjpc" podUID="77bdd7d8-7371-4293-a83d-893893480931" containerName="main" containerID="cri-o://55bab94aeaeecf7af9a04c5e2240ba9f7c13d150f9d151ec6875c8566b2fc137" gracePeriod=45
Jul 13 17:17:26.358640 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:26.358575    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0c5523/test-model-0c5523-kserve-677d88d89c-wqjpc"
Jul 13 17:17:26.408505 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:26.408477    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77bdd7d8-7371-4293-a83d-893893480931-home\") pod \"77bdd7d8-7371-4293-a83d-893893480931\" (UID: \"77bdd7d8-7371-4293-a83d-893893480931\") "
Jul 13 17:17:26.408698 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:26.408537    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77bdd7d8-7371-4293-a83d-893893480931-model-cache\") pod \"77bdd7d8-7371-4293-a83d-893893480931\" (UID: \"77bdd7d8-7371-4293-a83d-893893480931\") "
Jul 13 17:17:26.408698 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:26.408558    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77bdd7d8-7371-4293-a83d-893893480931-dshm\") pod \"77bdd7d8-7371-4293-a83d-893893480931\" (UID: \"77bdd7d8-7371-4293-a83d-893893480931\") "
Jul 13 17:17:26.408698 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:26.408605    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pvbnp\" (UniqueName: \"kubernetes.io/projected/77bdd7d8-7371-4293-a83d-893893480931-kube-api-access-pvbnp\") pod \"77bdd7d8-7371-4293-a83d-893893480931\" (UID: \"77bdd7d8-7371-4293-a83d-893893480931\") "
Jul 13 17:17:26.408698 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:26.408639    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77bdd7d8-7371-4293-a83d-893893480931-tls-certs\") pod \"77bdd7d8-7371-4293-a83d-893893480931\" (UID: \"77bdd7d8-7371-4293-a83d-893893480931\") "
Jul 13 17:17:26.408698 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:26.408681    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77bdd7d8-7371-4293-a83d-893893480931-tmp-dir\") pod \"77bdd7d8-7371-4293-a83d-893893480931\" (UID: \"77bdd7d8-7371-4293-a83d-893893480931\") "
Jul 13 17:17:26.408958 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:26.408776    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77bdd7d8-7371-4293-a83d-893893480931-home" (OuterVolumeSpecName: "home") pod "77bdd7d8-7371-4293-a83d-893893480931" (UID: "77bdd7d8-7371-4293-a83d-893893480931"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 17:17:26.408958 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:26.408791    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77bdd7d8-7371-4293-a83d-893893480931-model-cache" (OuterVolumeSpecName: "model-cache") pod "77bdd7d8-7371-4293-a83d-893893480931" (UID: "77bdd7d8-7371-4293-a83d-893893480931"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 17:17:26.409054 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:26.409019    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77bdd7d8-7371-4293-a83d-893893480931-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "77bdd7d8-7371-4293-a83d-893893480931" (UID: "77bdd7d8-7371-4293-a83d-893893480931"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 17:17:26.409108 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:26.409082    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77bdd7d8-7371-4293-a83d-893893480931-home\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:17:26.409108 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:26.409102    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77bdd7d8-7371-4293-a83d-893893480931-model-cache\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:17:26.410787 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:26.410767    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77bdd7d8-7371-4293-a83d-893893480931-dshm" (OuterVolumeSpecName: "dshm") pod "77bdd7d8-7371-4293-a83d-893893480931" (UID: "77bdd7d8-7371-4293-a83d-893893480931"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 17:17:26.410881 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:26.410830    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77bdd7d8-7371-4293-a83d-893893480931-kube-api-access-pvbnp" (OuterVolumeSpecName: "kube-api-access-pvbnp") pod "77bdd7d8-7371-4293-a83d-893893480931" (UID: "77bdd7d8-7371-4293-a83d-893893480931"). InnerVolumeSpecName "kube-api-access-pvbnp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 17:17:26.410934 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:26.410900    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77bdd7d8-7371-4293-a83d-893893480931-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "77bdd7d8-7371-4293-a83d-893893480931" (UID: "77bdd7d8-7371-4293-a83d-893893480931"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 17:17:26.510380 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:26.510335    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77bdd7d8-7371-4293-a83d-893893480931-tmp-dir\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:17:26.510380 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:26.510370    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77bdd7d8-7371-4293-a83d-893893480931-dshm\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:17:26.510380 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:26.510379    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvbnp\" (UniqueName: \"kubernetes.io/projected/77bdd7d8-7371-4293-a83d-893893480931-kube-api-access-pvbnp\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:17:26.510380 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:26.510388    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77bdd7d8-7371-4293-a83d-893893480931-tls-certs\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:17:26.833710 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:26.833672    2566 generic.go:356] "Generic (PLEG): container finished" podID="77bdd7d8-7371-4293-a83d-893893480931" containerID="55bab94aeaeecf7af9a04c5e2240ba9f7c13d150f9d151ec6875c8566b2fc137" exitCode=0
Jul 13 17:17:26.833893 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:26.833715    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0c5523/test-model-0c5523-kserve-677d88d89c-wqjpc"
Jul 13 17:17:26.833893 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:26.833757    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0c5523/test-model-0c5523-kserve-677d88d89c-wqjpc" event={"ID":"77bdd7d8-7371-4293-a83d-893893480931","Type":"ContainerDied","Data":"55bab94aeaeecf7af9a04c5e2240ba9f7c13d150f9d151ec6875c8566b2fc137"}
Jul 13 17:17:26.833893 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:26.833800    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0c5523/test-model-0c5523-kserve-677d88d89c-wqjpc" event={"ID":"77bdd7d8-7371-4293-a83d-893893480931","Type":"ContainerDied","Data":"4d0676560e23858f4fe7709d0ae1d4a77214eb20c1aa1754c733f9808114516c"}
Jul 13 17:17:26.833893 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:26.833818    2566 scope.go:117] "RemoveContainer" containerID="55bab94aeaeecf7af9a04c5e2240ba9f7c13d150f9d151ec6875c8566b2fc137"
Jul 13 17:17:26.843068 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:26.843052    2566 scope.go:117] "RemoveContainer" containerID="55bab94aeaeecf7af9a04c5e2240ba9f7c13d150f9d151ec6875c8566b2fc137"
Jul 13 17:17:26.843353 ip-10-0-128-173 kubenswrapper[2566]: E0713 17:17:26.843329    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55bab94aeaeecf7af9a04c5e2240ba9f7c13d150f9d151ec6875c8566b2fc137\": container with ID starting with 55bab94aeaeecf7af9a04c5e2240ba9f7c13d150f9d151ec6875c8566b2fc137 not found: ID does not exist" containerID="55bab94aeaeecf7af9a04c5e2240ba9f7c13d150f9d151ec6875c8566b2fc137"
Jul 13 17:17:26.843454 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:26.843357    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55bab94aeaeecf7af9a04c5e2240ba9f7c13d150f9d151ec6875c8566b2fc137"} err="failed to get container status \"55bab94aeaeecf7af9a04c5e2240ba9f7c13d150f9d151ec6875c8566b2fc137\": rpc error: code = NotFound desc = could not find container \"55bab94aeaeecf7af9a04c5e2240ba9f7c13d150f9d151ec6875c8566b2fc137\": container with ID starting with 55bab94aeaeecf7af9a04c5e2240ba9f7c13d150f9d151ec6875c8566b2fc137 not found: ID does not exist"
Jul 13 17:17:26.855063 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:26.855037    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-0c5523/test-model-0c5523-kserve-677d88d89c-wqjpc"]
Jul 13 17:17:26.858991 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:26.858970    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-0c5523/test-model-0c5523-kserve-677d88d89c-wqjpc"]
Jul 13 17:17:27.981476 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:27.981446    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77bdd7d8-7371-4293-a83d-893893480931" path="/var/lib/kubelet/pods/77bdd7d8-7371-4293-a83d-893893480931/volumes"
Jul 13 17:17:30.090083 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:30.090045    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:17:37.442009 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:37.441927    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0c5523-fxslf"]
Jul 13 17:17:37.442403 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:37.442345    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4669b8b6-22fa-4955-be56-1b222d8ed615" containerName="main"
Jul 13 17:17:37.442403 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:37.442357    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4669b8b6-22fa-4955-be56-1b222d8ed615" containerName="main"
Jul 13 17:17:37.442403 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:37.442367    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77bdd7d8-7371-4293-a83d-893893480931" containerName="main"
Jul 13 17:17:37.442403 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:37.442373    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="77bdd7d8-7371-4293-a83d-893893480931" containerName="main"
Jul 13 17:17:37.442531 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:37.442446    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="77bdd7d8-7371-4293-a83d-893893480931" containerName="main"
Jul 13 17:17:37.442531 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:37.442454    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="4669b8b6-22fa-4955-be56-1b222d8ed615" containerName="main"
Jul 13 17:17:37.446874 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:37.446853    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0c5523-fxslf"
Jul 13 17:17:37.449847 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:37.449811    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fkks4\""
Jul 13 17:17:37.449965 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:37.449852    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 17:17:37.454378 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:37.454352    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0c5523-fxslf"]
Jul 13 17:17:37.521593 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:37.521561    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6aefacc3-14fa-459c-adb3-cf5f79dded3e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-0c5523-fxslf\" (UID: \"6aefacc3-14fa-459c-adb3-cf5f79dded3e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0c5523-fxslf"
Jul 13 17:17:37.622844 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:37.622798    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6aefacc3-14fa-459c-adb3-cf5f79dded3e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-0c5523-fxslf\" (UID: \"6aefacc3-14fa-459c-adb3-cf5f79dded3e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0c5523-fxslf"
Jul 13 17:17:37.623697 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:37.623673    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6aefacc3-14fa-459c-adb3-cf5f79dded3e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-0c5523-fxslf\" (UID: \"6aefacc3-14fa-459c-adb3-cf5f79dded3e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0c5523-fxslf"
Jul 13 17:17:37.759665 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:37.759563    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0c5523-fxslf"
Jul 13 17:17:37.900935 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:37.900908    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0c5523-fxslf"]
Jul 13 17:17:37.903462 ip-10-0-128-173 kubenswrapper[2566]: W0713 17:17:37.903432    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6aefacc3_14fa_459c_adb3_cf5f79dded3e.slice/crio-7c4b4e5acdd14f769756f207f78d07e92474d2846824223a8aac1263ca4db208 WatchSource:0}: Error finding container 7c4b4e5acdd14f769756f207f78d07e92474d2846824223a8aac1263ca4db208: Status 404 returned error can't find the container with id 7c4b4e5acdd14f769756f207f78d07e92474d2846824223a8aac1263ca4db208
Jul 13 17:17:38.902376 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:38.902343    2566 generic.go:356] "Generic (PLEG): container finished" podID="6aefacc3-14fa-459c-adb3-cf5f79dded3e" containerID="4c9603750bb3f6a016dc82dbe3e5dd68cb42d09eff80735611e74d7bcb738a40" exitCode=0
Jul 13 17:17:38.902736 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:38.902432    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0c5523-fxslf" event={"ID":"6aefacc3-14fa-459c-adb3-cf5f79dded3e","Type":"ContainerDied","Data":"4c9603750bb3f6a016dc82dbe3e5dd68cb42d09eff80735611e74d7bcb738a40"}
Jul 13 17:17:38.902736 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:38.902480    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0c5523-fxslf" event={"ID":"6aefacc3-14fa-459c-adb3-cf5f79dded3e","Type":"ContainerStarted","Data":"7c4b4e5acdd14f769756f207f78d07e92474d2846824223a8aac1263ca4db208"}
Jul 13 17:17:40.039832 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:40.039805    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0c5523-fxslf"
Jul 13 17:17:40.149939 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:40.149899    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6aefacc3-14fa-459c-adb3-cf5f79dded3e-maas-api-service-ca\") pod \"6aefacc3-14fa-459c-adb3-cf5f79dded3e\" (UID: \"6aefacc3-14fa-459c-adb3-cf5f79dded3e\") "
Jul 13 17:17:40.150302 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:40.150279    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6aefacc3-14fa-459c-adb3-cf5f79dded3e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6aefacc3-14fa-459c-adb3-cf5f79dded3e" (UID: "6aefacc3-14fa-459c-adb3-cf5f79dded3e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 17:17:40.251219 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:40.251187    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6aefacc3-14fa-459c-adb3-cf5f79dded3e-maas-api-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:17:40.914863 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:40.914830    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0c5523-fxslf"
Jul 13 17:17:40.914863 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:40.914848    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-0c5523-fxslf" event={"ID":"6aefacc3-14fa-459c-adb3-cf5f79dded3e","Type":"ContainerDied","Data":"7c4b4e5acdd14f769756f207f78d07e92474d2846824223a8aac1263ca4db208"}
Jul 13 17:17:40.915076 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:40.914900    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7c4b4e5acdd14f769756f207f78d07e92474d2846824223a8aac1263ca4db208"
Jul 13 17:17:40.964864 ip-10-0-128-173 kubenswrapper[2566]: E0713 17:17:40.964828    2566 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6aefacc3_14fa_459c_adb3_cf5f79dded3e.slice\": RecentStats: unable to find data in memory cache]"
Jul 13 17:17:51.293838 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:17:51.293802    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:18:12.597149 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:12.597110    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:18:28.116673 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:28.116637    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2386aa-qjvl5"]
Jul 13 17:18:28.117115 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:28.117026    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6aefacc3-14fa-459c-adb3-cf5f79dded3e" containerName="revoke-keys"
Jul 13 17:18:28.117115 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:28.117037    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6aefacc3-14fa-459c-adb3-cf5f79dded3e" containerName="revoke-keys"
Jul 13 17:18:28.117187 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:28.117118    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6aefacc3-14fa-459c-adb3-cf5f79dded3e" containerName="revoke-keys"
Jul 13 17:18:28.120023 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:28.120006    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2386aa-qjvl5"
Jul 13 17:18:28.123416 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:28.123393    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fkks4\""
Jul 13 17:18:28.123550 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:28.123434    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 17:18:28.128171 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:28.128143    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2386aa-qjvl5"]
Jul 13 17:18:28.303455 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:28.303413    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3713018d-33c0-4c5d-918f-17559dba1070-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-2386aa-qjvl5\" (UID: \"3713018d-33c0-4c5d-918f-17559dba1070\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2386aa-qjvl5"
Jul 13 17:18:28.404532 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:28.404434    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3713018d-33c0-4c5d-918f-17559dba1070-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-2386aa-qjvl5\" (UID: \"3713018d-33c0-4c5d-918f-17559dba1070\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2386aa-qjvl5"
Jul 13 17:18:28.405112 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:28.405086    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3713018d-33c0-4c5d-918f-17559dba1070-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-2386aa-qjvl5\" (UID: \"3713018d-33c0-4c5d-918f-17559dba1070\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2386aa-qjvl5"
Jul 13 17:18:28.431698 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:28.431662    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2386aa-qjvl5"
Jul 13 17:18:28.578913 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:28.578884    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2386aa-qjvl5"]
Jul 13 17:18:28.580385 ip-10-0-128-173 kubenswrapper[2566]: W0713 17:18:28.580356    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3713018d_33c0_4c5d_918f_17559dba1070.slice/crio-8af71206eb8301888012f350470af86754d5c8a8204e167ef1f51d7521700993 WatchSource:0}: Error finding container 8af71206eb8301888012f350470af86754d5c8a8204e167ef1f51d7521700993: Status 404 returned error can't find the container with id 8af71206eb8301888012f350470af86754d5c8a8204e167ef1f51d7521700993
Jul 13 17:18:29.164052 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:29.163990    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2386aa-qjvl5" event={"ID":"3713018d-33c0-4c5d-918f-17559dba1070","Type":"ContainerStarted","Data":"9f895927c94a4e2985247bf5d02bfaf098766996e4677f5714c0047677a3d524"}
Jul 13 17:18:29.164052 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:29.164044    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2386aa-qjvl5" event={"ID":"3713018d-33c0-4c5d-918f-17559dba1070","Type":"ContainerStarted","Data":"8af71206eb8301888012f350470af86754d5c8a8204e167ef1f51d7521700993"}
Jul 13 17:18:29.182043 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:29.181993    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2386aa-qjvl5" podStartSLOduration=1.181976751 podStartE2EDuration="1.181976751s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:18:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:18:29.180746762 +0000 UTC m=+2295.722127193" watchObservedRunningTime="2026-07-13 17:18:29.181976751 +0000 UTC m=+2295.723357179"
Jul 13 17:18:29.403923 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:29.403889    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:18:30.170976 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:30.170937    2566 generic.go:356] "Generic (PLEG): container finished" podID="3713018d-33c0-4c5d-918f-17559dba1070" containerID="9f895927c94a4e2985247bf5d02bfaf098766996e4677f5714c0047677a3d524" exitCode=0
Jul 13 17:18:30.171394 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:30.171025    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2386aa-qjvl5" event={"ID":"3713018d-33c0-4c5d-918f-17559dba1070","Type":"ContainerDied","Data":"9f895927c94a4e2985247bf5d02bfaf098766996e4677f5714c0047677a3d524"}
Jul 13 17:18:31.316080 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:31.316046    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2386aa-qjvl5"
Jul 13 17:18:31.433445 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:31.433355    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3713018d-33c0-4c5d-918f-17559dba1070-maas-api-service-ca\") pod \"3713018d-33c0-4c5d-918f-17559dba1070\" (UID: \"3713018d-33c0-4c5d-918f-17559dba1070\") "
Jul 13 17:18:31.433762 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:31.433740    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3713018d-33c0-4c5d-918f-17559dba1070-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3713018d-33c0-4c5d-918f-17559dba1070" (UID: "3713018d-33c0-4c5d-918f-17559dba1070"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 17:18:31.534732 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:31.534696    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3713018d-33c0-4c5d-918f-17559dba1070-maas-api-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:18:32.182423 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:32.182396    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2386aa-qjvl5"
Jul 13 17:18:32.182610 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:32.182393    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-2386aa-qjvl5" event={"ID":"3713018d-33c0-4c5d-918f-17559dba1070","Type":"ContainerDied","Data":"8af71206eb8301888012f350470af86754d5c8a8204e167ef1f51d7521700993"}
Jul 13 17:18:32.182610 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:32.182523    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8af71206eb8301888012f350470af86754d5c8a8204e167ef1f51d7521700993"
Jul 13 17:18:41.998821 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:41.998785    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:18:54.273304 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:54.273271    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9c55b0-jfgv6"]
Jul 13 17:18:54.273686 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:54.273673    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3713018d-33c0-4c5d-918f-17559dba1070" containerName="revoke-keys"
Jul 13 17:18:54.273740 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:54.273686    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3713018d-33c0-4c5d-918f-17559dba1070" containerName="revoke-keys"
Jul 13 17:18:54.273780 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:54.273750    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3713018d-33c0-4c5d-918f-17559dba1070" containerName="revoke-keys"
Jul 13 17:18:54.276732 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:54.276715    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9c55b0-jfgv6"
Jul 13 17:18:54.279448 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:54.279429    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fkks4\""
Jul 13 17:18:54.279557 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:54.279434    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 17:18:54.283785 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:54.283750    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9c55b0-jfgv6"]
Jul 13 17:18:54.344039 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:54.343983    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/45676e36-19f3-4464-ba66-18fde5b698c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-9c55b0-jfgv6\" (UID: \"45676e36-19f3-4464-ba66-18fde5b698c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9c55b0-jfgv6"
Jul 13 17:18:54.444904 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:54.444866    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/45676e36-19f3-4464-ba66-18fde5b698c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-9c55b0-jfgv6\" (UID: \"45676e36-19f3-4464-ba66-18fde5b698c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9c55b0-jfgv6"
Jul 13 17:18:54.445523 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:54.445501    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/45676e36-19f3-4464-ba66-18fde5b698c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-9c55b0-jfgv6\" (UID: \"45676e36-19f3-4464-ba66-18fde5b698c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9c55b0-jfgv6"
Jul 13 17:18:54.588937 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:54.588835    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9c55b0-jfgv6"
Jul 13 17:18:54.722367 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:54.722342    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9c55b0-jfgv6"]
Jul 13 17:18:54.724156 ip-10-0-128-173 kubenswrapper[2566]: W0713 17:18:54.724131    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45676e36_19f3_4464_ba66_18fde5b698c9.slice/crio-e323b1571550eefda50411bca69e6dc896bad0e5a4e8d4e14311a75efc4e2735 WatchSource:0}: Error finding container e323b1571550eefda50411bca69e6dc896bad0e5a4e8d4e14311a75efc4e2735: Status 404 returned error can't find the container with id e323b1571550eefda50411bca69e6dc896bad0e5a4e8d4e14311a75efc4e2735
Jul 13 17:18:55.313426 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:55.313394    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9c55b0-jfgv6" event={"ID":"45676e36-19f3-4464-ba66-18fde5b698c9","Type":"ContainerStarted","Data":"96b79414bbd70e54d43dc45b92646e9ffe3d8f83e05e7b853e431a2f3454de9f"}
Jul 13 17:18:55.313829 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:55.313444    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9c55b0-jfgv6" event={"ID":"45676e36-19f3-4464-ba66-18fde5b698c9","Type":"ContainerStarted","Data":"e323b1571550eefda50411bca69e6dc896bad0e5a4e8d4e14311a75efc4e2735"}
Jul 13 17:18:55.329659 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:55.329617    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9c55b0-jfgv6" podStartSLOduration=1.329601803 podStartE2EDuration="1.329601803s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:18:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:18:55.328872673 +0000 UTC m=+2321.870253104" watchObservedRunningTime="2026-07-13 17:18:55.329601803 +0000 UTC m=+2321.870982232"
Jul 13 17:18:56.319476 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:56.319437    2566 generic.go:356] "Generic (PLEG): container finished" podID="45676e36-19f3-4464-ba66-18fde5b698c9" containerID="96b79414bbd70e54d43dc45b92646e9ffe3d8f83e05e7b853e431a2f3454de9f" exitCode=0
Jul 13 17:18:56.319873 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:56.319522    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9c55b0-jfgv6" event={"ID":"45676e36-19f3-4464-ba66-18fde5b698c9","Type":"ContainerDied","Data":"96b79414bbd70e54d43dc45b92646e9ffe3d8f83e05e7b853e431a2f3454de9f"}
Jul 13 17:18:57.455671 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:57.455647    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9c55b0-jfgv6"
Jul 13 17:18:57.577613 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:57.577541    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/45676e36-19f3-4464-ba66-18fde5b698c9-maas-api-service-ca\") pod \"45676e36-19f3-4464-ba66-18fde5b698c9\" (UID: \"45676e36-19f3-4464-ba66-18fde5b698c9\") "
Jul 13 17:18:57.577851 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:57.577829    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45676e36-19f3-4464-ba66-18fde5b698c9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "45676e36-19f3-4464-ba66-18fde5b698c9" (UID: "45676e36-19f3-4464-ba66-18fde5b698c9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 17:18:57.678734 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:57.678706    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/45676e36-19f3-4464-ba66-18fde5b698c9-maas-api-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:18:58.331730 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:58.331645    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9c55b0-jfgv6"
Jul 13 17:18:58.331730 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:58.331685    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-9c55b0-jfgv6" event={"ID":"45676e36-19f3-4464-ba66-18fde5b698c9","Type":"ContainerDied","Data":"e323b1571550eefda50411bca69e6dc896bad0e5a4e8d4e14311a75efc4e2735"}
Jul 13 17:18:58.331913 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:18:58.331753    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e323b1571550eefda50411bca69e6dc896bad0e5a4e8d4e14311a75efc4e2735"
Jul 13 17:19:07.897569 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:19:07.897488    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:19:29.703939 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:19:29.703901    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:19:45.603864 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:19:45.603835    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:20:00.401349 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:00.401311    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-aac012/auth-test-model-aac012-kserve-7cdc795c9f-cmkxb"]
Jul 13 17:20:00.401811 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:00.401707    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45676e36-19f3-4464-ba66-18fde5b698c9" containerName="revoke-keys"
Jul 13 17:20:00.401811 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:00.401719    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="45676e36-19f3-4464-ba66-18fde5b698c9" containerName="revoke-keys"
Jul 13 17:20:00.401811 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:00.401795    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="45676e36-19f3-4464-ba66-18fde5b698c9" containerName="revoke-keys"
Jul 13 17:20:00.404857 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:00.404840    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-aac012/auth-test-model-aac012-kserve-7cdc795c9f-cmkxb"
Jul 13 17:20:00.408240 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:00.408215    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-aac012\"/\"openshift-service-ca.crt\""
Jul 13 17:20:00.409600 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:00.409584    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-aac012\"/\"auth-test-model-aac012-kserve-self-signed-certs\""
Jul 13 17:20:00.409600 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:00.409596    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-aac012\"/\"kube-root-ca.crt\""
Jul 13 17:20:00.409724 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:00.409632    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-aac012\"/\"default-dockercfg-vrfqb\""
Jul 13 17:20:00.413954 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:00.413924    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-aac012/auth-test-model-aac012-kserve-7cdc795c9f-cmkxb"]
Jul 13 17:20:00.458587 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:00.458551    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b18a0b3-a54a-4041-a0f4-9776b4d640bc-model-cache\") pod \"auth-test-model-aac012-kserve-7cdc795c9f-cmkxb\" (UID: \"9b18a0b3-a54a-4041-a0f4-9776b4d640bc\") " pod="ai-tenant-e2e-shared-a-aac012/auth-test-model-aac012-kserve-7cdc795c9f-cmkxb"
Jul 13 17:20:00.458587 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:00.458596    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b18a0b3-a54a-4041-a0f4-9776b4d640bc-tmp-dir\") pod \"auth-test-model-aac012-kserve-7cdc795c9f-cmkxb\" (UID: \"9b18a0b3-a54a-4041-a0f4-9776b4d640bc\") " pod="ai-tenant-e2e-shared-a-aac012/auth-test-model-aac012-kserve-7cdc795c9f-cmkxb"
Jul 13 17:20:00.458808 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:00.458674    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjd2g\" (UniqueName: \"kubernetes.io/projected/9b18a0b3-a54a-4041-a0f4-9776b4d640bc-kube-api-access-jjd2g\") pod \"auth-test-model-aac012-kserve-7cdc795c9f-cmkxb\" (UID: \"9b18a0b3-a54a-4041-a0f4-9776b4d640bc\") " pod="ai-tenant-e2e-shared-a-aac012/auth-test-model-aac012-kserve-7cdc795c9f-cmkxb"
Jul 13 17:20:00.458808 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:00.458750    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b18a0b3-a54a-4041-a0f4-9776b4d640bc-tls-certs\") pod \"auth-test-model-aac012-kserve-7cdc795c9f-cmkxb\" (UID: \"9b18a0b3-a54a-4041-a0f4-9776b4d640bc\") " pod="ai-tenant-e2e-shared-a-aac012/auth-test-model-aac012-kserve-7cdc795c9f-cmkxb"
Jul 13 17:20:00.458808 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:00.458796    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b18a0b3-a54a-4041-a0f4-9776b4d640bc-dshm\") pod \"auth-test-model-aac012-kserve-7cdc795c9f-cmkxb\" (UID: \"9b18a0b3-a54a-4041-a0f4-9776b4d640bc\") " pod="ai-tenant-e2e-shared-a-aac012/auth-test-model-aac012-kserve-7cdc795c9f-cmkxb"
Jul 13 17:20:00.458930 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:00.458816    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b18a0b3-a54a-4041-a0f4-9776b4d640bc-home\") pod \"auth-test-model-aac012-kserve-7cdc795c9f-cmkxb\" (UID: \"9b18a0b3-a54a-4041-a0f4-9776b4d640bc\") " pod="ai-tenant-e2e-shared-a-aac012/auth-test-model-aac012-kserve-7cdc795c9f-cmkxb"
Jul 13 17:20:00.559961 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:00.559925    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b18a0b3-a54a-4041-a0f4-9776b4d640bc-model-cache\") pod \"auth-test-model-aac012-kserve-7cdc795c9f-cmkxb\" (UID: \"9b18a0b3-a54a-4041-a0f4-9776b4d640bc\") " pod="ai-tenant-e2e-shared-a-aac012/auth-test-model-aac012-kserve-7cdc795c9f-cmkxb"
Jul 13 17:20:00.559961 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:00.559962    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b18a0b3-a54a-4041-a0f4-9776b4d640bc-tmp-dir\") pod \"auth-test-model-aac012-kserve-7cdc795c9f-cmkxb\" (UID: \"9b18a0b3-a54a-4041-a0f4-9776b4d640bc\") " pod="ai-tenant-e2e-shared-a-aac012/auth-test-model-aac012-kserve-7cdc795c9f-cmkxb"
Jul 13 17:20:00.560281 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:00.559987    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjd2g\" (UniqueName: \"kubernetes.io/projected/9b18a0b3-a54a-4041-a0f4-9776b4d640bc-kube-api-access-jjd2g\") pod \"auth-test-model-aac012-kserve-7cdc795c9f-cmkxb\" (UID: \"9b18a0b3-a54a-4041-a0f4-9776b4d640bc\") " pod="ai-tenant-e2e-shared-a-aac012/auth-test-model-aac012-kserve-7cdc795c9f-cmkxb"
Jul 13 17:20:00.560281 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:00.560055    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b18a0b3-a54a-4041-a0f4-9776b4d640bc-tls-certs\") pod \"auth-test-model-aac012-kserve-7cdc795c9f-cmkxb\" (UID: \"9b18a0b3-a54a-4041-a0f4-9776b4d640bc\") " pod="ai-tenant-e2e-shared-a-aac012/auth-test-model-aac012-kserve-7cdc795c9f-cmkxb"
Jul 13 17:20:00.560281 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:00.560128    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b18a0b3-a54a-4041-a0f4-9776b4d640bc-dshm\") pod \"auth-test-model-aac012-kserve-7cdc795c9f-cmkxb\" (UID: \"9b18a0b3-a54a-4041-a0f4-9776b4d640bc\") " pod="ai-tenant-e2e-shared-a-aac012/auth-test-model-aac012-kserve-7cdc795c9f-cmkxb"
Jul 13 17:20:00.560281 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:00.560162    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b18a0b3-a54a-4041-a0f4-9776b4d640bc-home\") pod \"auth-test-model-aac012-kserve-7cdc795c9f-cmkxb\" (UID: \"9b18a0b3-a54a-4041-a0f4-9776b4d640bc\") " pod="ai-tenant-e2e-shared-a-aac012/auth-test-model-aac012-kserve-7cdc795c9f-cmkxb"
Jul 13 17:20:00.560498 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:00.560431    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b18a0b3-a54a-4041-a0f4-9776b4d640bc-tmp-dir\") pod \"auth-test-model-aac012-kserve-7cdc795c9f-cmkxb\" (UID: \"9b18a0b3-a54a-4041-a0f4-9776b4d640bc\") " pod="ai-tenant-e2e-shared-a-aac012/auth-test-model-aac012-kserve-7cdc795c9f-cmkxb"
Jul 13 17:20:00.560620 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:00.560580    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b18a0b3-a54a-4041-a0f4-9776b4d640bc-home\") pod \"auth-test-model-aac012-kserve-7cdc795c9f-cmkxb\" (UID: \"9b18a0b3-a54a-4041-a0f4-9776b4d640bc\") " pod="ai-tenant-e2e-shared-a-aac012/auth-test-model-aac012-kserve-7cdc795c9f-cmkxb"
Jul 13 17:20:00.560668 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:00.560645    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b18a0b3-a54a-4041-a0f4-9776b4d640bc-model-cache\") pod \"auth-test-model-aac012-kserve-7cdc795c9f-cmkxb\" (UID: \"9b18a0b3-a54a-4041-a0f4-9776b4d640bc\") " pod="ai-tenant-e2e-shared-a-aac012/auth-test-model-aac012-kserve-7cdc795c9f-cmkxb"
Jul 13 17:20:00.562432 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:00.562414    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b18a0b3-a54a-4041-a0f4-9776b4d640bc-dshm\") pod \"auth-test-model-aac012-kserve-7cdc795c9f-cmkxb\" (UID: \"9b18a0b3-a54a-4041-a0f4-9776b4d640bc\") " pod="ai-tenant-e2e-shared-a-aac012/auth-test-model-aac012-kserve-7cdc795c9f-cmkxb"
Jul 13 17:20:00.562503 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:00.562488    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b18a0b3-a54a-4041-a0f4-9776b4d640bc-tls-certs\") pod \"auth-test-model-aac012-kserve-7cdc795c9f-cmkxb\" (UID: \"9b18a0b3-a54a-4041-a0f4-9776b4d640bc\") " pod="ai-tenant-e2e-shared-a-aac012/auth-test-model-aac012-kserve-7cdc795c9f-cmkxb"
Jul 13 17:20:00.569342 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:00.569316    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjd2g\" (UniqueName: \"kubernetes.io/projected/9b18a0b3-a54a-4041-a0f4-9776b4d640bc-kube-api-access-jjd2g\") pod \"auth-test-model-aac012-kserve-7cdc795c9f-cmkxb\" (UID: \"9b18a0b3-a54a-4041-a0f4-9776b4d640bc\") " pod="ai-tenant-e2e-shared-a-aac012/auth-test-model-aac012-kserve-7cdc795c9f-cmkxb"
Jul 13 17:20:00.715181 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:00.715140    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-aac012/auth-test-model-aac012-kserve-7cdc795c9f-cmkxb"
Jul 13 17:20:00.854024 ip-10-0-128-173 kubenswrapper[2566]: W0713 17:20:00.853993    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9b18a0b3_a54a_4041_a0f4_9776b4d640bc.slice/crio-768e5d6d8ff3a278244b8b45bca14d15187a669ae3a5a5723ef5529bf7e51633 WatchSource:0}: Error finding container 768e5d6d8ff3a278244b8b45bca14d15187a669ae3a5a5723ef5529bf7e51633: Status 404 returned error can't find the container with id 768e5d6d8ff3a278244b8b45bca14d15187a669ae3a5a5723ef5529bf7e51633
Jul 13 17:20:00.854845 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:00.854798    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-aac012/auth-test-model-aac012-kserve-7cdc795c9f-cmkxb"]
Jul 13 17:20:01.673290 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:01.673228    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-aac012/auth-test-model-aac012-kserve-7cdc795c9f-cmkxb" event={"ID":"9b18a0b3-a54a-4041-a0f4-9776b4d640bc","Type":"ContainerStarted","Data":"713dc1a5917ead27d65f8d449ba3fd3ad9bb5fea8c66cf5fd057ab707930fbde"}
Jul 13 17:20:01.673748 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:01.673298    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-aac012/auth-test-model-aac012-kserve-7cdc795c9f-cmkxb" event={"ID":"9b18a0b3-a54a-4041-a0f4-9776b4d640bc","Type":"ContainerStarted","Data":"768e5d6d8ff3a278244b8b45bca14d15187a669ae3a5a5723ef5529bf7e51633"}
Jul 13 17:20:01.696219 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:01.696163    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-aac012/auth-test-model-aac012-kserve-7cdc795c9f-cmkxb" podStartSLOduration=1.696146454 podStartE2EDuration="1.696146454s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:20:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:20:01.695042731 +0000 UTC m=+2388.236423179" watchObservedRunningTime="2026-07-13 17:20:01.696146454 +0000 UTC m=+2388.237526883"
Jul 13 17:20:04.096087 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:04.096045    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:20:10.715322 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:10.715273    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-aac012/auth-test-model-aac012-kserve-7cdc795c9f-cmkxb"
Jul 13 17:20:10.715838 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:10.715337    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-aac012/auth-test-model-aac012-kserve-7cdc795c9f-cmkxb"
Jul 13 17:20:10.727966 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:10.727938    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-aac012/auth-test-model-aac012-kserve-7cdc795c9f-cmkxb"
Jul 13 17:20:10.739313 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:10.739290    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-aac012/auth-test-model-aac012-kserve-7cdc795c9f-cmkxb"
Jul 13 17:20:14.167605 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:14.167571    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gdqcb_6a33d4fc-2988-4f88-a92c-212d95ec2060/ovn-acl-logging/0.log"
Jul 13 17:20:14.171814 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:14.171791    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gdqcb_6a33d4fc-2988-4f88-a92c-212d95ec2060/ovn-acl-logging/0.log"
Jul 13 17:20:19.909684 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:19.909643    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-bd468d/auth-test-model-bd468d-kserve-58cd8bf987-wl246"]
Jul 13 17:20:19.912988 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:19.912970    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bd468d/auth-test-model-bd468d-kserve-58cd8bf987-wl246"
Jul 13 17:20:19.916350 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:19.916315    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-bd468d\"/\"openshift-service-ca.crt\""
Jul 13 17:20:19.917737 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:19.917714    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-bd468d\"/\"default-dockercfg-6pqs2\""
Jul 13 17:20:19.917866 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:19.917776    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-bd468d\"/\"auth-test-model-bd468d-kserve-self-signed-certs\""
Jul 13 17:20:19.917866 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:19.917791    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-bd468d\"/\"kube-root-ca.crt\""
Jul 13 17:20:19.924482 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:19.924456    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-bd468d/auth-test-model-bd468d-kserve-58cd8bf987-wl246"]
Jul 13 17:20:20.056637 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:20.056600    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05e5c70f-73bf-4fdc-b686-771a1b019744-tls-certs\") pod \"auth-test-model-bd468d-kserve-58cd8bf987-wl246\" (UID: \"05e5c70f-73bf-4fdc-b686-771a1b019744\") " pod="ai-tenant-e2e-shared-b-bd468d/auth-test-model-bd468d-kserve-58cd8bf987-wl246"
Jul 13 17:20:20.056637 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:20.056646    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05e5c70f-73bf-4fdc-b686-771a1b019744-tmp-dir\") pod \"auth-test-model-bd468d-kserve-58cd8bf987-wl246\" (UID: \"05e5c70f-73bf-4fdc-b686-771a1b019744\") " pod="ai-tenant-e2e-shared-b-bd468d/auth-test-model-bd468d-kserve-58cd8bf987-wl246"
Jul 13 17:20:20.056902 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:20.056681    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjqdn\" (UniqueName: \"kubernetes.io/projected/05e5c70f-73bf-4fdc-b686-771a1b019744-kube-api-access-zjqdn\") pod \"auth-test-model-bd468d-kserve-58cd8bf987-wl246\" (UID: \"05e5c70f-73bf-4fdc-b686-771a1b019744\") " pod="ai-tenant-e2e-shared-b-bd468d/auth-test-model-bd468d-kserve-58cd8bf987-wl246"
Jul 13 17:20:20.056902 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:20.056751    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05e5c70f-73bf-4fdc-b686-771a1b019744-home\") pod \"auth-test-model-bd468d-kserve-58cd8bf987-wl246\" (UID: \"05e5c70f-73bf-4fdc-b686-771a1b019744\") " pod="ai-tenant-e2e-shared-b-bd468d/auth-test-model-bd468d-kserve-58cd8bf987-wl246"
Jul 13 17:20:20.056902 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:20.056787    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05e5c70f-73bf-4fdc-b686-771a1b019744-dshm\") pod \"auth-test-model-bd468d-kserve-58cd8bf987-wl246\" (UID: \"05e5c70f-73bf-4fdc-b686-771a1b019744\") " pod="ai-tenant-e2e-shared-b-bd468d/auth-test-model-bd468d-kserve-58cd8bf987-wl246"
Jul 13 17:20:20.056902 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:20.056808    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05e5c70f-73bf-4fdc-b686-771a1b019744-model-cache\") pod \"auth-test-model-bd468d-kserve-58cd8bf987-wl246\" (UID: \"05e5c70f-73bf-4fdc-b686-771a1b019744\") " pod="ai-tenant-e2e-shared-b-bd468d/auth-test-model-bd468d-kserve-58cd8bf987-wl246"
Jul 13 17:20:20.158068 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:20.158030    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05e5c70f-73bf-4fdc-b686-771a1b019744-home\") pod \"auth-test-model-bd468d-kserve-58cd8bf987-wl246\" (UID: \"05e5c70f-73bf-4fdc-b686-771a1b019744\") " pod="ai-tenant-e2e-shared-b-bd468d/auth-test-model-bd468d-kserve-58cd8bf987-wl246"
Jul 13 17:20:20.158068 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:20.158070    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05e5c70f-73bf-4fdc-b686-771a1b019744-dshm\") pod \"auth-test-model-bd468d-kserve-58cd8bf987-wl246\" (UID: \"05e5c70f-73bf-4fdc-b686-771a1b019744\") " pod="ai-tenant-e2e-shared-b-bd468d/auth-test-model-bd468d-kserve-58cd8bf987-wl246"
Jul 13 17:20:20.158347 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:20.158094    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05e5c70f-73bf-4fdc-b686-771a1b019744-model-cache\") pod \"auth-test-model-bd468d-kserve-58cd8bf987-wl246\" (UID: \"05e5c70f-73bf-4fdc-b686-771a1b019744\") " pod="ai-tenant-e2e-shared-b-bd468d/auth-test-model-bd468d-kserve-58cd8bf987-wl246"
Jul 13 17:20:20.158347 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:20.158135    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05e5c70f-73bf-4fdc-b686-771a1b019744-tls-certs\") pod \"auth-test-model-bd468d-kserve-58cd8bf987-wl246\" (UID: \"05e5c70f-73bf-4fdc-b686-771a1b019744\") " pod="ai-tenant-e2e-shared-b-bd468d/auth-test-model-bd468d-kserve-58cd8bf987-wl246"
Jul 13 17:20:20.158347 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:20.158164    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05e5c70f-73bf-4fdc-b686-771a1b019744-tmp-dir\") pod \"auth-test-model-bd468d-kserve-58cd8bf987-wl246\" (UID: \"05e5c70f-73bf-4fdc-b686-771a1b019744\") " pod="ai-tenant-e2e-shared-b-bd468d/auth-test-model-bd468d-kserve-58cd8bf987-wl246"
Jul 13 17:20:20.158347 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:20.158204    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjqdn\" (UniqueName: \"kubernetes.io/projected/05e5c70f-73bf-4fdc-b686-771a1b019744-kube-api-access-zjqdn\") pod \"auth-test-model-bd468d-kserve-58cd8bf987-wl246\" (UID: \"05e5c70f-73bf-4fdc-b686-771a1b019744\") " pod="ai-tenant-e2e-shared-b-bd468d/auth-test-model-bd468d-kserve-58cd8bf987-wl246"
Jul 13 17:20:20.158565 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:20.158536    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05e5c70f-73bf-4fdc-b686-771a1b019744-home\") pod \"auth-test-model-bd468d-kserve-58cd8bf987-wl246\" (UID: \"05e5c70f-73bf-4fdc-b686-771a1b019744\") " pod="ai-tenant-e2e-shared-b-bd468d/auth-test-model-bd468d-kserve-58cd8bf987-wl246"
Jul 13 17:20:20.158629 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:20.158567    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05e5c70f-73bf-4fdc-b686-771a1b019744-model-cache\") pod \"auth-test-model-bd468d-kserve-58cd8bf987-wl246\" (UID: \"05e5c70f-73bf-4fdc-b686-771a1b019744\") " pod="ai-tenant-e2e-shared-b-bd468d/auth-test-model-bd468d-kserve-58cd8bf987-wl246"
Jul 13 17:20:20.158694 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:20.158677    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05e5c70f-73bf-4fdc-b686-771a1b019744-tmp-dir\") pod \"auth-test-model-bd468d-kserve-58cd8bf987-wl246\" (UID: \"05e5c70f-73bf-4fdc-b686-771a1b019744\") " pod="ai-tenant-e2e-shared-b-bd468d/auth-test-model-bd468d-kserve-58cd8bf987-wl246"
Jul 13 17:20:20.160529 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:20.160469    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05e5c70f-73bf-4fdc-b686-771a1b019744-dshm\") pod \"auth-test-model-bd468d-kserve-58cd8bf987-wl246\" (UID: \"05e5c70f-73bf-4fdc-b686-771a1b019744\") " pod="ai-tenant-e2e-shared-b-bd468d/auth-test-model-bd468d-kserve-58cd8bf987-wl246"
Jul 13 17:20:20.160665 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:20.160647    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05e5c70f-73bf-4fdc-b686-771a1b019744-tls-certs\") pod \"auth-test-model-bd468d-kserve-58cd8bf987-wl246\" (UID: \"05e5c70f-73bf-4fdc-b686-771a1b019744\") " pod="ai-tenant-e2e-shared-b-bd468d/auth-test-model-bd468d-kserve-58cd8bf987-wl246"
Jul 13 17:20:20.167699 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:20.167672    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjqdn\" (UniqueName: \"kubernetes.io/projected/05e5c70f-73bf-4fdc-b686-771a1b019744-kube-api-access-zjqdn\") pod \"auth-test-model-bd468d-kserve-58cd8bf987-wl246\" (UID: \"05e5c70f-73bf-4fdc-b686-771a1b019744\") " pod="ai-tenant-e2e-shared-b-bd468d/auth-test-model-bd468d-kserve-58cd8bf987-wl246"
Jul 13 17:20:20.222418 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:20.222376    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bd468d/auth-test-model-bd468d-kserve-58cd8bf987-wl246"
Jul 13 17:20:20.357676 ip-10-0-128-173 kubenswrapper[2566]: W0713 17:20:20.357641    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod05e5c70f_73bf_4fdc_b686_771a1b019744.slice/crio-983e4fe2cbd1a0f275e3dc573be60a9f9cf68ef678d029f6946f89ece8b883d9 WatchSource:0}: Error finding container 983e4fe2cbd1a0f275e3dc573be60a9f9cf68ef678d029f6946f89ece8b883d9: Status 404 returned error can't find the container with id 983e4fe2cbd1a0f275e3dc573be60a9f9cf68ef678d029f6946f89ece8b883d9
Jul 13 17:20:20.361147 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:20.358708    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-bd468d/auth-test-model-bd468d-kserve-58cd8bf987-wl246"]
Jul 13 17:20:20.776122 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:20.776032    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bd468d/auth-test-model-bd468d-kserve-58cd8bf987-wl246" event={"ID":"05e5c70f-73bf-4fdc-b686-771a1b019744","Type":"ContainerStarted","Data":"9ff661af31ffb771a1f4a24a4e20122d67a5cda99ebde742bb79b626cad21a85"}
Jul 13 17:20:20.776122 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:20.776075    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bd468d/auth-test-model-bd468d-kserve-58cd8bf987-wl246" event={"ID":"05e5c70f-73bf-4fdc-b686-771a1b019744","Type":"ContainerStarted","Data":"983e4fe2cbd1a0f275e3dc573be60a9f9cf68ef678d029f6946f89ece8b883d9"}
Jul 13 17:20:20.794752 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:20.794702    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-bd468d/auth-test-model-bd468d-kserve-58cd8bf987-wl246" podStartSLOduration=1.794687506 podStartE2EDuration="1.794687506s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:20:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:20:20.793478812 +0000 UTC m=+2407.334859241" watchObservedRunningTime="2026-07-13 17:20:20.794687506 +0000 UTC m=+2407.336067934"
Jul 13 17:20:23.504447 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:23.504411    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:20:30.222732 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:30.222688    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-bd468d/auth-test-model-bd468d-kserve-58cd8bf987-wl246"
Jul 13 17:20:30.223207 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:30.222748    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-bd468d/auth-test-model-bd468d-kserve-58cd8bf987-wl246"
Jul 13 17:20:30.235186 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:30.235158    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-bd468d/auth-test-model-bd468d-kserve-58cd8bf987-wl246"
Jul 13 17:20:30.840871 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:30.840838    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-bd468d/auth-test-model-bd468d-kserve-58cd8bf987-wl246"
Jul 13 17:20:38.404036 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:38.403958    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:20:56.992173 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:20:56.992142    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:21:16.299882 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:21:16.299846    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:21:31.002299 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:21:31.002265    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:21:47.290347 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:21:47.290308    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:22:06.397870 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:06.397793    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:22:21.243611 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:21.243579    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-aac012/auth-test-model-aac012-kserve-7cdc795c9f-cmkxb"]
Jul 13 17:22:22.116871 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:22.116835    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-bd468d/auth-test-model-bd468d-kserve-58cd8bf987-wl246"]
Jul 13 17:22:24.295696 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:24.295656    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:22:36.263591 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:36.263535    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-aac012/auth-test-model-aac012-kserve-7cdc795c9f-cmkxb" podUID="9b18a0b3-a54a-4041-a0f4-9776b4d640bc" containerName="main" containerID="cri-o://713dc1a5917ead27d65f8d449ba3fd3ad9bb5fea8c66cf5fd057ab707930fbde" gracePeriod=45
Jul 13 17:22:36.498161 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:36.498129    2566 generic.go:356] "Generic (PLEG): container finished" podID="9b18a0b3-a54a-4041-a0f4-9776b4d640bc" containerID="713dc1a5917ead27d65f8d449ba3fd3ad9bb5fea8c66cf5fd057ab707930fbde" exitCode=0
Jul 13 17:22:36.498221 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:36.498174    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-aac012/auth-test-model-aac012-kserve-7cdc795c9f-cmkxb" event={"ID":"9b18a0b3-a54a-4041-a0f4-9776b4d640bc","Type":"ContainerDied","Data":"713dc1a5917ead27d65f8d449ba3fd3ad9bb5fea8c66cf5fd057ab707930fbde"}
Jul 13 17:22:36.498221 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:36.498208    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-aac012/auth-test-model-aac012-kserve-7cdc795c9f-cmkxb" event={"ID":"9b18a0b3-a54a-4041-a0f4-9776b4d640bc","Type":"ContainerDied","Data":"768e5d6d8ff3a278244b8b45bca14d15187a669ae3a5a5723ef5529bf7e51633"}
Jul 13 17:22:36.498353 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:36.498222    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="768e5d6d8ff3a278244b8b45bca14d15187a669ae3a5a5723ef5529bf7e51633"
Jul 13 17:22:36.513658 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:36.513601    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-aac012/auth-test-model-aac012-kserve-7cdc795c9f-cmkxb"
Jul 13 17:22:36.525290 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:36.525267    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jjd2g\" (UniqueName: \"kubernetes.io/projected/9b18a0b3-a54a-4041-a0f4-9776b4d640bc-kube-api-access-jjd2g\") pod \"9b18a0b3-a54a-4041-a0f4-9776b4d640bc\" (UID: \"9b18a0b3-a54a-4041-a0f4-9776b4d640bc\") "
Jul 13 17:22:36.525355 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:36.525326    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b18a0b3-a54a-4041-a0f4-9776b4d640bc-home\") pod \"9b18a0b3-a54a-4041-a0f4-9776b4d640bc\" (UID: \"9b18a0b3-a54a-4041-a0f4-9776b4d640bc\") "
Jul 13 17:22:36.525393 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:36.525368    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b18a0b3-a54a-4041-a0f4-9776b4d640bc-model-cache\") pod \"9b18a0b3-a54a-4041-a0f4-9776b4d640bc\" (UID: \"9b18a0b3-a54a-4041-a0f4-9776b4d640bc\") "
Jul 13 17:22:36.525438 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:36.525393    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b18a0b3-a54a-4041-a0f4-9776b4d640bc-dshm\") pod \"9b18a0b3-a54a-4041-a0f4-9776b4d640bc\" (UID: \"9b18a0b3-a54a-4041-a0f4-9776b4d640bc\") "
Jul 13 17:22:36.525438 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:36.525433    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b18a0b3-a54a-4041-a0f4-9776b4d640bc-tmp-dir\") pod \"9b18a0b3-a54a-4041-a0f4-9776b4d640bc\" (UID: \"9b18a0b3-a54a-4041-a0f4-9776b4d640bc\") "
Jul 13 17:22:36.525532 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:36.525496    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b18a0b3-a54a-4041-a0f4-9776b4d640bc-tls-certs\") pod \"9b18a0b3-a54a-4041-a0f4-9776b4d640bc\" (UID: \"9b18a0b3-a54a-4041-a0f4-9776b4d640bc\") "
Jul 13 17:22:36.525589 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:36.525567    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b18a0b3-a54a-4041-a0f4-9776b4d640bc-model-cache" (OuterVolumeSpecName: "model-cache") pod "9b18a0b3-a54a-4041-a0f4-9776b4d640bc" (UID: "9b18a0b3-a54a-4041-a0f4-9776b4d640bc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 17:22:36.525647 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:36.525595    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b18a0b3-a54a-4041-a0f4-9776b4d640bc-home" (OuterVolumeSpecName: "home") pod "9b18a0b3-a54a-4041-a0f4-9776b4d640bc" (UID: "9b18a0b3-a54a-4041-a0f4-9776b4d640bc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 17:22:36.525733 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:36.525712    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b18a0b3-a54a-4041-a0f4-9776b4d640bc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9b18a0b3-a54a-4041-a0f4-9776b4d640bc" (UID: "9b18a0b3-a54a-4041-a0f4-9776b4d640bc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 17:22:36.525874 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:36.525856    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b18a0b3-a54a-4041-a0f4-9776b4d640bc-home\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:22:36.525925 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:36.525880    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b18a0b3-a54a-4041-a0f4-9776b4d640bc-model-cache\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:22:36.525925 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:36.525894    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b18a0b3-a54a-4041-a0f4-9776b4d640bc-tmp-dir\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:22:36.527562 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:36.527533    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b18a0b3-a54a-4041-a0f4-9776b4d640bc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9b18a0b3-a54a-4041-a0f4-9776b4d640bc" (UID: "9b18a0b3-a54a-4041-a0f4-9776b4d640bc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 17:22:36.527647 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:36.527555    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b18a0b3-a54a-4041-a0f4-9776b4d640bc-dshm" (OuterVolumeSpecName: "dshm") pod "9b18a0b3-a54a-4041-a0f4-9776b4d640bc" (UID: "9b18a0b3-a54a-4041-a0f4-9776b4d640bc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 17:22:36.527647 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:36.527608    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b18a0b3-a54a-4041-a0f4-9776b4d640bc-kube-api-access-jjd2g" (OuterVolumeSpecName: "kube-api-access-jjd2g") pod "9b18a0b3-a54a-4041-a0f4-9776b4d640bc" (UID: "9b18a0b3-a54a-4041-a0f4-9776b4d640bc"). InnerVolumeSpecName "kube-api-access-jjd2g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 17:22:36.626647 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:36.626614    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b18a0b3-a54a-4041-a0f4-9776b4d640bc-dshm\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:22:36.626647 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:36.626641    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b18a0b3-a54a-4041-a0f4-9776b4d640bc-tls-certs\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:22:36.626647 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:36.626651    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjd2g\" (UniqueName: \"kubernetes.io/projected/9b18a0b3-a54a-4041-a0f4-9776b4d640bc-kube-api-access-jjd2g\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:22:37.137686 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:37.137647    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-bd468d/auth-test-model-bd468d-kserve-58cd8bf987-wl246" podUID="05e5c70f-73bf-4fdc-b686-771a1b019744" containerName="main" containerID="cri-o://9ff661af31ffb771a1f4a24a4e20122d67a5cda99ebde742bb79b626cad21a85" gracePeriod=45
Jul 13 17:22:37.391485 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:37.391416    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bd468d/auth-test-model-bd468d-kserve-58cd8bf987-wl246"
Jul 13 17:22:37.434408 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:37.434378    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zjqdn\" (UniqueName: \"kubernetes.io/projected/05e5c70f-73bf-4fdc-b686-771a1b019744-kube-api-access-zjqdn\") pod \"05e5c70f-73bf-4fdc-b686-771a1b019744\" (UID: \"05e5c70f-73bf-4fdc-b686-771a1b019744\") "
Jul 13 17:22:37.434575 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:37.434422    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05e5c70f-73bf-4fdc-b686-771a1b019744-home\") pod \"05e5c70f-73bf-4fdc-b686-771a1b019744\" (UID: \"05e5c70f-73bf-4fdc-b686-771a1b019744\") "
Jul 13 17:22:37.434575 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:37.434479    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05e5c70f-73bf-4fdc-b686-771a1b019744-tmp-dir\") pod \"05e5c70f-73bf-4fdc-b686-771a1b019744\" (UID: \"05e5c70f-73bf-4fdc-b686-771a1b019744\") "
Jul 13 17:22:37.434575 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:37.434512    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05e5c70f-73bf-4fdc-b686-771a1b019744-tls-certs\") pod \"05e5c70f-73bf-4fdc-b686-771a1b019744\" (UID: \"05e5c70f-73bf-4fdc-b686-771a1b019744\") "
Jul 13 17:22:37.434704 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:37.434580    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05e5c70f-73bf-4fdc-b686-771a1b019744-model-cache\") pod \"05e5c70f-73bf-4fdc-b686-771a1b019744\" (UID: \"05e5c70f-73bf-4fdc-b686-771a1b019744\") "
Jul 13 17:22:37.434704 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:37.434634    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05e5c70f-73bf-4fdc-b686-771a1b019744-dshm\") pod \"05e5c70f-73bf-4fdc-b686-771a1b019744\" (UID: \"05e5c70f-73bf-4fdc-b686-771a1b019744\") "
Jul 13 17:22:37.434783 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:37.434708    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05e5c70f-73bf-4fdc-b686-771a1b019744-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "05e5c70f-73bf-4fdc-b686-771a1b019744" (UID: "05e5c70f-73bf-4fdc-b686-771a1b019744"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 17:22:37.434783 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:37.434728    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05e5c70f-73bf-4fdc-b686-771a1b019744-home" (OuterVolumeSpecName: "home") pod "05e5c70f-73bf-4fdc-b686-771a1b019744" (UID: "05e5c70f-73bf-4fdc-b686-771a1b019744"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 17:22:37.434901 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:37.434876    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05e5c70f-73bf-4fdc-b686-771a1b019744-model-cache" (OuterVolumeSpecName: "model-cache") pod "05e5c70f-73bf-4fdc-b686-771a1b019744" (UID: "05e5c70f-73bf-4fdc-b686-771a1b019744"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 17:22:37.434960 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:37.434929    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05e5c70f-73bf-4fdc-b686-771a1b019744-home\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:22:37.434960 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:37.434946    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05e5c70f-73bf-4fdc-b686-771a1b019744-tmp-dir\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:22:37.436765 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:37.436732    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05e5c70f-73bf-4fdc-b686-771a1b019744-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "05e5c70f-73bf-4fdc-b686-771a1b019744" (UID: "05e5c70f-73bf-4fdc-b686-771a1b019744"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 17:22:37.436895 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:37.436795    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05e5c70f-73bf-4fdc-b686-771a1b019744-dshm" (OuterVolumeSpecName: "dshm") pod "05e5c70f-73bf-4fdc-b686-771a1b019744" (UID: "05e5c70f-73bf-4fdc-b686-771a1b019744"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 17:22:37.436895 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:37.436874    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05e5c70f-73bf-4fdc-b686-771a1b019744-kube-api-access-zjqdn" (OuterVolumeSpecName: "kube-api-access-zjqdn") pod "05e5c70f-73bf-4fdc-b686-771a1b019744" (UID: "05e5c70f-73bf-4fdc-b686-771a1b019744"). InnerVolumeSpecName "kube-api-access-zjqdn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 17:22:37.505235 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:37.505196    2566 generic.go:356] "Generic (PLEG): container finished" podID="05e5c70f-73bf-4fdc-b686-771a1b019744" containerID="9ff661af31ffb771a1f4a24a4e20122d67a5cda99ebde742bb79b626cad21a85" exitCode=0
Jul 13 17:22:37.505502 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:37.505271    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bd468d/auth-test-model-bd468d-kserve-58cd8bf987-wl246"
Jul 13 17:22:37.505502 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:37.505288    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bd468d/auth-test-model-bd468d-kserve-58cd8bf987-wl246" event={"ID":"05e5c70f-73bf-4fdc-b686-771a1b019744","Type":"ContainerDied","Data":"9ff661af31ffb771a1f4a24a4e20122d67a5cda99ebde742bb79b626cad21a85"}
Jul 13 17:22:37.505502 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:37.505333    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bd468d/auth-test-model-bd468d-kserve-58cd8bf987-wl246" event={"ID":"05e5c70f-73bf-4fdc-b686-771a1b019744","Type":"ContainerDied","Data":"983e4fe2cbd1a0f275e3dc573be60a9f9cf68ef678d029f6946f89ece8b883d9"}
Jul 13 17:22:37.505502 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:37.505354    2566 scope.go:117] "RemoveContainer" containerID="9ff661af31ffb771a1f4a24a4e20122d67a5cda99ebde742bb79b626cad21a85"
Jul 13 17:22:37.505502 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:37.505401    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-aac012/auth-test-model-aac012-kserve-7cdc795c9f-cmkxb"
Jul 13 17:22:37.516496 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:37.516475    2566 scope.go:117] "RemoveContainer" containerID="9ff661af31ffb771a1f4a24a4e20122d67a5cda99ebde742bb79b626cad21a85"
Jul 13 17:22:37.516788 ip-10-0-128-173 kubenswrapper[2566]: E0713 17:22:37.516769    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ff661af31ffb771a1f4a24a4e20122d67a5cda99ebde742bb79b626cad21a85\": container with ID starting with 9ff661af31ffb771a1f4a24a4e20122d67a5cda99ebde742bb79b626cad21a85 not found: ID does not exist" containerID="9ff661af31ffb771a1f4a24a4e20122d67a5cda99ebde742bb79b626cad21a85"
Jul 13 17:22:37.516846 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:37.516798    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ff661af31ffb771a1f4a24a4e20122d67a5cda99ebde742bb79b626cad21a85"} err="failed to get container status \"9ff661af31ffb771a1f4a24a4e20122d67a5cda99ebde742bb79b626cad21a85\": rpc error: code = NotFound desc = could not find container \"9ff661af31ffb771a1f4a24a4e20122d67a5cda99ebde742bb79b626cad21a85\": container with ID starting with 9ff661af31ffb771a1f4a24a4e20122d67a5cda99ebde742bb79b626cad21a85 not found: ID does not exist"
Jul 13 17:22:37.532177 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:37.532143    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-aac012/auth-test-model-aac012-kserve-7cdc795c9f-cmkxb"]
Jul 13 17:22:37.535619 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:37.535597    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05e5c70f-73bf-4fdc-b686-771a1b019744-tls-certs\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:22:37.535619 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:37.535622    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05e5c70f-73bf-4fdc-b686-771a1b019744-model-cache\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:22:37.535755 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:37.535631    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05e5c70f-73bf-4fdc-b686-771a1b019744-dshm\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:22:37.535755 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:37.535640    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjqdn\" (UniqueName: \"kubernetes.io/projected/05e5c70f-73bf-4fdc-b686-771a1b019744-kube-api-access-zjqdn\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:22:37.537491 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:37.537465    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-aac012/auth-test-model-aac012-kserve-7cdc795c9f-cmkxb"]
Jul 13 17:22:37.557011 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:37.556988    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-bd468d/auth-test-model-bd468d-kserve-58cd8bf987-wl246"]
Jul 13 17:22:37.558904 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:37.558886    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-bd468d/auth-test-model-bd468d-kserve-58cd8bf987-wl246"]
Jul 13 17:22:37.980440 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:37.980410    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05e5c70f-73bf-4fdc-b686-771a1b019744" path="/var/lib/kubelet/pods/05e5c70f-73bf-4fdc-b686-771a1b019744/volumes"
Jul 13 17:22:37.980794 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:37.980782    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b18a0b3-a54a-4041-a0f4-9776b4d640bc" path="/var/lib/kubelet/pods/9b18a0b3-a54a-4041-a0f4-9776b4d640bc/volumes"
Jul 13 17:22:42.123126 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:42.123093    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:22:53.101617 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:53.101586    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-bd468d/sub-test-model-bd468d-kserve-6848ccbccf-2npzm"]
Jul 13 17:22:53.102109 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:53.102040    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b18a0b3-a54a-4041-a0f4-9776b4d640bc" containerName="main"
Jul 13 17:22:53.102109 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:53.102052    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b18a0b3-a54a-4041-a0f4-9776b4d640bc" containerName="main"
Jul 13 17:22:53.102109 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:53.102085    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05e5c70f-73bf-4fdc-b686-771a1b019744" containerName="main"
Jul 13 17:22:53.102109 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:53.102094    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="05e5c70f-73bf-4fdc-b686-771a1b019744" containerName="main"
Jul 13 17:22:53.102277 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:53.102158    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b18a0b3-a54a-4041-a0f4-9776b4d640bc" containerName="main"
Jul 13 17:22:53.102277 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:53.102171    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="05e5c70f-73bf-4fdc-b686-771a1b019744" containerName="main"
Jul 13 17:22:53.105064 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:53.105047    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bd468d/sub-test-model-bd468d-kserve-6848ccbccf-2npzm"
Jul 13 17:22:53.108701 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:53.108682    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-bd468d\"/\"kube-root-ca.crt\""
Jul 13 17:22:53.110108 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:53.110090    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-bd468d\"/\"default-dockercfg-6pqs2\""
Jul 13 17:22:53.110194 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:53.110118    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-bd468d\"/\"sub-test-model-bd468d-kserve-self-signed-certs\""
Jul 13 17:22:53.110194 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:53.110143    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-bd468d\"/\"openshift-service-ca.crt\""
Jul 13 17:22:53.120164 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:53.120140    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-bd468d/sub-test-model-bd468d-kserve-6848ccbccf-2npzm"]
Jul 13 17:22:53.187735 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:53.187702    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b4e5678-4d5e-464f-adac-5071d53ba2d1-tmp-dir\") pod \"sub-test-model-bd468d-kserve-6848ccbccf-2npzm\" (UID: \"4b4e5678-4d5e-464f-adac-5071d53ba2d1\") " pod="ai-tenant-e2e-shared-b-bd468d/sub-test-model-bd468d-kserve-6848ccbccf-2npzm"
Jul 13 17:22:53.187890 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:53.187749    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b4e5678-4d5e-464f-adac-5071d53ba2d1-home\") pod \"sub-test-model-bd468d-kserve-6848ccbccf-2npzm\" (UID: \"4b4e5678-4d5e-464f-adac-5071d53ba2d1\") " pod="ai-tenant-e2e-shared-b-bd468d/sub-test-model-bd468d-kserve-6848ccbccf-2npzm"
Jul 13 17:22:53.187890 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:53.187778    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b4e5678-4d5e-464f-adac-5071d53ba2d1-tls-certs\") pod \"sub-test-model-bd468d-kserve-6848ccbccf-2npzm\" (UID: \"4b4e5678-4d5e-464f-adac-5071d53ba2d1\") " pod="ai-tenant-e2e-shared-b-bd468d/sub-test-model-bd468d-kserve-6848ccbccf-2npzm"
Jul 13 17:22:53.187890 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:53.187823    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rd4ww\" (UniqueName: \"kubernetes.io/projected/4b4e5678-4d5e-464f-adac-5071d53ba2d1-kube-api-access-rd4ww\") pod \"sub-test-model-bd468d-kserve-6848ccbccf-2npzm\" (UID: \"4b4e5678-4d5e-464f-adac-5071d53ba2d1\") " pod="ai-tenant-e2e-shared-b-bd468d/sub-test-model-bd468d-kserve-6848ccbccf-2npzm"
Jul 13 17:22:53.187890 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:53.187855    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b4e5678-4d5e-464f-adac-5071d53ba2d1-model-cache\") pod \"sub-test-model-bd468d-kserve-6848ccbccf-2npzm\" (UID: \"4b4e5678-4d5e-464f-adac-5071d53ba2d1\") " pod="ai-tenant-e2e-shared-b-bd468d/sub-test-model-bd468d-kserve-6848ccbccf-2npzm"
Jul 13 17:22:53.188031 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:53.187989    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b4e5678-4d5e-464f-adac-5071d53ba2d1-dshm\") pod \"sub-test-model-bd468d-kserve-6848ccbccf-2npzm\" (UID: \"4b4e5678-4d5e-464f-adac-5071d53ba2d1\") " pod="ai-tenant-e2e-shared-b-bd468d/sub-test-model-bd468d-kserve-6848ccbccf-2npzm"
Jul 13 17:22:53.288869 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:53.288844    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rd4ww\" (UniqueName: \"kubernetes.io/projected/4b4e5678-4d5e-464f-adac-5071d53ba2d1-kube-api-access-rd4ww\") pod \"sub-test-model-bd468d-kserve-6848ccbccf-2npzm\" (UID: \"4b4e5678-4d5e-464f-adac-5071d53ba2d1\") " pod="ai-tenant-e2e-shared-b-bd468d/sub-test-model-bd468d-kserve-6848ccbccf-2npzm"
Jul 13 17:22:53.289011 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:53.288877    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b4e5678-4d5e-464f-adac-5071d53ba2d1-model-cache\") pod \"sub-test-model-bd468d-kserve-6848ccbccf-2npzm\" (UID: \"4b4e5678-4d5e-464f-adac-5071d53ba2d1\") " pod="ai-tenant-e2e-shared-b-bd468d/sub-test-model-bd468d-kserve-6848ccbccf-2npzm"
Jul 13 17:22:53.289011 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:53.288942    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b4e5678-4d5e-464f-adac-5071d53ba2d1-dshm\") pod \"sub-test-model-bd468d-kserve-6848ccbccf-2npzm\" (UID: \"4b4e5678-4d5e-464f-adac-5071d53ba2d1\") " pod="ai-tenant-e2e-shared-b-bd468d/sub-test-model-bd468d-kserve-6848ccbccf-2npzm"
Jul 13 17:22:53.289011 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:53.288979    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b4e5678-4d5e-464f-adac-5071d53ba2d1-tmp-dir\") pod \"sub-test-model-bd468d-kserve-6848ccbccf-2npzm\" (UID: \"4b4e5678-4d5e-464f-adac-5071d53ba2d1\") " pod="ai-tenant-e2e-shared-b-bd468d/sub-test-model-bd468d-kserve-6848ccbccf-2npzm"
Jul 13 17:22:53.289011 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:53.288999    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b4e5678-4d5e-464f-adac-5071d53ba2d1-home\") pod \"sub-test-model-bd468d-kserve-6848ccbccf-2npzm\" (UID: \"4b4e5678-4d5e-464f-adac-5071d53ba2d1\") " pod="ai-tenant-e2e-shared-b-bd468d/sub-test-model-bd468d-kserve-6848ccbccf-2npzm"
Jul 13 17:22:53.289219 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:53.289022    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b4e5678-4d5e-464f-adac-5071d53ba2d1-tls-certs\") pod \"sub-test-model-bd468d-kserve-6848ccbccf-2npzm\" (UID: \"4b4e5678-4d5e-464f-adac-5071d53ba2d1\") " pod="ai-tenant-e2e-shared-b-bd468d/sub-test-model-bd468d-kserve-6848ccbccf-2npzm"
Jul 13 17:22:53.289393 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:53.289368    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b4e5678-4d5e-464f-adac-5071d53ba2d1-model-cache\") pod \"sub-test-model-bd468d-kserve-6848ccbccf-2npzm\" (UID: \"4b4e5678-4d5e-464f-adac-5071d53ba2d1\") " pod="ai-tenant-e2e-shared-b-bd468d/sub-test-model-bd468d-kserve-6848ccbccf-2npzm"
Jul 13 17:22:53.289467 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:53.289420    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b4e5678-4d5e-464f-adac-5071d53ba2d1-home\") pod \"sub-test-model-bd468d-kserve-6848ccbccf-2npzm\" (UID: \"4b4e5678-4d5e-464f-adac-5071d53ba2d1\") " pod="ai-tenant-e2e-shared-b-bd468d/sub-test-model-bd468d-kserve-6848ccbccf-2npzm"
Jul 13 17:22:53.289512 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:53.289465    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b4e5678-4d5e-464f-adac-5071d53ba2d1-tmp-dir\") pod \"sub-test-model-bd468d-kserve-6848ccbccf-2npzm\" (UID: \"4b4e5678-4d5e-464f-adac-5071d53ba2d1\") " pod="ai-tenant-e2e-shared-b-bd468d/sub-test-model-bd468d-kserve-6848ccbccf-2npzm"
Jul 13 17:22:53.291227 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:53.291206    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b4e5678-4d5e-464f-adac-5071d53ba2d1-dshm\") pod \"sub-test-model-bd468d-kserve-6848ccbccf-2npzm\" (UID: \"4b4e5678-4d5e-464f-adac-5071d53ba2d1\") " pod="ai-tenant-e2e-shared-b-bd468d/sub-test-model-bd468d-kserve-6848ccbccf-2npzm"
Jul 13 17:22:53.291391 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:53.291374    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b4e5678-4d5e-464f-adac-5071d53ba2d1-tls-certs\") pod \"sub-test-model-bd468d-kserve-6848ccbccf-2npzm\" (UID: \"4b4e5678-4d5e-464f-adac-5071d53ba2d1\") " pod="ai-tenant-e2e-shared-b-bd468d/sub-test-model-bd468d-kserve-6848ccbccf-2npzm"
Jul 13 17:22:53.299819 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:53.299793    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rd4ww\" (UniqueName: \"kubernetes.io/projected/4b4e5678-4d5e-464f-adac-5071d53ba2d1-kube-api-access-rd4ww\") pod \"sub-test-model-bd468d-kserve-6848ccbccf-2npzm\" (UID: \"4b4e5678-4d5e-464f-adac-5071d53ba2d1\") " pod="ai-tenant-e2e-shared-b-bd468d/sub-test-model-bd468d-kserve-6848ccbccf-2npzm"
Jul 13 17:22:53.414716 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:53.414690    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bd468d/sub-test-model-bd468d-kserve-6848ccbccf-2npzm"
Jul 13 17:22:53.546664 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:53.546639    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-bd468d/sub-test-model-bd468d-kserve-6848ccbccf-2npzm"]
Jul 13 17:22:53.548264 ip-10-0-128-173 kubenswrapper[2566]: W0713 17:22:53.548222    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b4e5678_4d5e_464f_adac_5071d53ba2d1.slice/crio-ade4ddf60646485198037b230abd05ef7d5bd8986732d82929342663addaf25f WatchSource:0}: Error finding container ade4ddf60646485198037b230abd05ef7d5bd8986732d82929342663addaf25f: Status 404 returned error can't find the container with id ade4ddf60646485198037b230abd05ef7d5bd8986732d82929342663addaf25f
Jul 13 17:22:53.590683 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:53.590657    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bd468d/sub-test-model-bd468d-kserve-6848ccbccf-2npzm" event={"ID":"4b4e5678-4d5e-464f-adac-5071d53ba2d1","Type":"ContainerStarted","Data":"ade4ddf60646485198037b230abd05ef7d5bd8986732d82929342663addaf25f"}
Jul 13 17:22:54.598099 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:54.598004    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bd468d/sub-test-model-bd468d-kserve-6848ccbccf-2npzm" event={"ID":"4b4e5678-4d5e-464f-adac-5071d53ba2d1","Type":"ContainerStarted","Data":"28557eec3650eec02c0c964c4a6649dd3c5ec7fd1a34745d1a9062d7d8f1791e"}
Jul 13 17:22:54.618672 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:54.618621    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-bd468d/sub-test-model-bd468d-kserve-6848ccbccf-2npzm" podStartSLOduration=1.6186032849999998 podStartE2EDuration="1.618603285s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:22:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:22:54.617216204 +0000 UTC m=+2561.158596633" watchObservedRunningTime="2026-07-13 17:22:54.618603285 +0000 UTC m=+2561.159983716"
Jul 13 17:22:57.823080 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:22:57.823046    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:23:03.415180 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:03.415141    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-bd468d/sub-test-model-bd468d-kserve-6848ccbccf-2npzm"
Jul 13 17:23:03.415714 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:03.415198    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-bd468d/sub-test-model-bd468d-kserve-6848ccbccf-2npzm"
Jul 13 17:23:03.428388 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:03.428358    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-bd468d/sub-test-model-bd468d-kserve-6848ccbccf-2npzm"
Jul 13 17:23:03.657919 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:03.657889    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-bd468d/sub-test-model-bd468d-kserve-6848ccbccf-2npzm"
Jul 13 17:23:18.715314 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:18.715278    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:23:35.898170 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:35.898091    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-bd468d/sub-test-model-bd468d-kserve-6848ccbccf-2npzm"]
Jul 13 17:23:40.139608 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:40.139563    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:23:46.193366 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:46.193335    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-aac012/rate-test-model-aac012-kserve-9f8978579-zwvmt"]
Jul 13 17:23:46.197203 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:46.197184    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-aac012/rate-test-model-aac012-kserve-9f8978579-zwvmt"
Jul 13 17:23:46.200938 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:46.200910    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-aac012\"/\"kube-root-ca.crt\""
Jul 13 17:23:46.202105 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:46.202085    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-aac012\"/\"openshift-service-ca.crt\""
Jul 13 17:23:46.202105 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:46.202097    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-aac012\"/\"rate-test-model-aac012-kserve-self-signed-certs\""
Jul 13 17:23:46.202317 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:46.202141    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-aac012\"/\"default-dockercfg-vrfqb\""
Jul 13 17:23:46.207889 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:46.207863    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-aac012/rate-test-model-aac012-kserve-9f8978579-zwvmt"]
Jul 13 17:23:46.287687 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:46.287644    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gd948\" (UniqueName: \"kubernetes.io/projected/38f3f2a0-29ea-4d7a-8dd5-9b1499805427-kube-api-access-gd948\") pod \"rate-test-model-aac012-kserve-9f8978579-zwvmt\" (UID: \"38f3f2a0-29ea-4d7a-8dd5-9b1499805427\") " pod="ai-tenant-e2e-shared-a-aac012/rate-test-model-aac012-kserve-9f8978579-zwvmt"
Jul 13 17:23:46.287868 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:46.287697    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38f3f2a0-29ea-4d7a-8dd5-9b1499805427-tmp-dir\") pod \"rate-test-model-aac012-kserve-9f8978579-zwvmt\" (UID: \"38f3f2a0-29ea-4d7a-8dd5-9b1499805427\") " pod="ai-tenant-e2e-shared-a-aac012/rate-test-model-aac012-kserve-9f8978579-zwvmt"
Jul 13 17:23:46.287868 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:46.287717    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38f3f2a0-29ea-4d7a-8dd5-9b1499805427-tls-certs\") pod \"rate-test-model-aac012-kserve-9f8978579-zwvmt\" (UID: \"38f3f2a0-29ea-4d7a-8dd5-9b1499805427\") " pod="ai-tenant-e2e-shared-a-aac012/rate-test-model-aac012-kserve-9f8978579-zwvmt"
Jul 13 17:23:46.287868 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:46.287799    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38f3f2a0-29ea-4d7a-8dd5-9b1499805427-dshm\") pod \"rate-test-model-aac012-kserve-9f8978579-zwvmt\" (UID: \"38f3f2a0-29ea-4d7a-8dd5-9b1499805427\") " pod="ai-tenant-e2e-shared-a-aac012/rate-test-model-aac012-kserve-9f8978579-zwvmt"
Jul 13 17:23:46.287868 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:46.287850    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38f3f2a0-29ea-4d7a-8dd5-9b1499805427-model-cache\") pod \"rate-test-model-aac012-kserve-9f8978579-zwvmt\" (UID: \"38f3f2a0-29ea-4d7a-8dd5-9b1499805427\") " pod="ai-tenant-e2e-shared-a-aac012/rate-test-model-aac012-kserve-9f8978579-zwvmt"
Jul 13 17:23:46.288007 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:46.287881    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38f3f2a0-29ea-4d7a-8dd5-9b1499805427-home\") pod \"rate-test-model-aac012-kserve-9f8978579-zwvmt\" (UID: \"38f3f2a0-29ea-4d7a-8dd5-9b1499805427\") " pod="ai-tenant-e2e-shared-a-aac012/rate-test-model-aac012-kserve-9f8978579-zwvmt"
Jul 13 17:23:46.389416 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:46.389371    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gd948\" (UniqueName: \"kubernetes.io/projected/38f3f2a0-29ea-4d7a-8dd5-9b1499805427-kube-api-access-gd948\") pod \"rate-test-model-aac012-kserve-9f8978579-zwvmt\" (UID: \"38f3f2a0-29ea-4d7a-8dd5-9b1499805427\") " pod="ai-tenant-e2e-shared-a-aac012/rate-test-model-aac012-kserve-9f8978579-zwvmt"
Jul 13 17:23:46.389642 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:46.389429    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38f3f2a0-29ea-4d7a-8dd5-9b1499805427-tmp-dir\") pod \"rate-test-model-aac012-kserve-9f8978579-zwvmt\" (UID: \"38f3f2a0-29ea-4d7a-8dd5-9b1499805427\") " pod="ai-tenant-e2e-shared-a-aac012/rate-test-model-aac012-kserve-9f8978579-zwvmt"
Jul 13 17:23:46.389642 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:46.389456    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38f3f2a0-29ea-4d7a-8dd5-9b1499805427-tls-certs\") pod \"rate-test-model-aac012-kserve-9f8978579-zwvmt\" (UID: \"38f3f2a0-29ea-4d7a-8dd5-9b1499805427\") " pod="ai-tenant-e2e-shared-a-aac012/rate-test-model-aac012-kserve-9f8978579-zwvmt"
Jul 13 17:23:46.389642 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:46.389491    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38f3f2a0-29ea-4d7a-8dd5-9b1499805427-dshm\") pod \"rate-test-model-aac012-kserve-9f8978579-zwvmt\" (UID: \"38f3f2a0-29ea-4d7a-8dd5-9b1499805427\") " pod="ai-tenant-e2e-shared-a-aac012/rate-test-model-aac012-kserve-9f8978579-zwvmt"
Jul 13 17:23:46.389642 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:46.389536    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38f3f2a0-29ea-4d7a-8dd5-9b1499805427-model-cache\") pod \"rate-test-model-aac012-kserve-9f8978579-zwvmt\" (UID: \"38f3f2a0-29ea-4d7a-8dd5-9b1499805427\") " pod="ai-tenant-e2e-shared-a-aac012/rate-test-model-aac012-kserve-9f8978579-zwvmt"
Jul 13 17:23:46.389642 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:46.389562    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38f3f2a0-29ea-4d7a-8dd5-9b1499805427-home\") pod \"rate-test-model-aac012-kserve-9f8978579-zwvmt\" (UID: \"38f3f2a0-29ea-4d7a-8dd5-9b1499805427\") " pod="ai-tenant-e2e-shared-a-aac012/rate-test-model-aac012-kserve-9f8978579-zwvmt"
Jul 13 17:23:46.389918 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:46.389854    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38f3f2a0-29ea-4d7a-8dd5-9b1499805427-tmp-dir\") pod \"rate-test-model-aac012-kserve-9f8978579-zwvmt\" (UID: \"38f3f2a0-29ea-4d7a-8dd5-9b1499805427\") " pod="ai-tenant-e2e-shared-a-aac012/rate-test-model-aac012-kserve-9f8978579-zwvmt"
Jul 13 17:23:46.389918 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:46.389869    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38f3f2a0-29ea-4d7a-8dd5-9b1499805427-model-cache\") pod \"rate-test-model-aac012-kserve-9f8978579-zwvmt\" (UID: \"38f3f2a0-29ea-4d7a-8dd5-9b1499805427\") " pod="ai-tenant-e2e-shared-a-aac012/rate-test-model-aac012-kserve-9f8978579-zwvmt"
Jul 13 17:23:46.390026 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:46.389969    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38f3f2a0-29ea-4d7a-8dd5-9b1499805427-home\") pod \"rate-test-model-aac012-kserve-9f8978579-zwvmt\" (UID: \"38f3f2a0-29ea-4d7a-8dd5-9b1499805427\") " pod="ai-tenant-e2e-shared-a-aac012/rate-test-model-aac012-kserve-9f8978579-zwvmt"
Jul 13 17:23:46.391781 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:46.391759    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38f3f2a0-29ea-4d7a-8dd5-9b1499805427-dshm\") pod \"rate-test-model-aac012-kserve-9f8978579-zwvmt\" (UID: \"38f3f2a0-29ea-4d7a-8dd5-9b1499805427\") " pod="ai-tenant-e2e-shared-a-aac012/rate-test-model-aac012-kserve-9f8978579-zwvmt"
Jul 13 17:23:46.392133 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:46.392117    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38f3f2a0-29ea-4d7a-8dd5-9b1499805427-tls-certs\") pod \"rate-test-model-aac012-kserve-9f8978579-zwvmt\" (UID: \"38f3f2a0-29ea-4d7a-8dd5-9b1499805427\") " pod="ai-tenant-e2e-shared-a-aac012/rate-test-model-aac012-kserve-9f8978579-zwvmt"
Jul 13 17:23:46.399699 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:46.399676    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gd948\" (UniqueName: \"kubernetes.io/projected/38f3f2a0-29ea-4d7a-8dd5-9b1499805427-kube-api-access-gd948\") pod \"rate-test-model-aac012-kserve-9f8978579-zwvmt\" (UID: \"38f3f2a0-29ea-4d7a-8dd5-9b1499805427\") " pod="ai-tenant-e2e-shared-a-aac012/rate-test-model-aac012-kserve-9f8978579-zwvmt"
Jul 13 17:23:46.507533 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:46.507436    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-aac012/rate-test-model-aac012-kserve-9f8978579-zwvmt"
Jul 13 17:23:46.647698 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:46.647667    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-aac012/rate-test-model-aac012-kserve-9f8978579-zwvmt"]
Jul 13 17:23:46.649359 ip-10-0-128-173 kubenswrapper[2566]: W0713 17:23:46.649330    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod38f3f2a0_29ea_4d7a_8dd5_9b1499805427.slice/crio-ac0470f7a03607f1ba8f52648e890c44f35d8e1d4b1c56b8b9c692c2f3c24438 WatchSource:0}: Error finding container ac0470f7a03607f1ba8f52648e890c44f35d8e1d4b1c56b8b9c692c2f3c24438: Status 404 returned error can't find the container with id ac0470f7a03607f1ba8f52648e890c44f35d8e1d4b1c56b8b9c692c2f3c24438
Jul 13 17:23:46.880676 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:46.880585    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-aac012/rate-test-model-aac012-kserve-9f8978579-zwvmt" event={"ID":"38f3f2a0-29ea-4d7a-8dd5-9b1499805427","Type":"ContainerStarted","Data":"af9668ef5eb0bc480aee5b0804a425ad23f582a0caf56597b7ec73875eec6f32"}
Jul 13 17:23:46.880676 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:46.880629    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-aac012/rate-test-model-aac012-kserve-9f8978579-zwvmt" event={"ID":"38f3f2a0-29ea-4d7a-8dd5-9b1499805427","Type":"ContainerStarted","Data":"ac0470f7a03607f1ba8f52648e890c44f35d8e1d4b1c56b8b9c692c2f3c24438"}
Jul 13 17:23:46.900434 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:46.900380    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-aac012/rate-test-model-aac012-kserve-9f8978579-zwvmt" podStartSLOduration=0.900362843 podStartE2EDuration="900.362843ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:23:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:23:46.898418813 +0000 UTC m=+2613.439799267" watchObservedRunningTime="2026-07-13 17:23:46.900362843 +0000 UTC m=+2613.441743271"
Jul 13 17:23:50.918026 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:50.917979    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-bd468d/sub-test-model-bd468d-kserve-6848ccbccf-2npzm" podUID="4b4e5678-4d5e-464f-adac-5071d53ba2d1" containerName="main" containerID="cri-o://28557eec3650eec02c0c964c4a6649dd3c5ec7fd1a34745d1a9062d7d8f1791e" gracePeriod=45
Jul 13 17:23:51.185570 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:51.185544    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bd468d/sub-test-model-bd468d-kserve-6848ccbccf-2npzm"
Jul 13 17:23:51.235007 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:51.234968    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b4e5678-4d5e-464f-adac-5071d53ba2d1-model-cache\") pod \"4b4e5678-4d5e-464f-adac-5071d53ba2d1\" (UID: \"4b4e5678-4d5e-464f-adac-5071d53ba2d1\") "
Jul 13 17:23:51.235007 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:51.235005    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b4e5678-4d5e-464f-adac-5071d53ba2d1-tmp-dir\") pod \"4b4e5678-4d5e-464f-adac-5071d53ba2d1\" (UID: \"4b4e5678-4d5e-464f-adac-5071d53ba2d1\") "
Jul 13 17:23:51.235289 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:51.235036    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b4e5678-4d5e-464f-adac-5071d53ba2d1-home\") pod \"4b4e5678-4d5e-464f-adac-5071d53ba2d1\" (UID: \"4b4e5678-4d5e-464f-adac-5071d53ba2d1\") "
Jul 13 17:23:51.235289 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:51.235069    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b4e5678-4d5e-464f-adac-5071d53ba2d1-dshm\") pod \"4b4e5678-4d5e-464f-adac-5071d53ba2d1\" (UID: \"4b4e5678-4d5e-464f-adac-5071d53ba2d1\") "
Jul 13 17:23:51.235289 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:51.235148    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b4e5678-4d5e-464f-adac-5071d53ba2d1-tls-certs\") pod \"4b4e5678-4d5e-464f-adac-5071d53ba2d1\" (UID: \"4b4e5678-4d5e-464f-adac-5071d53ba2d1\") "
Jul 13 17:23:51.235289 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:51.235191    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rd4ww\" (UniqueName: \"kubernetes.io/projected/4b4e5678-4d5e-464f-adac-5071d53ba2d1-kube-api-access-rd4ww\") pod \"4b4e5678-4d5e-464f-adac-5071d53ba2d1\" (UID: \"4b4e5678-4d5e-464f-adac-5071d53ba2d1\") "
Jul 13 17:23:51.235519 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:51.235280    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b4e5678-4d5e-464f-adac-5071d53ba2d1-model-cache" (OuterVolumeSpecName: "model-cache") pod "4b4e5678-4d5e-464f-adac-5071d53ba2d1" (UID: "4b4e5678-4d5e-464f-adac-5071d53ba2d1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 17:23:51.235519 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:51.235308    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b4e5678-4d5e-464f-adac-5071d53ba2d1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4b4e5678-4d5e-464f-adac-5071d53ba2d1" (UID: "4b4e5678-4d5e-464f-adac-5071d53ba2d1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 17:23:51.235519 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:51.235386    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b4e5678-4d5e-464f-adac-5071d53ba2d1-home" (OuterVolumeSpecName: "home") pod "4b4e5678-4d5e-464f-adac-5071d53ba2d1" (UID: "4b4e5678-4d5e-464f-adac-5071d53ba2d1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 17:23:51.235665 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:51.235592    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b4e5678-4d5e-464f-adac-5071d53ba2d1-model-cache\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:23:51.235665 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:51.235613    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b4e5678-4d5e-464f-adac-5071d53ba2d1-tmp-dir\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:23:51.235665 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:51.235626    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b4e5678-4d5e-464f-adac-5071d53ba2d1-home\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:23:51.237366 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:51.237330    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b4e5678-4d5e-464f-adac-5071d53ba2d1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4b4e5678-4d5e-464f-adac-5071d53ba2d1" (UID: "4b4e5678-4d5e-464f-adac-5071d53ba2d1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 17:23:51.237478 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:51.237408    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b4e5678-4d5e-464f-adac-5071d53ba2d1-dshm" (OuterVolumeSpecName: "dshm") pod "4b4e5678-4d5e-464f-adac-5071d53ba2d1" (UID: "4b4e5678-4d5e-464f-adac-5071d53ba2d1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 17:23:51.237525 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:51.237466    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b4e5678-4d5e-464f-adac-5071d53ba2d1-kube-api-access-rd4ww" (OuterVolumeSpecName: "kube-api-access-rd4ww") pod "4b4e5678-4d5e-464f-adac-5071d53ba2d1" (UID: "4b4e5678-4d5e-464f-adac-5071d53ba2d1"). InnerVolumeSpecName "kube-api-access-rd4ww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 17:23:51.336489 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:51.336456    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b4e5678-4d5e-464f-adac-5071d53ba2d1-dshm\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:23:51.336489 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:51.336482    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b4e5678-4d5e-464f-adac-5071d53ba2d1-tls-certs\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:23:51.336489 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:51.336492    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rd4ww\" (UniqueName: \"kubernetes.io/projected/4b4e5678-4d5e-464f-adac-5071d53ba2d1-kube-api-access-rd4ww\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:23:51.910117 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:51.910082    2566 generic.go:356] "Generic (PLEG): container finished" podID="4b4e5678-4d5e-464f-adac-5071d53ba2d1" containerID="28557eec3650eec02c0c964c4a6649dd3c5ec7fd1a34745d1a9062d7d8f1791e" exitCode=0
Jul 13 17:23:51.910318 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:51.910125    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bd468d/sub-test-model-bd468d-kserve-6848ccbccf-2npzm"
Jul 13 17:23:51.910318 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:51.910135    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bd468d/sub-test-model-bd468d-kserve-6848ccbccf-2npzm" event={"ID":"4b4e5678-4d5e-464f-adac-5071d53ba2d1","Type":"ContainerDied","Data":"28557eec3650eec02c0c964c4a6649dd3c5ec7fd1a34745d1a9062d7d8f1791e"}
Jul 13 17:23:51.910318 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:51.910166    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bd468d/sub-test-model-bd468d-kserve-6848ccbccf-2npzm" event={"ID":"4b4e5678-4d5e-464f-adac-5071d53ba2d1","Type":"ContainerDied","Data":"ade4ddf60646485198037b230abd05ef7d5bd8986732d82929342663addaf25f"}
Jul 13 17:23:51.910318 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:51.910182    2566 scope.go:117] "RemoveContainer" containerID="28557eec3650eec02c0c964c4a6649dd3c5ec7fd1a34745d1a9062d7d8f1791e"
Jul 13 17:23:51.919755 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:51.919626    2566 scope.go:117] "RemoveContainer" containerID="28557eec3650eec02c0c964c4a6649dd3c5ec7fd1a34745d1a9062d7d8f1791e"
Jul 13 17:23:51.919952 ip-10-0-128-173 kubenswrapper[2566]: E0713 17:23:51.919896    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28557eec3650eec02c0c964c4a6649dd3c5ec7fd1a34745d1a9062d7d8f1791e\": container with ID starting with 28557eec3650eec02c0c964c4a6649dd3c5ec7fd1a34745d1a9062d7d8f1791e not found: ID does not exist" containerID="28557eec3650eec02c0c964c4a6649dd3c5ec7fd1a34745d1a9062d7d8f1791e"
Jul 13 17:23:51.919952 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:51.919922    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28557eec3650eec02c0c964c4a6649dd3c5ec7fd1a34745d1a9062d7d8f1791e"} err="failed to get container status \"28557eec3650eec02c0c964c4a6649dd3c5ec7fd1a34745d1a9062d7d8f1791e\": rpc error: code = NotFound desc = could not find container \"28557eec3650eec02c0c964c4a6649dd3c5ec7fd1a34745d1a9062d7d8f1791e\": container with ID starting with 28557eec3650eec02c0c964c4a6649dd3c5ec7fd1a34745d1a9062d7d8f1791e not found: ID does not exist"
Jul 13 17:23:51.932619 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:51.932583    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-bd468d/sub-test-model-bd468d-kserve-6848ccbccf-2npzm"]
Jul 13 17:23:51.935298 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:51.935274    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-bd468d/sub-test-model-bd468d-kserve-6848ccbccf-2npzm"]
Jul 13 17:23:51.981262 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:51.981220    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b4e5678-4d5e-464f-adac-5071d53ba2d1" path="/var/lib/kubelet/pods/4b4e5678-4d5e-464f-adac-5071d53ba2d1/volumes"
Jul 13 17:23:56.507816 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:56.507777    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-aac012/rate-test-model-aac012-kserve-9f8978579-zwvmt"
Jul 13 17:23:56.508216 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:56.507844    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-aac012/rate-test-model-aac012-kserve-9f8978579-zwvmt"
Jul 13 17:23:56.520219 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:56.520194    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-aac012/rate-test-model-aac012-kserve-9f8978579-zwvmt"
Jul 13 17:23:56.948862 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:56.948834    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-aac012/rate-test-model-aac012-kserve-9f8978579-zwvmt"
Jul 13 17:23:59.709149 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:23:59.709116    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:24:15.719033 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:15.719001    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-bd468d/rate-test-model-bd468d-kserve-775ffb75b7-dwvbm"]
Jul 13 17:24:15.719432 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:15.719419    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b4e5678-4d5e-464f-adac-5071d53ba2d1" containerName="main"
Jul 13 17:24:15.719432 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:15.719431    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b4e5678-4d5e-464f-adac-5071d53ba2d1" containerName="main"
Jul 13 17:24:15.719532 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:15.719514    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b4e5678-4d5e-464f-adac-5071d53ba2d1" containerName="main"
Jul 13 17:24:15.724008 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:15.723986    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bd468d/rate-test-model-bd468d-kserve-775ffb75b7-dwvbm"
Jul 13 17:24:15.729619 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:15.729598    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-bd468d\"/\"kube-root-ca.crt\""
Jul 13 17:24:15.730939 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:15.730706    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-bd468d\"/\"openshift-service-ca.crt\""
Jul 13 17:24:15.730939 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:15.730757    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-bd468d\"/\"default-dockercfg-6pqs2\""
Jul 13 17:24:15.731274 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:15.731256    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-bd468d\"/\"rate-test-model-bd468d-kserve-self-signed-certs\""
Jul 13 17:24:15.736429 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:15.736409    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-bd468d/rate-test-model-bd468d-kserve-775ffb75b7-dwvbm"]
Jul 13 17:24:15.876132 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:15.876098    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77dde529-3c0e-4684-a39f-bd4cb465270c-tls-certs\") pod \"rate-test-model-bd468d-kserve-775ffb75b7-dwvbm\" (UID: \"77dde529-3c0e-4684-a39f-bd4cb465270c\") " pod="ai-tenant-e2e-shared-b-bd468d/rate-test-model-bd468d-kserve-775ffb75b7-dwvbm"
Jul 13 17:24:15.876320 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:15.876140    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77dde529-3c0e-4684-a39f-bd4cb465270c-dshm\") pod \"rate-test-model-bd468d-kserve-775ffb75b7-dwvbm\" (UID: \"77dde529-3c0e-4684-a39f-bd4cb465270c\") " pod="ai-tenant-e2e-shared-b-bd468d/rate-test-model-bd468d-kserve-775ffb75b7-dwvbm"
Jul 13 17:24:15.876320 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:15.876155    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77dde529-3c0e-4684-a39f-bd4cb465270c-model-cache\") pod \"rate-test-model-bd468d-kserve-775ffb75b7-dwvbm\" (UID: \"77dde529-3c0e-4684-a39f-bd4cb465270c\") " pod="ai-tenant-e2e-shared-b-bd468d/rate-test-model-bd468d-kserve-775ffb75b7-dwvbm"
Jul 13 17:24:15.876320 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:15.876206    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqj6s\" (UniqueName: \"kubernetes.io/projected/77dde529-3c0e-4684-a39f-bd4cb465270c-kube-api-access-rqj6s\") pod \"rate-test-model-bd468d-kserve-775ffb75b7-dwvbm\" (UID: \"77dde529-3c0e-4684-a39f-bd4cb465270c\") " pod="ai-tenant-e2e-shared-b-bd468d/rate-test-model-bd468d-kserve-775ffb75b7-dwvbm"
Jul 13 17:24:15.876320 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:15.876281    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77dde529-3c0e-4684-a39f-bd4cb465270c-home\") pod \"rate-test-model-bd468d-kserve-775ffb75b7-dwvbm\" (UID: \"77dde529-3c0e-4684-a39f-bd4cb465270c\") " pod="ai-tenant-e2e-shared-b-bd468d/rate-test-model-bd468d-kserve-775ffb75b7-dwvbm"
Jul 13 17:24:15.876320 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:15.876312    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77dde529-3c0e-4684-a39f-bd4cb465270c-tmp-dir\") pod \"rate-test-model-bd468d-kserve-775ffb75b7-dwvbm\" (UID: \"77dde529-3c0e-4684-a39f-bd4cb465270c\") " pod="ai-tenant-e2e-shared-b-bd468d/rate-test-model-bd468d-kserve-775ffb75b7-dwvbm"
Jul 13 17:24:15.977593 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:15.977504    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqj6s\" (UniqueName: \"kubernetes.io/projected/77dde529-3c0e-4684-a39f-bd4cb465270c-kube-api-access-rqj6s\") pod \"rate-test-model-bd468d-kserve-775ffb75b7-dwvbm\" (UID: \"77dde529-3c0e-4684-a39f-bd4cb465270c\") " pod="ai-tenant-e2e-shared-b-bd468d/rate-test-model-bd468d-kserve-775ffb75b7-dwvbm"
Jul 13 17:24:15.977593 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:15.977563    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77dde529-3c0e-4684-a39f-bd4cb465270c-home\") pod \"rate-test-model-bd468d-kserve-775ffb75b7-dwvbm\" (UID: \"77dde529-3c0e-4684-a39f-bd4cb465270c\") " pod="ai-tenant-e2e-shared-b-bd468d/rate-test-model-bd468d-kserve-775ffb75b7-dwvbm"
Jul 13 17:24:15.977818 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:15.977602    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77dde529-3c0e-4684-a39f-bd4cb465270c-tmp-dir\") pod \"rate-test-model-bd468d-kserve-775ffb75b7-dwvbm\" (UID: \"77dde529-3c0e-4684-a39f-bd4cb465270c\") " pod="ai-tenant-e2e-shared-b-bd468d/rate-test-model-bd468d-kserve-775ffb75b7-dwvbm"
Jul 13 17:24:15.977818 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:15.977678    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77dde529-3c0e-4684-a39f-bd4cb465270c-tls-certs\") pod \"rate-test-model-bd468d-kserve-775ffb75b7-dwvbm\" (UID: \"77dde529-3c0e-4684-a39f-bd4cb465270c\") " pod="ai-tenant-e2e-shared-b-bd468d/rate-test-model-bd468d-kserve-775ffb75b7-dwvbm"
Jul 13 17:24:15.977818 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:15.977718    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77dde529-3c0e-4684-a39f-bd4cb465270c-dshm\") pod \"rate-test-model-bd468d-kserve-775ffb75b7-dwvbm\" (UID: \"77dde529-3c0e-4684-a39f-bd4cb465270c\") " pod="ai-tenant-e2e-shared-b-bd468d/rate-test-model-bd468d-kserve-775ffb75b7-dwvbm"
Jul 13 17:24:15.977818 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:15.977738    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77dde529-3c0e-4684-a39f-bd4cb465270c-model-cache\") pod \"rate-test-model-bd468d-kserve-775ffb75b7-dwvbm\" (UID: \"77dde529-3c0e-4684-a39f-bd4cb465270c\") " pod="ai-tenant-e2e-shared-b-bd468d/rate-test-model-bd468d-kserve-775ffb75b7-dwvbm"
Jul 13 17:24:15.978072 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:15.978041    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77dde529-3c0e-4684-a39f-bd4cb465270c-home\") pod \"rate-test-model-bd468d-kserve-775ffb75b7-dwvbm\" (UID: \"77dde529-3c0e-4684-a39f-bd4cb465270c\") " pod="ai-tenant-e2e-shared-b-bd468d/rate-test-model-bd468d-kserve-775ffb75b7-dwvbm"
Jul 13 17:24:15.978194 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:15.978142    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77dde529-3c0e-4684-a39f-bd4cb465270c-model-cache\") pod \"rate-test-model-bd468d-kserve-775ffb75b7-dwvbm\" (UID: \"77dde529-3c0e-4684-a39f-bd4cb465270c\") " pod="ai-tenant-e2e-shared-b-bd468d/rate-test-model-bd468d-kserve-775ffb75b7-dwvbm"
Jul 13 17:24:15.978194 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:15.978089    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77dde529-3c0e-4684-a39f-bd4cb465270c-tmp-dir\") pod \"rate-test-model-bd468d-kserve-775ffb75b7-dwvbm\" (UID: \"77dde529-3c0e-4684-a39f-bd4cb465270c\") " pod="ai-tenant-e2e-shared-b-bd468d/rate-test-model-bd468d-kserve-775ffb75b7-dwvbm"
Jul 13 17:24:15.979992 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:15.979971    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77dde529-3c0e-4684-a39f-bd4cb465270c-dshm\") pod \"rate-test-model-bd468d-kserve-775ffb75b7-dwvbm\" (UID: \"77dde529-3c0e-4684-a39f-bd4cb465270c\") " pod="ai-tenant-e2e-shared-b-bd468d/rate-test-model-bd468d-kserve-775ffb75b7-dwvbm"
Jul 13 17:24:15.980288 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:15.980237    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77dde529-3c0e-4684-a39f-bd4cb465270c-tls-certs\") pod \"rate-test-model-bd468d-kserve-775ffb75b7-dwvbm\" (UID: \"77dde529-3c0e-4684-a39f-bd4cb465270c\") " pod="ai-tenant-e2e-shared-b-bd468d/rate-test-model-bd468d-kserve-775ffb75b7-dwvbm"
Jul 13 17:24:15.986991 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:15.986966    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqj6s\" (UniqueName: \"kubernetes.io/projected/77dde529-3c0e-4684-a39f-bd4cb465270c-kube-api-access-rqj6s\") pod \"rate-test-model-bd468d-kserve-775ffb75b7-dwvbm\" (UID: \"77dde529-3c0e-4684-a39f-bd4cb465270c\") " pod="ai-tenant-e2e-shared-b-bd468d/rate-test-model-bd468d-kserve-775ffb75b7-dwvbm"
Jul 13 17:24:16.034014 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:16.033986    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bd468d/rate-test-model-bd468d-kserve-775ffb75b7-dwvbm"
Jul 13 17:24:16.167719 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:16.167691    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-bd468d/rate-test-model-bd468d-kserve-775ffb75b7-dwvbm"]
Jul 13 17:24:16.168726 ip-10-0-128-173 kubenswrapper[2566]: W0713 17:24:16.168701    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod77dde529_3c0e_4684_a39f_bd4cb465270c.slice/crio-7b93453017b4a6e2e7e4ece351424988e761337e38b2084e85f1616ade2b6b42 WatchSource:0}: Error finding container 7b93453017b4a6e2e7e4ece351424988e761337e38b2084e85f1616ade2b6b42: Status 404 returned error can't find the container with id 7b93453017b4a6e2e7e4ece351424988e761337e38b2084e85f1616ade2b6b42
Jul 13 17:24:17.046164 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:17.046126    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bd468d/rate-test-model-bd468d-kserve-775ffb75b7-dwvbm" event={"ID":"77dde529-3c0e-4684-a39f-bd4cb465270c","Type":"ContainerStarted","Data":"32c9dc54d64f4ae2e74af5dd9406e83e9e91edfd71cac3e3fa323d425db6ef97"}
Jul 13 17:24:17.046164 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:17.046167    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bd468d/rate-test-model-bd468d-kserve-775ffb75b7-dwvbm" event={"ID":"77dde529-3c0e-4684-a39f-bd4cb465270c","Type":"ContainerStarted","Data":"7b93453017b4a6e2e7e4ece351424988e761337e38b2084e85f1616ade2b6b42"}
Jul 13 17:24:17.068737 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:17.068688    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-bd468d/rate-test-model-bd468d-kserve-775ffb75b7-dwvbm" podStartSLOduration=2.068674012 podStartE2EDuration="2.068674012s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:24:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:24:17.066132661 +0000 UTC m=+2643.607513116" watchObservedRunningTime="2026-07-13 17:24:17.068674012 +0000 UTC m=+2643.610054440"
Jul 13 17:24:20.101662 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:20.101630    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:24:26.034576 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:26.034540    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-bd468d/rate-test-model-bd468d-kserve-775ffb75b7-dwvbm"
Jul 13 17:24:26.035144 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:26.034635    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-bd468d/rate-test-model-bd468d-kserve-775ffb75b7-dwvbm"
Jul 13 17:24:26.047446 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:26.047423    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-bd468d/rate-test-model-bd468d-kserve-775ffb75b7-dwvbm"
Jul 13 17:24:26.105571 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:26.105540    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-bd468d/rate-test-model-bd468d-kserve-775ffb75b7-dwvbm"
Jul 13 17:24:34.897762 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:34.897722    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:24:51.395353 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:51.395321    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:24:57.575838 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:57.575802    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-aac012/rate-test-model-aac012-kserve-9f8978579-zwvmt"]
Jul 13 17:24:58.587169 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:58.587134    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-bd468d/rate-test-model-bd468d-kserve-775ffb75b7-dwvbm"]
Jul 13 17:24:59.593978 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:59.593943    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7204-139899646560688"]
Jul 13 17:24:59.597563 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:59.597547    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7204-139899646560688"
Jul 13 17:24:59.601371 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:59.601349    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-d9vf2\""
Jul 13 17:24:59.606455 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:59.606434    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7204-139899646560688"]
Jul 13 17:24:59.671303 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:59.671241    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sh6f7\" (UniqueName: \"kubernetes.io/projected/a5296186-2d2b-4200-9acc-8fd888d60f9b-kube-api-access-sh6f7\") pod \"test-curl-7204-139899646560688\" (UID: \"a5296186-2d2b-4200-9acc-8fd888d60f9b\") " pod="openshift-ingress/test-curl-7204-139899646560688"
Jul 13 17:24:59.775880 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:59.775835    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sh6f7\" (UniqueName: \"kubernetes.io/projected/a5296186-2d2b-4200-9acc-8fd888d60f9b-kube-api-access-sh6f7\") pod \"test-curl-7204-139899646560688\" (UID: \"a5296186-2d2b-4200-9acc-8fd888d60f9b\") " pod="openshift-ingress/test-curl-7204-139899646560688"
Jul 13 17:24:59.786770 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:59.786736    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sh6f7\" (UniqueName: \"kubernetes.io/projected/a5296186-2d2b-4200-9acc-8fd888d60f9b-kube-api-access-sh6f7\") pod \"test-curl-7204-139899646560688\" (UID: \"a5296186-2d2b-4200-9acc-8fd888d60f9b\") " pod="openshift-ingress/test-curl-7204-139899646560688"
Jul 13 17:24:59.908399 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:24:59.908316    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7204-139899646560688"
Jul 13 17:25:00.041308 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:00.041276    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7204-139899646560688"]
Jul 13 17:25:00.042237 ip-10-0-128-173 kubenswrapper[2566]: W0713 17:25:00.042212    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda5296186_2d2b_4200_9acc_8fd888d60f9b.slice/crio-14962e69c60f3cb965c9727f679e9ead1443369716eaa40e72622a7e28bdc50a WatchSource:0}: Error finding container 14962e69c60f3cb965c9727f679e9ead1443369716eaa40e72622a7e28bdc50a: Status 404 returned error can't find the container with id 14962e69c60f3cb965c9727f679e9ead1443369716eaa40e72622a7e28bdc50a
Jul 13 17:25:00.043533 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:00.043513    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 17:25:00.272903 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:00.272872    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7204-139899646560688" event={"ID":"a5296186-2d2b-4200-9acc-8fd888d60f9b","Type":"ContainerStarted","Data":"14962e69c60f3cb965c9727f679e9ead1443369716eaa40e72622a7e28bdc50a"}
Jul 13 17:25:01.280333 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:01.280297    2566 generic.go:356] "Generic (PLEG): container finished" podID="a5296186-2d2b-4200-9acc-8fd888d60f9b" containerID="bdf98ab9f0259500854f595223f2a84405b1a68824476d2c370ff30e8ae463f2" exitCode=0
Jul 13 17:25:01.280893 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:01.280380    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7204-139899646560688" event={"ID":"a5296186-2d2b-4200-9acc-8fd888d60f9b","Type":"ContainerDied","Data":"bdf98ab9f0259500854f595223f2a84405b1a68824476d2c370ff30e8ae463f2"}
Jul 13 17:25:02.428930 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:02.428901    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7204-139899646560688"
Jul 13 17:25:02.468971 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:02.468939    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7204-139899646560688_a5296186-2d2b-4200-9acc-8fd888d60f9b/test-curl-7204-139899646560688/0.log"
Jul 13 17:25:02.494217 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:02.494184    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7204-139899646560688"]
Jul 13 17:25:02.498036 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:02.498008    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7204-139899646560688"]
Jul 13 17:25:02.503075 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:02.503056    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sh6f7\" (UniqueName: \"kubernetes.io/projected/a5296186-2d2b-4200-9acc-8fd888d60f9b-kube-api-access-sh6f7\") pod \"a5296186-2d2b-4200-9acc-8fd888d60f9b\" (UID: \"a5296186-2d2b-4200-9acc-8fd888d60f9b\") "
Jul 13 17:25:02.505220 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:02.505195    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5296186-2d2b-4200-9acc-8fd888d60f9b-kube-api-access-sh6f7" (OuterVolumeSpecName: "kube-api-access-sh6f7") pod "a5296186-2d2b-4200-9acc-8fd888d60f9b" (UID: "a5296186-2d2b-4200-9acc-8fd888d60f9b"). InnerVolumeSpecName "kube-api-access-sh6f7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 17:25:02.604599 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:02.604527    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sh6f7\" (UniqueName: \"kubernetes.io/projected/a5296186-2d2b-4200-9acc-8fd888d60f9b-kube-api-access-sh6f7\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:25:02.619287 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:02.619257    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7204-139899644948784"]
Jul 13 17:25:02.619676 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:02.619663    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5296186-2d2b-4200-9acc-8fd888d60f9b" containerName="test-curl-7204-139899646560688"
Jul 13 17:25:02.619716 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:02.619677    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5296186-2d2b-4200-9acc-8fd888d60f9b" containerName="test-curl-7204-139899646560688"
Jul 13 17:25:02.619764 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:02.619755    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5296186-2d2b-4200-9acc-8fd888d60f9b" containerName="test-curl-7204-139899646560688"
Jul 13 17:25:02.622788 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:02.622771    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7204-139899644948784"
Jul 13 17:25:02.630916 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:02.630897    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7204-139899644948784"]
Jul 13 17:25:02.705644 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:02.705615    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tp7zp\" (UniqueName: \"kubernetes.io/projected/8abc2a11-18e1-4fd9-88b6-c6fa9a14dbdb-kube-api-access-tp7zp\") pod \"test-curl-7204-139899644948784\" (UID: \"8abc2a11-18e1-4fd9-88b6-c6fa9a14dbdb\") " pod="openshift-ingress/test-curl-7204-139899644948784"
Jul 13 17:25:02.806920 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:02.806882    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tp7zp\" (UniqueName: \"kubernetes.io/projected/8abc2a11-18e1-4fd9-88b6-c6fa9a14dbdb-kube-api-access-tp7zp\") pod \"test-curl-7204-139899644948784\" (UID: \"8abc2a11-18e1-4fd9-88b6-c6fa9a14dbdb\") " pod="openshift-ingress/test-curl-7204-139899644948784"
Jul 13 17:25:02.816561 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:02.816539    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tp7zp\" (UniqueName: \"kubernetes.io/projected/8abc2a11-18e1-4fd9-88b6-c6fa9a14dbdb-kube-api-access-tp7zp\") pod \"test-curl-7204-139899644948784\" (UID: \"8abc2a11-18e1-4fd9-88b6-c6fa9a14dbdb\") " pod="openshift-ingress/test-curl-7204-139899644948784"
Jul 13 17:25:02.932855 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:02.932826    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7204-139899644948784"
Jul 13 17:25:03.061980 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:03.061951    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7204-139899644948784"]
Jul 13 17:25:03.063282 ip-10-0-128-173 kubenswrapper[2566]: W0713 17:25:03.063240    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8abc2a11_18e1_4fd9_88b6_c6fa9a14dbdb.slice/crio-1c399318868a2d7f4aad26e0df64c6a6b5717ae316f9241eb1cdbadafe8ca5bd WatchSource:0}: Error finding container 1c399318868a2d7f4aad26e0df64c6a6b5717ae316f9241eb1cdbadafe8ca5bd: Status 404 returned error can't find the container with id 1c399318868a2d7f4aad26e0df64c6a6b5717ae316f9241eb1cdbadafe8ca5bd
Jul 13 17:25:03.292022 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:03.291934    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="14962e69c60f3cb965c9727f679e9ead1443369716eaa40e72622a7e28bdc50a"
Jul 13 17:25:03.292022 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:03.291938    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7204-139899646560688"
Jul 13 17:25:03.293207 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:03.293168    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7204-139899644948784" event={"ID":"8abc2a11-18e1-4fd9-88b6-c6fa9a14dbdb","Type":"ContainerStarted","Data":"1c399318868a2d7f4aad26e0df64c6a6b5717ae316f9241eb1cdbadafe8ca5bd"}
Jul 13 17:25:03.302984 ip-10-0-128-173 kubenswrapper[2566]: E0713 17:25:03.302959    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7204-139899646560688\" is forbidden: User \"system:node:ip-10-0-128-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-128-173.ec2.internal' and this object" podUID="a5296186-2d2b-4200-9acc-8fd888d60f9b" pod="openshift-ingress/test-curl-7204-139899646560688"
Jul 13 17:25:03.985111 ip-10-0-128-173 kubenswrapper[2566]: E0713 17:25:03.985074    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7204-139899646560688\" is forbidden: User \"system:node:ip-10-0-128-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-128-173.ec2.internal' and this object" podUID="a5296186-2d2b-4200-9acc-8fd888d60f9b" pod="openshift-ingress/test-curl-7204-139899646560688"
Jul 13 17:25:03.988195 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:03.988160    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5296186-2d2b-4200-9acc-8fd888d60f9b" path="/var/lib/kubelet/pods/a5296186-2d2b-4200-9acc-8fd888d60f9b/volumes"
Jul 13 17:25:04.300737 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:04.300639    2566 generic.go:356] "Generic (PLEG): container finished" podID="8abc2a11-18e1-4fd9-88b6-c6fa9a14dbdb" containerID="2757b2c2488f295d4c293a31bf984abcf8932fc5904a1b8a5117d5fb2f81a364" exitCode=0
Jul 13 17:25:04.300880 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:04.300740    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7204-139899644948784" event={"ID":"8abc2a11-18e1-4fd9-88b6-c6fa9a14dbdb","Type":"ContainerDied","Data":"2757b2c2488f295d4c293a31bf984abcf8932fc5904a1b8a5117d5fb2f81a364"}
Jul 13 17:25:05.455145 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:05.455119    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7204-139899644948784"
Jul 13 17:25:05.476618 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:05.476590    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7204-139899644948784_8abc2a11-18e1-4fd9-88b6-c6fa9a14dbdb/test-curl-7204-139899644948784/0.log"
Jul 13 17:25:05.504237 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:05.504197    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7204-139899644948784"]
Jul 13 17:25:05.507565 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:05.507538    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7204-139899644948784"]
Jul 13 17:25:05.533162 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:05.533113    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tp7zp\" (UniqueName: \"kubernetes.io/projected/8abc2a11-18e1-4fd9-88b6-c6fa9a14dbdb-kube-api-access-tp7zp\") pod \"8abc2a11-18e1-4fd9-88b6-c6fa9a14dbdb\" (UID: \"8abc2a11-18e1-4fd9-88b6-c6fa9a14dbdb\") "
Jul 13 17:25:05.535284 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:05.535236    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8abc2a11-18e1-4fd9-88b6-c6fa9a14dbdb-kube-api-access-tp7zp" (OuterVolumeSpecName: "kube-api-access-tp7zp") pod "8abc2a11-18e1-4fd9-88b6-c6fa9a14dbdb" (UID: "8abc2a11-18e1-4fd9-88b6-c6fa9a14dbdb"). InnerVolumeSpecName "kube-api-access-tp7zp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 17:25:05.634927 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:05.634813    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tp7zp\" (UniqueName: \"kubernetes.io/projected/8abc2a11-18e1-4fd9-88b6-c6fa9a14dbdb-kube-api-access-tp7zp\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:25:05.671454 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:05.671419    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7204-139899644833840"]
Jul 13 17:25:05.671832 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:05.671819    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8abc2a11-18e1-4fd9-88b6-c6fa9a14dbdb" containerName="test-curl-7204-139899644948784"
Jul 13 17:25:05.671882 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:05.671834    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8abc2a11-18e1-4fd9-88b6-c6fa9a14dbdb" containerName="test-curl-7204-139899644948784"
Jul 13 17:25:05.671948 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:05.671934    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8abc2a11-18e1-4fd9-88b6-c6fa9a14dbdb" containerName="test-curl-7204-139899644948784"
Jul 13 17:25:05.675022 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:05.675002    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7204-139899644833840"
Jul 13 17:25:05.682110 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:05.682077    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7204-139899644833840"]
Jul 13 17:25:05.702172 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:05.702138    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:25:05.836438 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:05.836396    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqqv4\" (UniqueName: \"kubernetes.io/projected/ec9dbf45-dd3c-42fb-91d7-b1be4bdfe6f4-kube-api-access-sqqv4\") pod \"test-curl-7204-139899644833840\" (UID: \"ec9dbf45-dd3c-42fb-91d7-b1be4bdfe6f4\") " pod="openshift-ingress/test-curl-7204-139899644833840"
Jul 13 17:25:05.937929 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:05.937897    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqqv4\" (UniqueName: \"kubernetes.io/projected/ec9dbf45-dd3c-42fb-91d7-b1be4bdfe6f4-kube-api-access-sqqv4\") pod \"test-curl-7204-139899644833840\" (UID: \"ec9dbf45-dd3c-42fb-91d7-b1be4bdfe6f4\") " pod="openshift-ingress/test-curl-7204-139899644833840"
Jul 13 17:25:05.946351 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:05.946320    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqqv4\" (UniqueName: \"kubernetes.io/projected/ec9dbf45-dd3c-42fb-91d7-b1be4bdfe6f4-kube-api-access-sqqv4\") pod \"test-curl-7204-139899644833840\" (UID: \"ec9dbf45-dd3c-42fb-91d7-b1be4bdfe6f4\") " pod="openshift-ingress/test-curl-7204-139899644833840"
Jul 13 17:25:05.980530 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:05.980496    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8abc2a11-18e1-4fd9-88b6-c6fa9a14dbdb" path="/var/lib/kubelet/pods/8abc2a11-18e1-4fd9-88b6-c6fa9a14dbdb/volumes"
Jul 13 17:25:05.985855 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:05.985834    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7204-139899644833840"
Jul 13 17:25:06.117042 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:06.117018    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7204-139899644833840"]
Jul 13 17:25:06.119106 ip-10-0-128-173 kubenswrapper[2566]: W0713 17:25:06.119078    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podec9dbf45_dd3c_42fb_91d7_b1be4bdfe6f4.slice/crio-97c9679473dacfc4227c032002fc6881c31e62d50dfd373d564bbaecbeb8a2e0 WatchSource:0}: Error finding container 97c9679473dacfc4227c032002fc6881c31e62d50dfd373d564bbaecbeb8a2e0: Status 404 returned error can't find the container with id 97c9679473dacfc4227c032002fc6881c31e62d50dfd373d564bbaecbeb8a2e0
Jul 13 17:25:06.312183 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:06.312094    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7204-139899644948784"
Jul 13 17:25:06.312183 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:06.312160    2566 scope.go:117] "RemoveContainer" containerID="2757b2c2488f295d4c293a31bf984abcf8932fc5904a1b8a5117d5fb2f81a364"
Jul 13 17:25:06.313530 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:06.313511    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7204-139899644833840" event={"ID":"ec9dbf45-dd3c-42fb-91d7-b1be4bdfe6f4","Type":"ContainerStarted","Data":"97c9679473dacfc4227c032002fc6881c31e62d50dfd373d564bbaecbeb8a2e0"}
Jul 13 17:25:07.321688 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:07.321656    2566 generic.go:356] "Generic (PLEG): container finished" podID="ec9dbf45-dd3c-42fb-91d7-b1be4bdfe6f4" containerID="0e22fbf0645855269c9d4d8974671725aa13c16a00f6f9cf1c71ac9dc615735a" exitCode=0
Jul 13 17:25:07.322063 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:07.321749    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7204-139899644833840" event={"ID":"ec9dbf45-dd3c-42fb-91d7-b1be4bdfe6f4","Type":"ContainerDied","Data":"0e22fbf0645855269c9d4d8974671725aa13c16a00f6f9cf1c71ac9dc615735a"}
Jul 13 17:25:08.478704 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:08.478681    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7204-139899644833840"
Jul 13 17:25:08.499259 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:08.499222    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7204-139899644833840_ec9dbf45-dd3c-42fb-91d7-b1be4bdfe6f4/test-curl-7204-139899644833840/0.log"
Jul 13 17:25:08.523761 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:08.523737    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7204-139899644833840"]
Jul 13 17:25:08.530960 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:08.530940    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7204-139899644833840"]
Jul 13 17:25:08.563858 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:08.563840    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqqv4\" (UniqueName: \"kubernetes.io/projected/ec9dbf45-dd3c-42fb-91d7-b1be4bdfe6f4-kube-api-access-sqqv4\") pod \"ec9dbf45-dd3c-42fb-91d7-b1be4bdfe6f4\" (UID: \"ec9dbf45-dd3c-42fb-91d7-b1be4bdfe6f4\") "
Jul 13 17:25:08.565838 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:08.565812    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec9dbf45-dd3c-42fb-91d7-b1be4bdfe6f4-kube-api-access-sqqv4" (OuterVolumeSpecName: "kube-api-access-sqqv4") pod "ec9dbf45-dd3c-42fb-91d7-b1be4bdfe6f4" (UID: "ec9dbf45-dd3c-42fb-91d7-b1be4bdfe6f4"). InnerVolumeSpecName "kube-api-access-sqqv4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 17:25:08.657489 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:08.657421    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7204-139899644833840"]
Jul 13 17:25:08.657807 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:08.657795    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec9dbf45-dd3c-42fb-91d7-b1be4bdfe6f4" containerName="test-curl-7204-139899644833840"
Jul 13 17:25:08.657850 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:08.657808    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec9dbf45-dd3c-42fb-91d7-b1be4bdfe6f4" containerName="test-curl-7204-139899644833840"
Jul 13 17:25:08.657900 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:08.657891    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec9dbf45-dd3c-42fb-91d7-b1be4bdfe6f4" containerName="test-curl-7204-139899644833840"
Jul 13 17:25:08.660856 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:08.660841    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7204-139899644833840"
Jul 13 17:25:08.665382 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:08.665357    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqqv4\" (UniqueName: \"kubernetes.io/projected/ec9dbf45-dd3c-42fb-91d7-b1be4bdfe6f4-kube-api-access-sqqv4\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:25:08.667571 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:08.667547    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7204-139899644833840"]
Jul 13 17:25:08.766230 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:08.766182    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fwsg\" (UniqueName: \"kubernetes.io/projected/1d0ae452-e963-4533-8807-85ad8d357bd4-kube-api-access-4fwsg\") pod \"test-curl-7204-139899644833840\" (UID: \"1d0ae452-e963-4533-8807-85ad8d357bd4\") " pod="openshift-ingress/test-curl-7204-139899644833840"
Jul 13 17:25:08.866841 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:08.866805    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fwsg\" (UniqueName: \"kubernetes.io/projected/1d0ae452-e963-4533-8807-85ad8d357bd4-kube-api-access-4fwsg\") pod \"test-curl-7204-139899644833840\" (UID: \"1d0ae452-e963-4533-8807-85ad8d357bd4\") " pod="openshift-ingress/test-curl-7204-139899644833840"
Jul 13 17:25:08.875629 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:08.875600    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fwsg\" (UniqueName: \"kubernetes.io/projected/1d0ae452-e963-4533-8807-85ad8d357bd4-kube-api-access-4fwsg\") pod \"test-curl-7204-139899644833840\" (UID: \"1d0ae452-e963-4533-8807-85ad8d357bd4\") " pod="openshift-ingress/test-curl-7204-139899644833840"
Jul 13 17:25:08.970360 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:08.970334    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7204-139899644833840"
Jul 13 17:25:09.098995 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:09.098972    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7204-139899644833840"]
Jul 13 17:25:09.100350 ip-10-0-128-173 kubenswrapper[2566]: W0713 17:25:09.100323    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1d0ae452_e963_4533_8807_85ad8d357bd4.slice/crio-ebf0669490b2810119303180b9a890cdcd88be7b83d5626e30ccb5c37a72d9d7 WatchSource:0}: Error finding container ebf0669490b2810119303180b9a890cdcd88be7b83d5626e30ccb5c37a72d9d7: Status 404 returned error can't find the container with id ebf0669490b2810119303180b9a890cdcd88be7b83d5626e30ccb5c37a72d9d7
Jul 13 17:25:09.334825 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:09.334751    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="97c9679473dacfc4227c032002fc6881c31e62d50dfd373d564bbaecbeb8a2e0"
Jul 13 17:25:09.334825 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:09.334754    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7204-139899644833840"
Jul 13 17:25:09.335900 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:09.335870    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7204-139899644833840" event={"ID":"1d0ae452-e963-4533-8807-85ad8d357bd4","Type":"ContainerStarted","Data":"ebf0669490b2810119303180b9a890cdcd88be7b83d5626e30ccb5c37a72d9d7"}
Jul 13 17:25:09.348379 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:09.348351    2566 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="openshift-ingress/test-curl-7204-139899644833840" oldPodUID="ec9dbf45-dd3c-42fb-91d7-b1be4bdfe6f4" podUID="1d0ae452-e963-4533-8807-85ad8d357bd4"
Jul 13 17:25:09.980672 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:09.980641    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec9dbf45-dd3c-42fb-91d7-b1be4bdfe6f4" path="/var/lib/kubelet/pods/ec9dbf45-dd3c-42fb-91d7-b1be4bdfe6f4/volumes"
Jul 13 17:25:10.343024 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:10.342934    2566 generic.go:356] "Generic (PLEG): container finished" podID="1d0ae452-e963-4533-8807-85ad8d357bd4" containerID="079188f15224ee879e389d55a8f481ec5f6527ab63edc429379f118b1b8f0113" exitCode=0
Jul 13 17:25:10.343174 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:10.343019    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7204-139899644833840" event={"ID":"1d0ae452-e963-4533-8807-85ad8d357bd4","Type":"ContainerDied","Data":"079188f15224ee879e389d55a8f481ec5f6527ab63edc429379f118b1b8f0113"}
Jul 13 17:25:11.490958 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:11.490931    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7204-139899644833840"
Jul 13 17:25:11.514295 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:11.514273    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7204-139899644833840_1d0ae452-e963-4533-8807-85ad8d357bd4/test-curl-7204-139899644833840/0.log"
Jul 13 17:25:11.540902 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:11.540871    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7204-139899644833840"]
Jul 13 17:25:11.544431 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:11.544405    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7204-139899644833840"]
Jul 13 17:25:11.593127 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:11.593098    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4fwsg\" (UniqueName: \"kubernetes.io/projected/1d0ae452-e963-4533-8807-85ad8d357bd4-kube-api-access-4fwsg\") pod \"1d0ae452-e963-4533-8807-85ad8d357bd4\" (UID: \"1d0ae452-e963-4533-8807-85ad8d357bd4\") "
Jul 13 17:25:11.595125 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:11.595104    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d0ae452-e963-4533-8807-85ad8d357bd4-kube-api-access-4fwsg" (OuterVolumeSpecName: "kube-api-access-4fwsg") pod "1d0ae452-e963-4533-8807-85ad8d357bd4" (UID: "1d0ae452-e963-4533-8807-85ad8d357bd4"). InnerVolumeSpecName "kube-api-access-4fwsg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 17:25:11.678050 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:11.678017    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7204-139899644834848"]
Jul 13 17:25:11.678429 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:11.678417    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d0ae452-e963-4533-8807-85ad8d357bd4" containerName="test-curl-7204-139899644833840"
Jul 13 17:25:11.678475 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:11.678431    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d0ae452-e963-4533-8807-85ad8d357bd4" containerName="test-curl-7204-139899644833840"
Jul 13 17:25:11.678522 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:11.678513    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d0ae452-e963-4533-8807-85ad8d357bd4" containerName="test-curl-7204-139899644833840"
Jul 13 17:25:11.681648 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:11.681632    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7204-139899644834848"
Jul 13 17:25:11.688423 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:11.688397    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7204-139899644834848"]
Jul 13 17:25:11.694589 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:11.694565    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4fwsg\" (UniqueName: \"kubernetes.io/projected/1d0ae452-e963-4533-8807-85ad8d357bd4-kube-api-access-4fwsg\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:25:11.795457 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:11.795420    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tj5tv\" (UniqueName: \"kubernetes.io/projected/0c33e4b8-ab75-4dc7-b395-e862606ea55e-kube-api-access-tj5tv\") pod \"test-curl-7204-139899644834848\" (UID: \"0c33e4b8-ab75-4dc7-b395-e862606ea55e\") " pod="openshift-ingress/test-curl-7204-139899644834848"
Jul 13 17:25:11.896459 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:11.896362    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tj5tv\" (UniqueName: \"kubernetes.io/projected/0c33e4b8-ab75-4dc7-b395-e862606ea55e-kube-api-access-tj5tv\") pod \"test-curl-7204-139899644834848\" (UID: \"0c33e4b8-ab75-4dc7-b395-e862606ea55e\") " pod="openshift-ingress/test-curl-7204-139899644834848"
Jul 13 17:25:11.905424 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:11.905402    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tj5tv\" (UniqueName: \"kubernetes.io/projected/0c33e4b8-ab75-4dc7-b395-e862606ea55e-kube-api-access-tj5tv\") pod \"test-curl-7204-139899644834848\" (UID: \"0c33e4b8-ab75-4dc7-b395-e862606ea55e\") " pod="openshift-ingress/test-curl-7204-139899644834848"
Jul 13 17:25:11.981592 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:11.981562    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d0ae452-e963-4533-8807-85ad8d357bd4" path="/var/lib/kubelet/pods/1d0ae452-e963-4533-8807-85ad8d357bd4/volumes"
Jul 13 17:25:11.991668 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:11.991650    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7204-139899644834848"
Jul 13 17:25:12.127727 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:12.127704    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7204-139899644834848"]
Jul 13 17:25:12.129472 ip-10-0-128-173 kubenswrapper[2566]: W0713 17:25:12.129447    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0c33e4b8_ab75_4dc7_b395_e862606ea55e.slice/crio-3492fb15fbbbc9061dfcdc4ec142cd99e5e38101cddc1896c2fecf46da8d0f9d WatchSource:0}: Error finding container 3492fb15fbbbc9061dfcdc4ec142cd99e5e38101cddc1896c2fecf46da8d0f9d: Status 404 returned error can't find the container with id 3492fb15fbbbc9061dfcdc4ec142cd99e5e38101cddc1896c2fecf46da8d0f9d
Jul 13 17:25:12.355312 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:12.355283    2566 scope.go:117] "RemoveContainer" containerID="079188f15224ee879e389d55a8f481ec5f6527ab63edc429379f118b1b8f0113"
Jul 13 17:25:12.355510 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:12.355280    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7204-139899644833840"
Jul 13 17:25:12.356760 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:12.356727    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7204-139899644834848" event={"ID":"0c33e4b8-ab75-4dc7-b395-e862606ea55e","Type":"ContainerStarted","Data":"3492fb15fbbbc9061dfcdc4ec142cd99e5e38101cddc1896c2fecf46da8d0f9d"}
Jul 13 17:25:12.595477 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:12.595439    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-aac012/rate-test-model-aac012-kserve-9f8978579-zwvmt" podUID="38f3f2a0-29ea-4d7a-8dd5-9b1499805427" containerName="main" containerID="cri-o://af9668ef5eb0bc480aee5b0804a425ad23f582a0caf56597b7ec73875eec6f32" gracePeriod=45
Jul 13 17:25:12.851521 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:12.851496    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-aac012/rate-test-model-aac012-kserve-9f8978579-zwvmt"
Jul 13 17:25:13.007888 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:13.007811    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38f3f2a0-29ea-4d7a-8dd5-9b1499805427-model-cache\") pod \"38f3f2a0-29ea-4d7a-8dd5-9b1499805427\" (UID: \"38f3f2a0-29ea-4d7a-8dd5-9b1499805427\") "
Jul 13 17:25:13.007888 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:13.007843    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38f3f2a0-29ea-4d7a-8dd5-9b1499805427-dshm\") pod \"38f3f2a0-29ea-4d7a-8dd5-9b1499805427\" (UID: \"38f3f2a0-29ea-4d7a-8dd5-9b1499805427\") "
Jul 13 17:25:13.008112 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:13.007901    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38f3f2a0-29ea-4d7a-8dd5-9b1499805427-tmp-dir\") pod \"38f3f2a0-29ea-4d7a-8dd5-9b1499805427\" (UID: \"38f3f2a0-29ea-4d7a-8dd5-9b1499805427\") "
Jul 13 17:25:13.008112 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:13.007931    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38f3f2a0-29ea-4d7a-8dd5-9b1499805427-tls-certs\") pod \"38f3f2a0-29ea-4d7a-8dd5-9b1499805427\" (UID: \"38f3f2a0-29ea-4d7a-8dd5-9b1499805427\") "
Jul 13 17:25:13.008112 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:13.007987    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gd948\" (UniqueName: \"kubernetes.io/projected/38f3f2a0-29ea-4d7a-8dd5-9b1499805427-kube-api-access-gd948\") pod \"38f3f2a0-29ea-4d7a-8dd5-9b1499805427\" (UID: \"38f3f2a0-29ea-4d7a-8dd5-9b1499805427\") "
Jul 13 17:25:13.008112 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:13.008013    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38f3f2a0-29ea-4d7a-8dd5-9b1499805427-home\") pod \"38f3f2a0-29ea-4d7a-8dd5-9b1499805427\" (UID: \"38f3f2a0-29ea-4d7a-8dd5-9b1499805427\") "
Jul 13 17:25:13.008112 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:13.008048    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38f3f2a0-29ea-4d7a-8dd5-9b1499805427-model-cache" (OuterVolumeSpecName: "model-cache") pod "38f3f2a0-29ea-4d7a-8dd5-9b1499805427" (UID: "38f3f2a0-29ea-4d7a-8dd5-9b1499805427"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 17:25:13.008388 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:13.008195    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38f3f2a0-29ea-4d7a-8dd5-9b1499805427-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "38f3f2a0-29ea-4d7a-8dd5-9b1499805427" (UID: "38f3f2a0-29ea-4d7a-8dd5-9b1499805427"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 17:25:13.008388 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:13.008338    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/38f3f2a0-29ea-4d7a-8dd5-9b1499805427-model-cache\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:25:13.008388 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:13.008348    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38f3f2a0-29ea-4d7a-8dd5-9b1499805427-home" (OuterVolumeSpecName: "home") pod "38f3f2a0-29ea-4d7a-8dd5-9b1499805427" (UID: "38f3f2a0-29ea-4d7a-8dd5-9b1499805427"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 17:25:13.008388 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:13.008360    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38f3f2a0-29ea-4d7a-8dd5-9b1499805427-tmp-dir\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:25:13.010061 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:13.010036    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38f3f2a0-29ea-4d7a-8dd5-9b1499805427-dshm" (OuterVolumeSpecName: "dshm") pod "38f3f2a0-29ea-4d7a-8dd5-9b1499805427" (UID: "38f3f2a0-29ea-4d7a-8dd5-9b1499805427"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 17:25:13.010165 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:13.010087    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38f3f2a0-29ea-4d7a-8dd5-9b1499805427-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "38f3f2a0-29ea-4d7a-8dd5-9b1499805427" (UID: "38f3f2a0-29ea-4d7a-8dd5-9b1499805427"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 17:25:13.010218 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:13.010165    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38f3f2a0-29ea-4d7a-8dd5-9b1499805427-kube-api-access-gd948" (OuterVolumeSpecName: "kube-api-access-gd948") pod "38f3f2a0-29ea-4d7a-8dd5-9b1499805427" (UID: "38f3f2a0-29ea-4d7a-8dd5-9b1499805427"). InnerVolumeSpecName "kube-api-access-gd948". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 17:25:13.109388 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:13.109356    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/38f3f2a0-29ea-4d7a-8dd5-9b1499805427-home\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:25:13.109388 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:13.109379    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/38f3f2a0-29ea-4d7a-8dd5-9b1499805427-dshm\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:25:13.109388 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:13.109388    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/38f3f2a0-29ea-4d7a-8dd5-9b1499805427-tls-certs\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:25:13.109388 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:13.109397    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gd948\" (UniqueName: \"kubernetes.io/projected/38f3f2a0-29ea-4d7a-8dd5-9b1499805427-kube-api-access-gd948\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:25:13.364399 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:13.364302    2566 generic.go:356] "Generic (PLEG): container finished" podID="0c33e4b8-ab75-4dc7-b395-e862606ea55e" containerID="f9c44a484fd0bdcce899a9efbe5bcbb7ac7c90cde0648f0c55f11656d7021e76" exitCode=0
Jul 13 17:25:13.364575 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:13.364398    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7204-139899644834848" event={"ID":"0c33e4b8-ab75-4dc7-b395-e862606ea55e","Type":"ContainerDied","Data":"f9c44a484fd0bdcce899a9efbe5bcbb7ac7c90cde0648f0c55f11656d7021e76"}
Jul 13 17:25:13.365705 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:13.365681    2566 generic.go:356] "Generic (PLEG): container finished" podID="38f3f2a0-29ea-4d7a-8dd5-9b1499805427" containerID="af9668ef5eb0bc480aee5b0804a425ad23f582a0caf56597b7ec73875eec6f32" exitCode=0
Jul 13 17:25:13.365826 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:13.365731    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-aac012/rate-test-model-aac012-kserve-9f8978579-zwvmt"
Jul 13 17:25:13.365826 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:13.365766    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-aac012/rate-test-model-aac012-kserve-9f8978579-zwvmt" event={"ID":"38f3f2a0-29ea-4d7a-8dd5-9b1499805427","Type":"ContainerDied","Data":"af9668ef5eb0bc480aee5b0804a425ad23f582a0caf56597b7ec73875eec6f32"}
Jul 13 17:25:13.365826 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:13.365810    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-aac012/rate-test-model-aac012-kserve-9f8978579-zwvmt" event={"ID":"38f3f2a0-29ea-4d7a-8dd5-9b1499805427","Type":"ContainerDied","Data":"ac0470f7a03607f1ba8f52648e890c44f35d8e1d4b1c56b8b9c692c2f3c24438"}
Jul 13 17:25:13.365980 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:13.365832    2566 scope.go:117] "RemoveContainer" containerID="af9668ef5eb0bc480aee5b0804a425ad23f582a0caf56597b7ec73875eec6f32"
Jul 13 17:25:13.374736 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:13.374722    2566 scope.go:117] "RemoveContainer" containerID="af9668ef5eb0bc480aee5b0804a425ad23f582a0caf56597b7ec73875eec6f32"
Jul 13 17:25:13.374967 ip-10-0-128-173 kubenswrapper[2566]: E0713 17:25:13.374951    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af9668ef5eb0bc480aee5b0804a425ad23f582a0caf56597b7ec73875eec6f32\": container with ID starting with af9668ef5eb0bc480aee5b0804a425ad23f582a0caf56597b7ec73875eec6f32 not found: ID does not exist" containerID="af9668ef5eb0bc480aee5b0804a425ad23f582a0caf56597b7ec73875eec6f32"
Jul 13 17:25:13.375011 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:13.374973    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af9668ef5eb0bc480aee5b0804a425ad23f582a0caf56597b7ec73875eec6f32"} err="failed to get container status \"af9668ef5eb0bc480aee5b0804a425ad23f582a0caf56597b7ec73875eec6f32\": rpc error: code = NotFound desc = could not find container \"af9668ef5eb0bc480aee5b0804a425ad23f582a0caf56597b7ec73875eec6f32\": container with ID starting with af9668ef5eb0bc480aee5b0804a425ad23f582a0caf56597b7ec73875eec6f32 not found: ID does not exist"
Jul 13 17:25:13.394985 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:13.394964    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-aac012/rate-test-model-aac012-kserve-9f8978579-zwvmt"]
Jul 13 17:25:13.398195 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:13.398172    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-aac012/rate-test-model-aac012-kserve-9f8978579-zwvmt"]
Jul 13 17:25:13.607144 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:13.607094    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-bd468d/rate-test-model-bd468d-kserve-775ffb75b7-dwvbm" podUID="77dde529-3c0e-4684-a39f-bd4cb465270c" containerName="main" containerID="cri-o://32c9dc54d64f4ae2e74af5dd9406e83e9e91edfd71cac3e3fa323d425db6ef97" gracePeriod=45
Jul 13 17:25:13.859296 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:13.859269    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bd468d/rate-test-model-bd468d-kserve-775ffb75b7-dwvbm"
Jul 13 17:25:13.981676 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:13.981647    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38f3f2a0-29ea-4d7a-8dd5-9b1499805427" path="/var/lib/kubelet/pods/38f3f2a0-29ea-4d7a-8dd5-9b1499805427/volumes"
Jul 13 17:25:14.018260 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.018219    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77dde529-3c0e-4684-a39f-bd4cb465270c-tls-certs\") pod \"77dde529-3c0e-4684-a39f-bd4cb465270c\" (UID: \"77dde529-3c0e-4684-a39f-bd4cb465270c\") "
Jul 13 17:25:14.018372 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.018280    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77dde529-3c0e-4684-a39f-bd4cb465270c-tmp-dir\") pod \"77dde529-3c0e-4684-a39f-bd4cb465270c\" (UID: \"77dde529-3c0e-4684-a39f-bd4cb465270c\") "
Jul 13 17:25:14.018372 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.018308    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77dde529-3c0e-4684-a39f-bd4cb465270c-dshm\") pod \"77dde529-3c0e-4684-a39f-bd4cb465270c\" (UID: \"77dde529-3c0e-4684-a39f-bd4cb465270c\") "
Jul 13 17:25:14.018372 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.018342    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77dde529-3c0e-4684-a39f-bd4cb465270c-model-cache\") pod \"77dde529-3c0e-4684-a39f-bd4cb465270c\" (UID: \"77dde529-3c0e-4684-a39f-bd4cb465270c\") "
Jul 13 17:25:14.018495 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.018426    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rqj6s\" (UniqueName: \"kubernetes.io/projected/77dde529-3c0e-4684-a39f-bd4cb465270c-kube-api-access-rqj6s\") pod \"77dde529-3c0e-4684-a39f-bd4cb465270c\" (UID: \"77dde529-3c0e-4684-a39f-bd4cb465270c\") "
Jul 13 17:25:14.018495 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.018449    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77dde529-3c0e-4684-a39f-bd4cb465270c-home\") pod \"77dde529-3c0e-4684-a39f-bd4cb465270c\" (UID: \"77dde529-3c0e-4684-a39f-bd4cb465270c\") "
Jul 13 17:25:14.018578 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.018550    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77dde529-3c0e-4684-a39f-bd4cb465270c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "77dde529-3c0e-4684-a39f-bd4cb465270c" (UID: "77dde529-3c0e-4684-a39f-bd4cb465270c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 17:25:14.018688 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.018663    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77dde529-3c0e-4684-a39f-bd4cb465270c-model-cache" (OuterVolumeSpecName: "model-cache") pod "77dde529-3c0e-4684-a39f-bd4cb465270c" (UID: "77dde529-3c0e-4684-a39f-bd4cb465270c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 17:25:14.018772 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.018748    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77dde529-3c0e-4684-a39f-bd4cb465270c-home" (OuterVolumeSpecName: "home") pod "77dde529-3c0e-4684-a39f-bd4cb465270c" (UID: "77dde529-3c0e-4684-a39f-bd4cb465270c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 17:25:14.018823 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.018777    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77dde529-3c0e-4684-a39f-bd4cb465270c-tmp-dir\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:25:14.018823 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.018797    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77dde529-3c0e-4684-a39f-bd4cb465270c-model-cache\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:25:14.020324 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.020301    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77dde529-3c0e-4684-a39f-bd4cb465270c-kube-api-access-rqj6s" (OuterVolumeSpecName: "kube-api-access-rqj6s") pod "77dde529-3c0e-4684-a39f-bd4cb465270c" (UID: "77dde529-3c0e-4684-a39f-bd4cb465270c"). InnerVolumeSpecName "kube-api-access-rqj6s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 17:25:14.020394 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.020306    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77dde529-3c0e-4684-a39f-bd4cb465270c-dshm" (OuterVolumeSpecName: "dshm") pod "77dde529-3c0e-4684-a39f-bd4cb465270c" (UID: "77dde529-3c0e-4684-a39f-bd4cb465270c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 17:25:14.020394 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.020345    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77dde529-3c0e-4684-a39f-bd4cb465270c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "77dde529-3c0e-4684-a39f-bd4cb465270c" (UID: "77dde529-3c0e-4684-a39f-bd4cb465270c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 17:25:14.120063 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.120028    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77dde529-3c0e-4684-a39f-bd4cb465270c-tls-certs\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:25:14.120221 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.120073    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77dde529-3c0e-4684-a39f-bd4cb465270c-dshm\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:25:14.120221 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.120153    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rqj6s\" (UniqueName: \"kubernetes.io/projected/77dde529-3c0e-4684-a39f-bd4cb465270c-kube-api-access-rqj6s\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:25:14.120221 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.120176    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77dde529-3c0e-4684-a39f-bd4cb465270c-home\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:25:14.201966 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.201933    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gdqcb_6a33d4fc-2988-4f88-a92c-212d95ec2060/ovn-acl-logging/0.log"
Jul 13 17:25:14.208083 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.208061    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gdqcb_6a33d4fc-2988-4f88-a92c-212d95ec2060/ovn-acl-logging/0.log"
Jul 13 17:25:14.373089 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.373005    2566 generic.go:356] "Generic (PLEG): container finished" podID="77dde529-3c0e-4684-a39f-bd4cb465270c" containerID="32c9dc54d64f4ae2e74af5dd9406e83e9e91edfd71cac3e3fa323d425db6ef97" exitCode=0
Jul 13 17:25:14.373089 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.373051    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-bd468d/rate-test-model-bd468d-kserve-775ffb75b7-dwvbm"
Jul 13 17:25:14.373332 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.373092    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bd468d/rate-test-model-bd468d-kserve-775ffb75b7-dwvbm" event={"ID":"77dde529-3c0e-4684-a39f-bd4cb465270c","Type":"ContainerDied","Data":"32c9dc54d64f4ae2e74af5dd9406e83e9e91edfd71cac3e3fa323d425db6ef97"}
Jul 13 17:25:14.373332 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.373137    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-bd468d/rate-test-model-bd468d-kserve-775ffb75b7-dwvbm" event={"ID":"77dde529-3c0e-4684-a39f-bd4cb465270c","Type":"ContainerDied","Data":"7b93453017b4a6e2e7e4ece351424988e761337e38b2084e85f1616ade2b6b42"}
Jul 13 17:25:14.373332 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.373163    2566 scope.go:117] "RemoveContainer" containerID="32c9dc54d64f4ae2e74af5dd9406e83e9e91edfd71cac3e3fa323d425db6ef97"
Jul 13 17:25:14.385022 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.385001    2566 scope.go:117] "RemoveContainer" containerID="32c9dc54d64f4ae2e74af5dd9406e83e9e91edfd71cac3e3fa323d425db6ef97"
Jul 13 17:25:14.385375 ip-10-0-128-173 kubenswrapper[2566]: E0713 17:25:14.385354    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32c9dc54d64f4ae2e74af5dd9406e83e9e91edfd71cac3e3fa323d425db6ef97\": container with ID starting with 32c9dc54d64f4ae2e74af5dd9406e83e9e91edfd71cac3e3fa323d425db6ef97 not found: ID does not exist" containerID="32c9dc54d64f4ae2e74af5dd9406e83e9e91edfd71cac3e3fa323d425db6ef97"
Jul 13 17:25:14.385444 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.385388    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32c9dc54d64f4ae2e74af5dd9406e83e9e91edfd71cac3e3fa323d425db6ef97"} err="failed to get container status \"32c9dc54d64f4ae2e74af5dd9406e83e9e91edfd71cac3e3fa323d425db6ef97\": rpc error: code = NotFound desc = could not find container \"32c9dc54d64f4ae2e74af5dd9406e83e9e91edfd71cac3e3fa323d425db6ef97\": container with ID starting with 32c9dc54d64f4ae2e74af5dd9406e83e9e91edfd71cac3e3fa323d425db6ef97 not found: ID does not exist"
Jul 13 17:25:14.401476 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.401439    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-bd468d/rate-test-model-bd468d-kserve-775ffb75b7-dwvbm"]
Jul 13 17:25:14.407345 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.407315    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-bd468d/rate-test-model-bd468d-kserve-775ffb75b7-dwvbm"]
Jul 13 17:25:14.522284 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.522239    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7204-139899644834848"
Jul 13 17:25:14.546986 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.546962    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7204-139899644834848_0c33e4b8-ab75-4dc7-b395-e862606ea55e/test-curl-7204-139899644834848/0.log"
Jul 13 17:25:14.575499 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.575470    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7204-139899644834848"]
Jul 13 17:25:14.577587 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.577566    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7204-139899644834848"]
Jul 13 17:25:14.625229 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.625143    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tj5tv\" (UniqueName: \"kubernetes.io/projected/0c33e4b8-ab75-4dc7-b395-e862606ea55e-kube-api-access-tj5tv\") pod \"0c33e4b8-ab75-4dc7-b395-e862606ea55e\" (UID: \"0c33e4b8-ab75-4dc7-b395-e862606ea55e\") "
Jul 13 17:25:14.627207 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.627181    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c33e4b8-ab75-4dc7-b395-e862606ea55e-kube-api-access-tj5tv" (OuterVolumeSpecName: "kube-api-access-tj5tv") pod "0c33e4b8-ab75-4dc7-b395-e862606ea55e" (UID: "0c33e4b8-ab75-4dc7-b395-e862606ea55e"). InnerVolumeSpecName "kube-api-access-tj5tv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 17:25:14.699420 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.699383    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7204-139899644834848"]
Jul 13 17:25:14.699810 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.699797    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77dde529-3c0e-4684-a39f-bd4cb465270c" containerName="main"
Jul 13 17:25:14.699856 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.699811    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="77dde529-3c0e-4684-a39f-bd4cb465270c" containerName="main"
Jul 13 17:25:14.699856 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.699818    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c33e4b8-ab75-4dc7-b395-e862606ea55e" containerName="test-curl-7204-139899644834848"
Jul 13 17:25:14.699856 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.699825    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c33e4b8-ab75-4dc7-b395-e862606ea55e" containerName="test-curl-7204-139899644834848"
Jul 13 17:25:14.699856 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.699852    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38f3f2a0-29ea-4d7a-8dd5-9b1499805427" containerName="main"
Jul 13 17:25:14.699856 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.699857    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="38f3f2a0-29ea-4d7a-8dd5-9b1499805427" containerName="main"
Jul 13 17:25:14.700007 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.699933    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="38f3f2a0-29ea-4d7a-8dd5-9b1499805427" containerName="main"
Jul 13 17:25:14.700007 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.699939    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="77dde529-3c0e-4684-a39f-bd4cb465270c" containerName="main"
Jul 13 17:25:14.700007 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.699946    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c33e4b8-ab75-4dc7-b395-e862606ea55e" containerName="test-curl-7204-139899644834848"
Jul 13 17:25:14.702862 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.702846    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7204-139899644834848"
Jul 13 17:25:14.708080 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.708058    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7204-139899644834848"]
Jul 13 17:25:14.726063 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.726035    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tj5tv\" (UniqueName: \"kubernetes.io/projected/0c33e4b8-ab75-4dc7-b395-e862606ea55e-kube-api-access-tj5tv\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:25:14.827215 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.827172    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5thzb\" (UniqueName: \"kubernetes.io/projected/d9e5487a-dfe5-4455-9d48-33311d332203-kube-api-access-5thzb\") pod \"test-curl-7204-139899644834848\" (UID: \"d9e5487a-dfe5-4455-9d48-33311d332203\") " pod="openshift-ingress/test-curl-7204-139899644834848"
Jul 13 17:25:14.928762 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.928729    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5thzb\" (UniqueName: \"kubernetes.io/projected/d9e5487a-dfe5-4455-9d48-33311d332203-kube-api-access-5thzb\") pod \"test-curl-7204-139899644834848\" (UID: \"d9e5487a-dfe5-4455-9d48-33311d332203\") " pod="openshift-ingress/test-curl-7204-139899644834848"
Jul 13 17:25:14.938499 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:14.938468    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5thzb\" (UniqueName: \"kubernetes.io/projected/d9e5487a-dfe5-4455-9d48-33311d332203-kube-api-access-5thzb\") pod \"test-curl-7204-139899644834848\" (UID: \"d9e5487a-dfe5-4455-9d48-33311d332203\") " pod="openshift-ingress/test-curl-7204-139899644834848"
Jul 13 17:25:15.012741 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:15.012710    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7204-139899644834848"
Jul 13 17:25:15.143973 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:15.143925    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7204-139899644834848"]
Jul 13 17:25:15.146271 ip-10-0-128-173 kubenswrapper[2566]: W0713 17:25:15.146222    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd9e5487a_dfe5_4455_9d48_33311d332203.slice/crio-045dc2a2ad910770435e7d49e7177a0a7fa64124000fcff3b80b49b18d023d7a WatchSource:0}: Error finding container 045dc2a2ad910770435e7d49e7177a0a7fa64124000fcff3b80b49b18d023d7a: Status 404 returned error can't find the container with id 045dc2a2ad910770435e7d49e7177a0a7fa64124000fcff3b80b49b18d023d7a
Jul 13 17:25:15.379805 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:15.379775    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7204-139899644834848" event={"ID":"d9e5487a-dfe5-4455-9d48-33311d332203","Type":"ContainerStarted","Data":"045dc2a2ad910770435e7d49e7177a0a7fa64124000fcff3b80b49b18d023d7a"}
Jul 13 17:25:15.380976 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:15.380955    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3492fb15fbbbc9061dfcdc4ec142cd99e5e38101cddc1896c2fecf46da8d0f9d"
Jul 13 17:25:15.381073 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:15.380977    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7204-139899644834848"
Jul 13 17:25:15.394552 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:15.394526    2566 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="openshift-ingress/test-curl-7204-139899644834848" oldPodUID="0c33e4b8-ab75-4dc7-b395-e862606ea55e" podUID="d9e5487a-dfe5-4455-9d48-33311d332203"
Jul 13 17:25:15.980204 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:15.980171    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c33e4b8-ab75-4dc7-b395-e862606ea55e" path="/var/lib/kubelet/pods/0c33e4b8-ab75-4dc7-b395-e862606ea55e/volumes"
Jul 13 17:25:15.980608 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:15.980594    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77dde529-3c0e-4684-a39f-bd4cb465270c" path="/var/lib/kubelet/pods/77dde529-3c0e-4684-a39f-bd4cb465270c/volumes"
Jul 13 17:25:16.388440 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:16.388353    2566 generic.go:356] "Generic (PLEG): container finished" podID="d9e5487a-dfe5-4455-9d48-33311d332203" containerID="e94dc8ba9458df8eb4b7feb91523b3d97539b905d9aa1becce93bc5bef0eaa75" exitCode=0
Jul 13 17:25:16.388587 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:16.388445    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7204-139899644834848" event={"ID":"d9e5487a-dfe5-4455-9d48-33311d332203","Type":"ContainerDied","Data":"e94dc8ba9458df8eb4b7feb91523b3d97539b905d9aa1becce93bc5bef0eaa75"}
Jul 13 17:25:17.548539 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:17.548512    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7204-139899644834848"
Jul 13 17:25:17.552242 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:17.552224    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5thzb\" (UniqueName: \"kubernetes.io/projected/d9e5487a-dfe5-4455-9d48-33311d332203-kube-api-access-5thzb\") pod \"d9e5487a-dfe5-4455-9d48-33311d332203\" (UID: \"d9e5487a-dfe5-4455-9d48-33311d332203\") "
Jul 13 17:25:17.554183 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:17.554160    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9e5487a-dfe5-4455-9d48-33311d332203-kube-api-access-5thzb" (OuterVolumeSpecName: "kube-api-access-5thzb") pod "d9e5487a-dfe5-4455-9d48-33311d332203" (UID: "d9e5487a-dfe5-4455-9d48-33311d332203"). InnerVolumeSpecName "kube-api-access-5thzb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 17:25:17.570387 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:17.570365    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7204-139899644834848_d9e5487a-dfe5-4455-9d48-33311d332203/test-curl-7204-139899644834848/0.log"
Jul 13 17:25:17.599912 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:17.599878    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7204-139899644834848"]
Jul 13 17:25:17.607587 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:17.607561    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7204-139899644834848"]
Jul 13 17:25:17.653331 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:17.653236    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5thzb\" (UniqueName: \"kubernetes.io/projected/d9e5487a-dfe5-4455-9d48-33311d332203-kube-api-access-5thzb\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:25:17.793736 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:17.793697    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-aac012-qwqtl"]
Jul 13 17:25:17.794317 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:17.794300    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9e5487a-dfe5-4455-9d48-33311d332203" containerName="test-curl-7204-139899644834848"
Jul 13 17:25:17.794373 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:17.794320    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9e5487a-dfe5-4455-9d48-33311d332203" containerName="test-curl-7204-139899644834848"
Jul 13 17:25:17.794456 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:17.794445    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9e5487a-dfe5-4455-9d48-33311d332203" containerName="test-curl-7204-139899644834848"
Jul 13 17:25:17.797941 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:17.797923    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-aac012-qwqtl"
Jul 13 17:25:17.801279 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:17.801227    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fkks4\""
Jul 13 17:25:17.801393 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:17.801352    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 17:25:17.804564 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:17.804541    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-aac012-qwqtl"]
Jul 13 17:25:17.855615 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:17.855584    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26b8ad23-69dd-44f5-be9b-facf3c9711a3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-aac012-qwqtl\" (UID: \"26b8ad23-69dd-44f5-be9b-facf3c9711a3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-aac012-qwqtl"
Jul 13 17:25:17.956930 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:17.956890    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26b8ad23-69dd-44f5-be9b-facf3c9711a3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-aac012-qwqtl\" (UID: \"26b8ad23-69dd-44f5-be9b-facf3c9711a3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-aac012-qwqtl"
Jul 13 17:25:17.957535 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:17.957517    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26b8ad23-69dd-44f5-be9b-facf3c9711a3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-aac012-qwqtl\" (UID: \"26b8ad23-69dd-44f5-be9b-facf3c9711a3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-aac012-qwqtl"
Jul 13 17:25:17.980895 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:17.980865    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9e5487a-dfe5-4455-9d48-33311d332203" path="/var/lib/kubelet/pods/d9e5487a-dfe5-4455-9d48-33311d332203/volumes"
Jul 13 17:25:18.111154 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:18.111121    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-aac012-qwqtl"
Jul 13 17:25:18.241853 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:18.241830    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-aac012-qwqtl"]
Jul 13 17:25:18.243559 ip-10-0-128-173 kubenswrapper[2566]: W0713 17:25:18.243520    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26b8ad23_69dd_44f5_be9b_facf3c9711a3.slice/crio-7749b0dd2f7330df14d51dab12da39ea79ccd3cb283857ce30f5683017629855 WatchSource:0}: Error finding container 7749b0dd2f7330df14d51dab12da39ea79ccd3cb283857ce30f5683017629855: Status 404 returned error can't find the container with id 7749b0dd2f7330df14d51dab12da39ea79ccd3cb283857ce30f5683017629855
Jul 13 17:25:18.400152 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:18.400123    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7204-139899644834848"
Jul 13 17:25:18.400367 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:18.400130    2566 scope.go:117] "RemoveContainer" containerID="e94dc8ba9458df8eb4b7feb91523b3d97539b905d9aa1becce93bc5bef0eaa75"
Jul 13 17:25:18.401696 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:18.401668    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-aac012-qwqtl" event={"ID":"26b8ad23-69dd-44f5-be9b-facf3c9711a3","Type":"ContainerStarted","Data":"7749b0dd2f7330df14d51dab12da39ea79ccd3cb283857ce30f5683017629855"}
Jul 13 17:25:19.415891 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:19.415861    2566 generic.go:356] "Generic (PLEG): container finished" podID="26b8ad23-69dd-44f5-be9b-facf3c9711a3" containerID="c8d332a3032a610b08ca626d75f9d761404907a122ad2d324cdb0596afc08c7d" exitCode=0
Jul 13 17:25:19.416293 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:19.415946    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-aac012-qwqtl" event={"ID":"26b8ad23-69dd-44f5-be9b-facf3c9711a3","Type":"ContainerDied","Data":"c8d332a3032a610b08ca626d75f9d761404907a122ad2d324cdb0596afc08c7d"}
Jul 13 17:25:20.553612 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:20.553589    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-aac012-qwqtl"
Jul 13 17:25:20.581106 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:20.581076    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26b8ad23-69dd-44f5-be9b-facf3c9711a3-maas-api-service-ca\") pod \"26b8ad23-69dd-44f5-be9b-facf3c9711a3\" (UID: \"26b8ad23-69dd-44f5-be9b-facf3c9711a3\") "
Jul 13 17:25:20.581524 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:20.581495    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26b8ad23-69dd-44f5-be9b-facf3c9711a3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "26b8ad23-69dd-44f5-be9b-facf3c9711a3" (UID: "26b8ad23-69dd-44f5-be9b-facf3c9711a3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 17:25:20.682704 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:20.682625    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26b8ad23-69dd-44f5-be9b-facf3c9711a3-maas-api-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:25:21.427688 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:21.427659    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-aac012-qwqtl"
Jul 13 17:25:21.427879 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:21.427707    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-aac012-qwqtl" event={"ID":"26b8ad23-69dd-44f5-be9b-facf3c9711a3","Type":"ContainerDied","Data":"7749b0dd2f7330df14d51dab12da39ea79ccd3cb283857ce30f5683017629855"}
Jul 13 17:25:21.427879 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:21.427753    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7749b0dd2f7330df14d51dab12da39ea79ccd3cb283857ce30f5683017629855"
Jul 13 17:25:32.096929 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:32.096896    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:25:43.994052 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:43.994015    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bd468d-7jwf5"]
Jul 13 17:25:43.994465 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:43.994453    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26b8ad23-69dd-44f5-be9b-facf3c9711a3" containerName="revoke-keys"
Jul 13 17:25:43.994530 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:43.994468    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="26b8ad23-69dd-44f5-be9b-facf3c9711a3" containerName="revoke-keys"
Jul 13 17:25:43.994593 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:43.994535    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="26b8ad23-69dd-44f5-be9b-facf3c9711a3" containerName="revoke-keys"
Jul 13 17:25:43.999788 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:43.999770    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bd468d-7jwf5"
Jul 13 17:25:44.003364 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:44.003342    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 17:25:44.003424 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:44.003402    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fkks4\""
Jul 13 17:25:44.009355 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:44.009327    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bd468d-7jwf5"]
Jul 13 17:25:44.103045 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:44.103007    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a766ce5d-29c8-43b1-808d-cab34a63e375-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-bd468d-7jwf5\" (UID: \"a766ce5d-29c8-43b1-808d-cab34a63e375\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bd468d-7jwf5"
Jul 13 17:25:44.204749 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:44.204707    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a766ce5d-29c8-43b1-808d-cab34a63e375-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-bd468d-7jwf5\" (UID: \"a766ce5d-29c8-43b1-808d-cab34a63e375\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bd468d-7jwf5"
Jul 13 17:25:44.205424 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:44.205404    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a766ce5d-29c8-43b1-808d-cab34a63e375-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-bd468d-7jwf5\" (UID: \"a766ce5d-29c8-43b1-808d-cab34a63e375\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bd468d-7jwf5"
Jul 13 17:25:44.311220 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:44.311126    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bd468d-7jwf5"
Jul 13 17:25:44.447771 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:44.447740    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bd468d-7jwf5"]
Jul 13 17:25:44.449188 ip-10-0-128-173 kubenswrapper[2566]: W0713 17:25:44.449155    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda766ce5d_29c8_43b1_808d_cab34a63e375.slice/crio-626b6e183c80fe500e57246ab9baf8038937c322814d29f23e225520df2b640a WatchSource:0}: Error finding container 626b6e183c80fe500e57246ab9baf8038937c322814d29f23e225520df2b640a: Status 404 returned error can't find the container with id 626b6e183c80fe500e57246ab9baf8038937c322814d29f23e225520df2b640a
Jul 13 17:25:44.551301 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:44.551228    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bd468d-7jwf5" event={"ID":"a766ce5d-29c8-43b1-808d-cab34a63e375","Type":"ContainerStarted","Data":"626b6e183c80fe500e57246ab9baf8038937c322814d29f23e225520df2b640a"}
Jul 13 17:25:45.558897 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:45.558800    2566 generic.go:356] "Generic (PLEG): container finished" podID="a766ce5d-29c8-43b1-808d-cab34a63e375" containerID="69e8a644955edea5d85fe95a744dc489744dea041aeb4d6a4c563dd3a1bf23b4" exitCode=0
Jul 13 17:25:45.558897 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:45.558884    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bd468d-7jwf5" event={"ID":"a766ce5d-29c8-43b1-808d-cab34a63e375","Type":"ContainerDied","Data":"69e8a644955edea5d85fe95a744dc489744dea041aeb4d6a4c563dd3a1bf23b4"}
Jul 13 17:25:46.700945 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:46.700916    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bd468d-7jwf5"
Jul 13 17:25:46.829327 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:46.829223    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a766ce5d-29c8-43b1-808d-cab34a63e375-maas-api-service-ca\") pod \"a766ce5d-29c8-43b1-808d-cab34a63e375\" (UID: \"a766ce5d-29c8-43b1-808d-cab34a63e375\") "
Jul 13 17:25:46.829661 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:46.829631    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a766ce5d-29c8-43b1-808d-cab34a63e375-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a766ce5d-29c8-43b1-808d-cab34a63e375" (UID: "a766ce5d-29c8-43b1-808d-cab34a63e375"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 17:25:46.930393 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:46.930355    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a766ce5d-29c8-43b1-808d-cab34a63e375-maas-api-service-ca\") on node \"ip-10-0-128-173.ec2.internal\" DevicePath \"\""
Jul 13 17:25:47.571131 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:47.571094    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bd468d-7jwf5"
Jul 13 17:25:47.571359 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:47.571133    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bd468d-7jwf5" event={"ID":"a766ce5d-29c8-43b1-808d-cab34a63e375","Type":"ContainerDied","Data":"626b6e183c80fe500e57246ab9baf8038937c322814d29f23e225520df2b640a"}
Jul 13 17:25:47.571359 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:47.571193    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="626b6e183c80fe500e57246ab9baf8038937c322814d29f23e225520df2b640a"
Jul 13 17:25:57.494315 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:25:57.494284    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-pfdvl"]
Jul 13 17:26:14.225805 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:14.225775    2566 scope.go:117] "RemoveContainer" containerID="713dc1a5917ead27d65f8d449ba3fd3ad9bb5fea8c66cf5fd057ab707930fbde"
Jul 13 17:26:16.134448 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:16.134417    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5857456dfc-fdgtp_e33cc5db-422b-42db-805c-cb9a18300419/manager/0.log"
Jul 13 17:26:16.142767 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:16.142743    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5857456dfc-fdgtp_e33cc5db-422b-42db-805c-cb9a18300419/copy-manifests/0.log"
Jul 13 17:26:17.161591 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:17.161558    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-97e65980-bx5n8_fd6e5402-33e7-45d0-847f-231117b037cc/revoke-keys/1.log"
Jul 13 17:26:17.279346 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:17.279317    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-a5ed8a38-kmcdm_d1caba73-cc08-4f09-b6ff-0a345bddf06d/revoke-keys/1.log"
Jul 13 17:26:17.390778 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:17.390753    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-2386aa-qjvl5_3713018d-33c0-4c5d-918f-17559dba1070/revoke-keys/0.log"
Jul 13 17:26:17.512358 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:17.512329    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-b-9c55b0-jfgv6_45676e36-19f3-4464-ba66-18fde5b698c9/revoke-keys/0.log"
Jul 13 17:26:17.620569 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:17.620537    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-derive-1890c174-rsbdc_74d76adb-3cb6-42b0-9882-b2268517180d/revoke-keys/0.log"
Jul 13 17:26:17.850001 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:17.849925    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-0c5523-fxslf_6aefacc3-14fa-459c-adb3-cf5f79dded3e/revoke-keys/0.log"
Jul 13 17:26:17.962651 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:17.962617    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-migrate-51bf4736-znh2n_68afc24e-ba04-48e4-b379-83e8e7b49498/revoke-keys/0.log"
Jul 13 17:26:18.078314 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:18.078278    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-45a43c1d-6wkg5_76ef5af9-5cb6-410b-ba12-7f0f58362ab0/revoke-keys/0.log"
Jul 13 17:26:18.202754 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:18.202723    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-54358301-x6hrn_e716680e-2c26-48af-93ee-de99c913f649/revoke-keys/0.log"
Jul 13 17:26:18.321375 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:18.321339    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-a-aac012-qwqtl_26b8ad23-69dd-44f5-be9b-facf3c9711a3/revoke-keys/0.log"
Jul 13 17:26:18.435411 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:18.435381    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-bd468d-7jwf5_a766ce5d-29c8-43b1-808d-cab34a63e375/revoke-keys/0.log"
Jul 13 17:26:20.250241 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:20.250210    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-gvm4k_2e80eca0-f8d1-4e84-8298-c5c501729a23/kuadrant-console-plugin/0.log"
Jul 13 17:26:20.638600 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:20.638522    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-pfdvl_814ec693-bc16-4ee1-a9a0-787b82cd9f6a/limitador/0.log"
Jul 13 17:26:21.374529 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:21.374500    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8b9cf5f88-7jl4m_96b28493-8066-4d87-a7d7-fd7667387c48/kube-auth-proxy/0.log"
Jul 13 17:26:21.737775 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:21.737733    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-jpzhd_2430d611-a99a-4205-b431-1565dc09982f/payload-pre-processing/0.log"
Jul 13 17:26:22.431888 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:22.431861    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-zzv62_6d596154-f0e8-4473-b049-790311b438a8/main/0.log"
Jul 13 17:26:22.775395 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:22.775322    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-2mkqj_24a2fe54-5509-4c12-a86e-d5e77351f94b/main/0.log"
Jul 13 17:26:27.263848 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:27.263819    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mqxjc/must-gather-22wmx"]
Jul 13 17:26:27.264318 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:27.264302    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a766ce5d-29c8-43b1-808d-cab34a63e375" containerName="revoke-keys"
Jul 13 17:26:27.264373 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:27.264320    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a766ce5d-29c8-43b1-808d-cab34a63e375" containerName="revoke-keys"
Jul 13 17:26:27.264408 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:27.264385    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a766ce5d-29c8-43b1-808d-cab34a63e375" containerName="revoke-keys"
Jul 13 17:26:27.267635 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:27.267615    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mqxjc/must-gather-22wmx"
Jul 13 17:26:27.270740 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:27.270723    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mqxjc\"/\"kube-root-ca.crt\""
Jul 13 17:26:27.270807 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:27.270772    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-mqxjc\"/\"default-dockercfg-8zq4g\""
Jul 13 17:26:27.271022 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:27.271009    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mqxjc\"/\"openshift-service-ca.crt\""
Jul 13 17:26:27.281569 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:27.281547    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mqxjc/must-gather-22wmx"]
Jul 13 17:26:27.408136 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:27.408098    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmnvz\" (UniqueName: \"kubernetes.io/projected/b944e99d-918a-4f18-af11-d5949d04ef39-kube-api-access-jmnvz\") pod \"must-gather-22wmx\" (UID: \"b944e99d-918a-4f18-af11-d5949d04ef39\") " pod="openshift-must-gather-mqxjc/must-gather-22wmx"
Jul 13 17:26:27.408324 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:27.408216    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b944e99d-918a-4f18-af11-d5949d04ef39-must-gather-output\") pod \"must-gather-22wmx\" (UID: \"b944e99d-918a-4f18-af11-d5949d04ef39\") " pod="openshift-must-gather-mqxjc/must-gather-22wmx"
Jul 13 17:26:27.509276 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:27.509224    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b944e99d-918a-4f18-af11-d5949d04ef39-must-gather-output\") pod \"must-gather-22wmx\" (UID: \"b944e99d-918a-4f18-af11-d5949d04ef39\") " pod="openshift-must-gather-mqxjc/must-gather-22wmx"
Jul 13 17:26:27.509463 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:27.509318    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmnvz\" (UniqueName: \"kubernetes.io/projected/b944e99d-918a-4f18-af11-d5949d04ef39-kube-api-access-jmnvz\") pod \"must-gather-22wmx\" (UID: \"b944e99d-918a-4f18-af11-d5949d04ef39\") " pod="openshift-must-gather-mqxjc/must-gather-22wmx"
Jul 13 17:26:27.509544 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:27.509526    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b944e99d-918a-4f18-af11-d5949d04ef39-must-gather-output\") pod \"must-gather-22wmx\" (UID: \"b944e99d-918a-4f18-af11-d5949d04ef39\") " pod="openshift-must-gather-mqxjc/must-gather-22wmx"
Jul 13 17:26:27.521467 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:27.521414    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmnvz\" (UniqueName: \"kubernetes.io/projected/b944e99d-918a-4f18-af11-d5949d04ef39-kube-api-access-jmnvz\") pod \"must-gather-22wmx\" (UID: \"b944e99d-918a-4f18-af11-d5949d04ef39\") " pod="openshift-must-gather-mqxjc/must-gather-22wmx"
Jul 13 17:26:27.576769 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:27.576744    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mqxjc/must-gather-22wmx"
Jul 13 17:26:27.710701 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:27.707666    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mqxjc/must-gather-22wmx"]
Jul 13 17:26:27.789592 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:27.789503    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mqxjc/must-gather-22wmx" event={"ID":"b944e99d-918a-4f18-af11-d5949d04ef39","Type":"ContainerStarted","Data":"72e9121a59a7c47c378408fae4b2029c9454dd48fcd3fb7cac64c5e2644f55af"}
Jul 13 17:26:28.798501 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:28.798400    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mqxjc/must-gather-22wmx" event={"ID":"b944e99d-918a-4f18-af11-d5949d04ef39","Type":"ContainerStarted","Data":"085e8f0da13ea2450001ffbb6300077175b28f84fa95157792f34e02d280badc"}
Jul 13 17:26:28.798501 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:28.798447    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mqxjc/must-gather-22wmx" event={"ID":"b944e99d-918a-4f18-af11-d5949d04ef39","Type":"ContainerStarted","Data":"a2f94c2a0a58e9d9c6f36a91bfe265944e0a058aee04beec70114d84669b4a0a"}
Jul 13 17:26:28.821986 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:28.821921    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-mqxjc/must-gather-22wmx" podStartSLOduration=1.032221183 podStartE2EDuration="1.821899604s" totalImagesPullingTime="789.678421ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 17:26:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:26:28.817073191 +0000 UTC m=+2775.358453623" watchObservedRunningTime="2026-07-13 17:26:28.821899604 +0000 UTC m=+2775.363280034"
Jul 13 17:26:30.038800 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:30.038762    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-cx6bw_5cdf4449-5e1c-453d-b7f0-ae71276fd373/global-pull-secret-syncer/0.log"
Jul 13 17:26:30.206892 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:30.206850    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-lrq6k_de1a001e-9d1f-4b04-ad7c-c186a513e940/konnectivity-agent/0.log"
Jul 13 17:26:30.261435 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:30.261400    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-173.ec2.internal_f5c60a2d059f0ab2782f7f7ac54db0b9/haproxy/0.log"
Jul 13 17:26:35.490449 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:35.490423    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-gvm4k_2e80eca0-f8d1-4e84-8298-c5c501729a23/kuadrant-console-plugin/0.log"
Jul 13 17:26:35.710837 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:35.710810    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-pfdvl_814ec693-bc16-4ee1-a9a0-787b82cd9f6a/limitador/0.log"
Jul 13 17:26:37.674825 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:37.674797    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6a578cc2-bbf1-4f21-86fa-980c61491d7b/alertmanager/0.log"
Jul 13 17:26:37.701818 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:37.701785    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6a578cc2-bbf1-4f21-86fa-980c61491d7b/config-reloader/0.log"
Jul 13 17:26:37.732660 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:37.732625    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6a578cc2-bbf1-4f21-86fa-980c61491d7b/kube-rbac-proxy-web/0.log"
Jul 13 17:26:37.761723 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:37.761675    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6a578cc2-bbf1-4f21-86fa-980c61491d7b/kube-rbac-proxy/0.log"
Jul 13 17:26:37.788178 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:37.788124    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6a578cc2-bbf1-4f21-86fa-980c61491d7b/kube-rbac-proxy-metric/0.log"
Jul 13 17:26:37.818269 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:37.818202    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6a578cc2-bbf1-4f21-86fa-980c61491d7b/prom-label-proxy/0.log"
Jul 13 17:26:37.855270 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:37.855007    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6a578cc2-bbf1-4f21-86fa-980c61491d7b/init-config-reloader/0.log"
Jul 13 17:26:37.977980 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:37.977895    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-m6wbp_d757b55d-630e-4ecb-8380-1925f6da02d5/kube-state-metrics/0.log"
Jul 13 17:26:38.010669 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:38.010634    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-m6wbp_d757b55d-630e-4ecb-8380-1925f6da02d5/kube-rbac-proxy-main/0.log"
Jul 13 17:26:38.044623 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:38.044580    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-m6wbp_d757b55d-630e-4ecb-8380-1925f6da02d5/kube-rbac-proxy-self/0.log"
Jul 13 17:26:38.084543 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:38.084513    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5b48f74765-gbsc7_c86f5354-b9d8-4f7e-b02c-6603f3b217d1/metrics-server/0.log"
Jul 13 17:26:38.125766 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:38.125733    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-frs55_1dbd2873-477a-48e1-aff4-f5ad6342fd9b/monitoring-plugin/0.log"
Jul 13 17:26:38.260971 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:38.260884    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5m2pj_4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899/node-exporter/0.log"
Jul 13 17:26:38.287002 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:38.286962    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5m2pj_4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899/kube-rbac-proxy/0.log"
Jul 13 17:26:38.314309 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:38.314275    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5m2pj_4249a8a0-d7f7-4ac1-8653-8f8dc7ceb899/init-textfile/0.log"
Jul 13 17:26:38.438703 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:38.438670    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-mmkvq_79f7dffb-624b-4874-be76-01f1a6d59a2d/kube-rbac-proxy-main/0.log"
Jul 13 17:26:38.464702 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:38.464665    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-mmkvq_79f7dffb-624b-4874-be76-01f1a6d59a2d/kube-rbac-proxy-self/0.log"
Jul 13 17:26:38.492646 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:38.492607    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-mmkvq_79f7dffb-624b-4874-be76-01f1a6d59a2d/openshift-state-metrics/0.log"
Jul 13 17:26:38.541655 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:38.541558    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_47e1ddb3-5111-448e-a9fd-93903f35e886/prometheus/0.log"
Jul 13 17:26:38.567093 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:38.567059    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_47e1ddb3-5111-448e-a9fd-93903f35e886/config-reloader/0.log"
Jul 13 17:26:38.596166 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:38.596133    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_47e1ddb3-5111-448e-a9fd-93903f35e886/thanos-sidecar/0.log"
Jul 13 17:26:38.624498 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:38.624467    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_47e1ddb3-5111-448e-a9fd-93903f35e886/kube-rbac-proxy-web/0.log"
Jul 13 17:26:38.650752 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:38.650723    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_47e1ddb3-5111-448e-a9fd-93903f35e886/kube-rbac-proxy/0.log"
Jul 13 17:26:38.677517 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:38.677480    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_47e1ddb3-5111-448e-a9fd-93903f35e886/kube-rbac-proxy-thanos/0.log"
Jul 13 17:26:38.706360 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:38.706237    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_47e1ddb3-5111-448e-a9fd-93903f35e886/init-config-reloader/0.log"
Jul 13 17:26:38.750544 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:38.750514    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-zw42s_9d54ffd4-e8a2-4745-bfc5-bdcb7abd45b2/prometheus-operator/0.log"
Jul 13 17:26:38.775649 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:38.775621    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-zw42s_9d54ffd4-e8a2-4745-bfc5-bdcb7abd45b2/kube-rbac-proxy/0.log"
Jul 13 17:26:38.804582 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:38.804552    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-gnnw7_a38eb5f0-9cea-4996-891d-a86a96d4d88c/prometheus-operator-admission-webhook/0.log"
Jul 13 17:26:38.847015 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:38.846980    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5b75c7698b-445mg_a68dd61e-fe5e-46c6-9a48-745740af28b6/telemeter-client/0.log"
Jul 13 17:26:38.874318 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:38.874288    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5b75c7698b-445mg_a68dd61e-fe5e-46c6-9a48-745740af28b6/reload/0.log"
Jul 13 17:26:38.899490 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:38.899425    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5b75c7698b-445mg_a68dd61e-fe5e-46c6-9a48-745740af28b6/kube-rbac-proxy/0.log"
Jul 13 17:26:38.938152 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:38.938115    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-686f4d8bc6-d6fv8_c4959268-4527-4c5f-aae4-6d47b89c3410/thanos-query/0.log"
Jul 13 17:26:38.960792 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:38.960757    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-686f4d8bc6-d6fv8_c4959268-4527-4c5f-aae4-6d47b89c3410/kube-rbac-proxy-web/0.log"
Jul 13 17:26:38.988096 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:38.988064    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-686f4d8bc6-d6fv8_c4959268-4527-4c5f-aae4-6d47b89c3410/kube-rbac-proxy/0.log"
Jul 13 17:26:39.013838 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:39.013801    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-686f4d8bc6-d6fv8_c4959268-4527-4c5f-aae4-6d47b89c3410/prom-label-proxy/0.log"
Jul 13 17:26:39.038895 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:39.038822    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-686f4d8bc6-d6fv8_c4959268-4527-4c5f-aae4-6d47b89c3410/kube-rbac-proxy-rules/0.log"
Jul 13 17:26:39.064903 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:39.064795    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-686f4d8bc6-d6fv8_c4959268-4527-4c5f-aae4-6d47b89c3410/kube-rbac-proxy-metrics/0.log"
Jul 13 17:26:40.560198 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:40.560162    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mqxjc/perf-node-gather-daemonset-2qpqv"]
Jul 13 17:26:40.567096 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:40.567070    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2qpqv"
Jul 13 17:26:40.575377 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:40.575347    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mqxjc/perf-node-gather-daemonset-2qpqv"]
Jul 13 17:26:40.664698 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:40.664659    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mngn9\" (UniqueName: \"kubernetes.io/projected/2e6003e9-eef7-45f8-a005-d49d4d605bd8-kube-api-access-mngn9\") pod \"perf-node-gather-daemonset-2qpqv\" (UID: \"2e6003e9-eef7-45f8-a005-d49d4d605bd8\") " pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2qpqv"
Jul 13 17:26:40.664886 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:40.664718    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2e6003e9-eef7-45f8-a005-d49d4d605bd8-podres\") pod \"perf-node-gather-daemonset-2qpqv\" (UID: \"2e6003e9-eef7-45f8-a005-d49d4d605bd8\") " pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2qpqv"
Jul 13 17:26:40.664886 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:40.664810    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2e6003e9-eef7-45f8-a005-d49d4d605bd8-lib-modules\") pod \"perf-node-gather-daemonset-2qpqv\" (UID: \"2e6003e9-eef7-45f8-a005-d49d4d605bd8\") " pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2qpqv"
Jul 13 17:26:40.664886 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:40.664847    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2e6003e9-eef7-45f8-a005-d49d4d605bd8-sys\") pod \"perf-node-gather-daemonset-2qpqv\" (UID: \"2e6003e9-eef7-45f8-a005-d49d4d605bd8\") " pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2qpqv"
Jul 13 17:26:40.665069 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:40.664891    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2e6003e9-eef7-45f8-a005-d49d4d605bd8-proc\") pod \"perf-node-gather-daemonset-2qpqv\" (UID: \"2e6003e9-eef7-45f8-a005-d49d4d605bd8\") " pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2qpqv"
Jul 13 17:26:40.765720 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:40.765684    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2e6003e9-eef7-45f8-a005-d49d4d605bd8-proc\") pod \"perf-node-gather-daemonset-2qpqv\" (UID: \"2e6003e9-eef7-45f8-a005-d49d4d605bd8\") " pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2qpqv"
Jul 13 17:26:40.766012 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:40.765996    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mngn9\" (UniqueName: \"kubernetes.io/projected/2e6003e9-eef7-45f8-a005-d49d4d605bd8-kube-api-access-mngn9\") pod \"perf-node-gather-daemonset-2qpqv\" (UID: \"2e6003e9-eef7-45f8-a005-d49d4d605bd8\") " pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2qpqv"
Jul 13 17:26:40.766543 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:40.766521    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2e6003e9-eef7-45f8-a005-d49d4d605bd8-podres\") pod \"perf-node-gather-daemonset-2qpqv\" (UID: \"2e6003e9-eef7-45f8-a005-d49d4d605bd8\") " pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2qpqv"
Jul 13 17:26:40.766917 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:40.766899    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2e6003e9-eef7-45f8-a005-d49d4d605bd8-lib-modules\") pod \"perf-node-gather-daemonset-2qpqv\" (UID: \"2e6003e9-eef7-45f8-a005-d49d4d605bd8\") " pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2qpqv"
Jul 13 17:26:40.767171 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:40.767158    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2e6003e9-eef7-45f8-a005-d49d4d605bd8-sys\") pod \"perf-node-gather-daemonset-2qpqv\" (UID: \"2e6003e9-eef7-45f8-a005-d49d4d605bd8\") " pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2qpqv"
Jul 13 17:26:40.767396 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:40.765821    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2e6003e9-eef7-45f8-a005-d49d4d605bd8-proc\") pod \"perf-node-gather-daemonset-2qpqv\" (UID: \"2e6003e9-eef7-45f8-a005-d49d4d605bd8\") " pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2qpqv"
Jul 13 17:26:40.767508 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:40.767103    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2e6003e9-eef7-45f8-a005-d49d4d605bd8-lib-modules\") pod \"perf-node-gather-daemonset-2qpqv\" (UID: \"2e6003e9-eef7-45f8-a005-d49d4d605bd8\") " pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2qpqv"
Jul 13 17:26:40.767600 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:40.766780    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2e6003e9-eef7-45f8-a005-d49d4d605bd8-podres\") pod \"perf-node-gather-daemonset-2qpqv\" (UID: \"2e6003e9-eef7-45f8-a005-d49d4d605bd8\") " pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2qpqv"
Jul 13 17:26:40.767684 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:40.767306    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2e6003e9-eef7-45f8-a005-d49d4d605bd8-sys\") pod \"perf-node-gather-daemonset-2qpqv\" (UID: \"2e6003e9-eef7-45f8-a005-d49d4d605bd8\") " pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2qpqv"
Jul 13 17:26:40.781454 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:40.781425    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mngn9\" (UniqueName: \"kubernetes.io/projected/2e6003e9-eef7-45f8-a005-d49d4d605bd8-kube-api-access-mngn9\") pod \"perf-node-gather-daemonset-2qpqv\" (UID: \"2e6003e9-eef7-45f8-a005-d49d4d605bd8\") " pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2qpqv"
Jul 13 17:26:40.884531 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:40.884444    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2qpqv"
Jul 13 17:26:41.093794 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:41.093697    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mqxjc/perf-node-gather-daemonset-2qpqv"]
Jul 13 17:26:41.098741 ip-10-0-128-173 kubenswrapper[2566]: W0713 17:26:41.098708    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2e6003e9_eef7_45f8_a005_d49d4d605bd8.slice/crio-73daa4143e9dc5faa567c440f17d90b12a3154a94e6bc2cffb5586c538996f29 WatchSource:0}: Error finding container 73daa4143e9dc5faa567c440f17d90b12a3154a94e6bc2cffb5586c538996f29: Status 404 returned error can't find the container with id 73daa4143e9dc5faa567c440f17d90b12a3154a94e6bc2cffb5586c538996f29
Jul 13 17:26:41.433207 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:41.433183    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6b7d9d866f-2n8rn_4de289db-da0f-47fa-ac8e-24268efe09c8/console/0.log"
Jul 13 17:26:41.903089 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:41.902995    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2qpqv" event={"ID":"2e6003e9-eef7-45f8-a005-d49d4d605bd8","Type":"ContainerStarted","Data":"8d2b6e3377db9415994a2bbe63f144c4ef27136b05cc1be6a64a6941c52e6135"}
Jul 13 17:26:41.903089 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:41.903062    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2qpqv" event={"ID":"2e6003e9-eef7-45f8-a005-d49d4d605bd8","Type":"ContainerStarted","Data":"73daa4143e9dc5faa567c440f17d90b12a3154a94e6bc2cffb5586c538996f29"}
Jul 13 17:26:41.903662 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:41.903408    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2qpqv"
Jul 13 17:26:41.922619 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:41.922552    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2qpqv" podStartSLOduration=1.922531247 podStartE2EDuration="1.922531247s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 17:26:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 17:26:41.919643452 +0000 UTC m=+2788.461023885" watchObservedRunningTime="2026-07-13 17:26:41.922531247 +0000 UTC m=+2788.463911677"
Jul 13 17:26:43.156802 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:43.156777    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pmlmx_0d2910d0-a6df-45ed-87ba-991eb6179dde/dns/0.log"
Jul 13 17:26:43.180417 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:43.180389    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pmlmx_0d2910d0-a6df-45ed-87ba-991eb6179dde/kube-rbac-proxy/0.log"
Jul 13 17:26:43.286331 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:43.286304    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-m6gh9_2fdaea84-a3f8-4b68-bf69-ed84f92fcdbb/dns-node-resolver/0.log"
Jul 13 17:26:43.990289 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:43.990263    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-j2pwp_559d8901-967f-4100-a1df-0600fd3d305a/node-ca/0.log"
Jul 13 17:26:45.256293 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:45.256232    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8b9cf5f88-7jl4m_96b28493-8066-4d87-a7d7-fd7667387c48/kube-auth-proxy/0.log"
Jul 13 17:26:45.415168 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:45.415141    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-jpzhd_2430d611-a99a-4205-b431-1565dc09982f/payload-pre-processing/0.log"
Jul 13 17:26:46.236688 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:46.236658    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kcg7j_62d5929c-f868-4354-b47a-4ae2b62e1362/serve-healthcheck-canary/0.log"
Jul 13 17:26:46.780149 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:46.780118    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-798sv_2f8a3a0f-9ff7-4a01-b0b2-c9edac2e964e/kube-rbac-proxy/0.log"
Jul 13 17:26:46.806443 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:46.806403    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-798sv_2f8a3a0f-9ff7-4a01-b0b2-c9edac2e964e/exporter/0.log"
Jul 13 17:26:46.830421 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:46.830394    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-798sv_2f8a3a0f-9ff7-4a01-b0b2-c9edac2e964e/extractor/0.log"
Jul 13 17:26:47.923471 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:47.923437    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-mqxjc/perf-node-gather-daemonset-2qpqv"
Jul 13 17:26:49.617045 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:49.616924    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5857456dfc-fdgtp_e33cc5db-422b-42db-805c-cb9a18300419/manager/0.log"
Jul 13 17:26:49.647111 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:49.647084    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5857456dfc-fdgtp_e33cc5db-422b-42db-805c-cb9a18300419/copy-manifests/0.log"
Jul 13 17:26:51.179711 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:51.179681    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8b9cf5f88-7jl4m_96b28493-8066-4d87-a7d7-fd7667387c48/kube-auth-proxy/0.log"
Jul 13 17:26:51.272180 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:51.272146    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-jpzhd_2430d611-a99a-4205-b431-1565dc09982f/payload-pre-processing/0.log"
Jul 13 17:26:51.335276 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:51.335217    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-78cc96c4cb-llcvt_9e74dc84-65ad-41c1-a938-66ceb207e631/manager/0.log"
Jul 13 17:26:52.720970 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:52.720933    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8b9cf5f88-7jl4m_96b28493-8066-4d87-a7d7-fd7667387c48/kube-auth-proxy/0.log"
Jul 13 17:26:52.823562 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:52.823483    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-jpzhd_2430d611-a99a-4205-b431-1565dc09982f/payload-pre-processing/0.log"
Jul 13 17:26:57.897652 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:57.897603    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-4wq9f_ebb4c244-b823-48ea-a747-09b0330ce4e7/kube-multus/0.log"
Jul 13 17:26:57.959056 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:57.959026    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6lnrp_650632b5-fb83-42dd-9274-4be5ff84ffe9/kube-multus-additional-cni-plugins/0.log"
Jul 13 17:26:57.985238 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:57.985205    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6lnrp_650632b5-fb83-42dd-9274-4be5ff84ffe9/egress-router-binary-copy/0.log"
Jul 13 17:26:58.010276 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:58.010222    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6lnrp_650632b5-fb83-42dd-9274-4be5ff84ffe9/cni-plugins/0.log"
Jul 13 17:26:58.035483 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:58.035456    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6lnrp_650632b5-fb83-42dd-9274-4be5ff84ffe9/bond-cni-plugin/0.log"
Jul 13 17:26:58.059125 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:58.059097    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6lnrp_650632b5-fb83-42dd-9274-4be5ff84ffe9/routeoverride-cni/0.log"
Jul 13 17:26:58.082949 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:58.082872    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6lnrp_650632b5-fb83-42dd-9274-4be5ff84ffe9/whereabouts-cni-bincopy/0.log"
Jul 13 17:26:58.107884 ip-10-0-128-173 kubenswrapper[2566]: I0713 17:26:58.107857    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6lnrp_650632b5-fb83-42dd-9274-4be5ff84ffe9/whereabouts-cni/0.log"