Jul 28 10:05:33.173665 ip-10-0-130-235 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 28 10:05:33.173681 ip-10-0-130-235 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 28 10:05:33.173709 ip-10-0-130-235 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 28 10:05:33.174072 ip-10-0-130-235 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 28 10:05:43.367222 ip-10-0-130-235 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 28 10:05:43.367241 ip-10-0-130-235 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 13a7e8d9f74b46f6ac755995dbc2fa16 --
Jul 28 10:07:36.434675 ip-10-0-130-235 systemd[1]: Starting Kubernetes Kubelet...
Jul 28 10:07:36.881482 ip-10-0-130-235 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 10:07:36.881482 ip-10-0-130-235 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 28 10:07:36.881482 ip-10-0-130-235 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 10:07:36.881482 ip-10-0-130-235 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 28 10:07:36.881482 ip-10-0-130-235 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 10:07:36.882721 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.882534    2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 28 10:07:36.889381 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889337    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 10:07:36.889381 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889378    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 10:07:36.889381 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889382    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 10:07:36.889381 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889386    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 10:07:36.889538 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889389    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 10:07:36.889538 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889393    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 10:07:36.889538 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889396    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 10:07:36.889538 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889399    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 10:07:36.889538 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889402    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 10:07:36.889538 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889404    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 10:07:36.889538 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889407    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 10:07:36.889538 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889412    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 10:07:36.889538 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889416    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 10:07:36.889538 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889419    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 10:07:36.889538 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889423    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 10:07:36.889538 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889436    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 10:07:36.889538 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889439    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 10:07:36.889538 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889442    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 10:07:36.889538 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889466    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 10:07:36.889538 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889469    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 10:07:36.889538 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889473    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 10:07:36.889538 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889477    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 10:07:36.889538 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889479    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 10:07:36.890106 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889483    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 10:07:36.890106 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889486    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 10:07:36.890106 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889489    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 10:07:36.890106 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889492    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 10:07:36.890106 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889494    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 10:07:36.890106 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889497    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 10:07:36.890106 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889501    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 10:07:36.890106 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889503    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 10:07:36.890106 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889506    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 10:07:36.890106 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889509    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 10:07:36.890106 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889512    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 10:07:36.890106 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889514    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 10:07:36.890106 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889517    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 10:07:36.890106 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889520    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 10:07:36.890106 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889523    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 10:07:36.890106 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889526    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 10:07:36.890106 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889529    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 10:07:36.890106 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889532    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 10:07:36.890106 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889534    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 10:07:36.890851 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889539    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 10:07:36.890851 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889544    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 10:07:36.890851 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889547    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 10:07:36.890851 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889551    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 10:07:36.890851 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889554    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 10:07:36.890851 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889557    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 10:07:36.890851 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889560    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 10:07:36.890851 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889564    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 10:07:36.890851 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889567    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 10:07:36.890851 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889571    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 10:07:36.890851 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889574    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 10:07:36.890851 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889577    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 10:07:36.890851 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889580    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 10:07:36.890851 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889582    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 10:07:36.890851 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889586    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 10:07:36.890851 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889588    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 10:07:36.890851 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889591    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 10:07:36.890851 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889594    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 10:07:36.890851 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889597    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 10:07:36.891350 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889599    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 10:07:36.891350 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889602    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 10:07:36.891350 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889605    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 10:07:36.891350 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889608    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 10:07:36.891350 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889610    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 10:07:36.891350 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889613    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 10:07:36.891350 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889616    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 10:07:36.891350 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889619    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 10:07:36.891350 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889622    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 10:07:36.891350 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889624    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 10:07:36.891350 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889627    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 10:07:36.891350 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889633    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 10:07:36.891350 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889635    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 10:07:36.891350 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889638    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 10:07:36.891350 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889640    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 10:07:36.891350 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889643    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 10:07:36.891350 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889646    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 10:07:36.891350 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889649    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 10:07:36.891350 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889652    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 10:07:36.891350 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889655    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 10:07:36.891350 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889657    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 10:07:36.892099 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889660    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 10:07:36.892099 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889663    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 10:07:36.892099 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889666    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 10:07:36.892099 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889668    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 10:07:36.892099 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889671    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 10:07:36.892099 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889673    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 10:07:36.892099 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889677    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 10:07:36.892099 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889679    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 10:07:36.892099 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889682    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 10:07:36.892099 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889684    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 10:07:36.892099 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889687    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 10:07:36.892099 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889689    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 10:07:36.892099 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889692    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 10:07:36.892099 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889695    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 10:07:36.892099 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889698    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 10:07:36.892099 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889700    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 10:07:36.892099 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.889704    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 10:07:36.892549 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892465    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 10:07:36.892549 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892486    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 10:07:36.892549 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892489    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 10:07:36.892549 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892492    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 10:07:36.892549 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892496    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 10:07:36.892549 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892499    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 10:07:36.892549 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892503    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 10:07:36.892549 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892505    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 10:07:36.892549 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892508    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 10:07:36.892549 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892511    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 10:07:36.892549 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892514    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 10:07:36.892549 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892516    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 10:07:36.892549 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892519    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 10:07:36.892549 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892521    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 10:07:36.892549 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892524    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 10:07:36.892549 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892526    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 10:07:36.892549 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892529    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 10:07:36.892549 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892531    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 10:07:36.892549 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892541    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 10:07:36.892549 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892544    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 10:07:36.893055 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892547    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 10:07:36.893055 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892550    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 10:07:36.893055 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892552    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 10:07:36.893055 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892556    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 10:07:36.893055 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892559    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 10:07:36.893055 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892562    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 10:07:36.893055 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892565    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 10:07:36.893055 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892567    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 10:07:36.893055 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892570    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 10:07:36.893055 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892573    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 10:07:36.893055 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892577    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 10:07:36.893055 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892581    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 10:07:36.893055 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892584    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 10:07:36.893055 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892586    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 10:07:36.893055 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892589    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 10:07:36.893055 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892593    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 10:07:36.893055 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892597    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 10:07:36.893055 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892600    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 10:07:36.893055 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892603    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 10:07:36.893676 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892606    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 10:07:36.893676 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892608    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 10:07:36.893676 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892611    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 10:07:36.893676 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892614    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 10:07:36.893676 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892616    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 10:07:36.893676 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892619    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 10:07:36.893676 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892622    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 10:07:36.893676 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892626    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 10:07:36.893676 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892628    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 10:07:36.893676 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892631    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 10:07:36.893676 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892634    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 10:07:36.893676 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892637    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 10:07:36.893676 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892640    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 10:07:36.893676 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892643    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 10:07:36.893676 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892646    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 10:07:36.893676 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892648    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 10:07:36.893676 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892651    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 10:07:36.893676 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892654    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 10:07:36.893676 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892657    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 10:07:36.894148 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892660    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 10:07:36.894148 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892662    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 10:07:36.894148 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892665    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 10:07:36.894148 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892668    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 10:07:36.894148 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892670    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 10:07:36.894148 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892673    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 10:07:36.894148 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892676    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 10:07:36.894148 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892678    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 10:07:36.894148 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892681    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 10:07:36.894148 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892684    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 10:07:36.894148 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892686    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 10:07:36.894148 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892689    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 10:07:36.894148 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892692    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 10:07:36.894148 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892694    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 10:07:36.894148 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892697    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 10:07:36.894148 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892700    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 10:07:36.894148 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892702    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 10:07:36.894148 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892705    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 10:07:36.894148 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892707    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 10:07:36.894148 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892710    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 10:07:36.894651 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892714    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 10:07:36.894651 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892717    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 10:07:36.894651 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892720    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 10:07:36.894651 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892723    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 10:07:36.894651 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892726    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 10:07:36.894651 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892729    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 10:07:36.894651 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892732    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 10:07:36.894651 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892735    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 10:07:36.894651 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892738    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 10:07:36.894651 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892741    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 10:07:36.894651 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892744    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 10:07:36.894651 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892747    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 10:07:36.894651 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892751    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 10:07:36.894651 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892754    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 10:07:36.894651 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892757    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 10:07:36.894651 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892760    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 10:07:36.894651 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892763    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 10:07:36.894651 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892765    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 10:07:36.894651 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892768    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 10:07:36.894651 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892771    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 10:07:36.895142 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.892774    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 10:07:36.895142 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893230    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 10:07:36.895142 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893236    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 10:07:36.895142 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893239    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 10:07:36.895142 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893242    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 10:07:36.895142 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893245    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 10:07:36.895142 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893248    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 10:07:36.895142 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893251    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 10:07:36.895142 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893254    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 10:07:36.895142 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893256    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 10:07:36.895142 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893259    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 10:07:36.895142 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893263    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 10:07:36.895142 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893266    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 10:07:36.895142 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893269    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 10:07:36.895142 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893273    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 10:07:36.895142 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893277    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 10:07:36.895142 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893280    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 10:07:36.895142 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893285    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 10:07:36.895142 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893287    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 10:07:36.895142 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893291    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 10:07:36.895681 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893294    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 10:07:36.895681 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893297    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 10:07:36.895681 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893300    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 10:07:36.895681 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893303    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 10:07:36.895681 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893306    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 10:07:36.895681 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893309    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 10:07:36.895681 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893311    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 10:07:36.895681 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893314    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 10:07:36.895681 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893317    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 10:07:36.895681 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893320    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 10:07:36.895681 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893322    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 10:07:36.895681 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893326    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 10:07:36.895681 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893328    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 10:07:36.895681 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893331    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 10:07:36.895681 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893334    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 10:07:36.895681 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893336    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 10:07:36.895681 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893339    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 10:07:36.895681 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893342    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 10:07:36.895681 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893344    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 10:07:36.895681 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893347    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 10:07:36.896166 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893350    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 10:07:36.896166 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893352    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 10:07:36.896166 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893374    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 10:07:36.896166 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893377    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 10:07:36.896166 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893380    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 10:07:36.896166 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893383    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 10:07:36.896166 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893386    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 10:07:36.896166 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893388    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 10:07:36.896166 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893391    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 10:07:36.896166 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893394    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 10:07:36.896166 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893398    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 10:07:36.896166 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893401    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 10:07:36.896166 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893404    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 10:07:36.896166 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893407    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 10:07:36.896166 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893410    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 10:07:36.896166 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893412    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 10:07:36.896166 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893415    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 10:07:36.896166 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893418    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 10:07:36.896166 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893420    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 10:07:36.896663 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893424    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 10:07:36.896663 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893428    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 10:07:36.896663 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893431    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 10:07:36.896663 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893433    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 10:07:36.896663 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893436    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 10:07:36.896663 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893439    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 10:07:36.896663 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893441    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 10:07:36.896663 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893444    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 10:07:36.896663 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893447    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 10:07:36.896663 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893450    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 10:07:36.896663 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893453    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 10:07:36.896663 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893456    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 10:07:36.896663 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893458    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 10:07:36.896663 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893461    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 10:07:36.896663 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893463    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 10:07:36.896663 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893466    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 10:07:36.896663 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893469    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 10:07:36.896663 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893471    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 10:07:36.896663 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893474    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 10:07:36.896663 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893477    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 10:07:36.897198 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893479    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 10:07:36.897198 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893482    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 10:07:36.897198 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893485    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 10:07:36.897198 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893487    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 10:07:36.897198 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893492    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 10:07:36.897198 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893495    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 10:07:36.897198 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893498    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 10:07:36.897198 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893501    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 10:07:36.897198 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893505    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 10:07:36.897198 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893508    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 10:07:36.897198 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893511    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 10:07:36.897198 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893513    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 10:07:36.897198 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893516    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 10:07:36.897198 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893519    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 10:07:36.897198 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893522    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 10:07:36.897198 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893525    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 10:07:36.897198 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893527    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 10:07:36.897198 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893530    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 10:07:36.897198 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893532    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 10:07:36.897198 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893535    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 10:07:36.897710 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.893537    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 10:07:36.897710 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893629    2563 flags.go:64] FLAG: --address="0.0.0.0"
Jul 28 10:07:36.897710 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893647    2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 28 10:07:36.897710 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893656    2563 flags.go:64] FLAG: --anonymous-auth="true"
Jul 28 10:07:36.897710 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893662    2563 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 28 10:07:36.897710 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893667    2563 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 28 10:07:36.897710 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893670    2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 28 10:07:36.897710 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893675    2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 28 10:07:36.897710 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893680    2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 28 10:07:36.897710 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893683    2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 28 10:07:36.897710 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893687    2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 28 10:07:36.897710 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893691    2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 28 10:07:36.897710 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893694    2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 28 10:07:36.897710 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893698    2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 28 10:07:36.897710 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893701    2563 flags.go:64] FLAG: --cgroup-root=""
Jul 28 10:07:36.897710 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893704    2563 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 28 10:07:36.897710 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893708    2563 flags.go:64] FLAG: --client-ca-file=""
Jul 28 10:07:36.897710 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893712    2563 flags.go:64] FLAG: --cloud-provider="external"
Jul 28 10:07:36.897710 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893715    2563 flags.go:64] FLAG: --cluster-dns="[]"
Jul 28 10:07:36.897710 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893720    2563 flags.go:64] FLAG: --cluster-domain=""
Jul 28 10:07:36.897710 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893724    2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 28 10:07:36.897710 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893727    2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 28 10:07:36.897710 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893730    2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 28 10:07:36.897710 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893734    2563 flags.go:64] FLAG: --container-log-max-files="5"
Jul 28 10:07:36.898313 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893738    2563 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 28 10:07:36.898313 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893741    2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 28 10:07:36.898313 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893744    2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 28 10:07:36.898313 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893748    2563 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 28 10:07:36.898313 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893751    2563 flags.go:64] FLAG: --contention-profiling="false"
Jul 28 10:07:36.898313 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893754    2563 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 28 10:07:36.898313 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893757    2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 28 10:07:36.898313 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893760    2563 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 28 10:07:36.898313 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893763    2563 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 28 10:07:36.898313 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893768    2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 28 10:07:36.898313 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893771    2563 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 28 10:07:36.898313 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893774    2563 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 28 10:07:36.898313 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893777    2563 flags.go:64] FLAG: --enable-load-reader="false"
Jul 28 10:07:36.898313 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893780    2563 flags.go:64] FLAG: --enable-server="true"
Jul 28 10:07:36.898313 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893783    2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 28 10:07:36.898313 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893788    2563 flags.go:64] FLAG: --event-burst="100"
Jul 28 10:07:36.898313 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893792    2563 flags.go:64] FLAG: --event-qps="50"
Jul 28 10:07:36.898313 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893794    2563 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 28 10:07:36.898313 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893798    2563 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 28 10:07:36.898313 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893801    2563 flags.go:64] FLAG: --eviction-hard=""
Jul 28 10:07:36.898313 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893805    2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 28 10:07:36.898313 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893808    2563 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 28 10:07:36.898313 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893811    2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 28 10:07:36.898313 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893814    2563 flags.go:64] FLAG: --eviction-soft=""
Jul 28 10:07:36.898313 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893817    2563 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 28 10:07:36.898938 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893820    2563 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 28 10:07:36.898938 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893824    2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 28 10:07:36.898938 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893827    2563 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 28 10:07:36.898938 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893830    2563 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 28 10:07:36.898938 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893833    2563 flags.go:64] FLAG: --fail-swap-on="true"
Jul 28 10:07:36.898938 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893837    2563 flags.go:64] FLAG: --feature-gates=""
Jul 28 10:07:36.898938 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893841    2563 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 28 10:07:36.898938 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893844    2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 28 10:07:36.898938 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893848    2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 28 10:07:36.898938 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893851    2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 28 10:07:36.898938 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893855    2563 flags.go:64] FLAG: --healthz-port="10248"
Jul 28 10:07:36.898938 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893858    2563 flags.go:64] FLAG: --help="false"
Jul 28 10:07:36.898938 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893861    2563 flags.go:64] FLAG: --hostname-override="ip-10-0-130-235.ec2.internal"
Jul 28 10:07:36.898938 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893864    2563 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 28 10:07:36.898938 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893867    2563 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 28 10:07:36.898938 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893871    2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 28 10:07:36.898938 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893875    2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 28 10:07:36.898938 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893878    2563 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 28 10:07:36.898938 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893881    2563 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 28 10:07:36.898938 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893884    2563 flags.go:64] FLAG: --image-service-endpoint=""
Jul 28 10:07:36.898938 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893887    2563 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 28 10:07:36.898938 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893890    2563 flags.go:64] FLAG: --kube-api-burst="100"
Jul 28 10:07:36.898938 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893893    2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 28 10:07:36.899531 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893897    2563 flags.go:64] FLAG: --kube-api-qps="50"
Jul 28 10:07:36.899531 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893900    2563 flags.go:64] FLAG: --kube-reserved=""
Jul 28 10:07:36.899531 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893903    2563 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 28 10:07:36.899531 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893906    2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 28 10:07:36.899531 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893910    2563 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 28 10:07:36.899531 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893913    2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 28 10:07:36.899531 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893916    2563 flags.go:64] FLAG: --lock-file=""
Jul 28 10:07:36.899531 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893919    2563 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 28 10:07:36.899531 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893922    2563 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 28 10:07:36.899531 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893925    2563 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 28 10:07:36.899531 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893931    2563 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 28 10:07:36.899531 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893934    2563 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 28 10:07:36.899531 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893938    2563 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 28 10:07:36.899531 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893941    2563 flags.go:64] FLAG: --logging-format="text"
Jul 28 10:07:36.899531 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893943    2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 28 10:07:36.899531 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893947    2563 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 28 10:07:36.899531 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893950    2563 flags.go:64] FLAG: --manifest-url=""
Jul 28 10:07:36.899531 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893953    2563 flags.go:64] FLAG: --manifest-url-header=""
Jul 28 10:07:36.899531 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893959    2563 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 28 10:07:36.899531 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893962    2563 flags.go:64] FLAG: --max-open-files="1000000"
Jul 28 10:07:36.899531 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893966    2563 flags.go:64] FLAG: --max-pods="110"
Jul 28 10:07:36.899531 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893970    2563 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 28 10:07:36.899531 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893973    2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 28 10:07:36.899531 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893976    2563 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 28 10:07:36.899531 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893980    2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 28 10:07:36.900151 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893983    2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 28 10:07:36.900151 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893986    2563 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 28 10:07:36.900151 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893990    2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 28 10:07:36.900151 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.893998    2563 flags.go:64] FLAG: --node-status-max-images="50"
Jul 28 10:07:36.900151 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894001    2563 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 28 10:07:36.900151 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894004    2563 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 28 10:07:36.900151 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894008    2563 flags.go:64] FLAG: --pod-cidr=""
Jul 28 10:07:36.900151 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894011    2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 28 10:07:36.900151 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894018    2563 flags.go:64] FLAG: --pod-manifest-path=""
Jul 28 10:07:36.900151 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894021    2563 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 28 10:07:36.900151 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894024    2563 flags.go:64] FLAG: --pods-per-core="0"
Jul 28 10:07:36.900151 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894027    2563 flags.go:64] FLAG: --port="10250"
Jul 28 10:07:36.900151 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894030    2563 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 28 10:07:36.900151 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894033    2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ea6f7c8622e3f1f8"
Jul 28 10:07:36.900151 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894037    2563 flags.go:64] FLAG: --qos-reserved=""
Jul 28 10:07:36.900151 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894040    2563 flags.go:64] FLAG: --read-only-port="10255"
Jul 28 10:07:36.900151 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894043    2563 flags.go:64] FLAG: --register-node="true"
Jul 28 10:07:36.900151 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894046    2563 flags.go:64] FLAG: --register-with-taints=""
Jul 28 10:07:36.900151 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894050    2563 flags.go:64] FLAG: --registry-burst="10"
Jul 28 10:07:36.900151 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894054    2563 flags.go:64] FLAG: --registry-qps="5"
Jul 28 10:07:36.900151 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894056    2563 flags.go:64] FLAG: --reserved-cpus=""
Jul 28 10:07:36.900151 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894059    2563 flags.go:64] FLAG: --reserved-memory=""
Jul 28 10:07:36.900151 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894063    2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 28 10:07:36.900151 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894066    2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 28 10:07:36.900151 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894070    2563 flags.go:64] FLAG: --rotate-certificates="false"
Jul 28 10:07:36.900867 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894073    2563 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 28 10:07:36.900867 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894076    2563 flags.go:64] FLAG: --runonce="false"
Jul 28 10:07:36.900867 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894079    2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 28 10:07:36.900867 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894082    2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 28 10:07:36.900867 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894086    2563 flags.go:64] FLAG: --seccomp-default="false"
Jul 28 10:07:36.900867 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894089    2563 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 28 10:07:36.900867 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894092    2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 28 10:07:36.900867 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894095    2563 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 28 10:07:36.900867 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894098    2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 28 10:07:36.900867 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894101    2563 flags.go:64] FLAG: --storage-driver-password="root"
Jul 28 10:07:36.900867 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894104    2563 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 28 10:07:36.900867 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894108    2563 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 28 10:07:36.900867 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894111    2563 flags.go:64] FLAG: --storage-driver-user="root"
Jul 28 10:07:36.900867 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894114    2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 28 10:07:36.900867 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894117    2563 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 28 10:07:36.900867 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894120    2563 flags.go:64] FLAG: --system-cgroups=""
Jul 28 10:07:36.900867 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894123    2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 28 10:07:36.900867 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894131    2563 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 28 10:07:36.900867 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894134    2563 flags.go:64] FLAG: --tls-cert-file=""
Jul 28 10:07:36.900867 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894137    2563 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 28 10:07:36.900867 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894142    2563 flags.go:64] FLAG: --tls-min-version=""
Jul 28 10:07:36.900867 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894145    2563 flags.go:64] FLAG: --tls-private-key-file=""
Jul 28 10:07:36.900867 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894148    2563 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 28 10:07:36.900867 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894152    2563 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 28 10:07:36.900867 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894155    2563 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 28 10:07:36.901568 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894158    2563 flags.go:64] FLAG: --v="2"
Jul 28 10:07:36.901568 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894163    2563 flags.go:64] FLAG: --version="false"
Jul 28 10:07:36.901568 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894168    2563 flags.go:64] FLAG: --vmodule=""
Jul 28 10:07:36.901568 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894172    2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 28 10:07:36.901568 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894175    2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 28 10:07:36.901568 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894325    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 10:07:36.901568 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894330    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 10:07:36.901568 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894333    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 10:07:36.901568 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894336    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 10:07:36.901568 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894340    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 10:07:36.901568 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894342    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 10:07:36.901568 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894347    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 10:07:36.901568 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894350    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 10:07:36.901568 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894371    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 10:07:36.901568 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894377    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 10:07:36.901568 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894380    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 10:07:36.901568 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894383    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 10:07:36.901568 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894387    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 10:07:36.901568 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894390    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 10:07:36.901568 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894393    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 10:07:36.901568 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894395    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 10:07:36.902104 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894399    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 10:07:36.902104 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894402    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 10:07:36.902104 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894404    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 10:07:36.902104 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894409    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 10:07:36.902104 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894411    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 10:07:36.902104 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894414    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 10:07:36.902104 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894417    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 10:07:36.902104 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894420    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 10:07:36.902104 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894422    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 10:07:36.902104 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894425    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 10:07:36.902104 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894428    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 10:07:36.902104 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894430    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 10:07:36.902104 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894434    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 10:07:36.902104 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894439    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 10:07:36.902104 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894442    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 10:07:36.902104 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894445    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 10:07:36.902104 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894448    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 10:07:36.902104 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894451    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 10:07:36.902104 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894453    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 10:07:36.902605 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894456    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 10:07:36.902605 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894458    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 10:07:36.902605 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894461    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 10:07:36.902605 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894466    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 10:07:36.902605 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894468    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 10:07:36.902605 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894471    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 10:07:36.902605 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894474    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 10:07:36.902605 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894477    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 10:07:36.902605 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894479    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 10:07:36.902605 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894481    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 10:07:36.902605 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894486    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 10:07:36.902605 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894489    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 10:07:36.902605 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894492    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 10:07:36.902605 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894496    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 10:07:36.902605 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894498    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 10:07:36.902605 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894501    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 10:07:36.902605 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894505    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 10:07:36.902605 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894508    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 10:07:36.902605 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894510    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 10:07:36.902605 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894513    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 10:07:36.903109 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894516    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 10:07:36.903109 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894518    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 10:07:36.903109 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894521    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 10:07:36.903109 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894524    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 10:07:36.903109 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894526    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 10:07:36.903109 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894529    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 10:07:36.903109 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894531    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 10:07:36.903109 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894534    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 10:07:36.903109 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894537    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 10:07:36.903109 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894540    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 10:07:36.903109 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894543    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 10:07:36.903109 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894545    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 10:07:36.903109 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894548    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 10:07:36.903109 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894550    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 10:07:36.903109 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894553    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 10:07:36.903109 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894558    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 10:07:36.903109 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894560    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 10:07:36.903109 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894563    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 10:07:36.903109 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894565    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 10:07:36.903109 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894568    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 10:07:36.903623 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894571    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 10:07:36.903623 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894573    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 10:07:36.903623 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894576    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 10:07:36.903623 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894578    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 10:07:36.903623 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894581    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 10:07:36.903623 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894584    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 10:07:36.903623 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894586    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 10:07:36.903623 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894589    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 10:07:36.903623 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894593    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 10:07:36.903623 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894596    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 10:07:36.903623 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894598    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 10:07:36.903623 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894601    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 10:07:36.903623 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894603    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 10:07:36.903623 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894606    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 10:07:36.903623 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894608    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 10:07:36.903623 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894611    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 10:07:36.903623 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894614    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 10:07:36.903623 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894617    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 10:07:36.903623 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894619    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 10:07:36.904090 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894622    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 10:07:36.904090 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894625    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 10:07:36.904090 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894628    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 10:07:36.904090 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894630    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 10:07:36.904090 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.894633    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 10:07:36.904090 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.894642    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 10:07:36.904090 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.902030    2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 28 10:07:36.904090 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.902055    2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 28 10:07:36.904090 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.902911    2563 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 28 10:07:36.904090 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.902923    2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 28 10:07:36.904090 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.902994    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 10:07:36.904090 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903001    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 10:07:36.904090 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903005    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 10:07:36.904090 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903008    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 10:07:36.904090 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903011    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 10:07:36.904492 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903014    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 10:07:36.904492 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903017    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 10:07:36.904492 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903021    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 10:07:36.904492 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903024    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 10:07:36.904492 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903027    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 10:07:36.904492 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903030    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 10:07:36.904492 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903033    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 10:07:36.904492 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903038    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 10:07:36.904492 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903042    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 10:07:36.904492 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903045    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 10:07:36.904492 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903048    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 10:07:36.904492 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903051    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 10:07:36.904492 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903054    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 10:07:36.904492 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903057    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 10:07:36.904492 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903059    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 10:07:36.904492 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903062    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 10:07:36.904492 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903065    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 10:07:36.904492 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903067    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 10:07:36.904492 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903070    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 10:07:36.904492 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903073    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 10:07:36.905013 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903075    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 10:07:36.905013 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903078    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 10:07:36.905013 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903081    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 10:07:36.905013 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903084    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 10:07:36.905013 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903088    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 10:07:36.905013 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903091    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 10:07:36.905013 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903094    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 10:07:36.905013 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903096    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 10:07:36.905013 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903099    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 10:07:36.905013 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903102    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 10:07:36.905013 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903104    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 10:07:36.905013 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903107    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 10:07:36.905013 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903110    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 10:07:36.905013 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903113    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 10:07:36.905013 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903116    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 10:07:36.905013 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903118    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 10:07:36.905013 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903122    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 10:07:36.905013 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903124    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 10:07:36.905013 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903127    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 10:07:36.905013 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903130    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 10:07:36.905013 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903132    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 10:07:36.905559 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903135    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 10:07:36.905559 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903138    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 10:07:36.905559 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903141    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 10:07:36.905559 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903144    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 10:07:36.905559 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903146    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 10:07:36.905559 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903149    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 10:07:36.905559 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903151    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 10:07:36.905559 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903154    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 10:07:36.905559 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903158    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 10:07:36.905559 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903160    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 10:07:36.905559 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903163    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 10:07:36.905559 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903166    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 10:07:36.905559 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903169    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 10:07:36.905559 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903173    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 10:07:36.905559 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903178    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 10:07:36.905559 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903181    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 10:07:36.905559 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903185    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 10:07:36.905559 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903188    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 10:07:36.905559 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903191    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 10:07:36.906022 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903194    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 10:07:36.906022 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903196    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 10:07:36.906022 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903200    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 10:07:36.906022 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903204    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 10:07:36.906022 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903207    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 10:07:36.906022 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903210    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 10:07:36.906022 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903212    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 10:07:36.906022 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903215    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 10:07:36.906022 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903219    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 10:07:36.906022 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903222    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 10:07:36.906022 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903224    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 10:07:36.906022 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903227    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 10:07:36.906022 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903229    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 10:07:36.906022 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903232    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 10:07:36.906022 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903235    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 10:07:36.906022 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903238    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 10:07:36.906022 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903241    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 10:07:36.906022 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903244    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 10:07:36.906022 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903246    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 10:07:36.906585 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903249    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 10:07:36.906585 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903251    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 10:07:36.906585 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903254    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 10:07:36.906585 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903257    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 10:07:36.906585 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903259    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 10:07:36.906585 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903262    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 10:07:36.906585 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903264    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 10:07:36.906585 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903268    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 10:07:36.906585 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903272    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 10:07:36.906585 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903275    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 10:07:36.906585 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903278    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 10:07:36.906585 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903281    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 10:07:36.906585 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903284    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 10:07:36.906585 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903286    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 10:07:36.906585 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903289    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 10:07:36.906972 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.903294    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 10:07:36.906972 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903444    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 10:07:36.906972 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903451    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 10:07:36.906972 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903455    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 10:07:36.906972 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903458    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 10:07:36.906972 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903461    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 10:07:36.906972 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903464    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 10:07:36.906972 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903467    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 10:07:36.906972 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903469    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 10:07:36.906972 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903472    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 10:07:36.906972 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903474    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 10:07:36.906972 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903477    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 10:07:36.906972 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903480    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 10:07:36.906972 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903483    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 10:07:36.907327 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903487    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 10:07:36.907327 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903492    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 10:07:36.907327 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903495    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 10:07:36.907327 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903498    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 10:07:36.907327 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903500    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 10:07:36.907327 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903503    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 10:07:36.907327 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903506    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 10:07:36.907327 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903509    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 10:07:36.907327 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903511    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 10:07:36.907327 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903514    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 10:07:36.907327 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903517    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 10:07:36.907327 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903520    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 10:07:36.907327 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903523    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 10:07:36.907327 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903526    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 10:07:36.907327 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903529    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 10:07:36.907327 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903532    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 10:07:36.907327 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903535    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 10:07:36.907327 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903537    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 10:07:36.907327 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903540    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 10:07:36.907327 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903543    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 10:07:36.907851 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903545    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 10:07:36.907851 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903548    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 10:07:36.907851 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903551    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 10:07:36.907851 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903553    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 10:07:36.907851 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903556    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 10:07:36.907851 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903559    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 10:07:36.907851 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903561    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 10:07:36.907851 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903564    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 10:07:36.907851 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903567    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 10:07:36.907851 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903569    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 10:07:36.907851 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903572    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 10:07:36.907851 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903574    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 10:07:36.907851 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903577    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 10:07:36.907851 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903580    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 10:07:36.907851 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903582    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 10:07:36.907851 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903585    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 10:07:36.907851 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903587    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 10:07:36.907851 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903590    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 10:07:36.907851 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903592    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 10:07:36.907851 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903595    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 10:07:36.908346 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903597    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 10:07:36.908346 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903600    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 10:07:36.908346 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903602    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 10:07:36.908346 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903605    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 10:07:36.908346 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903608    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 10:07:36.908346 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903611    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 10:07:36.908346 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903614    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 10:07:36.908346 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903616    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 10:07:36.908346 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903622    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 10:07:36.908346 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903624    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 10:07:36.908346 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903627    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 10:07:36.908346 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903630    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 10:07:36.908346 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903633    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 10:07:36.908346 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903637    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 10:07:36.908346 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903640    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 10:07:36.908346 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903643    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 10:07:36.908346 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903646    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 10:07:36.908346 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903649    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 10:07:36.908346 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903651    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 10:07:36.908346 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903654    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 10:07:36.908855 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903657    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 10:07:36.908855 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903659    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 10:07:36.908855 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903663    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 10:07:36.908855 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903666    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 10:07:36.908855 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903670    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 10:07:36.908855 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903673    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 10:07:36.908855 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903676    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 10:07:36.908855 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903678    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 10:07:36.908855 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903681    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 10:07:36.908855 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903684    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 10:07:36.908855 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903686    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 10:07:36.908855 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903689    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 10:07:36.908855 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903692    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 10:07:36.908855 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903695    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 10:07:36.908855 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903697    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 10:07:36.908855 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903700    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 10:07:36.908855 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903703    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 10:07:36.908855 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903706    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 10:07:36.908855 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903708    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 10:07:36.909321 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903711    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 10:07:36.909321 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903713    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 10:07:36.909321 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903717    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 10:07:36.909321 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903719    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 10:07:36.909321 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903722    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 10:07:36.909321 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903725    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 10:07:36.909321 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:36.903727    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 10:07:36.909321 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.903732    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 10:07:36.909321 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.903883    2563 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 28 10:07:36.909321 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.906155    2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 28 10:07:36.909321 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.907591    2563 server.go:1019] "Starting client certificate rotation"
Jul 28 10:07:36.909321 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.907703    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 10:07:36.909321 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.907741    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 10:07:36.910443 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.910426    2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 10:07:36.915862 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.915833    2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 10:07:36.927474 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.927441    2563 log.go:25] "Validated CRI v1 runtime API"
Jul 28 10:07:36.929583 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.929561    2563 log.go:25] "Validated CRI v1 image API"
Jul 28 10:07:36.930530 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.930505    2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 28 10:07:36.933048 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.933017    2563 fs.go:135] Filesystem UUIDs: map[00cb0d0f-4c80-45bb-a4d9-13d16fb641ef:/dev/nvme0n1p3 06429d42-885b-4801-b8f7-d30d0cb509d9:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Jul 28 10:07:36.933140 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.933044    2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 28 10:07:36.938600 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.938440    2563 manager.go:216] Machine: {Timestamp:2026-07-28 10:07:36.937010611 +0000 UTC m=+0.378551959 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098688 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26bc9c5c6760227d63bdb3b74ee425 SystemUUID:ec26bc9c-5c67-6022-7d63-bdb3b74ee425 BootID:13a7e8d9-f74b-46f6-ac75-5995dbc2fa16 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:61:2d:8d:06:7d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:61:2d:8d:06:7d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:9a:c8:e8:eb:1c:95 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 28 10:07:36.938600 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.938579    2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 28 10:07:36.938814 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.938722    2563 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 28 10:07:36.941830 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.941795    2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 28 10:07:36.941979 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.941831    2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-235.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 28 10:07:36.942635 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.942622    2563 topology_manager.go:138] "Creating topology manager with none policy"
Jul 28 10:07:36.942690 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.942638    2563 container_manager_linux.go:309] "Creating device plugin manager"
Jul 28 10:07:36.942690 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.942646    2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 10:07:36.943481 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.943468    2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 10:07:36.944091 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.944070    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 10:07:36.944303 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.944289    2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 28 10:07:36.946301 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.946287    2563 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 10:07:36.946464 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.946454    2563 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 28 10:07:36.946509 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.946490    2563 kubelet.go:486] "Attempting to sync node with API server"
Jul 28 10:07:36.946538 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.946517    2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 28 10:07:36.946565 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.946537    2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 28 10:07:36.946565 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.946551    2563 kubelet.go:398] "Adding apiserver pod source"
Jul 28 10:07:36.946565 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.946564    2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 28 10:07:36.947102 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.947088    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 10:07:36.947148 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.947112    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 10:07:36.949337 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.949317    2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 28 10:07:36.951183 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.951168    2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 28 10:07:36.951223 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.951200    2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 28 10:07:36.954605 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.954586    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 28 10:07:36.954684 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.954610    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 28 10:07:36.954684 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.954616    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 28 10:07:36.954684 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.954622    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 28 10:07:36.954684 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.954630    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 28 10:07:36.954684 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.954636    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 28 10:07:36.954684 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.954645    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 28 10:07:36.954684 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.954651    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 28 10:07:36.954684 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.954659    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 28 10:07:36.954684 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.954664    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 28 10:07:36.954684 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.954673    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 28 10:07:36.954684 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.954678    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 28 10:07:36.955519 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.955507    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 28 10:07:36.955519 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.955520    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 28 10:07:36.959550 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.959521    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 28 10:07:36.959550 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:36.959518    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-235.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 28 10:07:36.959740 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:36.959550    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 28 10:07:36.959740 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.959574    2563 server.go:1268] "Started kubelet"
Jul 28 10:07:36.959740 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.959662    2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 28 10:07:36.959740 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.959733    2563 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 28 10:07:36.959913 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.959809    2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 28 10:07:36.960477 ip-10-0-130-235 systemd[1]: Started Kubernetes Kubelet.
Jul 28 10:07:36.961845 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.961831    2563 server.go:310] "Adding debug handlers to kubelet server"
Jul 28 10:07:36.962252 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.962232    2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 28 10:07:36.965624 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:36.965592    2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 28 10:07:36.967616 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.967596    2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 28 10:07:36.967616 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.967604    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 28 10:07:36.967758 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:36.967702    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-235.ec2.internal\" not found"
Jul 28 10:07:36.968224 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.968210    2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 28 10:07:36.968387 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.968222    2563 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 28 10:07:36.968387 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.968384    2563 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 28 10:07:36.968556 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.968454    2563 factory.go:55] Registering systemd factory
Jul 28 10:07:36.968556 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.968469    2563 factory.go:223] Registration of the systemd container factory successfully
Jul 28 10:07:36.968556 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.968531    2563 reconstruct.go:97] "Volume reconstruction finished"
Jul 28 10:07:36.968556 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.968538    2563 reconciler.go:29] "Reconciler: start to sync state"
Jul 28 10:07:36.968736 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.968722    2563 factory.go:153] Registering CRI-O factory
Jul 28 10:07:36.968793 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.968741    2563 factory.go:223] Registration of the crio container factory successfully
Jul 28 10:07:36.968840 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.968795    2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 28 10:07:36.968840 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.968823    2563 factory.go:103] Registering Raw factory
Jul 28 10:07:36.968840 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.968838    2563 manager.go:1196] Started watching for new ooms in manager
Jul 28 10:07:36.969383 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.969338    2563 manager.go:349] Starting recovery of all containers
Jul 28 10:07:36.975031 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:36.974989    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 28 10:07:36.975159 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:36.975047    2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-235.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 28 10:07:36.976063 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:36.974978    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-235.ec2.internal.18c66c304a496910  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-235.ec2.internal,UID:ip-10-0-130-235.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-235.ec2.internal,},FirstTimestamp:2026-07-28 10:07:36.959535376 +0000 UTC m=+0.401076704,LastTimestamp:2026-07-28 10:07:36.959535376 +0000 UTC m=+0.401076704,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-235.ec2.internal,}"
Jul 28 10:07:36.979486 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.979446    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 28 10:07:36.980264 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.980238    2563 manager.go:354] Recovery completed
Jul 28 10:07:36.986144 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.986123    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 10:07:36.989084 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.989062    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 10:07:36.989167 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.989098    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 10:07:36.989167 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.989109    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-235.ec2.internal" event="NodeHasSufficientPID"
Jul 28 10:07:36.989662 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.989648    2563 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 28 10:07:36.989728 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.989663    2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 28 10:07:36.989728 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.989666    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 10:07:36.989728 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.989685    2563 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 10:07:36.990540 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.990516    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 10:07:36.990588 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.990544    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 10:07:36.990588 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.990554    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-235.ec2.internal" event="NodeHasSufficientPID"
Jul 28 10:07:36.991422 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.991406    2563 policy_none.go:49] "None policy: Start"
Jul 28 10:07:36.991485 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.991428    2563 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 28 10:07:36.991485 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.991445    2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 28 10:07:36.991702 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:36.991619    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-235.ec2.internal.18c66c304c0c45d9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-235.ec2.internal,UID:ip-10-0-130-235.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-235.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-235.ec2.internal,},FirstTimestamp:2026-07-28 10:07:36.989083097 +0000 UTC m=+0.430624428,LastTimestamp:2026-07-28 10:07:36.989083097 +0000 UTC m=+0.430624428,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-235.ec2.internal,}"
Jul 28 10:07:36.992436 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.992420    2563 policy_none.go:47] "Start"
Jul 28 10:07:36.998717 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:36.998693    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-l8m2s"
Jul 28 10:07:37.000451 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:37.000342    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-235.ec2.internal.18c66c304c0c934a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-235.ec2.internal,UID:ip-10-0-130-235.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-130-235.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-130-235.ec2.internal,},FirstTimestamp:2026-07-28 10:07:36.989102922 +0000 UTC m=+0.430644251,LastTimestamp:2026-07-28 10:07:36.989102922 +0000 UTC m=+0.430644251,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-235.ec2.internal,}"
Jul 28 10:07:37.009265 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.009093    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-l8m2s"
Jul 28 10:07:37.010574 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:37.010448    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-235.ec2.internal.18c66c304c0cb92b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-235.ec2.internal,UID:ip-10-0-130-235.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-130-235.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-130-235.ec2.internal,},FirstTimestamp:2026-07-28 10:07:36.989112619 +0000 UTC m=+0.430653947,LastTimestamp:2026-07-28 10:07:36.989112619 +0000 UTC m=+0.430653947,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-235.ec2.internal,}"
Jul 28 10:07:37.027090 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.027069    2563 manager.go:337] "Starting Device Plugin manager"
Jul 28 10:07:37.027208 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:37.027122    2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 28 10:07:37.027208 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.027137    2563 server.go:87] "Starting device plugin registration server"
Jul 28 10:07:37.027459 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.027447    2563 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 28 10:07:37.027498 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.027462    2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 28 10:07:37.027587 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.027560    2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 28 10:07:37.027697 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.027661    2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 28 10:07:37.027697 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.027671    2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 28 10:07:37.028286 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:37.028267    2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 28 10:07:37.028390 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:37.028324    2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-235.ec2.internal\" not found"
Jul 28 10:07:37.123983 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.123941    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 28 10:07:37.123983 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.123991    2563 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 28 10:07:37.126064 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.124013    2563 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 28 10:07:37.126064 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:37.124053    2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 28 10:07:37.126709 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.126692    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 10:07:37.128099 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.128083    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 10:07:37.129025 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.129006    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 10:07:37.129149 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.129039    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 10:07:37.129149 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.129050    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-235.ec2.internal" event="NodeHasSufficientPID"
Jul 28 10:07:37.129149 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.129075    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-235.ec2.internal"
Jul 28 10:07:37.140856 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.140778    2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-235.ec2.internal"
Jul 28 10:07:37.140856 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:37.140814    2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-235.ec2.internal\": node \"ip-10-0-130-235.ec2.internal\" not found"
Jul 28 10:07:37.172041 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:37.172007    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-235.ec2.internal\" not found"
Jul 28 10:07:37.225197 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.225145    2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-235.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-235.ec2.internal"]
Jul 28 10:07:37.225414 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.225235    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 10:07:37.228012 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.227995    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 10:07:37.228108 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.228033    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 10:07:37.228108 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.228050    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-235.ec2.internal" event="NodeHasSufficientPID"
Jul 28 10:07:37.230383 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.230347    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 10:07:37.230506 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.230464    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-235.ec2.internal"
Jul 28 10:07:37.230506 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.230493    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 10:07:37.233242 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.233219    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 10:07:37.233346 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.233253    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 10:07:37.233346 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.233264    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-235.ec2.internal" event="NodeHasSufficientPID"
Jul 28 10:07:37.233346 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.233224    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 10:07:37.233346 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.233314    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 10:07:37.233346 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.233327    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-235.ec2.internal" event="NodeHasSufficientPID"
Jul 28 10:07:37.235689 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.235670    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-235.ec2.internal"
Jul 28 10:07:37.235774 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.235706    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 10:07:37.236759 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.236743    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 10:07:37.236848 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.236771    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 10:07:37.236848 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.236782    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-235.ec2.internal" event="NodeHasSufficientPID"
Jul 28 10:07:37.256279 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:37.256242    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-235.ec2.internal\" not found" node="ip-10-0-130-235.ec2.internal"
Jul 28 10:07:37.260360 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:37.260335    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-235.ec2.internal\" not found" node="ip-10-0-130-235.ec2.internal"
Jul 28 10:07:37.271067 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.271033    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/aef8770b7b9299f2dc08df0481ffe72d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-235.ec2.internal\" (UID: \"aef8770b7b9299f2dc08df0481ffe72d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-235.ec2.internal"
Jul 28 10:07:37.271181 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.271073    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aef8770b7b9299f2dc08df0481ffe72d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-235.ec2.internal\" (UID: \"aef8770b7b9299f2dc08df0481ffe72d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-235.ec2.internal"
Jul 28 10:07:37.271181 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.271094    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b79e08bd50776ee222be14a701a056dc-config\") pod \"kube-apiserver-proxy-ip-10-0-130-235.ec2.internal\" (UID: \"b79e08bd50776ee222be14a701a056dc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-235.ec2.internal"
Jul 28 10:07:37.272176 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:37.272147    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-235.ec2.internal\" not found"
Jul 28 10:07:37.371661 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.371618    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/aef8770b7b9299f2dc08df0481ffe72d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-235.ec2.internal\" (UID: \"aef8770b7b9299f2dc08df0481ffe72d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-235.ec2.internal"
Jul 28 10:07:37.371758 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.371661    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aef8770b7b9299f2dc08df0481ffe72d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-235.ec2.internal\" (UID: \"aef8770b7b9299f2dc08df0481ffe72d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-235.ec2.internal"
Jul 28 10:07:37.371758 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.371684    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b79e08bd50776ee222be14a701a056dc-config\") pod \"kube-apiserver-proxy-ip-10-0-130-235.ec2.internal\" (UID: \"b79e08bd50776ee222be14a701a056dc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-235.ec2.internal"
Jul 28 10:07:37.371758 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.371715    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b79e08bd50776ee222be14a701a056dc-config\") pod \"kube-apiserver-proxy-ip-10-0-130-235.ec2.internal\" (UID: \"b79e08bd50776ee222be14a701a056dc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-235.ec2.internal"
Jul 28 10:07:37.371758 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.371727    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/aef8770b7b9299f2dc08df0481ffe72d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-235.ec2.internal\" (UID: \"aef8770b7b9299f2dc08df0481ffe72d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-235.ec2.internal"
Jul 28 10:07:37.371881 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.371729    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aef8770b7b9299f2dc08df0481ffe72d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-235.ec2.internal\" (UID: \"aef8770b7b9299f2dc08df0481ffe72d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-235.ec2.internal"
Jul 28 10:07:37.372746 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:37.372727    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-235.ec2.internal\" not found"
Jul 28 10:07:37.473085 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:37.473006    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-235.ec2.internal\" not found"
Jul 28 10:07:37.558791 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.558742    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-235.ec2.internal"
Jul 28 10:07:37.565488 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.565455    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-235.ec2.internal"
Jul 28 10:07:37.573410 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:37.573382    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-235.ec2.internal\" not found"
Jul 28 10:07:37.674514 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:37.674462    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-235.ec2.internal\" not found"
Jul 28 10:07:37.775353 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:37.775262    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-235.ec2.internal\" not found"
Jul 28 10:07:37.876006 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:37.875970    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-235.ec2.internal\" not found"
Jul 28 10:07:37.908067 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.908024    2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 28 10:07:37.908569 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:37.908251    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 10:07:37.976451 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:37.976410    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-235.ec2.internal\" not found"
Jul 28 10:07:38.014620 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.014554    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 10:02:37 +0000 UTC" deadline="2028-02-12 22:13:52.724177623 +0000 UTC"
Jul 28 10:07:38.014620 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.014616    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13548h6m14.709565962s"
Jul 28 10:07:38.076559 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:38.076519    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-235.ec2.internal\" not found"
Jul 28 10:07:38.108825 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.108787    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 10:07:38.110308 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.110284    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 10:07:38.110460 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.110318    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 10:07:38.110460 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.110328    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-235.ec2.internal" event="NodeHasSufficientPID"
Jul 28 10:07:38.159640 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.159611    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 10:07:38.168174 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.168146    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-235.ec2.internal"
Jul 28 10:07:38.179202 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.179175    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 10:07:38.181188 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.181162    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-235.ec2.internal"
Jul 28 10:07:38.194696 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.194669    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 10:07:38.196334 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.196310    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 10:07:38.234998 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.234969    2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 10:07:38.244217 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:38.244165    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaef8770b7b9299f2dc08df0481ffe72d.slice/crio-39dd61f0e62ec95066ac48729aa32725b5847ec932c96e8d465be3e4b6025f45 WatchSource:0}: Error finding container 39dd61f0e62ec95066ac48729aa32725b5847ec932c96e8d465be3e4b6025f45: Status 404 returned error can't find the container with id 39dd61f0e62ec95066ac48729aa32725b5847ec932c96e8d465be3e4b6025f45
Jul 28 10:07:38.244835 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:38.244816    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb79e08bd50776ee222be14a701a056dc.slice/crio-3f2380db8dd14516cabb87b6ed3516f5737a223f5877ecdc592fc3257498113e WatchSource:0}: Error finding container 3f2380db8dd14516cabb87b6ed3516f5737a223f5877ecdc592fc3257498113e: Status 404 returned error can't find the container with id 3f2380db8dd14516cabb87b6ed3516f5737a223f5877ecdc592fc3257498113e
Jul 28 10:07:38.250422 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.250400    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 10:07:38.948516 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.948480    2563 apiserver.go:52] "Watching apiserver"
Jul 28 10:07:38.956719 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.956668    2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 28 10:07:38.957230 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.957202    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-cmrjd","openshift-network-operator/iptables-alerter-bzknt","kube-system/konnectivity-agent-8bh2v","kube-system/kube-apiserver-proxy-ip-10-0-130-235.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2h2b9","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-235.ec2.internal","openshift-multus/multus-additional-cni-plugins-bmkbw","openshift-multus/network-metrics-daemon-299qd","openshift-ovn-kubernetes/ovnkube-node-4sg48","kube-system/global-pull-secret-syncer-nz956","openshift-cluster-node-tuning-operator/tuned-fmf4x","openshift-dns/node-resolver-hpntr","openshift-image-registry/node-ca-d48vp","openshift-multus/multus-29wrb"]
Jul 28 10:07:38.959948 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.959913    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nz956"
Jul 28 10:07:38.960092 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:38.960014    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nz956" podUID="5f9bc3b4-d2b8-40de-8c04-846085757396"
Jul 28 10:07:38.962277 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.962251    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-d48vp"
Jul 28 10:07:38.964687 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.964651    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8bh2v"
Jul 28 10:07:38.965539 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.965509    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 28 10:07:38.965871 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.965842    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 28 10:07:38.966111 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.966094    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 28 10:07:38.966479 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.966457    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-dfdvg\""
Jul 28 10:07:38.966797 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.966776    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 28 10:07:38.967401 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.967097    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 28 10:07:38.967401 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.967378    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-29wrb"
Jul 28 10:07:38.968375 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.968319    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-2dxng\""
Jul 28 10:07:38.971392 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.971367    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 28 10:07:38.971691 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.971671    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 28 10:07:38.971895 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.971874    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 28 10:07:38.972109 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.972089    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-q7gqj\""
Jul 28 10:07:38.972338 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.972319    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 28 10:07:38.973249 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.973218    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-299qd"
Jul 28 10:07:38.973249 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.973248    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 28 10:07:38.973438 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:38.973300    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-299qd" podUID="bfe1e00b-5d1b-4127-b270-abe1d44d3dd0"
Jul 28 10:07:38.974003 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.973218    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2h2b9"
Jul 28 10:07:38.975516 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.975495    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:38.977721 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.977694    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bmkbw"
Jul 28 10:07:38.980345 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.978925    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-var-lib-openvswitch\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:38.980345 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.978963    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-multus-cni-dir\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:38.980345 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.978988    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-host-var-lib-cni-bin\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:38.980345 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.979009    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-host-var-lib-cni-multus\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:38.980345 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.979043    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-log-socket\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:38.980345 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.979065    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rqpp\" (UniqueName: \"kubernetes.io/projected/53d373c1-808b-4f32-ab54-d47bbeb3a151-kube-api-access-2rqpp\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:38.980345 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.979088    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c4838d1b-fa7d-4770-a990-66c67c14c8b9-konnectivity-ca\") pod \"konnectivity-agent-8bh2v\" (UID: \"c4838d1b-fa7d-4770-a990-66c67c14c8b9\") " pod="kube-system/konnectivity-agent-8bh2v"
Jul 28 10:07:38.980345 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.979112    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5f9bc3b4-d2b8-40de-8c04-846085757396-dbus\") pod \"global-pull-secret-syncer-nz956\" (UID: \"5f9bc3b4-d2b8-40de-8c04-846085757396\") " pod="kube-system/global-pull-secret-syncer-nz956"
Jul 28 10:07:38.980345 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.979134    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5f8ff51d-90c2-4146-b040-a71013fed650-cni-binary-copy\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:38.980345 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.979157    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-multus-conf-dir\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:38.980345 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.979178    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5f9bc3b4-d2b8-40de-8c04-846085757396-kubelet-config\") pod \"global-pull-secret-syncer-nz956\" (UID: \"5f9bc3b4-d2b8-40de-8c04-846085757396\") " pod="kube-system/global-pull-secret-syncer-nz956"
Jul 28 10:07:38.980345 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.979200    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-host-var-lib-kubelet\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:38.980345 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.979240    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-host-run-multus-certs\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:38.980345 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.979263    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-host-cni-netd\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:38.980345 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.979284    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-os-release\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:38.980345 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.979305    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-multus-socket-dir-parent\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:38.980345 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.979347    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b5d7a916-bec0-44bd-945a-ead38981eedb-device-dir\") pod \"aws-ebs-csi-driver-node-2h2b9\" (UID: \"b5d7a916-bec0-44bd-945a-ead38981eedb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2h2b9"
Jul 28 10:07:38.981510 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.979394    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfe1e00b-5d1b-4127-b270-abe1d44d3dd0-metrics-certs\") pod \"network-metrics-daemon-299qd\" (UID: \"bfe1e00b-5d1b-4127-b270-abe1d44d3dd0\") " pod="openshift-multus/network-metrics-daemon-299qd"
Jul 28 10:07:38.981510 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.979434    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4b1075b5-7b36-420f-aa9f-90022e20d840-host\") pod \"node-ca-d48vp\" (UID: \"4b1075b5-7b36-420f-aa9f-90022e20d840\") " pod="openshift-image-registry/node-ca-d48vp"
Jul 28 10:07:38.981510 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.979459    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/53d373c1-808b-4f32-ab54-d47bbeb3a151-ovnkube-config\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:38.981510 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.979488    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/53d373c1-808b-4f32-ab54-d47bbeb3a151-ovnkube-script-lib\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:38.981510 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.979512    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b5d7a916-bec0-44bd-945a-ead38981eedb-socket-dir\") pod \"aws-ebs-csi-driver-node-2h2b9\" (UID: \"b5d7a916-bec0-44bd-945a-ead38981eedb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2h2b9"
Jul 28 10:07:38.981510 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.979537    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b5d7a916-bec0-44bd-945a-ead38981eedb-etc-selinux\") pod \"aws-ebs-csi-driver-node-2h2b9\" (UID: \"b5d7a916-bec0-44bd-945a-ead38981eedb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2h2b9"
Jul 28 10:07:38.981510 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.979588    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-etc-openvswitch\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:38.981510 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.979624    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-run-ovn\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:38.981510 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.979651    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/53d373c1-808b-4f32-ab54-d47bbeb3a151-ovn-node-metrics-cert\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:38.981510 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.979693    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-host-run-netns\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:38.981510 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.979757    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b5d7a916-bec0-44bd-945a-ead38981eedb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2h2b9\" (UID: \"b5d7a916-bec0-44bd-945a-ead38981eedb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2h2b9"
Jul 28 10:07:38.981510 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.979804    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w942v\" (UniqueName: \"kubernetes.io/projected/4b1075b5-7b36-420f-aa9f-90022e20d840-kube-api-access-w942v\") pod \"node-ca-d48vp\" (UID: \"4b1075b5-7b36-420f-aa9f-90022e20d840\") " pod="openshift-image-registry/node-ca-d48vp"
Jul 28 10:07:38.981510 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.979832    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/53d373c1-808b-4f32-ab54-d47bbeb3a151-env-overrides\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:38.981510 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.979895    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvdbd\" (UniqueName: \"kubernetes.io/projected/bfe1e00b-5d1b-4127-b270-abe1d44d3dd0-kube-api-access-zvdbd\") pod \"network-metrics-daemon-299qd\" (UID: \"bfe1e00b-5d1b-4127-b270-abe1d44d3dd0\") " pod="openshift-multus/network-metrics-daemon-299qd"
Jul 28 10:07:38.981510 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.979959    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-host-run-ovn-kubernetes\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:38.981510 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.979978    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-host-run-k8s-cni-cncf-io\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:38.982450 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.980026    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5f8ff51d-90c2-4146-b040-a71013fed650-multus-daemon-config\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:38.982450 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.980046    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-run-systemd\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:38.982450 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.980081    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4b1075b5-7b36-420f-aa9f-90022e20d840-serviceca\") pod \"node-ca-d48vp\" (UID: \"4b1075b5-7b36-420f-aa9f-90022e20d840\") " pod="openshift-image-registry/node-ca-d48vp"
Jul 28 10:07:38.982450 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.980110    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-cnibin\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:38.982450 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.980142    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-host-kubelet\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:38.982450 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.980158    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-host-slash\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:38.982450 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.980173    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-node-log\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:38.982450 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.980192    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-system-cni-dir\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:38.982450 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.980230    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-etc-kubernetes\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:38.982450 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.980247    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b5d7a916-bec0-44bd-945a-ead38981eedb-registration-dir\") pod \"aws-ebs-csi-driver-node-2h2b9\" (UID: \"b5d7a916-bec0-44bd-945a-ead38981eedb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2h2b9"
Jul 28 10:07:38.982450 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.980270    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-host-run-netns\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:38.982450 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.980285    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-run-openvswitch\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:38.982450 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.980352    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f9bc3b4-d2b8-40de-8c04-846085757396-original-pull-secret\") pod \"global-pull-secret-syncer-nz956\" (UID: \"5f9bc3b4-d2b8-40de-8c04-846085757396\") " pod="kube-system/global-pull-secret-syncer-nz956"
Jul 28 10:07:38.982450 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.980465    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c4838d1b-fa7d-4770-a990-66c67c14c8b9-agent-certs\") pod \"konnectivity-agent-8bh2v\" (UID: \"c4838d1b-fa7d-4770-a990-66c67c14c8b9\") " pod="kube-system/konnectivity-agent-8bh2v"
Jul 28 10:07:38.982450 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.980575    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tpdx\" (UniqueName: \"kubernetes.io/projected/5f8ff51d-90c2-4146-b040-a71013fed650-kube-api-access-9tpdx\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:38.982450 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.980609    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-systemd-units\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:38.982450 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.980635    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-host-cni-bin\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:38.983189 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.980664    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:38.983189 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.980714    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-hostroot\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:38.983189 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.980739    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b5d7a916-bec0-44bd-945a-ead38981eedb-sys-fs\") pod \"aws-ebs-csi-driver-node-2h2b9\" (UID: \"b5d7a916-bec0-44bd-945a-ead38981eedb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2h2b9"
Jul 28 10:07:38.983189 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.980781    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxc65\" (UniqueName: \"kubernetes.io/projected/b5d7a916-bec0-44bd-945a-ead38981eedb-kube-api-access-sxc65\") pod \"aws-ebs-csi-driver-node-2h2b9\" (UID: \"b5d7a916-bec0-44bd-945a-ead38981eedb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2h2b9"
Jul 28 10:07:38.983189 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.980913    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xgb8w\""
Jul 28 10:07:38.983189 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.981155    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bzknt"
Jul 28 10:07:38.983189 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.981205    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 28 10:07:38.983189 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.981276    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-xbfxp\""
Jul 28 10:07:38.983189 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.981293    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 28 10:07:38.983189 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.981496    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 28 10:07:38.983189 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.981604    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 28 10:07:38.983189 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.981629    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 28 10:07:38.983189 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.981910    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 28 10:07:38.983189 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.982049    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-rcwzq\""
Jul 28 10:07:38.983189 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.982137    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 28 10:07:38.983189 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.982233    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 28 10:07:38.983189 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.982345    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 28 10:07:38.983189 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.982395    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 28 10:07:38.983189 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.982773    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 28 10:07:38.984812 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.984789    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmrjd"
Jul 28 10:07:38.984942 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:38.984889    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmrjd" podUID="598884b3-5fd8-42ae-87ee-4f24d6c4bdb3"
Jul 28 10:07:38.986336 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.986309    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 28 10:07:38.986541 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.986519    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 28 10:07:38.986669 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.986527    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 28 10:07:38.987159 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.987128    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:38.989298 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.988593    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-p92mv\""
Jul 28 10:07:38.990401 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.990348    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hpntr"
Jul 28 10:07:38.990547 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.990447    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 28 10:07:38.990679 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.990659    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 28 10:07:38.990780 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.990762    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xxcmz\""
Jul 28 10:07:38.994291 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.992920    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 28 10:07:38.994291 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.993147    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 10:07:38.994291 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.993382    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4wbtr\""
Jul 28 10:07:38.994291 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:38.993887    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 28 10:07:39.020495 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.020470    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-s2mcm"
Jul 28 10:07:39.034520 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.034488    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-s2mcm"
Jul 28 10:07:39.070041 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.070014    2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 28 10:07:39.081043 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.080999    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxmcx\" (UniqueName: \"kubernetes.io/projected/598884b3-5fd8-42ae-87ee-4f24d6c4bdb3-kube-api-access-cxmcx\") pod \"network-check-target-cmrjd\" (UID: \"598884b3-5fd8-42ae-87ee-4f24d6c4bdb3\") " pod="openshift-network-diagnostics/network-check-target-cmrjd"
Jul 28 10:07:39.081043 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.081044    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9e2fbc65-eba0-483e-ad62-d8f17b9bd452-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bmkbw\" (UID: \"9e2fbc65-eba0-483e-ad62-d8f17b9bd452\") " pod="openshift-multus/multus-additional-cni-plugins-bmkbw"
Jul 28 10:07:39.081336 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.081075    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9e2fbc65-eba0-483e-ad62-d8f17b9bd452-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bmkbw\" (UID: \"9e2fbc65-eba0-483e-ad62-d8f17b9bd452\") " pod="openshift-multus/multus-additional-cni-plugins-bmkbw"
Jul 28 10:07:39.081336 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.081109    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-run-systemd\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:39.081336 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.081138    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4b1075b5-7b36-420f-aa9f-90022e20d840-serviceca\") pod \"node-ca-d48vp\" (UID: \"4b1075b5-7b36-420f-aa9f-90022e20d840\") " pod="openshift-image-registry/node-ca-d48vp"
Jul 28 10:07:39.081336 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.081165    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-cnibin\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:39.081336 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.081190    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2tmz\" (UniqueName: \"kubernetes.io/projected/9e2fbc65-eba0-483e-ad62-d8f17b9bd452-kube-api-access-r2tmz\") pod \"multus-additional-cni-plugins-bmkbw\" (UID: \"9e2fbc65-eba0-483e-ad62-d8f17b9bd452\") " pod="openshift-multus/multus-additional-cni-plugins-bmkbw"
Jul 28 10:07:39.081336 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.081217    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-host-kubelet\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:39.081336 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.081243    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-host-slash\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:39.081336 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.081267    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-node-log\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:39.081336 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.081293    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-system-cni-dir\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:39.081336 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.081318    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-etc-kubernetes\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:39.081941 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.081342    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b5d7a916-bec0-44bd-945a-ead38981eedb-registration-dir\") pod \"aws-ebs-csi-driver-node-2h2b9\" (UID: \"b5d7a916-bec0-44bd-945a-ead38981eedb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2h2b9"
Jul 28 10:07:39.081941 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.081369    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9e2fbc65-eba0-483e-ad62-d8f17b9bd452-cnibin\") pod \"multus-additional-cni-plugins-bmkbw\" (UID: \"9e2fbc65-eba0-483e-ad62-d8f17b9bd452\") " pod="openshift-multus/multus-additional-cni-plugins-bmkbw"
Jul 28 10:07:39.081941 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.081394    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-host-run-netns\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:39.081941 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.081436    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-run-openvswitch\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:39.081941 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.081461    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f9bc3b4-d2b8-40de-8c04-846085757396-original-pull-secret\") pod \"global-pull-secret-syncer-nz956\" (UID: \"5f9bc3b4-d2b8-40de-8c04-846085757396\") " pod="kube-system/global-pull-secret-syncer-nz956"
Jul 28 10:07:39.081941 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.081498    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c4838d1b-fa7d-4770-a990-66c67c14c8b9-agent-certs\") pod \"konnectivity-agent-8bh2v\" (UID: \"c4838d1b-fa7d-4770-a990-66c67c14c8b9\") " pod="kube-system/konnectivity-agent-8bh2v"
Jul 28 10:07:39.081941 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.081515    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tpdx\" (UniqueName: \"kubernetes.io/projected/5f8ff51d-90c2-4146-b040-a71013fed650-kube-api-access-9tpdx\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:39.081941 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.081531    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85j4l\" (UniqueName: \"kubernetes.io/projected/493a6ea0-265b-4330-8e11-b4911d3a3cb3-kube-api-access-85j4l\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.081941 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.081550    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgldr\" (UniqueName: \"kubernetes.io/projected/af6b006d-6148-4b30-a1e9-bd2e14118b06-kube-api-access-rgldr\") pod \"iptables-alerter-bzknt\" (UID: \"af6b006d-6148-4b30-a1e9-bd2e14118b06\") " pod="openshift-network-operator/iptables-alerter-bzknt"
Jul 28 10:07:39.081941 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.081565    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/493a6ea0-265b-4330-8e11-b4911d3a3cb3-var-lib-kubelet\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.081941 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.081598    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9e2fbc65-eba0-483e-ad62-d8f17b9bd452-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bmkbw\" (UID: \"9e2fbc65-eba0-483e-ad62-d8f17b9bd452\") " pod="openshift-multus/multus-additional-cni-plugins-bmkbw"
Jul 28 10:07:39.081941 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.081623    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-systemd-units\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:39.081941 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.081645    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-host-cni-bin\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:39.081941 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.081672    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:39.081941 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.081695    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-hostroot\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:39.081941 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.081711    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b5d7a916-bec0-44bd-945a-ead38981eedb-sys-fs\") pod \"aws-ebs-csi-driver-node-2h2b9\" (UID: \"b5d7a916-bec0-44bd-945a-ead38981eedb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2h2b9"
Jul 28 10:07:39.082756 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.081727    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxc65\" (UniqueName: \"kubernetes.io/projected/b5d7a916-bec0-44bd-945a-ead38981eedb-kube-api-access-sxc65\") pod \"aws-ebs-csi-driver-node-2h2b9\" (UID: \"b5d7a916-bec0-44bd-945a-ead38981eedb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2h2b9"
Jul 28 10:07:39.082756 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.081743    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/493a6ea0-265b-4330-8e11-b4911d3a3cb3-sys\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.082756 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.081760    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-var-lib-openvswitch\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:39.082756 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.081785    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-multus-cni-dir\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:39.082756 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.081824    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-host-var-lib-cni-bin\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:39.082756 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.081856    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-host-var-lib-cni-multus\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:39.082756 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.081892    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hk2h4\" (UniqueName: \"kubernetes.io/projected/0f2a806a-05a5-4fd1-be4f-be8b64a5013b-kube-api-access-hk2h4\") pod \"node-resolver-hpntr\" (UID: \"0f2a806a-05a5-4fd1-be4f-be8b64a5013b\") " pod="openshift-dns/node-resolver-hpntr"
Jul 28 10:07:39.082756 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.081914    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/493a6ea0-265b-4330-8e11-b4911d3a3cb3-host\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.082756 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.081943    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/493a6ea0-265b-4330-8e11-b4911d3a3cb3-tmp\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.082756 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.081980    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-log-socket\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:39.082756 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.082009    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rqpp\" (UniqueName: \"kubernetes.io/projected/53d373c1-808b-4f32-ab54-d47bbeb3a151-kube-api-access-2rqpp\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:39.082756 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.082040    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c4838d1b-fa7d-4770-a990-66c67c14c8b9-konnectivity-ca\") pod \"konnectivity-agent-8bh2v\" (UID: \"c4838d1b-fa7d-4770-a990-66c67c14c8b9\") " pod="kube-system/konnectivity-agent-8bh2v"
Jul 28 10:07:39.082756 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.082068    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5f9bc3b4-d2b8-40de-8c04-846085757396-dbus\") pod \"global-pull-secret-syncer-nz956\" (UID: \"5f9bc3b4-d2b8-40de-8c04-846085757396\") " pod="kube-system/global-pull-secret-syncer-nz956"
Jul 28 10:07:39.082756 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.082096    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5f8ff51d-90c2-4146-b040-a71013fed650-cni-binary-copy\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:39.082756 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.082123    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-multus-conf-dir\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:39.082756 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.082150    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5f9bc3b4-d2b8-40de-8c04-846085757396-kubelet-config\") pod \"global-pull-secret-syncer-nz956\" (UID: \"5f9bc3b4-d2b8-40de-8c04-846085757396\") " pod="kube-system/global-pull-secret-syncer-nz956"
Jul 28 10:07:39.082756 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.082176    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-host-var-lib-kubelet\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:39.083649 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.082207    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-host-run-multus-certs\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:39.083649 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.082234    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/493a6ea0-265b-4330-8e11-b4911d3a3cb3-etc-systemd\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.083649 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.082265    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/493a6ea0-265b-4330-8e11-b4911d3a3cb3-lib-modules\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.083649 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.082294    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/493a6ea0-265b-4330-8e11-b4911d3a3cb3-etc-tuned\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.083649 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.082323    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-host-cni-netd\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:39.083649 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.082351    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-os-release\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:39.083649 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.082378    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-multus-socket-dir-parent\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:39.083649 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.082408    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b5d7a916-bec0-44bd-945a-ead38981eedb-device-dir\") pod \"aws-ebs-csi-driver-node-2h2b9\" (UID: \"b5d7a916-bec0-44bd-945a-ead38981eedb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2h2b9"
Jul 28 10:07:39.083649 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.082429    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfe1e00b-5d1b-4127-b270-abe1d44d3dd0-metrics-certs\") pod \"network-metrics-daemon-299qd\" (UID: \"bfe1e00b-5d1b-4127-b270-abe1d44d3dd0\") " pod="openshift-multus/network-metrics-daemon-299qd"
Jul 28 10:07:39.083649 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.082475    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/493a6ea0-265b-4330-8e11-b4911d3a3cb3-etc-modprobe-d\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.083649 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.082502    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4b1075b5-7b36-420f-aa9f-90022e20d840-host\") pod \"node-ca-d48vp\" (UID: \"4b1075b5-7b36-420f-aa9f-90022e20d840\") " pod="openshift-image-registry/node-ca-d48vp"
Jul 28 10:07:39.083649 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.082553    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/53d373c1-808b-4f32-ab54-d47bbeb3a151-ovnkube-config\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:39.083649 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.082576    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-etc-kubernetes\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:39.083649 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.082638    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-host-run-netns\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:39.083649 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.082694    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-host-slash\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:39.083649 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.082697    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-node-log\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:39.083649 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.082744    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-run-openvswitch\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:39.083649 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.082758    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-system-cni-dir\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:39.084237 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.082656    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-cnibin\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:39.084237 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.082813    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b5d7a916-bec0-44bd-945a-ead38981eedb-registration-dir\") pod \"aws-ebs-csi-driver-node-2h2b9\" (UID: \"b5d7a916-bec0-44bd-945a-ead38981eedb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2h2b9"
Jul 28 10:07:39.084237 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.082578    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:39.084237 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.083083    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4b1075b5-7b36-420f-aa9f-90022e20d840-serviceca\") pod \"node-ca-d48vp\" (UID: \"4b1075b5-7b36-420f-aa9f-90022e20d840\") " pod="openshift-image-registry/node-ca-d48vp"
Jul 28 10:07:39.084237 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.083159    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/53d373c1-808b-4f32-ab54-d47bbeb3a151-ovnkube-script-lib\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:39.084237 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.083167    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/53d373c1-808b-4f32-ab54-d47bbeb3a151-ovnkube-config\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:39.084237 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.083188    2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 28 10:07:39.084237 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.083196    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b5d7a916-bec0-44bd-945a-ead38981eedb-socket-dir\") pod \"aws-ebs-csi-driver-node-2h2b9\" (UID: \"b5d7a916-bec0-44bd-945a-ead38981eedb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2h2b9"
Jul 28 10:07:39.084237 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.083230    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b5d7a916-bec0-44bd-945a-ead38981eedb-etc-selinux\") pod \"aws-ebs-csi-driver-node-2h2b9\" (UID: \"b5d7a916-bec0-44bd-945a-ead38981eedb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2h2b9"
Jul 28 10:07:39.084237 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.083250    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-multus-conf-dir\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:39.084237 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.083262    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/af6b006d-6148-4b30-a1e9-bd2e14118b06-iptables-alerter-script\") pod \"iptables-alerter-bzknt\" (UID: \"af6b006d-6148-4b30-a1e9-bd2e14118b06\") " pod="openshift-network-operator/iptables-alerter-bzknt"
Jul 28 10:07:39.084237 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.083297    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/af6b006d-6148-4b30-a1e9-bd2e14118b06-host-slash\") pod \"iptables-alerter-bzknt\" (UID: \"af6b006d-6148-4b30-a1e9-bd2e14118b06\") " pod="openshift-network-operator/iptables-alerter-bzknt"
Jul 28 10:07:39.084237 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.083304    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5f9bc3b4-d2b8-40de-8c04-846085757396-kubelet-config\") pod \"global-pull-secret-syncer-nz956\" (UID: \"5f9bc3b4-d2b8-40de-8c04-846085757396\") " pod="kube-system/global-pull-secret-syncer-nz956"
Jul 28 10:07:39.084237 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.083340    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-etc-openvswitch\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:39.084237 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.083358    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-host-var-lib-kubelet\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:39.084237 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.083370    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/493a6ea0-265b-4330-8e11-b4911d3a3cb3-etc-sysconfig\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.084237 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.083397    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/493a6ea0-265b-4330-8e11-b4911d3a3cb3-run\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.084990 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.083401    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-host-run-multus-certs\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:39.084990 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.083448    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-run-ovn\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:39.084990 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.083477    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/493a6ea0-265b-4330-8e11-b4911d3a3cb3-etc-sysctl-d\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.084990 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.083483    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-host-cni-netd\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:39.084990 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.083541    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-run-systemd\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:39.084990 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.083552    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b5d7a916-bec0-44bd-945a-ead38981eedb-device-dir\") pod \"aws-ebs-csi-driver-node-2h2b9\" (UID: \"b5d7a916-bec0-44bd-945a-ead38981eedb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2h2b9"
Jul 28 10:07:39.084990 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:39.083572    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 10:07:39.084990 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:39.083655    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfe1e00b-5d1b-4127-b270-abe1d44d3dd0-metrics-certs podName:bfe1e00b-5d1b-4127-b270-abe1d44d3dd0 nodeName:}" failed. No retries permitted until 2026-07-28 10:07:39.583614818 +0000 UTC m=+3.025156139 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bfe1e00b-5d1b-4127-b270-abe1d44d3dd0-metrics-certs") pod "network-metrics-daemon-299qd" (UID: "bfe1e00b-5d1b-4127-b270-abe1d44d3dd0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 10:07:39.084990 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.083667    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b5d7a916-bec0-44bd-945a-ead38981eedb-socket-dir\") pod \"aws-ebs-csi-driver-node-2h2b9\" (UID: \"b5d7a916-bec0-44bd-945a-ead38981eedb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2h2b9"
Jul 28 10:07:39.084990 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.083877    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-etc-openvswitch\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:39.084990 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:39.083915    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 10:07:39.084990 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.083944    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4b1075b5-7b36-420f-aa9f-90022e20d840-host\") pod \"node-ca-d48vp\" (UID: \"4b1075b5-7b36-420f-aa9f-90022e20d840\") " pod="openshift-image-registry/node-ca-d48vp"
Jul 28 10:07:39.084990 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:39.083964    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f9bc3b4-d2b8-40de-8c04-846085757396-original-pull-secret podName:5f9bc3b4-d2b8-40de-8c04-846085757396 nodeName:}" failed. No retries permitted until 2026-07-28 10:07:39.583947657 +0000 UTC m=+3.025488977 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f9bc3b4-d2b8-40de-8c04-846085757396-original-pull-secret") pod "global-pull-secret-syncer-nz956" (UID: "5f9bc3b4-d2b8-40de-8c04-846085757396") : object "kube-system"/"original-pull-secret" not registered
Jul 28 10:07:39.084990 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.084059    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b5d7a916-bec0-44bd-945a-ead38981eedb-etc-selinux\") pod \"aws-ebs-csi-driver-node-2h2b9\" (UID: \"b5d7a916-bec0-44bd-945a-ead38981eedb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2h2b9"
Jul 28 10:07:39.084990 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.084102    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/53d373c1-808b-4f32-ab54-d47bbeb3a151-ovnkube-script-lib\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:39.084990 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.084117    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-systemd-units\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:39.084990 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.084159    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e2fbc65-eba0-483e-ad62-d8f17b9bd452-system-cni-dir\") pod \"multus-additional-cni-plugins-bmkbw\" (UID: \"9e2fbc65-eba0-483e-ad62-d8f17b9bd452\") " pod="openshift-multus/multus-additional-cni-plugins-bmkbw"
Jul 28 10:07:39.085964 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.084180    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-os-release\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:39.085964 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.084197    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5f8ff51d-90c2-4146-b040-a71013fed650-cni-binary-copy\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:39.085964 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.084207    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9e2fbc65-eba0-483e-ad62-d8f17b9bd452-cni-binary-copy\") pod \"multus-additional-cni-plugins-bmkbw\" (UID: \"9e2fbc65-eba0-483e-ad62-d8f17b9bd452\") " pod="openshift-multus/multus-additional-cni-plugins-bmkbw"
Jul 28 10:07:39.085964 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.084227    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-host-kubelet\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:39.085964 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.084246    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/53d373c1-808b-4f32-ab54-d47bbeb3a151-ovn-node-metrics-cert\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:39.085964 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.084280    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-host-run-netns\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:39.085964 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.084305    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b5d7a916-bec0-44bd-945a-ead38981eedb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2h2b9\" (UID: \"b5d7a916-bec0-44bd-945a-ead38981eedb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2h2b9"
Jul 28 10:07:39.085964 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.084307    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-host-var-lib-cni-multus\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:39.085964 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.084337    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/493a6ea0-265b-4330-8e11-b4911d3a3cb3-etc-sysctl-conf\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.085964 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.084361    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-multus-socket-dir-parent\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:39.085964 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.084374    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w942v\" (UniqueName: \"kubernetes.io/projected/4b1075b5-7b36-420f-aa9f-90022e20d840-kube-api-access-w942v\") pod \"node-ca-d48vp\" (UID: \"4b1075b5-7b36-420f-aa9f-90022e20d840\") " pod="openshift-image-registry/node-ca-d48vp"
Jul 28 10:07:39.085964 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.084412    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/53d373c1-808b-4f32-ab54-d47bbeb3a151-env-overrides\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:39.085964 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.084438    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-multus-cni-dir\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:39.085964 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.084281    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-host-var-lib-cni-bin\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:39.085964 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.084475    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5f9bc3b4-d2b8-40de-8c04-846085757396-dbus\") pod \"global-pull-secret-syncer-nz956\" (UID: \"5f9bc3b4-d2b8-40de-8c04-846085757396\") " pod="kube-system/global-pull-secret-syncer-nz956"
Jul 28 10:07:39.085964 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.084495    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b5d7a916-bec0-44bd-945a-ead38981eedb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2h2b9\" (UID: \"b5d7a916-bec0-44bd-945a-ead38981eedb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2h2b9"
Jul 28 10:07:39.085964 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.084471    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvdbd\" (UniqueName: \"kubernetes.io/projected/bfe1e00b-5d1b-4127-b270-abe1d44d3dd0-kube-api-access-zvdbd\") pod \"network-metrics-daemon-299qd\" (UID: \"bfe1e00b-5d1b-4127-b270-abe1d44d3dd0\") " pod="openshift-multus/network-metrics-daemon-299qd"
Jul 28 10:07:39.086962 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.084558    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b5d7a916-bec0-44bd-945a-ead38981eedb-sys-fs\") pod \"aws-ebs-csi-driver-node-2h2b9\" (UID: \"b5d7a916-bec0-44bd-945a-ead38981eedb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2h2b9"
Jul 28 10:07:39.086962 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.084568    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-log-socket\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:39.086962 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.084599    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-var-lib-openvswitch\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:39.086962 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.084614    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-host-run-netns\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:39.086962 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.084680    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-run-ovn\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:39.086962 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.084740    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/493a6ea0-265b-4330-8e11-b4911d3a3cb3-etc-kubernetes\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.086962 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.084782    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9e2fbc65-eba0-483e-ad62-d8f17b9bd452-os-release\") pod \"multus-additional-cni-plugins-bmkbw\" (UID: \"9e2fbc65-eba0-483e-ad62-d8f17b9bd452\") " pod="openshift-multus/multus-additional-cni-plugins-bmkbw"
Jul 28 10:07:39.086962 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.084832    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-host-run-ovn-kubernetes\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:39.086962 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.084862    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-host-cni-bin\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:39.086962 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.084881    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-host-run-k8s-cni-cncf-io\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:39.086962 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.084915    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/53d373c1-808b-4f32-ab54-d47bbeb3a151-host-run-ovn-kubernetes\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:39.086962 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.084915    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5f8ff51d-90c2-4146-b040-a71013fed650-multus-daemon-config\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:39.086962 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.084960    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0f2a806a-05a5-4fd1-be4f-be8b64a5013b-hosts-file\") pod \"node-resolver-hpntr\" (UID: \"0f2a806a-05a5-4fd1-be4f-be8b64a5013b\") " pod="openshift-dns/node-resolver-hpntr"
Jul 28 10:07:39.086962 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.084991    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f2a806a-05a5-4fd1-be4f-be8b64a5013b-tmp-dir\") pod \"node-resolver-hpntr\" (UID: \"0f2a806a-05a5-4fd1-be4f-be8b64a5013b\") " pod="openshift-dns/node-resolver-hpntr"
Jul 28 10:07:39.086962 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.085089    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/53d373c1-808b-4f32-ab54-d47bbeb3a151-env-overrides\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:39.086962 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.085104    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-host-run-k8s-cni-cncf-io\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:39.086962 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.085152    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c4838d1b-fa7d-4770-a990-66c67c14c8b9-konnectivity-ca\") pod \"konnectivity-agent-8bh2v\" (UID: \"c4838d1b-fa7d-4770-a990-66c67c14c8b9\") " pod="kube-system/konnectivity-agent-8bh2v"
Jul 28 10:07:39.087947 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.085363    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5f8ff51d-90c2-4146-b040-a71013fed650-multus-daemon-config\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:39.087947 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.085378    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5f8ff51d-90c2-4146-b040-a71013fed650-hostroot\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:39.089497 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.089468    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/53d373c1-808b-4f32-ab54-d47bbeb3a151-ovn-node-metrics-cert\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:39.089616 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.089589    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c4838d1b-fa7d-4770-a990-66c67c14c8b9-agent-certs\") pod \"konnectivity-agent-8bh2v\" (UID: \"c4838d1b-fa7d-4770-a990-66c67c14c8b9\") " pod="kube-system/konnectivity-agent-8bh2v"
Jul 28 10:07:39.093156 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.093124    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tpdx\" (UniqueName: \"kubernetes.io/projected/5f8ff51d-90c2-4146-b040-a71013fed650-kube-api-access-9tpdx\") pod \"multus-29wrb\" (UID: \"5f8ff51d-90c2-4146-b040-a71013fed650\") " pod="openshift-multus/multus-29wrb"
Jul 28 10:07:39.095555 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.095525    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxc65\" (UniqueName: \"kubernetes.io/projected/b5d7a916-bec0-44bd-945a-ead38981eedb-kube-api-access-sxc65\") pod \"aws-ebs-csi-driver-node-2h2b9\" (UID: \"b5d7a916-bec0-44bd-945a-ead38981eedb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2h2b9"
Jul 28 10:07:39.096445 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.096401    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rqpp\" (UniqueName: \"kubernetes.io/projected/53d373c1-808b-4f32-ab54-d47bbeb3a151-kube-api-access-2rqpp\") pod \"ovnkube-node-4sg48\" (UID: \"53d373c1-808b-4f32-ab54-d47bbeb3a151\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:39.096960 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.096935    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvdbd\" (UniqueName: \"kubernetes.io/projected/bfe1e00b-5d1b-4127-b270-abe1d44d3dd0-kube-api-access-zvdbd\") pod \"network-metrics-daemon-299qd\" (UID: \"bfe1e00b-5d1b-4127-b270-abe1d44d3dd0\") " pod="openshift-multus/network-metrics-daemon-299qd"
Jul 28 10:07:39.097039 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.096973    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w942v\" (UniqueName: \"kubernetes.io/projected/4b1075b5-7b36-420f-aa9f-90022e20d840-kube-api-access-w942v\") pod \"node-ca-d48vp\" (UID: \"4b1075b5-7b36-420f-aa9f-90022e20d840\") " pod="openshift-image-registry/node-ca-d48vp"
Jul 28 10:07:39.131904 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.131822    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-235.ec2.internal" event={"ID":"aef8770b7b9299f2dc08df0481ffe72d","Type":"ContainerStarted","Data":"39dd61f0e62ec95066ac48729aa32725b5847ec932c96e8d465be3e4b6025f45"}
Jul 28 10:07:39.134388 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.134351    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-235.ec2.internal" event={"ID":"b79e08bd50776ee222be14a701a056dc","Type":"ContainerStarted","Data":"3f2380db8dd14516cabb87b6ed3516f5737a223f5877ecdc592fc3257498113e"}
Jul 28 10:07:39.185347 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.185301    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgldr\" (UniqueName: \"kubernetes.io/projected/af6b006d-6148-4b30-a1e9-bd2e14118b06-kube-api-access-rgldr\") pod \"iptables-alerter-bzknt\" (UID: \"af6b006d-6148-4b30-a1e9-bd2e14118b06\") " pod="openshift-network-operator/iptables-alerter-bzknt"
Jul 28 10:07:39.185347 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.185356    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/493a6ea0-265b-4330-8e11-b4911d3a3cb3-var-lib-kubelet\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.185607 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.185455    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/493a6ea0-265b-4330-8e11-b4911d3a3cb3-var-lib-kubelet\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.185607 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.185572    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9e2fbc65-eba0-483e-ad62-d8f17b9bd452-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bmkbw\" (UID: \"9e2fbc65-eba0-483e-ad62-d8f17b9bd452\") " pod="openshift-multus/multus-additional-cni-plugins-bmkbw"
Jul 28 10:07:39.185692 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.185609    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/493a6ea0-265b-4330-8e11-b4911d3a3cb3-sys\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.185692 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.185638    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hk2h4\" (UniqueName: \"kubernetes.io/projected/0f2a806a-05a5-4fd1-be4f-be8b64a5013b-kube-api-access-hk2h4\") pod \"node-resolver-hpntr\" (UID: \"0f2a806a-05a5-4fd1-be4f-be8b64a5013b\") " pod="openshift-dns/node-resolver-hpntr"
Jul 28 10:07:39.185692 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.185667    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/493a6ea0-265b-4330-8e11-b4911d3a3cb3-host\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.185811 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.185691    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/493a6ea0-265b-4330-8e11-b4911d3a3cb3-tmp\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.185811 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.185723    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/493a6ea0-265b-4330-8e11-b4911d3a3cb3-etc-systemd\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.185811 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.185726    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/493a6ea0-265b-4330-8e11-b4911d3a3cb3-sys\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.185811 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.185748    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/493a6ea0-265b-4330-8e11-b4911d3a3cb3-lib-modules\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.185811 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.185771    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/493a6ea0-265b-4330-8e11-b4911d3a3cb3-etc-tuned\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.185999 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.185816    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/493a6ea0-265b-4330-8e11-b4911d3a3cb3-etc-modprobe-d\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.185999 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.185848    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/af6b006d-6148-4b30-a1e9-bd2e14118b06-iptables-alerter-script\") pod \"iptables-alerter-bzknt\" (UID: \"af6b006d-6148-4b30-a1e9-bd2e14118b06\") " pod="openshift-network-operator/iptables-alerter-bzknt"
Jul 28 10:07:39.185999 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.185873    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/af6b006d-6148-4b30-a1e9-bd2e14118b06-host-slash\") pod \"iptables-alerter-bzknt\" (UID: \"af6b006d-6148-4b30-a1e9-bd2e14118b06\") " pod="openshift-network-operator/iptables-alerter-bzknt"
Jul 28 10:07:39.185999 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.185912    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/493a6ea0-265b-4330-8e11-b4911d3a3cb3-etc-sysconfig\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.185999 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.185937    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/493a6ea0-265b-4330-8e11-b4911d3a3cb3-run\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.185999 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.185965    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/493a6ea0-265b-4330-8e11-b4911d3a3cb3-etc-sysctl-d\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.185999 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.185995    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e2fbc65-eba0-483e-ad62-d8f17b9bd452-system-cni-dir\") pod \"multus-additional-cni-plugins-bmkbw\" (UID: \"9e2fbc65-eba0-483e-ad62-d8f17b9bd452\") " pod="openshift-multus/multus-additional-cni-plugins-bmkbw"
Jul 28 10:07:39.186245 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.186021    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9e2fbc65-eba0-483e-ad62-d8f17b9bd452-cni-binary-copy\") pod \"multus-additional-cni-plugins-bmkbw\" (UID: \"9e2fbc65-eba0-483e-ad62-d8f17b9bd452\") " pod="openshift-multus/multus-additional-cni-plugins-bmkbw"
Jul 28 10:07:39.186245 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.186049    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/493a6ea0-265b-4330-8e11-b4911d3a3cb3-etc-sysctl-conf\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.186245 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.186078    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/493a6ea0-265b-4330-8e11-b4911d3a3cb3-etc-kubernetes\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.186245 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.186102    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9e2fbc65-eba0-483e-ad62-d8f17b9bd452-os-release\") pod \"multus-additional-cni-plugins-bmkbw\" (UID: \"9e2fbc65-eba0-483e-ad62-d8f17b9bd452\") " pod="openshift-multus/multus-additional-cni-plugins-bmkbw"
Jul 28 10:07:39.186245 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.186133    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0f2a806a-05a5-4fd1-be4f-be8b64a5013b-hosts-file\") pod \"node-resolver-hpntr\" (UID: \"0f2a806a-05a5-4fd1-be4f-be8b64a5013b\") " pod="openshift-dns/node-resolver-hpntr"
Jul 28 10:07:39.186245 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.186159    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f2a806a-05a5-4fd1-be4f-be8b64a5013b-tmp-dir\") pod \"node-resolver-hpntr\" (UID: \"0f2a806a-05a5-4fd1-be4f-be8b64a5013b\") " pod="openshift-dns/node-resolver-hpntr"
Jul 28 10:07:39.186245 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.186185    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxmcx\" (UniqueName: \"kubernetes.io/projected/598884b3-5fd8-42ae-87ee-4f24d6c4bdb3-kube-api-access-cxmcx\") pod \"network-check-target-cmrjd\" (UID: \"598884b3-5fd8-42ae-87ee-4f24d6c4bdb3\") " pod="openshift-network-diagnostics/network-check-target-cmrjd"
Jul 28 10:07:39.186245 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.186212    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9e2fbc65-eba0-483e-ad62-d8f17b9bd452-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bmkbw\" (UID: \"9e2fbc65-eba0-483e-ad62-d8f17b9bd452\") " pod="openshift-multus/multus-additional-cni-plugins-bmkbw"
Jul 28 10:07:39.186245 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.186239    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9e2fbc65-eba0-483e-ad62-d8f17b9bd452-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bmkbw\" (UID: \"9e2fbc65-eba0-483e-ad62-d8f17b9bd452\") " pod="openshift-multus/multus-additional-cni-plugins-bmkbw"
Jul 28 10:07:39.186690 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.186271    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2tmz\" (UniqueName: \"kubernetes.io/projected/9e2fbc65-eba0-483e-ad62-d8f17b9bd452-kube-api-access-r2tmz\") pod \"multus-additional-cni-plugins-bmkbw\" (UID: \"9e2fbc65-eba0-483e-ad62-d8f17b9bd452\") " pod="openshift-multus/multus-additional-cni-plugins-bmkbw"
Jul 28 10:07:39.186690 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.186305    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9e2fbc65-eba0-483e-ad62-d8f17b9bd452-cnibin\") pod \"multus-additional-cni-plugins-bmkbw\" (UID: \"9e2fbc65-eba0-483e-ad62-d8f17b9bd452\") " pod="openshift-multus/multus-additional-cni-plugins-bmkbw"
Jul 28 10:07:39.186690 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.186347    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85j4l\" (UniqueName: \"kubernetes.io/projected/493a6ea0-265b-4330-8e11-b4911d3a3cb3-kube-api-access-85j4l\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.186690 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.186390    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9e2fbc65-eba0-483e-ad62-d8f17b9bd452-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bmkbw\" (UID: \"9e2fbc65-eba0-483e-ad62-d8f17b9bd452\") " pod="openshift-multus/multus-additional-cni-plugins-bmkbw"
Jul 28 10:07:39.186690 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.186553    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0f2a806a-05a5-4fd1-be4f-be8b64a5013b-hosts-file\") pod \"node-resolver-hpntr\" (UID: \"0f2a806a-05a5-4fd1-be4f-be8b64a5013b\") " pod="openshift-dns/node-resolver-hpntr"
Jul 28 10:07:39.186690 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.186585    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/493a6ea0-265b-4330-8e11-b4911d3a3cb3-etc-sysctl-d\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.186690 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.186639    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/493a6ea0-265b-4330-8e11-b4911d3a3cb3-etc-kubernetes\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.186690 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.186661    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/493a6ea0-265b-4330-8e11-b4911d3a3cb3-etc-sysctl-conf\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.186690 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.186669    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/493a6ea0-265b-4330-8e11-b4911d3a3cb3-etc-modprobe-d\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.187126 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.186721    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9e2fbc65-eba0-483e-ad62-d8f17b9bd452-os-release\") pod \"multus-additional-cni-plugins-bmkbw\" (UID: \"9e2fbc65-eba0-483e-ad62-d8f17b9bd452\") " pod="openshift-multus/multus-additional-cni-plugins-bmkbw"
Jul 28 10:07:39.187126 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.186744    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9e2fbc65-eba0-483e-ad62-d8f17b9bd452-system-cni-dir\") pod \"multus-additional-cni-plugins-bmkbw\" (UID: \"9e2fbc65-eba0-483e-ad62-d8f17b9bd452\") " pod="openshift-multus/multus-additional-cni-plugins-bmkbw"
Jul 28 10:07:39.187126 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.186839    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f2a806a-05a5-4fd1-be4f-be8b64a5013b-tmp-dir\") pod \"node-resolver-hpntr\" (UID: \"0f2a806a-05a5-4fd1-be4f-be8b64a5013b\") " pod="openshift-dns/node-resolver-hpntr"
Jul 28 10:07:39.187280 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.187137    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/493a6ea0-265b-4330-8e11-b4911d3a3cb3-etc-sysconfig\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.187280 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.187167    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9e2fbc65-eba0-483e-ad62-d8f17b9bd452-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bmkbw\" (UID: \"9e2fbc65-eba0-483e-ad62-d8f17b9bd452\") " pod="openshift-multus/multus-additional-cni-plugins-bmkbw"
Jul 28 10:07:39.187280 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.187183    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/af6b006d-6148-4b30-a1e9-bd2e14118b06-iptables-alerter-script\") pod \"iptables-alerter-bzknt\" (UID: \"af6b006d-6148-4b30-a1e9-bd2e14118b06\") " pod="openshift-network-operator/iptables-alerter-bzknt"
Jul 28 10:07:39.187280 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.187198    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/af6b006d-6148-4b30-a1e9-bd2e14118b06-host-slash\") pod \"iptables-alerter-bzknt\" (UID: \"af6b006d-6148-4b30-a1e9-bd2e14118b06\") " pod="openshift-network-operator/iptables-alerter-bzknt"
Jul 28 10:07:39.187280 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.187204    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9e2fbc65-eba0-483e-ad62-d8f17b9bd452-cnibin\") pod \"multus-additional-cni-plugins-bmkbw\" (UID: \"9e2fbc65-eba0-483e-ad62-d8f17b9bd452\") " pod="openshift-multus/multus-additional-cni-plugins-bmkbw"
Jul 28 10:07:39.187280 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.187257    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/493a6ea0-265b-4330-8e11-b4911d3a3cb3-run\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.187663 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.187328    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/493a6ea0-265b-4330-8e11-b4911d3a3cb3-etc-systemd\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.187663 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.187457    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/493a6ea0-265b-4330-8e11-b4911d3a3cb3-lib-modules\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.187663 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.187500    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/493a6ea0-265b-4330-8e11-b4911d3a3cb3-host\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.188059 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.188014    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9e2fbc65-eba0-483e-ad62-d8f17b9bd452-cni-binary-copy\") pod \"multus-additional-cni-plugins-bmkbw\" (UID: \"9e2fbc65-eba0-483e-ad62-d8f17b9bd452\") " pod="openshift-multus/multus-additional-cni-plugins-bmkbw"
Jul 28 10:07:39.188059 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.188041    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9e2fbc65-eba0-483e-ad62-d8f17b9bd452-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bmkbw\" (UID: \"9e2fbc65-eba0-483e-ad62-d8f17b9bd452\") " pod="openshift-multus/multus-additional-cni-plugins-bmkbw"
Jul 28 10:07:39.189516 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.189487    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/493a6ea0-265b-4330-8e11-b4911d3a3cb3-etc-tuned\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.189976 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.189951    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/493a6ea0-265b-4330-8e11-b4911d3a3cb3-tmp\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.194742 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:39.194703    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 10:07:39.194742 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:39.194732    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 10:07:39.194742 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:39.194748    2563 projected.go:196] Error preparing data for projected volume kube-api-access-cxmcx for pod openshift-network-diagnostics/network-check-target-cmrjd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 10:07:39.194990 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:39.194826    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/598884b3-5fd8-42ae-87ee-4f24d6c4bdb3-kube-api-access-cxmcx podName:598884b3-5fd8-42ae-87ee-4f24d6c4bdb3 nodeName:}" failed. No retries permitted until 2026-07-28 10:07:39.694806069 +0000 UTC m=+3.136347404 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-cxmcx" (UniqueName: "kubernetes.io/projected/598884b3-5fd8-42ae-87ee-4f24d6c4bdb3-kube-api-access-cxmcx") pod "network-check-target-cmrjd" (UID: "598884b3-5fd8-42ae-87ee-4f24d6c4bdb3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 10:07:39.196010 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.195973    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85j4l\" (UniqueName: \"kubernetes.io/projected/493a6ea0-265b-4330-8e11-b4911d3a3cb3-kube-api-access-85j4l\") pod \"tuned-fmf4x\" (UID: \"493a6ea0-265b-4330-8e11-b4911d3a3cb3\") " pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.197001 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.196967    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hk2h4\" (UniqueName: \"kubernetes.io/projected/0f2a806a-05a5-4fd1-be4f-be8b64a5013b-kube-api-access-hk2h4\") pod \"node-resolver-hpntr\" (UID: \"0f2a806a-05a5-4fd1-be4f-be8b64a5013b\") " pod="openshift-dns/node-resolver-hpntr"
Jul 28 10:07:39.197336 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.197315    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgldr\" (UniqueName: \"kubernetes.io/projected/af6b006d-6148-4b30-a1e9-bd2e14118b06-kube-api-access-rgldr\") pod \"iptables-alerter-bzknt\" (UID: \"af6b006d-6148-4b30-a1e9-bd2e14118b06\") " pod="openshift-network-operator/iptables-alerter-bzknt"
Jul 28 10:07:39.197582 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.197555    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2tmz\" (UniqueName: \"kubernetes.io/projected/9e2fbc65-eba0-483e-ad62-d8f17b9bd452-kube-api-access-r2tmz\") pod \"multus-additional-cni-plugins-bmkbw\" (UID: \"9e2fbc65-eba0-483e-ad62-d8f17b9bd452\") " pod="openshift-multus/multus-additional-cni-plugins-bmkbw"
Jul 28 10:07:39.224085 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.223986    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 10:07:39.286173 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.286133    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-d48vp"
Jul 28 10:07:39.295121 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.295089    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8bh2v"
Jul 28 10:07:39.299280 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:39.299240    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b1075b5_7b36_420f_aa9f_90022e20d840.slice/crio-99145aa28fc86bf7b07108de6894f60c29cfaf7e08c62ab2ef232873e4cfe9e5 WatchSource:0}: Error finding container 99145aa28fc86bf7b07108de6894f60c29cfaf7e08c62ab2ef232873e4cfe9e5: Status 404 returned error can't find the container with id 99145aa28fc86bf7b07108de6894f60c29cfaf7e08c62ab2ef232873e4cfe9e5
Jul 28 10:07:39.307807 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.307782    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-29wrb"
Jul 28 10:07:39.308660 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:39.308621    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4838d1b_fa7d_4770_a990_66c67c14c8b9.slice/crio-499bfbd16acb3e35acb97b46685422e64d3fd29639c7b8e04962438a5e051e8a WatchSource:0}: Error finding container 499bfbd16acb3e35acb97b46685422e64d3fd29639c7b8e04962438a5e051e8a: Status 404 returned error can't find the container with id 499bfbd16acb3e35acb97b46685422e64d3fd29639c7b8e04962438a5e051e8a
Jul 28 10:07:39.316752 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.316529    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2h2b9"
Jul 28 10:07:39.320362 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:39.320333    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f8ff51d_90c2_4146_b040_a71013fed650.slice/crio-cfb14674e7e78e2292d01acc31e41aa743ae3f3b1e3df8bb7518b1732fdf9cd4 WatchSource:0}: Error finding container cfb14674e7e78e2292d01acc31e41aa743ae3f3b1e3df8bb7518b1732fdf9cd4: Status 404 returned error can't find the container with id cfb14674e7e78e2292d01acc31e41aa743ae3f3b1e3df8bb7518b1732fdf9cd4
Jul 28 10:07:39.324844 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.324817    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:07:39.327977 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:39.327937    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5d7a916_bec0_44bd_945a_ead38981eedb.slice/crio-f5f9ed2a10be7b369d37761418c7f0ecf165aaea7b6a4ef2fc4968c3e8dfe13c WatchSource:0}: Error finding container f5f9ed2a10be7b369d37761418c7f0ecf165aaea7b6a4ef2fc4968c3e8dfe13c: Status 404 returned error can't find the container with id f5f9ed2a10be7b369d37761418c7f0ecf165aaea7b6a4ef2fc4968c3e8dfe13c
Jul 28 10:07:39.332831 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.332810    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bmkbw"
Jul 28 10:07:39.336214 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:39.336186    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53d373c1_808b_4f32_ab54_d47bbeb3a151.slice/crio-e8a148e1051537d0f66f6079ed2c9f6e5c78858c2711f8a5b27e03e0ae41438d WatchSource:0}: Error finding container e8a148e1051537d0f66f6079ed2c9f6e5c78858c2711f8a5b27e03e0ae41438d: Status 404 returned error can't find the container with id e8a148e1051537d0f66f6079ed2c9f6e5c78858c2711f8a5b27e03e0ae41438d
Jul 28 10:07:39.348271 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.342239    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bzknt"
Jul 28 10:07:39.350747 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.350719    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fmf4x"
Jul 28 10:07:39.359550 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.359518    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hpntr"
Jul 28 10:07:39.360542 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:39.360510    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e2fbc65_eba0_483e_ad62_d8f17b9bd452.slice/crio-3d50b9e30b0a24eb72eb268302f25a08925e9eb7191d66eb9453c5391455e36f WatchSource:0}: Error finding container 3d50b9e30b0a24eb72eb268302f25a08925e9eb7191d66eb9453c5391455e36f: Status 404 returned error can't find the container with id 3d50b9e30b0a24eb72eb268302f25a08925e9eb7191d66eb9453c5391455e36f
Jul 28 10:07:39.367469 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:39.367425    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf6b006d_6148_4b30_a1e9_bd2e14118b06.slice/crio-5ae77ac050c23b25f2fd5f5d7764df8faa95a6b0d8c7213c9d1dd1dbf11c957a WatchSource:0}: Error finding container 5ae77ac050c23b25f2fd5f5d7764df8faa95a6b0d8c7213c9d1dd1dbf11c957a: Status 404 returned error can't find the container with id 5ae77ac050c23b25f2fd5f5d7764df8faa95a6b0d8c7213c9d1dd1dbf11c957a
Jul 28 10:07:39.372705 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:39.372643    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod493a6ea0_265b_4330_8e11_b4911d3a3cb3.slice/crio-9b864ea0f797ef053ba48733fed5672c9dcd4b7eb0fed61d758b23c45a37a10c WatchSource:0}: Error finding container 9b864ea0f797ef053ba48733fed5672c9dcd4b7eb0fed61d758b23c45a37a10c: Status 404 returned error can't find the container with id 9b864ea0f797ef053ba48733fed5672c9dcd4b7eb0fed61d758b23c45a37a10c
Jul 28 10:07:39.376874 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:07:39.376843    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f2a806a_05a5_4fd1_be4f_be8b64a5013b.slice/crio-94b3fba7b85b8158b51cc1d1f65e9c27bc3d0c3529baf9ba9d1474c6f588ee62 WatchSource:0}: Error finding container 94b3fba7b85b8158b51cc1d1f65e9c27bc3d0c3529baf9ba9d1474c6f588ee62: Status 404 returned error can't find the container with id 94b3fba7b85b8158b51cc1d1f65e9c27bc3d0c3529baf9ba9d1474c6f588ee62
Jul 28 10:07:39.588937 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.588892    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f9bc3b4-d2b8-40de-8c04-846085757396-original-pull-secret\") pod \"global-pull-secret-syncer-nz956\" (UID: \"5f9bc3b4-d2b8-40de-8c04-846085757396\") " pod="kube-system/global-pull-secret-syncer-nz956"
Jul 28 10:07:39.589140 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.588974    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfe1e00b-5d1b-4127-b270-abe1d44d3dd0-metrics-certs\") pod \"network-metrics-daemon-299qd\" (UID: \"bfe1e00b-5d1b-4127-b270-abe1d44d3dd0\") " pod="openshift-multus/network-metrics-daemon-299qd"
Jul 28 10:07:39.589223 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:39.589159    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 10:07:39.589223 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:39.589220    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfe1e00b-5d1b-4127-b270-abe1d44d3dd0-metrics-certs podName:bfe1e00b-5d1b-4127-b270-abe1d44d3dd0 nodeName:}" failed. No retries permitted until 2026-07-28 10:07:40.589203672 +0000 UTC m=+4.030744988 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bfe1e00b-5d1b-4127-b270-abe1d44d3dd0-metrics-certs") pod "network-metrics-daemon-299qd" (UID: "bfe1e00b-5d1b-4127-b270-abe1d44d3dd0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 10:07:39.589656 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:39.589628    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 10:07:39.589807 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:39.589690    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f9bc3b4-d2b8-40de-8c04-846085757396-original-pull-secret podName:5f9bc3b4-d2b8-40de-8c04-846085757396 nodeName:}" failed. No retries permitted until 2026-07-28 10:07:40.58967395 +0000 UTC m=+4.031215270 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f9bc3b4-d2b8-40de-8c04-846085757396-original-pull-secret") pod "global-pull-secret-syncer-nz956" (UID: "5f9bc3b4-d2b8-40de-8c04-846085757396") : object "kube-system"/"original-pull-secret" not registered
Jul 28 10:07:39.791288 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:39.791194    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxmcx\" (UniqueName: \"kubernetes.io/projected/598884b3-5fd8-42ae-87ee-4f24d6c4bdb3-kube-api-access-cxmcx\") pod \"network-check-target-cmrjd\" (UID: \"598884b3-5fd8-42ae-87ee-4f24d6c4bdb3\") " pod="openshift-network-diagnostics/network-check-target-cmrjd"
Jul 28 10:07:39.791475 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:39.791450    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 10:07:39.791561 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:39.791480    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 10:07:39.791561 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:39.791494    2563 projected.go:196] Error preparing data for projected volume kube-api-access-cxmcx for pod openshift-network-diagnostics/network-check-target-cmrjd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 10:07:39.791670 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:39.791571    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/598884b3-5fd8-42ae-87ee-4f24d6c4bdb3-kube-api-access-cxmcx podName:598884b3-5fd8-42ae-87ee-4f24d6c4bdb3 nodeName:}" failed. No retries permitted until 2026-07-28 10:07:40.791547769 +0000 UTC m=+4.233089101 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-cxmcx" (UniqueName: "kubernetes.io/projected/598884b3-5fd8-42ae-87ee-4f24d6c4bdb3-kube-api-access-cxmcx") pod "network-check-target-cmrjd" (UID: "598884b3-5fd8-42ae-87ee-4f24d6c4bdb3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 10:07:40.038895 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:40.038851    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 10:02:39 +0000 UTC" deadline="2027-12-21 22:00:29.215839902 +0000 UTC"
Jul 28 10:07:40.038895 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:40.038890    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12275h52m49.176952998s"
Jul 28 10:07:40.137754 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:40.137711    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bzknt" event={"ID":"af6b006d-6148-4b30-a1e9-bd2e14118b06","Type":"ContainerStarted","Data":"5ae77ac050c23b25f2fd5f5d7764df8faa95a6b0d8c7213c9d1dd1dbf11c957a"}
Jul 28 10:07:40.139128 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:40.139095    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bmkbw" event={"ID":"9e2fbc65-eba0-483e-ad62-d8f17b9bd452","Type":"ContainerStarted","Data":"3d50b9e30b0a24eb72eb268302f25a08925e9eb7191d66eb9453c5391455e36f"}
Jul 28 10:07:40.140402 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:40.140354    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8bh2v" event={"ID":"c4838d1b-fa7d-4770-a990-66c67c14c8b9","Type":"ContainerStarted","Data":"499bfbd16acb3e35acb97b46685422e64d3fd29639c7b8e04962438a5e051e8a"}
Jul 28 10:07:40.141609 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:40.141581    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-d48vp" event={"ID":"4b1075b5-7b36-420f-aa9f-90022e20d840","Type":"ContainerStarted","Data":"99145aa28fc86bf7b07108de6894f60c29cfaf7e08c62ab2ef232873e4cfe9e5"}
Jul 28 10:07:40.142914 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:40.142891    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hpntr" event={"ID":"0f2a806a-05a5-4fd1-be4f-be8b64a5013b","Type":"ContainerStarted","Data":"94b3fba7b85b8158b51cc1d1f65e9c27bc3d0c3529baf9ba9d1474c6f588ee62"}
Jul 28 10:07:40.144167 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:40.144140    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fmf4x" event={"ID":"493a6ea0-265b-4330-8e11-b4911d3a3cb3","Type":"ContainerStarted","Data":"9b864ea0f797ef053ba48733fed5672c9dcd4b7eb0fed61d758b23c45a37a10c"}
Jul 28 10:07:40.145455 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:40.145406    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4sg48" event={"ID":"53d373c1-808b-4f32-ab54-d47bbeb3a151","Type":"ContainerStarted","Data":"e8a148e1051537d0f66f6079ed2c9f6e5c78858c2711f8a5b27e03e0ae41438d"}
Jul 28 10:07:40.146682 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:40.146645    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2h2b9" event={"ID":"b5d7a916-bec0-44bd-945a-ead38981eedb","Type":"ContainerStarted","Data":"f5f9ed2a10be7b369d37761418c7f0ecf165aaea7b6a4ef2fc4968c3e8dfe13c"}
Jul 28 10:07:40.147784 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:40.147761    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-29wrb" event={"ID":"5f8ff51d-90c2-4146-b040-a71013fed650","Type":"ContainerStarted","Data":"cfb14674e7e78e2292d01acc31e41aa743ae3f3b1e3df8bb7518b1732fdf9cd4"}
Jul 28 10:07:40.599033 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:40.598125    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfe1e00b-5d1b-4127-b270-abe1d44d3dd0-metrics-certs\") pod \"network-metrics-daemon-299qd\" (UID: \"bfe1e00b-5d1b-4127-b270-abe1d44d3dd0\") " pod="openshift-multus/network-metrics-daemon-299qd"
Jul 28 10:07:40.599033 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:40.598223    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f9bc3b4-d2b8-40de-8c04-846085757396-original-pull-secret\") pod \"global-pull-secret-syncer-nz956\" (UID: \"5f9bc3b4-d2b8-40de-8c04-846085757396\") " pod="kube-system/global-pull-secret-syncer-nz956"
Jul 28 10:07:40.599033 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:40.598356    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 10:07:40.599033 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:40.598443    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f9bc3b4-d2b8-40de-8c04-846085757396-original-pull-secret podName:5f9bc3b4-d2b8-40de-8c04-846085757396 nodeName:}" failed. No retries permitted until 2026-07-28 10:07:42.598423707 +0000 UTC m=+6.039965046 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f9bc3b4-d2b8-40de-8c04-846085757396-original-pull-secret") pod "global-pull-secret-syncer-nz956" (UID: "5f9bc3b4-d2b8-40de-8c04-846085757396") : object "kube-system"/"original-pull-secret" not registered
Jul 28 10:07:40.599033 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:40.598583    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 10:07:40.599033 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:40.598624    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfe1e00b-5d1b-4127-b270-abe1d44d3dd0-metrics-certs podName:bfe1e00b-5d1b-4127-b270-abe1d44d3dd0 nodeName:}" failed. No retries permitted until 2026-07-28 10:07:42.598610066 +0000 UTC m=+6.040151389 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bfe1e00b-5d1b-4127-b270-abe1d44d3dd0-metrics-certs") pod "network-metrics-daemon-299qd" (UID: "bfe1e00b-5d1b-4127-b270-abe1d44d3dd0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 10:07:40.800076 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:40.800035    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxmcx\" (UniqueName: \"kubernetes.io/projected/598884b3-5fd8-42ae-87ee-4f24d6c4bdb3-kube-api-access-cxmcx\") pod \"network-check-target-cmrjd\" (UID: \"598884b3-5fd8-42ae-87ee-4f24d6c4bdb3\") " pod="openshift-network-diagnostics/network-check-target-cmrjd"
Jul 28 10:07:40.800270 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:40.800231    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 10:07:40.800270 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:40.800252    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 10:07:40.800270 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:40.800264    2563 projected.go:196] Error preparing data for projected volume kube-api-access-cxmcx for pod openshift-network-diagnostics/network-check-target-cmrjd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 10:07:40.800464 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:40.800327    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/598884b3-5fd8-42ae-87ee-4f24d6c4bdb3-kube-api-access-cxmcx podName:598884b3-5fd8-42ae-87ee-4f24d6c4bdb3 nodeName:}" failed. No retries permitted until 2026-07-28 10:07:42.800307407 +0000 UTC m=+6.241848727 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-cxmcx" (UniqueName: "kubernetes.io/projected/598884b3-5fd8-42ae-87ee-4f24d6c4bdb3-kube-api-access-cxmcx") pod "network-check-target-cmrjd" (UID: "598884b3-5fd8-42ae-87ee-4f24d6c4bdb3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 10:07:41.039156 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:41.039058    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 10:02:39 +0000 UTC" deadline="2028-01-11 13:34:33.106763708 +0000 UTC"
Jul 28 10:07:41.039156 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:41.039113    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12771h26m52.067655032s"
Jul 28 10:07:41.127666 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:41.127630    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nz956"
Jul 28 10:07:41.127810 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:41.127779    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nz956" podUID="5f9bc3b4-d2b8-40de-8c04-846085757396"
Jul 28 10:07:41.128216 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:41.128192    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-299qd"
Jul 28 10:07:41.128386 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:41.128343    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-299qd" podUID="bfe1e00b-5d1b-4127-b270-abe1d44d3dd0"
Jul 28 10:07:41.128471 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:41.128427    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmrjd"
Jul 28 10:07:41.128559 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:41.128537    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmrjd" podUID="598884b3-5fd8-42ae-87ee-4f24d6c4bdb3"
Jul 28 10:07:42.169451 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:42.168232    2563 generic.go:356] "Generic (PLEG): container finished" podID="aef8770b7b9299f2dc08df0481ffe72d" containerID="75cf7e253f2186bdb0038915b316e23933d6d59e69d69007dd592db864d23360" exitCode=0
Jul 28 10:07:42.169940 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:42.169348    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-235.ec2.internal" event={"ID":"aef8770b7b9299f2dc08df0481ffe72d","Type":"ContainerDied","Data":"75cf7e253f2186bdb0038915b316e23933d6d59e69d69007dd592db864d23360"}
Jul 28 10:07:42.173852 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:42.173821    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-235.ec2.internal" event={"ID":"b79e08bd50776ee222be14a701a056dc","Type":"ContainerStarted","Data":"71a3a69045cd0c15d1b317f249974df2d441016b0bcc7e8132123934187afd90"}
Jul 28 10:07:42.203280 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:42.203211    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-235.ec2.internal" podStartSLOduration=4.203188388 podStartE2EDuration="4.203188388s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 10:07:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:07:42.202739529 +0000 UTC m=+5.644280885" watchObservedRunningTime="2026-07-28 10:07:42.203188388 +0000 UTC m=+5.644729726"
Jul 28 10:07:42.616657 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:42.616621    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfe1e00b-5d1b-4127-b270-abe1d44d3dd0-metrics-certs\") pod \"network-metrics-daemon-299qd\" (UID: \"bfe1e00b-5d1b-4127-b270-abe1d44d3dd0\") " pod="openshift-multus/network-metrics-daemon-299qd"
Jul 28 10:07:42.616856 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:42.616703    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f9bc3b4-d2b8-40de-8c04-846085757396-original-pull-secret\") pod \"global-pull-secret-syncer-nz956\" (UID: \"5f9bc3b4-d2b8-40de-8c04-846085757396\") " pod="kube-system/global-pull-secret-syncer-nz956"
Jul 28 10:07:42.616856 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:42.616816    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 10:07:42.616963 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:42.616874    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f9bc3b4-d2b8-40de-8c04-846085757396-original-pull-secret podName:5f9bc3b4-d2b8-40de-8c04-846085757396 nodeName:}" failed. No retries permitted until 2026-07-28 10:07:46.616857165 +0000 UTC m=+10.058398492 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f9bc3b4-d2b8-40de-8c04-846085757396-original-pull-secret") pod "global-pull-secret-syncer-nz956" (UID: "5f9bc3b4-d2b8-40de-8c04-846085757396") : object "kube-system"/"original-pull-secret" not registered
Jul 28 10:07:42.617689 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:42.617513    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 10:07:42.617689 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:42.617569    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfe1e00b-5d1b-4127-b270-abe1d44d3dd0-metrics-certs podName:bfe1e00b-5d1b-4127-b270-abe1d44d3dd0 nodeName:}" failed. No retries permitted until 2026-07-28 10:07:46.617552876 +0000 UTC m=+10.059094214 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bfe1e00b-5d1b-4127-b270-abe1d44d3dd0-metrics-certs") pod "network-metrics-daemon-299qd" (UID: "bfe1e00b-5d1b-4127-b270-abe1d44d3dd0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 10:07:42.818220 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:42.818181    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxmcx\" (UniqueName: \"kubernetes.io/projected/598884b3-5fd8-42ae-87ee-4f24d6c4bdb3-kube-api-access-cxmcx\") pod \"network-check-target-cmrjd\" (UID: \"598884b3-5fd8-42ae-87ee-4f24d6c4bdb3\") " pod="openshift-network-diagnostics/network-check-target-cmrjd"
Jul 28 10:07:42.818418 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:42.818399    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 10:07:42.818418 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:42.818419    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 10:07:42.818542 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:42.818434    2563 projected.go:196] Error preparing data for projected volume kube-api-access-cxmcx for pod openshift-network-diagnostics/network-check-target-cmrjd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 10:07:42.818542 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:42.818494    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/598884b3-5fd8-42ae-87ee-4f24d6c4bdb3-kube-api-access-cxmcx podName:598884b3-5fd8-42ae-87ee-4f24d6c4bdb3 nodeName:}" failed. No retries permitted until 2026-07-28 10:07:46.81847489 +0000 UTC m=+10.260016224 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-cxmcx" (UniqueName: "kubernetes.io/projected/598884b3-5fd8-42ae-87ee-4f24d6c4bdb3-kube-api-access-cxmcx") pod "network-check-target-cmrjd" (UID: "598884b3-5fd8-42ae-87ee-4f24d6c4bdb3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 10:07:43.125495 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:43.125461    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nz956"
Jul 28 10:07:43.125688 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:43.125503    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmrjd"
Jul 28 10:07:43.125688 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:43.125597    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nz956" podUID="5f9bc3b4-d2b8-40de-8c04-846085757396"
Jul 28 10:07:43.125688 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:43.125471    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-299qd"
Jul 28 10:07:43.125857 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:43.125835    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-299qd" podUID="bfe1e00b-5d1b-4127-b270-abe1d44d3dd0"
Jul 28 10:07:43.125994 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:43.125915    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmrjd" podUID="598884b3-5fd8-42ae-87ee-4f24d6c4bdb3"
Jul 28 10:07:45.127310 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:45.127210    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nz956"
Jul 28 10:07:45.127837 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:45.127346    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nz956" podUID="5f9bc3b4-d2b8-40de-8c04-846085757396"
Jul 28 10:07:45.127837 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:45.127769    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-299qd"
Jul 28 10:07:45.127943 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:45.127879    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-299qd" podUID="bfe1e00b-5d1b-4127-b270-abe1d44d3dd0"
Jul 28 10:07:45.127995 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:45.127943    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmrjd"
Jul 28 10:07:45.128050 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:45.128023    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmrjd" podUID="598884b3-5fd8-42ae-87ee-4f24d6c4bdb3"
Jul 28 10:07:46.653746 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:46.653704    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfe1e00b-5d1b-4127-b270-abe1d44d3dd0-metrics-certs\") pod \"network-metrics-daemon-299qd\" (UID: \"bfe1e00b-5d1b-4127-b270-abe1d44d3dd0\") " pod="openshift-multus/network-metrics-daemon-299qd"
Jul 28 10:07:46.654232 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:46.653793    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f9bc3b4-d2b8-40de-8c04-846085757396-original-pull-secret\") pod \"global-pull-secret-syncer-nz956\" (UID: \"5f9bc3b4-d2b8-40de-8c04-846085757396\") " pod="kube-system/global-pull-secret-syncer-nz956"
Jul 28 10:07:46.654232 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:46.653963    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 10:07:46.654232 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:46.654036    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f9bc3b4-d2b8-40de-8c04-846085757396-original-pull-secret podName:5f9bc3b4-d2b8-40de-8c04-846085757396 nodeName:}" failed. No retries permitted until 2026-07-28 10:07:54.654010068 +0000 UTC m=+18.095551402 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f9bc3b4-d2b8-40de-8c04-846085757396-original-pull-secret") pod "global-pull-secret-syncer-nz956" (UID: "5f9bc3b4-d2b8-40de-8c04-846085757396") : object "kube-system"/"original-pull-secret" not registered
Jul 28 10:07:46.654497 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:46.654474    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 10:07:46.654585 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:46.654533    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfe1e00b-5d1b-4127-b270-abe1d44d3dd0-metrics-certs podName:bfe1e00b-5d1b-4127-b270-abe1d44d3dd0 nodeName:}" failed. No retries permitted until 2026-07-28 10:07:54.654517272 +0000 UTC m=+18.096058604 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bfe1e00b-5d1b-4127-b270-abe1d44d3dd0-metrics-certs") pod "network-metrics-daemon-299qd" (UID: "bfe1e00b-5d1b-4127-b270-abe1d44d3dd0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 10:07:46.855081 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:46.855038    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxmcx\" (UniqueName: \"kubernetes.io/projected/598884b3-5fd8-42ae-87ee-4f24d6c4bdb3-kube-api-access-cxmcx\") pod \"network-check-target-cmrjd\" (UID: \"598884b3-5fd8-42ae-87ee-4f24d6c4bdb3\") " pod="openshift-network-diagnostics/network-check-target-cmrjd"
Jul 28 10:07:46.855285 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:46.855251    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 10:07:46.855285 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:46.855269    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 10:07:46.855285 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:46.855278    2563 projected.go:196] Error preparing data for projected volume kube-api-access-cxmcx for pod openshift-network-diagnostics/network-check-target-cmrjd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 10:07:46.855486 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:46.855321    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/598884b3-5fd8-42ae-87ee-4f24d6c4bdb3-kube-api-access-cxmcx podName:598884b3-5fd8-42ae-87ee-4f24d6c4bdb3 nodeName:}" failed. No retries permitted until 2026-07-28 10:07:54.855308217 +0000 UTC m=+18.296849534 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-cxmcx" (UniqueName: "kubernetes.io/projected/598884b3-5fd8-42ae-87ee-4f24d6c4bdb3-kube-api-access-cxmcx") pod "network-check-target-cmrjd" (UID: "598884b3-5fd8-42ae-87ee-4f24d6c4bdb3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 10:07:47.125922 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:47.125234    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-299qd"
Jul 28 10:07:47.125922 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:47.125388    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-299qd" podUID="bfe1e00b-5d1b-4127-b270-abe1d44d3dd0"
Jul 28 10:07:47.125922 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:47.125781    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nz956"
Jul 28 10:07:47.125922 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:47.125876    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nz956" podUID="5f9bc3b4-d2b8-40de-8c04-846085757396"
Jul 28 10:07:47.126282 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:47.126033    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmrjd"
Jul 28 10:07:47.126282 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:47.126130    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmrjd" podUID="598884b3-5fd8-42ae-87ee-4f24d6c4bdb3"
Jul 28 10:07:49.127774 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:49.127738    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nz956"
Jul 28 10:07:49.128244 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:49.127874    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nz956" podUID="5f9bc3b4-d2b8-40de-8c04-846085757396"
Jul 28 10:07:49.128783 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:49.128401    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-299qd"
Jul 28 10:07:49.128783 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:49.128549    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-299qd" podUID="bfe1e00b-5d1b-4127-b270-abe1d44d3dd0"
Jul 28 10:07:49.128783 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:49.128617    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmrjd"
Jul 28 10:07:49.128783 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:49.128725    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmrjd" podUID="598884b3-5fd8-42ae-87ee-4f24d6c4bdb3"
Jul 28 10:07:51.124259 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:51.124220    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-299qd"
Jul 28 10:07:51.124259 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:51.124220    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nz956"
Jul 28 10:07:51.124812 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:51.124218    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmrjd"
Jul 28 10:07:51.124812 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:51.124422    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nz956" podUID="5f9bc3b4-d2b8-40de-8c04-846085757396"
Jul 28 10:07:51.124812 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:51.124498    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-299qd" podUID="bfe1e00b-5d1b-4127-b270-abe1d44d3dd0"
Jul 28 10:07:51.124812 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:51.124583    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmrjd" podUID="598884b3-5fd8-42ae-87ee-4f24d6c4bdb3"
Jul 28 10:07:53.124928 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:53.124893    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-299qd"
Jul 28 10:07:53.125391 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:53.124894    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nz956"
Jul 28 10:07:53.125391 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:53.125033    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-299qd" podUID="bfe1e00b-5d1b-4127-b270-abe1d44d3dd0"
Jul 28 10:07:53.125391 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:53.124893    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmrjd"
Jul 28 10:07:53.125391 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:53.125122    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nz956" podUID="5f9bc3b4-d2b8-40de-8c04-846085757396"
Jul 28 10:07:53.125391 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:53.125180    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmrjd" podUID="598884b3-5fd8-42ae-87ee-4f24d6c4bdb3"
Jul 28 10:07:54.716832 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:54.716791    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f9bc3b4-d2b8-40de-8c04-846085757396-original-pull-secret\") pod \"global-pull-secret-syncer-nz956\" (UID: \"5f9bc3b4-d2b8-40de-8c04-846085757396\") " pod="kube-system/global-pull-secret-syncer-nz956"
Jul 28 10:07:54.717488 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:54.716859    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfe1e00b-5d1b-4127-b270-abe1d44d3dd0-metrics-certs\") pod \"network-metrics-daemon-299qd\" (UID: \"bfe1e00b-5d1b-4127-b270-abe1d44d3dd0\") " pod="openshift-multus/network-metrics-daemon-299qd"
Jul 28 10:07:54.717488 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:54.716914    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 10:07:54.717488 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:54.716935    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 10:07:54.717488 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:54.716989    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f9bc3b4-d2b8-40de-8c04-846085757396-original-pull-secret podName:5f9bc3b4-d2b8-40de-8c04-846085757396 nodeName:}" failed. No retries permitted until 2026-07-28 10:08:10.716972233 +0000 UTC m=+34.158513562 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f9bc3b4-d2b8-40de-8c04-846085757396-original-pull-secret") pod "global-pull-secret-syncer-nz956" (UID: "5f9bc3b4-d2b8-40de-8c04-846085757396") : object "kube-system"/"original-pull-secret" not registered
Jul 28 10:07:54.717488 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:54.717006    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfe1e00b-5d1b-4127-b270-abe1d44d3dd0-metrics-certs podName:bfe1e00b-5d1b-4127-b270-abe1d44d3dd0 nodeName:}" failed. No retries permitted until 2026-07-28 10:08:10.716998346 +0000 UTC m=+34.158539663 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bfe1e00b-5d1b-4127-b270-abe1d44d3dd0-metrics-certs") pod "network-metrics-daemon-299qd" (UID: "bfe1e00b-5d1b-4127-b270-abe1d44d3dd0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 10:07:54.918137 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:54.918100    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxmcx\" (UniqueName: \"kubernetes.io/projected/598884b3-5fd8-42ae-87ee-4f24d6c4bdb3-kube-api-access-cxmcx\") pod \"network-check-target-cmrjd\" (UID: \"598884b3-5fd8-42ae-87ee-4f24d6c4bdb3\") " pod="openshift-network-diagnostics/network-check-target-cmrjd"
Jul 28 10:07:54.918408 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:54.918303    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 10:07:54.918408 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:54.918332    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 10:07:54.918408 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:54.918344    2563 projected.go:196] Error preparing data for projected volume kube-api-access-cxmcx for pod openshift-network-diagnostics/network-check-target-cmrjd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 10:07:54.918580 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:54.918429    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/598884b3-5fd8-42ae-87ee-4f24d6c4bdb3-kube-api-access-cxmcx podName:598884b3-5fd8-42ae-87ee-4f24d6c4bdb3 nodeName:}" failed. No retries permitted until 2026-07-28 10:08:10.918410263 +0000 UTC m=+34.359951585 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-cxmcx" (UniqueName: "kubernetes.io/projected/598884b3-5fd8-42ae-87ee-4f24d6c4bdb3-kube-api-access-cxmcx") pod "network-check-target-cmrjd" (UID: "598884b3-5fd8-42ae-87ee-4f24d6c4bdb3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 10:07:55.127097 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:55.127070    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-299qd"
Jul 28 10:07:55.127269 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:55.127179    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-299qd" podUID="bfe1e00b-5d1b-4127-b270-abe1d44d3dd0"
Jul 28 10:07:55.127269 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:55.127250    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmrjd"
Jul 28 10:07:55.127376 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:55.127345    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmrjd" podUID="598884b3-5fd8-42ae-87ee-4f24d6c4bdb3"
Jul 28 10:07:55.127425 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:55.127396    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nz956"
Jul 28 10:07:55.127465 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:55.127451    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nz956" podUID="5f9bc3b4-d2b8-40de-8c04-846085757396"
Jul 28 10:07:57.127643 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:57.127607    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmrjd"
Jul 28 10:07:57.127643 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:57.127633    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nz956"
Jul 28 10:07:57.128168 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:57.127607    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-299qd"
Jul 28 10:07:57.128168 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:57.127718    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmrjd" podUID="598884b3-5fd8-42ae-87ee-4f24d6c4bdb3"
Jul 28 10:07:57.128168 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:57.127846    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-299qd" podUID="bfe1e00b-5d1b-4127-b270-abe1d44d3dd0"
Jul 28 10:07:57.128168 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:57.127949    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nz956" podUID="5f9bc3b4-d2b8-40de-8c04-846085757396"
Jul 28 10:07:59.127051 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:59.126967    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nz956"
Jul 28 10:07:59.127299 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:59.127098    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nz956" podUID="5f9bc3b4-d2b8-40de-8c04-846085757396"
Jul 28 10:07:59.127868 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:59.127524    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-299qd"
Jul 28 10:07:59.127868 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:59.127639    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-299qd" podUID="bfe1e00b-5d1b-4127-b270-abe1d44d3dd0"
Jul 28 10:07:59.127868 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:59.127682    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmrjd"
Jul 28 10:07:59.127868 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:07:59.127741    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmrjd" podUID="598884b3-5fd8-42ae-87ee-4f24d6c4bdb3"
Jul 28 10:07:59.227544 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:59.226730    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fmf4x" event={"ID":"493a6ea0-265b-4330-8e11-b4911d3a3cb3","Type":"ContainerStarted","Data":"5fbe06f7401ee2de74272ea9d8edabe67d67d9a8ea1e0c4b348431a37c761772"}
Jul 28 10:07:59.235375 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:59.234534    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-235.ec2.internal" event={"ID":"aef8770b7b9299f2dc08df0481ffe72d","Type":"ContainerStarted","Data":"da7b82127094e1d688824d5e11f812c900b4c0c427e6e33387e5991de77a3cbe"}
Jul 28 10:07:59.275826 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:59.275744    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-235.ec2.internal" podStartSLOduration=21.275721679 podStartE2EDuration="21.275721679s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 10:07:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:07:59.275433273 +0000 UTC m=+22.716974632" watchObservedRunningTime="2026-07-28 10:07:59.275721679 +0000 UTC m=+22.717263016"
Jul 28 10:07:59.276846 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:07:59.276272    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-fmf4x" podStartSLOduration=2.583787653 podStartE2EDuration="22.276260385s" totalImagesPullingTime="19.692472732s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 10:07:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:07:59.253227186 +0000 UTC m=+22.694768504" watchObservedRunningTime="2026-07-28 10:07:59.276260385 +0000 UTC m=+22.717801722"
Jul 28 10:08:00.244504 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:00.244270    2563 generic.go:356] "Generic (PLEG): container finished" podID="9e2fbc65-eba0-483e-ad62-d8f17b9bd452" containerID="3c0db5afe28f0d67fcac2a373a9786d7920162204b04ef6514ccd685c3488f0f" exitCode=0
Jul 28 10:08:00.245239 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:00.244349    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bmkbw" event={"ID":"9e2fbc65-eba0-483e-ad62-d8f17b9bd452","Type":"ContainerDied","Data":"3c0db5afe28f0d67fcac2a373a9786d7920162204b04ef6514ccd685c3488f0f"}
Jul 28 10:08:00.245879 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:00.245841    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8bh2v" event={"ID":"c4838d1b-fa7d-4770-a990-66c67c14c8b9","Type":"ContainerStarted","Data":"4115befc3305e4c55cb263d9a5067a4a703a4f6291c2351f65ae60a48629f8ea"}
Jul 28 10:08:00.247212 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:00.247190    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-d48vp" event={"ID":"4b1075b5-7b36-420f-aa9f-90022e20d840","Type":"ContainerStarted","Data":"41ba90b47929a33d903dcbe6cdac483cc1441b4b2cf5f97fd7f71065d4e87d7c"}
Jul 28 10:08:00.248654 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:00.248631    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hpntr" event={"ID":"0f2a806a-05a5-4fd1-be4f-be8b64a5013b","Type":"ContainerStarted","Data":"5865ad8f5c728d0b72ab1d0ee9ae33c74113f79e819b110ffdebb80c2df2f4bc"}
Jul 28 10:08:00.251506 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:00.251487    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4sg48" event={"ID":"53d373c1-808b-4f32-ab54-d47bbeb3a151","Type":"ContainerStarted","Data":"e2e112ed1c0e22e51eef2a46a3338cae32cfc92b09248828ea81e39e59f04d43"}
Jul 28 10:08:00.251581 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:00.251554    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4sg48" event={"ID":"53d373c1-808b-4f32-ab54-d47bbeb3a151","Type":"ContainerStarted","Data":"4a16960716897bb433550b620a96ce90434ee205d4d5c532a1b1ca3f812f1f4a"}
Jul 28 10:08:00.251632 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:00.251602    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4sg48" event={"ID":"53d373c1-808b-4f32-ab54-d47bbeb3a151","Type":"ContainerStarted","Data":"b655a03baf5f6fef742a87f6f2f65f1bc99608927ce93768deed097006bebf97"}
Jul 28 10:08:00.251667 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:00.251636    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4sg48" event={"ID":"53d373c1-808b-4f32-ab54-d47bbeb3a151","Type":"ContainerStarted","Data":"ee440119c68ff3dbd64fa2fe48d56d6b1dc563035101d2c7ce6fde0b6ac4736b"}
Jul 28 10:08:00.251712 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:00.251687    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4sg48" event={"ID":"53d373c1-808b-4f32-ab54-d47bbeb3a151","Type":"ContainerStarted","Data":"59019f5964d46e9290a531d99bc2e5e1dbda9b63168c88753c5d0e20f59f2422"}
Jul 28 10:08:00.251755 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:00.251724    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4sg48" event={"ID":"53d373c1-808b-4f32-ab54-d47bbeb3a151","Type":"ContainerStarted","Data":"5bed137f33fdbbd59a901ff7ffad5cfd9341e78be42484a18f01ef7243fc7ff4"}
Jul 28 10:08:00.252817 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:00.252796    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2h2b9" event={"ID":"b5d7a916-bec0-44bd-945a-ead38981eedb","Type":"ContainerStarted","Data":"2ee12b0b80815693fa8a7c947caf2687c2b1c1a28a0cd202bd5daf17beebd0c9"}
Jul 28 10:08:00.254067 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:00.254046    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-29wrb" event={"ID":"5f8ff51d-90c2-4146-b040-a71013fed650","Type":"ContainerStarted","Data":"069950d17ccd9d23f79891376d2452b001c1b623ff0dbc3fe2101547dd3a5ae6"}
Jul 28 10:08:00.281885 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:00.281830    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8bh2v" podStartSLOduration=3.537483639 podStartE2EDuration="23.281814493s" totalImagesPullingTime="19.744330854s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:07:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:08:00.281455854 +0000 UTC m=+23.722997191" watchObservedRunningTime="2026-07-28 10:08:00.281814493 +0000 UTC m=+23.723355829"
Jul 28 10:08:00.300744 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:00.300697    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-29wrb" podStartSLOduration=3.488631142 podStartE2EDuration="23.300678938s" totalImagesPullingTime="19.812047796s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 10:07:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:08:00.300418468 +0000 UTC m=+23.741959825" watchObservedRunningTime="2026-07-28 10:08:00.300678938 +0000 UTC m=+23.742220280"
Jul 28 10:08:00.314157 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:00.314106    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-d48vp" podStartSLOduration=3.560996828 podStartE2EDuration="23.314090182s" totalImagesPullingTime="19.753093354s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 10:07:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:08:00.313770268 +0000 UTC m=+23.755311605" watchObservedRunningTime="2026-07-28 10:08:00.314090182 +0000 UTC m=+23.755631518"
Jul 28 10:08:00.328550 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:00.328500    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-hpntr" podStartSLOduration=3.651326586 podStartE2EDuration="23.328482629s" totalImagesPullingTime="19.677156043s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 10:07:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:08:00.327894501 +0000 UTC m=+23.769435859" watchObservedRunningTime="2026-07-28 10:08:00.328482629 +0000 UTC m=+23.770023966"
Jul 28 10:08:00.522857 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:00.522823    2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 28 10:08:01.042563 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:01.042453    2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T10:08:00.522850847Z","UUID":"327f6d97-1f9e-4c5a-b2d8-eb1d87027b5c","Handler":null,"Name":"","Endpoint":""}
Jul 28 10:08:01.044246 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:01.044214    2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 28 10:08:01.044412 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:01.044263    2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 28 10:08:01.125224 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:01.125177    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nz956"
Jul 28 10:08:01.125224 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:01.125220    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmrjd"
Jul 28 10:08:01.125540 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:01.125190    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-299qd"
Jul 28 10:08:01.125540 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:01.125331    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmrjd" podUID="598884b3-5fd8-42ae-87ee-4f24d6c4bdb3"
Jul 28 10:08:01.125540 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:01.125418    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nz956" podUID="5f9bc3b4-d2b8-40de-8c04-846085757396"
Jul 28 10:08:01.125698 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:01.125527    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-299qd" podUID="bfe1e00b-5d1b-4127-b270-abe1d44d3dd0"
Jul 28 10:08:01.258668 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:01.258628    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bzknt" event={"ID":"af6b006d-6148-4b30-a1e9-bd2e14118b06","Type":"ContainerStarted","Data":"0b0f56c24ab8b5035f7c420e10b1a67af2aa0a900512aa0fdf995cada86a51c0"}
Jul 28 10:08:01.260910 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:01.260871    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2h2b9" event={"ID":"b5d7a916-bec0-44bd-945a-ead38981eedb","Type":"ContainerStarted","Data":"1c3af92c4d0ee960dcd5598e5970a302ebc26cf0e92325cca97c3da254d7a9a8"}
Jul 28 10:08:01.274476 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:01.274427    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bzknt" podStartSLOduration=4.5901262 podStartE2EDuration="24.274412519s" totalImagesPullingTime="19.684286319s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 10:07:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:08:01.274311131 +0000 UTC m=+24.715852468" watchObservedRunningTime="2026-07-28 10:08:01.274412519 +0000 UTC m=+24.715953872"
Jul 28 10:08:01.677774 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:01.677718    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8bh2v"
Jul 28 10:08:01.678546 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:01.678328    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8bh2v"
Jul 28 10:08:02.267495 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:02.267412    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4sg48" event={"ID":"53d373c1-808b-4f32-ab54-d47bbeb3a151","Type":"ContainerStarted","Data":"71aa01e02a59a6f67c1ca8fc9053876e133cae8a3cca79385b9951ec927db2a6"}
Jul 28 10:08:02.270159 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:02.270114    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2h2b9" event={"ID":"b5d7a916-bec0-44bd-945a-ead38981eedb","Type":"ContainerStarted","Data":"373f08a599fee5b98e0af741bee9b5410cbcd18265e7bb81e1a47ea42d52622c"}
Jul 28 10:08:02.270706 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:02.270644    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8bh2v"
Jul 28 10:08:02.271149 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:02.271128    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8bh2v"
Jul 28 10:08:02.289283 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:02.289223    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2h2b9" podStartSLOduration=3.340562758 podStartE2EDuration="25.289202018s" totalImagesPullingTime="21.94863926s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 10:07:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:08:02.288885467 +0000 UTC m=+25.730426830" watchObservedRunningTime="2026-07-28 10:08:02.289202018 +0000 UTC m=+25.730743381"
Jul 28 10:08:03.125243 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:03.125199    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-299qd"
Jul 28 10:08:03.125451 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:03.125199    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmrjd"
Jul 28 10:08:03.125451 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:03.125374    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-299qd" podUID="bfe1e00b-5d1b-4127-b270-abe1d44d3dd0"
Jul 28 10:08:03.125550 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:03.125199    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nz956"
Jul 28 10:08:03.125550 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:03.125498    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmrjd" podUID="598884b3-5fd8-42ae-87ee-4f24d6c4bdb3"
Jul 28 10:08:03.125652 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:03.125614    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nz956" podUID="5f9bc3b4-d2b8-40de-8c04-846085757396"
Jul 28 10:08:05.125336 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:05.125156    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nz956"
Jul 28 10:08:05.126096 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:05.125177    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-299qd"
Jul 28 10:08:05.126096 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:05.125226    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmrjd"
Jul 28 10:08:05.126096 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:05.125479    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nz956" podUID="5f9bc3b4-d2b8-40de-8c04-846085757396"
Jul 28 10:08:05.126096 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:05.125602    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-299qd" podUID="bfe1e00b-5d1b-4127-b270-abe1d44d3dd0"
Jul 28 10:08:05.126096 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:05.125702    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmrjd" podUID="598884b3-5fd8-42ae-87ee-4f24d6c4bdb3"
Jul 28 10:08:05.280605 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:05.280527    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4sg48" event={"ID":"53d373c1-808b-4f32-ab54-d47bbeb3a151","Type":"ContainerStarted","Data":"e7726382f649451c8d27d21cf7654ba58fc924a92dcec2e50ebec51a74231e57"}
Jul 28 10:08:05.280944 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:05.280918    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:08:05.281065 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:05.280991    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:08:05.281065 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:05.281039    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:08:05.283180 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:05.283157    2563 generic.go:356] "Generic (PLEG): container finished" podID="9e2fbc65-eba0-483e-ad62-d8f17b9bd452" containerID="9a8cbefacf2f7ed0eecb9d009f0cb977242a43cdbcdda88b450b5cdd2d608bef" exitCode=0
Jul 28 10:08:05.283622 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:05.283192    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bmkbw" event={"ID":"9e2fbc65-eba0-483e-ad62-d8f17b9bd452","Type":"ContainerDied","Data":"9a8cbefacf2f7ed0eecb9d009f0cb977242a43cdbcdda88b450b5cdd2d608bef"}
Jul 28 10:08:05.298988 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:05.298815    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:08:05.299134 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:05.299069    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:08:05.307345 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:05.307299    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4sg48" podStartSLOduration=8.51131899 podStartE2EDuration="28.307284462s" totalImagesPullingTime="19.795965472s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 10:07:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:08:05.307099525 +0000 UTC m=+28.748640862" watchObservedRunningTime="2026-07-28 10:08:05.307284462 +0000 UTC m=+28.748825798"
Jul 28 10:08:07.126127 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:07.126087    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nz956"
Jul 28 10:08:07.126608 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:07.126186    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-299qd"
Jul 28 10:08:07.126608 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:07.126228    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmrjd"
Jul 28 10:08:07.126608 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:07.126245    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nz956" podUID="5f9bc3b4-d2b8-40de-8c04-846085757396"
Jul 28 10:08:07.126608 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:07.126299    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmrjd" podUID="598884b3-5fd8-42ae-87ee-4f24d6c4bdb3"
Jul 28 10:08:07.126608 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:07.126414    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-299qd" podUID="bfe1e00b-5d1b-4127-b270-abe1d44d3dd0"
Jul 28 10:08:07.252563 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:07.251019    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cmrjd"]
Jul 28 10:08:07.254099 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:07.253906    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nz956"]
Jul 28 10:08:07.255333 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:07.255288    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-299qd"]
Jul 28 10:08:07.291432 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:07.291397    2563 generic.go:356] "Generic (PLEG): container finished" podID="9e2fbc65-eba0-483e-ad62-d8f17b9bd452" containerID="1cdf940edb4e829ddd4fb1bca38d537a5d407bd6dfa51f0e9589945ad3230e8b" exitCode=0
Jul 28 10:08:07.291596 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:07.291561    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmrjd"
Jul 28 10:08:07.291596 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:07.291512    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bmkbw" event={"ID":"9e2fbc65-eba0-483e-ad62-d8f17b9bd452","Type":"ContainerDied","Data":"1cdf940edb4e829ddd4fb1bca38d537a5d407bd6dfa51f0e9589945ad3230e8b"}
Jul 28 10:08:07.291868 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:07.291841    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmrjd" podUID="598884b3-5fd8-42ae-87ee-4f24d6c4bdb3"
Jul 28 10:08:07.292166 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:07.292140    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-299qd"
Jul 28 10:08:07.292292 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:07.292253    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-299qd" podUID="bfe1e00b-5d1b-4127-b270-abe1d44d3dd0"
Jul 28 10:08:07.292292 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:07.292281    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nz956"
Jul 28 10:08:07.292478 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:07.292327    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nz956" podUID="5f9bc3b4-d2b8-40de-8c04-846085757396"
Jul 28 10:08:09.124443 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:09.124402    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-299qd"
Jul 28 10:08:09.124443 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:09.124428    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmrjd"
Jul 28 10:08:09.124971 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:09.124402    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nz956"
Jul 28 10:08:09.124971 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:09.124540    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-299qd" podUID="bfe1e00b-5d1b-4127-b270-abe1d44d3dd0"
Jul 28 10:08:09.124971 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:09.124645    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nz956" podUID="5f9bc3b4-d2b8-40de-8c04-846085757396"
Jul 28 10:08:09.124971 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:09.124719    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmrjd" podUID="598884b3-5fd8-42ae-87ee-4f24d6c4bdb3"
Jul 28 10:08:09.298036 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:09.297985    2563 generic.go:356] "Generic (PLEG): container finished" podID="9e2fbc65-eba0-483e-ad62-d8f17b9bd452" containerID="79bd5c88b88346888463d2634c9c537aeefd815ce776ba6599a76a12e55e2cfe" exitCode=0
Jul 28 10:08:09.298231 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:09.298048    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bmkbw" event={"ID":"9e2fbc65-eba0-483e-ad62-d8f17b9bd452","Type":"ContainerDied","Data":"79bd5c88b88346888463d2634c9c537aeefd815ce776ba6599a76a12e55e2cfe"}
Jul 28 10:08:10.739798 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:10.739705    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f9bc3b4-d2b8-40de-8c04-846085757396-original-pull-secret\") pod \"global-pull-secret-syncer-nz956\" (UID: \"5f9bc3b4-d2b8-40de-8c04-846085757396\") " pod="kube-system/global-pull-secret-syncer-nz956"
Jul 28 10:08:10.739798 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:10.739776    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfe1e00b-5d1b-4127-b270-abe1d44d3dd0-metrics-certs\") pod \"network-metrics-daemon-299qd\" (UID: \"bfe1e00b-5d1b-4127-b270-abe1d44d3dd0\") " pod="openshift-multus/network-metrics-daemon-299qd"
Jul 28 10:08:10.740250 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:10.739892    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 10:08:10.740250 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:10.739902    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 10:08:10.740250 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:10.739975    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f9bc3b4-d2b8-40de-8c04-846085757396-original-pull-secret podName:5f9bc3b4-d2b8-40de-8c04-846085757396 nodeName:}" failed. No retries permitted until 2026-07-28 10:08:42.739952217 +0000 UTC m=+66.181493532 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f9bc3b4-d2b8-40de-8c04-846085757396-original-pull-secret") pod "global-pull-secret-syncer-nz956" (UID: "5f9bc3b4-d2b8-40de-8c04-846085757396") : object "kube-system"/"original-pull-secret" not registered
Jul 28 10:08:10.740250 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:10.739997    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfe1e00b-5d1b-4127-b270-abe1d44d3dd0-metrics-certs podName:bfe1e00b-5d1b-4127-b270-abe1d44d3dd0 nodeName:}" failed. No retries permitted until 2026-07-28 10:08:42.739986373 +0000 UTC m=+66.181527698 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bfe1e00b-5d1b-4127-b270-abe1d44d3dd0-metrics-certs") pod "network-metrics-daemon-299qd" (UID: "bfe1e00b-5d1b-4127-b270-abe1d44d3dd0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 10:08:10.942110 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:10.942074    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxmcx\" (UniqueName: \"kubernetes.io/projected/598884b3-5fd8-42ae-87ee-4f24d6c4bdb3-kube-api-access-cxmcx\") pod \"network-check-target-cmrjd\" (UID: \"598884b3-5fd8-42ae-87ee-4f24d6c4bdb3\") " pod="openshift-network-diagnostics/network-check-target-cmrjd"
Jul 28 10:08:10.942377 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:10.942334    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 10:08:10.942522 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:10.942461    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 10:08:10.942522 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:10.942479    2563 projected.go:196] Error preparing data for projected volume kube-api-access-cxmcx for pod openshift-network-diagnostics/network-check-target-cmrjd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 10:08:10.942638 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:10.942548    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/598884b3-5fd8-42ae-87ee-4f24d6c4bdb3-kube-api-access-cxmcx podName:598884b3-5fd8-42ae-87ee-4f24d6c4bdb3 nodeName:}" failed. No retries permitted until 2026-07-28 10:08:42.942527768 +0000 UTC m=+66.384069105 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-cxmcx" (UniqueName: "kubernetes.io/projected/598884b3-5fd8-42ae-87ee-4f24d6c4bdb3-kube-api-access-cxmcx") pod "network-check-target-cmrjd" (UID: "598884b3-5fd8-42ae-87ee-4f24d6c4bdb3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 10:08:11.125033 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.125000    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-299qd"
Jul 28 10:08:11.125033 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.125030    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmrjd"
Jul 28 10:08:11.125253 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.125000    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nz956"
Jul 28 10:08:11.125253 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:11.125138    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-299qd" podUID="bfe1e00b-5d1b-4127-b270-abe1d44d3dd0"
Jul 28 10:08:11.125253 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:11.125182    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nz956" podUID="5f9bc3b4-d2b8-40de-8c04-846085757396"
Jul 28 10:08:11.125253 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:11.125246    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cmrjd" podUID="598884b3-5fd8-42ae-87ee-4f24d6c4bdb3"
Jul 28 10:08:11.923329 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.923027    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-235.ec2.internal" event="NodeReady"
Jul 28 10:08:11.923722 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.923433    2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 28 10:08:11.961581 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.961539    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-nljwq"]
Jul 28 10:08:11.964516 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.964165    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-nljwq"
Jul 28 10:08:11.967682 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.966415    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vjpcz"]
Jul 28 10:08:11.967682 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.967283    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 28 10:08:11.968558 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.967996    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-gsdrm\""
Jul 28 10:08:11.968558 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.968468    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 28 10:08:11.968836 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.968815    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-65b54959b5-rhznz"]
Jul 28 10:08:11.972374 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.970059    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vjpcz"
Jul 28 10:08:11.972374 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.971202    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-g7s47"]
Jul 28 10:08:11.972848 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.972820    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-65b54959b5-rhznz"
Jul 28 10:08:11.973411 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.973386    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-czhcl"]
Jul 28 10:08:11.974416 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.974395    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-g7s47"
Jul 28 10:08:11.975129 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.974780    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 28 10:08:11.975129 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.974801    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 28 10:08:11.975334 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.975266    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 28 10:08:11.976010 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.975977    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-m29b4\""
Jul 28 10:08:11.976097 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.976055    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 28 10:08:11.976166 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.976097    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 28 10:08:11.976166 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.976055    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 28 10:08:11.976475 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.976456    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-f68cp\""
Jul 28 10:08:11.976567 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.976499    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 28 10:08:11.976567 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.976530    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7f9765899f-nlhq4"]
Jul 28 10:08:11.976567 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.976461    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 28 10:08:11.976714 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.976597    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 28 10:08:11.976714 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.976612    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-czhcl"
Jul 28 10:08:11.977090 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.977072    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-phskb\""
Jul 28 10:08:11.977352 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.977334    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 28 10:08:11.977352 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.977348    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 28 10:08:11.978881 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.978859    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-r6ptx"]
Jul 28 10:08:11.979019 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.978977    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f9765899f-nlhq4"
Jul 28 10:08:11.981124 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.981102    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 28 10:08:11.981295 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.981279    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 28 10:08:11.981392 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.981375    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-x6bhh"]
Jul 28 10:08:11.981464 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.981395    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-xfpxp\""
Jul 28 10:08:11.981519 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.981503    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 28 10:08:11.981630 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.981612    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 28 10:08:11.981721 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.981704    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 28 10:08:11.981927 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.981908    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 28 10:08:11.982026 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.982006    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 28 10:08:11.985802 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.985100    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-fn5hp\""
Jul 28 10:08:11.986173 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.986155    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 28 10:08:11.990507 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.986770    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-r6ptx"
Jul 28 10:08:11.990507 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.986930    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gn75p"]
Jul 28 10:08:11.990507 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.987387    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-x6bhh"
Jul 28 10:08:11.993045 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.993022    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 28 10:08:11.994320 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.994292    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 28 10:08:11.994668 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.994645    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 28 10:08:11.994894 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.994867    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 28 10:08:11.994990 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.994931    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-zgtl6\""
Jul 28 10:08:11.995108 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.995090    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 28 10:08:11.995252 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.995236    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 28 10:08:11.995675 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.995654    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 28 10:08:11.996024 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.995876    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-6pzkk\""
Jul 28 10:08:11.996135 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.996119    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 28 10:08:11.996785 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.996765    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 28 10:08:11.997560 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.997537    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-nljwq"]
Jul 28 10:08:12.000561 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.997571    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vjpcz"]
Jul 28 10:08:12.000561 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.997590    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-czhcl"]
Jul 28 10:08:12.000561 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.997603    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-x6bhh"]
Jul 28 10:08:12.000561 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.997616    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-r6ptx"]
Jul 28 10:08:12.000561 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.997676    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-65b54959b5-rhznz"]
Jul 28 10:08:12.000561 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.997692    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gn75p"
Jul 28 10:08:12.000561 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.998658    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7f9765899f-nlhq4"]
Jul 28 10:08:12.000561 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:11.999700    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 28 10:08:12.000561 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.000270    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 28 10:08:12.000561 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.000297    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 28 10:08:12.001069 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.000803    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 28 10:08:12.001069 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.000855    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-v6m4n\""
Jul 28 10:08:12.002239 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.002179    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 28 10:08:12.005148 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.005118    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 28 10:08:12.005578 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.005550    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 28 10:08:12.006497 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.006476    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-287lq"]
Jul 28 10:08:12.009421 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.009394    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-287lq"
Jul 28 10:08:12.012216 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.012185    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 28 10:08:12.012338 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.012222    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-dfkl4\""
Jul 28 10:08:12.012494 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.012472    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 28 10:08:12.012628 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.012543    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 28 10:08:12.013650 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.013628    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-thhvn"]
Jul 28 10:08:12.017874 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.017848    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-g7s47"]
Jul 28 10:08:12.018053 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.017943    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-thhvn"
Jul 28 10:08:12.018127 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.018107    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gn75p"]
Jul 28 10:08:12.020149 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.020127    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 28 10:08:12.020269 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.020135    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dv6fr\""
Jul 28 10:08:12.020860 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.020834    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 28 10:08:12.024427 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.024331    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-287lq"]
Jul 28 10:08:12.026074 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.026009    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-thhvn"]
Jul 28 10:08:12.050792 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.050760    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25qt9\" (UniqueName: \"kubernetes.io/projected/9e06bdeb-9ead-430e-a448-950a1d18cc7a-kube-api-access-25qt9\") pod \"image-registry-7f9765899f-nlhq4\" (UID: \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\") " pod="openshift-image-registry/image-registry-7f9765899f-nlhq4"
Jul 28 10:08:12.050965 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.050805    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/70a2f95e-67d1-4353-b4d9-d59a5c23f970-default-certificate\") pod \"router-default-65b54959b5-rhznz\" (UID: \"70a2f95e-67d1-4353-b4d9-d59a5c23f970\") " pod="openshift-ingress/router-default-65b54959b5-rhznz"
Jul 28 10:08:12.050965 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.050889    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9e06bdeb-9ead-430e-a448-950a1d18cc7a-bound-sa-token\") pod \"image-registry-7f9765899f-nlhq4\" (UID: \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\") " pod="openshift-image-registry/image-registry-7f9765899f-nlhq4"
Jul 28 10:08:12.050965 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.050919    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f6411b5d-e39d-4922-a1fc-67c4952bbb01-tmp\") pod \"insights-operator-5885fdb759-r6ptx\" (UID: \"f6411b5d-e39d-4922-a1fc-67c4952bbb01\") " pod="openshift-insights/insights-operator-5885fdb759-r6ptx"
Jul 28 10:08:12.050965 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.050945    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9e06bdeb-9ead-430e-a448-950a1d18cc7a-registry-tls\") pod \"image-registry-7f9765899f-nlhq4\" (UID: \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\") " pod="openshift-image-registry/image-registry-7f9765899f-nlhq4"
Jul 28 10:08:12.051153 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.050970    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6411b5d-e39d-4922-a1fc-67c4952bbb01-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-r6ptx\" (UID: \"f6411b5d-e39d-4922-a1fc-67c4952bbb01\") " pod="openshift-insights/insights-operator-5885fdb759-r6ptx"
Jul 28 10:08:12.051153 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.051104    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9e06bdeb-9ead-430e-a448-950a1d18cc7a-image-registry-private-configuration\") pod \"image-registry-7f9765899f-nlhq4\" (UID: \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\") " pod="openshift-image-registry/image-registry-7f9765899f-nlhq4"
Jul 28 10:08:12.051220 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.051168    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8m26\" (UniqueName: \"kubernetes.io/projected/f8b50686-28d7-4749-9430-54bd0f88f2f2-kube-api-access-p8m26\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-czhcl\" (UID: \"f8b50686-28d7-4749-9430-54bd0f88f2f2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-czhcl"
Jul 28 10:08:12.051220 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.051197    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9e06bdeb-9ead-430e-a448-950a1d18cc7a-installation-pull-secrets\") pod \"image-registry-7f9765899f-nlhq4\" (UID: \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\") " pod="openshift-image-registry/image-registry-7f9765899f-nlhq4"
Jul 28 10:08:12.051292 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.051234    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70a2f95e-67d1-4353-b4d9-d59a5c23f970-service-ca-bundle\") pod \"router-default-65b54959b5-rhznz\" (UID: \"70a2f95e-67d1-4353-b4d9-d59a5c23f970\") " pod="openshift-ingress/router-default-65b54959b5-rhznz"
Jul 28 10:08:12.051292 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.051264    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6411b5d-e39d-4922-a1fc-67c4952bbb01-service-ca-bundle\") pod \"insights-operator-5885fdb759-r6ptx\" (UID: \"f6411b5d-e39d-4922-a1fc-67c4952bbb01\") " pod="openshift-insights/insights-operator-5885fdb759-r6ptx"
Jul 28 10:08:12.051375 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.051292    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsvx7\" (UniqueName: \"kubernetes.io/projected/7f4d862e-1588-468a-bfce-d45f2a036b02-kube-api-access-nsvx7\") pod \"volume-data-source-validator-549c6d54cf-nljwq\" (UID: \"7f4d862e-1588-468a-bfce-d45f2a036b02\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-nljwq"
Jul 28 10:08:12.051375 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.051317    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75t4j\" (UniqueName: \"kubernetes.io/projected/70a2f95e-67d1-4353-b4d9-d59a5c23f970-kube-api-access-75t4j\") pod \"router-default-65b54959b5-rhznz\" (UID: \"70a2f95e-67d1-4353-b4d9-d59a5c23f970\") " pod="openshift-ingress/router-default-65b54959b5-rhznz"
Jul 28 10:08:12.051375 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.051342    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f6411b5d-e39d-4922-a1fc-67c4952bbb01-serving-cert\") pod \"insights-operator-5885fdb759-r6ptx\" (UID: \"f6411b5d-e39d-4922-a1fc-67c4952bbb01\") " pod="openshift-insights/insights-operator-5885fdb759-r6ptx"
Jul 28 10:08:12.051489 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.051399    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9e06bdeb-9ead-430e-a448-950a1d18cc7a-registry-certificates\") pod \"image-registry-7f9765899f-nlhq4\" (UID: \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\") " pod="openshift-image-registry/image-registry-7f9765899f-nlhq4"
Jul 28 10:08:12.051489 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.051454    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70a2f95e-67d1-4353-b4d9-d59a5c23f970-metrics-certs\") pod \"router-default-65b54959b5-rhznz\" (UID: \"70a2f95e-67d1-4353-b4d9-d59a5c23f970\") " pod="openshift-ingress/router-default-65b54959b5-rhznz"
Jul 28 10:08:12.051489 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.051474    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/f6411b5d-e39d-4922-a1fc-67c4952bbb01-snapshots\") pod \"insights-operator-5885fdb759-r6ptx\" (UID: \"f6411b5d-e39d-4922-a1fc-67c4952bbb01\") " pod="openshift-insights/insights-operator-5885fdb759-r6ptx"
Jul 28 10:08:12.051571 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.051493    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkd8s\" (UniqueName: \"kubernetes.io/projected/94b8b045-295f-4194-a022-913c35c0529f-kube-api-access-zkd8s\") pod \"cluster-samples-operator-6748489656-vjpcz\" (UID: \"94b8b045-295f-4194-a022-913c35c0529f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vjpcz"
Jul 28 10:08:12.051571 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.051518    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f8b50686-28d7-4749-9430-54bd0f88f2f2-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-czhcl\" (UID: \"f8b50686-28d7-4749-9430-54bd0f88f2f2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-czhcl"
Jul 28 10:08:12.051571 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.051536    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/70a2f95e-67d1-4353-b4d9-d59a5c23f970-stats-auth\") pod \"router-default-65b54959b5-rhznz\" (UID: \"70a2f95e-67d1-4353-b4d9-d59a5c23f970\") " pod="openshift-ingress/router-default-65b54959b5-rhznz"
Jul 28 10:08:12.051571 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.051551    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bebcadc4-87cd-4b77-997e-bf1daa3bdfb5-config\") pod \"console-operator-584dcb6b54-g7s47\" (UID: \"bebcadc4-87cd-4b77-997e-bf1daa3bdfb5\") " pod="openshift-console-operator/console-operator-584dcb6b54-g7s47"
Jul 28 10:08:12.051677 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.051570    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bebcadc4-87cd-4b77-997e-bf1daa3bdfb5-trusted-ca\") pod \"console-operator-584dcb6b54-g7s47\" (UID: \"bebcadc4-87cd-4b77-997e-bf1daa3bdfb5\") " pod="openshift-console-operator/console-operator-584dcb6b54-g7s47"
Jul 28 10:08:12.051677 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.051594    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bebcadc4-87cd-4b77-997e-bf1daa3bdfb5-serving-cert\") pod \"console-operator-584dcb6b54-g7s47\" (UID: \"bebcadc4-87cd-4b77-997e-bf1daa3bdfb5\") " pod="openshift-console-operator/console-operator-584dcb6b54-g7s47"
Jul 28 10:08:12.051677 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.051619    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9e06bdeb-9ead-430e-a448-950a1d18cc7a-trusted-ca\") pod \"image-registry-7f9765899f-nlhq4\" (UID: \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\") " pod="openshift-image-registry/image-registry-7f9765899f-nlhq4"
Jul 28 10:08:12.051677 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.051650    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/94b8b045-295f-4194-a022-913c35c0529f-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vjpcz\" (UID: \"94b8b045-295f-4194-a022-913c35c0529f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vjpcz"
Jul 28 10:08:12.051677 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.051668    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzmpf\" (UniqueName: \"kubernetes.io/projected/bebcadc4-87cd-4b77-997e-bf1daa3bdfb5-kube-api-access-kzmpf\") pod \"console-operator-584dcb6b54-g7s47\" (UID: \"bebcadc4-87cd-4b77-997e-bf1daa3bdfb5\") " pod="openshift-console-operator/console-operator-584dcb6b54-g7s47"
Jul 28 10:08:12.051879 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.051687    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f8b50686-28d7-4749-9430-54bd0f88f2f2-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-czhcl\" (UID: \"f8b50686-28d7-4749-9430-54bd0f88f2f2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-czhcl"
Jul 28 10:08:12.051879 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.051702    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9e06bdeb-9ead-430e-a448-950a1d18cc7a-ca-trust-extracted\") pod \"image-registry-7f9765899f-nlhq4\" (UID: \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\") " pod="openshift-image-registry/image-registry-7f9765899f-nlhq4"
Jul 28 10:08:12.051879 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.051728    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2f8jk\" (UniqueName: \"kubernetes.io/projected/f6411b5d-e39d-4922-a1fc-67c4952bbb01-kube-api-access-2f8jk\") pod \"insights-operator-5885fdb759-r6ptx\" (UID: \"f6411b5d-e39d-4922-a1fc-67c4952bbb01\") " pod="openshift-insights/insights-operator-5885fdb759-r6ptx"
Jul 28 10:08:12.152650 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.152611    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/898e925d-8503-4a54-adef-e90195218e36-config\") pod \"service-ca-operator-5d6c75f84f-gn75p\" (UID: \"898e925d-8503-4a54-adef-e90195218e36\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gn75p"
Jul 28 10:08:12.152850 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.152663    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3014f727-2c79-465d-ba86-bbea763014e5-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-x6bhh\" (UID: \"3014f727-2c79-465d-ba86-bbea763014e5\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-x6bhh"
Jul 28 10:08:12.152850 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.152688    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/826225a0-88f8-4113-9f7d-05a75796d40d-config-volume\") pod \"dns-default-thhvn\" (UID: \"826225a0-88f8-4113-9f7d-05a75796d40d\") " pod="openshift-dns/dns-default-thhvn"
Jul 28 10:08:12.152850 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.152785    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70a2f95e-67d1-4353-b4d9-d59a5c23f970-service-ca-bundle\") pod \"router-default-65b54959b5-rhznz\" (UID: \"70a2f95e-67d1-4353-b4d9-d59a5c23f970\") " pod="openshift-ingress/router-default-65b54959b5-rhznz"
Jul 28 10:08:12.152850 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.152826    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6411b5d-e39d-4922-a1fc-67c4952bbb01-service-ca-bundle\") pod \"insights-operator-5885fdb759-r6ptx\" (UID: \"f6411b5d-e39d-4922-a1fc-67c4952bbb01\") " pod="openshift-insights/insights-operator-5885fdb759-r6ptx"
Jul 28 10:08:12.153049 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.152857    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsvx7\" (UniqueName: \"kubernetes.io/projected/7f4d862e-1588-468a-bfce-d45f2a036b02-kube-api-access-nsvx7\") pod \"volume-data-source-validator-549c6d54cf-nljwq\" (UID: \"7f4d862e-1588-468a-bfce-d45f2a036b02\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-nljwq"
Jul 28 10:08:12.153049 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.152885    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75t4j\" (UniqueName: \"kubernetes.io/projected/70a2f95e-67d1-4353-b4d9-d59a5c23f970-kube-api-access-75t4j\") pod \"router-default-65b54959b5-rhznz\" (UID: \"70a2f95e-67d1-4353-b4d9-d59a5c23f970\") " pod="openshift-ingress/router-default-65b54959b5-rhznz"
Jul 28 10:08:12.153049 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.152909    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f6411b5d-e39d-4922-a1fc-67c4952bbb01-serving-cert\") pod \"insights-operator-5885fdb759-r6ptx\" (UID: \"f6411b5d-e39d-4922-a1fc-67c4952bbb01\") " pod="openshift-insights/insights-operator-5885fdb759-r6ptx"
Jul 28 10:08:12.153049 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.152937    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/898e925d-8503-4a54-adef-e90195218e36-serving-cert\") pod \"service-ca-operator-5d6c75f84f-gn75p\" (UID: \"898e925d-8503-4a54-adef-e90195218e36\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gn75p"
Jul 28 10:08:12.153049 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.152970    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9e06bdeb-9ead-430e-a448-950a1d18cc7a-registry-certificates\") pod \"image-registry-7f9765899f-nlhq4\" (UID: \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\") " pod="openshift-image-registry/image-registry-7f9765899f-nlhq4"
Jul 28 10:08:12.153049 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:12.152989    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/70a2f95e-67d1-4353-b4d9-d59a5c23f970-service-ca-bundle podName:70a2f95e-67d1-4353-b4d9-d59a5c23f970 nodeName:}" failed. No retries permitted until 2026-07-28 10:08:12.652969446 +0000 UTC m=+36.094510764 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/70a2f95e-67d1-4353-b4d9-d59a5c23f970-service-ca-bundle") pod "router-default-65b54959b5-rhznz" (UID: "70a2f95e-67d1-4353-b4d9-d59a5c23f970") : configmap references non-existent config key: service-ca.crt
Jul 28 10:08:12.153049 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.153027    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70a2f95e-67d1-4353-b4d9-d59a5c23f970-metrics-certs\") pod \"router-default-65b54959b5-rhznz\" (UID: \"70a2f95e-67d1-4353-b4d9-d59a5c23f970\") " pod="openshift-ingress/router-default-65b54959b5-rhznz"
Jul 28 10:08:12.153049 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.153047    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/f6411b5d-e39d-4922-a1fc-67c4952bbb01-snapshots\") pod \"insights-operator-5885fdb759-r6ptx\" (UID: \"f6411b5d-e39d-4922-a1fc-67c4952bbb01\") " pod="openshift-insights/insights-operator-5885fdb759-r6ptx"
Jul 28 10:08:12.153471 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.153068    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkd8s\" (UniqueName: \"kubernetes.io/projected/94b8b045-295f-4194-a022-913c35c0529f-kube-api-access-zkd8s\") pod \"cluster-samples-operator-6748489656-vjpcz\" (UID: \"94b8b045-295f-4194-a022-913c35c0529f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vjpcz"
Jul 28 10:08:12.153471 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.153087    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f8b50686-28d7-4749-9430-54bd0f88f2f2-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-czhcl\" (UID: \"f8b50686-28d7-4749-9430-54bd0f88f2f2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-czhcl"
Jul 28 10:08:12.153471 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.153112    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3014f727-2c79-465d-ba86-bbea763014e5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-x6bhh\" (UID: \"3014f727-2c79-465d-ba86-bbea763014e5\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-x6bhh"
Jul 28 10:08:12.153471 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.153134    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/70a2f95e-67d1-4353-b4d9-d59a5c23f970-stats-auth\") pod \"router-default-65b54959b5-rhznz\" (UID: \"70a2f95e-67d1-4353-b4d9-d59a5c23f970\") " pod="openshift-ingress/router-default-65b54959b5-rhznz"
Jul 28 10:08:12.153471 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.153150    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bebcadc4-87cd-4b77-997e-bf1daa3bdfb5-config\") pod \"console-operator-584dcb6b54-g7s47\" (UID: \"bebcadc4-87cd-4b77-997e-bf1daa3bdfb5\") " pod="openshift-console-operator/console-operator-584dcb6b54-g7s47"
Jul 28 10:08:12.153471 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.153171    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bebcadc4-87cd-4b77-997e-bf1daa3bdfb5-trusted-ca\") pod \"console-operator-584dcb6b54-g7s47\" (UID: \"bebcadc4-87cd-4b77-997e-bf1daa3bdfb5\") " pod="openshift-console-operator/console-operator-584dcb6b54-g7s47"
Jul 28 10:08:12.153471 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.153190    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bebcadc4-87cd-4b77-997e-bf1daa3bdfb5-serving-cert\") pod \"console-operator-584dcb6b54-g7s47\" (UID: \"bebcadc4-87cd-4b77-997e-bf1daa3bdfb5\") " pod="openshift-console-operator/console-operator-584dcb6b54-g7s47"
Jul 28 10:08:12.153471 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.153207    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krwqj\" (UniqueName: \"kubernetes.io/projected/898e925d-8503-4a54-adef-e90195218e36-kube-api-access-krwqj\") pod \"service-ca-operator-5d6c75f84f-gn75p\" (UID: \"898e925d-8503-4a54-adef-e90195218e36\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gn75p"
Jul 28 10:08:12.153471 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.153228    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6813c6f-ee42-4551-af42-d4fd8af7cf98-cert\") pod \"ingress-canary-287lq\" (UID: \"e6813c6f-ee42-4551-af42-d4fd8af7cf98\") " pod="openshift-ingress-canary/ingress-canary-287lq"
Jul 28 10:08:12.153471 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.153247    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqvzc\" (UniqueName: \"kubernetes.io/projected/3014f727-2c79-465d-ba86-bbea763014e5-kube-api-access-qqvzc\") pod \"cluster-monitoring-operator-548b489cb8-x6bhh\" (UID: \"3014f727-2c79-465d-ba86-bbea763014e5\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-x6bhh"
Jul 28 10:08:12.153471 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.153266    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9e06bdeb-9ead-430e-a448-950a1d18cc7a-trusted-ca\") pod \"image-registry-7f9765899f-nlhq4\" (UID: \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\") " pod="openshift-image-registry/image-registry-7f9765899f-nlhq4"
Jul 28 10:08:12.153471 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.153284    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5whz\" (UniqueName: \"kubernetes.io/projected/826225a0-88f8-4113-9f7d-05a75796d40d-kube-api-access-h5whz\") pod \"dns-default-thhvn\" (UID: \"826225a0-88f8-4113-9f7d-05a75796d40d\") " pod="openshift-dns/dns-default-thhvn"
Jul 28 10:08:12.153471 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.153312    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/94b8b045-295f-4194-a022-913c35c0529f-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vjpcz\" (UID: \"94b8b045-295f-4194-a022-913c35c0529f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vjpcz"
Jul 28 10:08:12.153471 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.153331    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzmpf\" (UniqueName: \"kubernetes.io/projected/bebcadc4-87cd-4b77-997e-bf1daa3bdfb5-kube-api-access-kzmpf\") pod \"console-operator-584dcb6b54-g7s47\" (UID: \"bebcadc4-87cd-4b77-997e-bf1daa3bdfb5\") " pod="openshift-console-operator/console-operator-584dcb6b54-g7s47"
Jul 28 10:08:12.153471 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.153348    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/826225a0-88f8-4113-9f7d-05a75796d40d-metrics-tls\") pod \"dns-default-thhvn\" (UID: \"826225a0-88f8-4113-9f7d-05a75796d40d\") " pod="openshift-dns/dns-default-thhvn"
Jul 28 10:08:12.154162 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.153401    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f8b50686-28d7-4749-9430-54bd0f88f2f2-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-czhcl\" (UID: \"f8b50686-28d7-4749-9430-54bd0f88f2f2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-czhcl"
Jul 28 10:08:12.154162 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.153420    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9e06bdeb-9ead-430e-a448-950a1d18cc7a-ca-trust-extracted\") pod \"image-registry-7f9765899f-nlhq4\" (UID: \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\") " pod="openshift-image-registry/image-registry-7f9765899f-nlhq4"
Jul 28 10:08:12.154162 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.153452    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2f8jk\" (UniqueName: \"kubernetes.io/projected/f6411b5d-e39d-4922-a1fc-67c4952bbb01-kube-api-access-2f8jk\") pod \"insights-operator-5885fdb759-r6ptx\" (UID: \"f6411b5d-e39d-4922-a1fc-67c4952bbb01\") " pod="openshift-insights/insights-operator-5885fdb759-r6ptx"
Jul 28 10:08:12.154162 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.153453    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9e06bdeb-9ead-430e-a448-950a1d18cc7a-registry-certificates\") pod \"image-registry-7f9765899f-nlhq4\" (UID: \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\") " pod="openshift-image-registry/image-registry-7f9765899f-nlhq4"
Jul 28 10:08:12.154162 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.153474    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25qt9\" (UniqueName: \"kubernetes.io/projected/9e06bdeb-9ead-430e-a448-950a1d18cc7a-kube-api-access-25qt9\") pod \"image-registry-7f9765899f-nlhq4\" (UID: \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\") " pod="openshift-image-registry/image-registry-7f9765899f-nlhq4"
Jul 28 10:08:12.154162 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.153493    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/70a2f95e-67d1-4353-b4d9-d59a5c23f970-default-certificate\") pod \"router-default-65b54959b5-rhznz\" (UID: \"70a2f95e-67d1-4353-b4d9-d59a5c23f970\") " pod="openshift-ingress/router-default-65b54959b5-rhznz"
Jul 28 10:08:12.154162 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.153509    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/826225a0-88f8-4113-9f7d-05a75796d40d-tmp-dir\") pod \"dns-default-thhvn\" (UID: \"826225a0-88f8-4113-9f7d-05a75796d40d\") " pod="openshift-dns/dns-default-thhvn"
Jul 28 10:08:12.154162 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.153518    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6411b5d-e39d-4922-a1fc-67c4952bbb01-service-ca-bundle\") pod \"insights-operator-5885fdb759-r6ptx\" (UID: \"f6411b5d-e39d-4922-a1fc-67c4952bbb01\") " pod="openshift-insights/insights-operator-5885fdb759-r6ptx"
Jul 28 10:08:12.154162 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.153530    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9e06bdeb-9ead-430e-a448-950a1d18cc7a-bound-sa-token\") pod \"image-registry-7f9765899f-nlhq4\" (UID: \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\") " pod="openshift-image-registry/image-registry-7f9765899f-nlhq4"
Jul 28 10:08:12.154162 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.153546    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f6411b5d-e39d-4922-a1fc-67c4952bbb01-tmp\") pod \"insights-operator-5885fdb759-r6ptx\" (UID: \"f6411b5d-e39d-4922-a1fc-67c4952bbb01\") " pod="openshift-insights/insights-operator-5885fdb759-r6ptx"
Jul 28 10:08:12.154162 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.153565    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9e06bdeb-9ead-430e-a448-950a1d18cc7a-registry-tls\") pod \"image-registry-7f9765899f-nlhq4\" (UID: \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\") " pod="openshift-image-registry/image-registry-7f9765899f-nlhq4"
Jul 28 10:08:12.154162 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.153583    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6411b5d-e39d-4922-a1fc-67c4952bbb01-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-r6ptx\" (UID: \"f6411b5d-e39d-4922-a1fc-67c4952bbb01\") " pod="openshift-insights/insights-operator-5885fdb759-r6ptx"
Jul 28 10:08:12.154162 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.153610    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9e06bdeb-9ead-430e-a448-950a1d18cc7a-image-registry-private-configuration\") pod \"image-registry-7f9765899f-nlhq4\" (UID: \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\") " pod="openshift-image-registry/image-registry-7f9765899f-nlhq4"
Jul 28 10:08:12.154162 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.153629    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8m26\" (UniqueName: \"kubernetes.io/projected/f8b50686-28d7-4749-9430-54bd0f88f2f2-kube-api-access-p8m26\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-czhcl\" (UID: \"f8b50686-28d7-4749-9430-54bd0f88f2f2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-czhcl"
Jul 28 10:08:12.154162 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.153648    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9e06bdeb-9ead-430e-a448-950a1d18cc7a-installation-pull-secrets\") pod \"image-registry-7f9765899f-nlhq4\" (UID: \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\") " pod="openshift-image-registry/image-registry-7f9765899f-nlhq4"
Jul 28 10:08:12.154162 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.153664    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwpqk\" (UniqueName: \"kubernetes.io/projected/e6813c6f-ee42-4551-af42-d4fd8af7cf98-kube-api-access-vwpqk\") pod \"ingress-canary-287lq\" (UID: \"e6813c6f-ee42-4551-af42-d4fd8af7cf98\") " pod="openshift-ingress-canary/ingress-canary-287lq"
Jul 28 10:08:12.154819 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:12.153891    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 10:08:12.154819 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:12.153964    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70a2f95e-67d1-4353-b4d9-d59a5c23f970-metrics-certs podName:70a2f95e-67d1-4353-b4d9-d59a5c23f970 nodeName:}" failed. No retries permitted until 2026-07-28 10:08:12.65394137 +0000 UTC m=+36.095482700 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/70a2f95e-67d1-4353-b4d9-d59a5c23f970-metrics-certs") pod "router-default-65b54959b5-rhznz" (UID: "70a2f95e-67d1-4353-b4d9-d59a5c23f970") : secret "router-metrics-certs-default" not found
Jul 28 10:08:12.154819 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.154241    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bebcadc4-87cd-4b77-997e-bf1daa3bdfb5-trusted-ca\") pod \"console-operator-584dcb6b54-g7s47\" (UID: \"bebcadc4-87cd-4b77-997e-bf1daa3bdfb5\") " pod="openshift-console-operator/console-operator-584dcb6b54-g7s47"
Jul 28 10:08:12.154819 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.154286    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9e06bdeb-9ead-430e-a448-950a1d18cc7a-trusted-ca\") pod \"image-registry-7f9765899f-nlhq4\" (UID: \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\") " pod="openshift-image-registry/image-registry-7f9765899f-nlhq4"
Jul 28 10:08:12.154819 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:12.154386    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 10:08:12.154819 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:12.154443    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94b8b045-295f-4194-a022-913c35c0529f-samples-operator-tls podName:94b8b045-295f-4194-a022-913c35c0529f nodeName:}" failed. No retries permitted until 2026-07-28 10:08:12.654426644 +0000 UTC m=+36.095967959 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/94b8b045-295f-4194-a022-913c35c0529f-samples-operator-tls") pod "cluster-samples-operator-6748489656-vjpcz" (UID: "94b8b045-295f-4194-a022-913c35c0529f") : secret "samples-operator-tls" not found
Jul 28 10:08:12.154819 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.154583    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f6411b5d-e39d-4922-a1fc-67c4952bbb01-tmp\") pod \"insights-operator-5885fdb759-r6ptx\" (UID: \"f6411b5d-e39d-4922-a1fc-67c4952bbb01\") " pod="openshift-insights/insights-operator-5885fdb759-r6ptx"
Jul 28 10:08:12.154819 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.154638    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/f6411b5d-e39d-4922-a1fc-67c4952bbb01-snapshots\") pod \"insights-operator-5885fdb759-r6ptx\" (UID: \"f6411b5d-e39d-4922-a1fc-67c4952bbb01\") " pod="openshift-insights/insights-operator-5885fdb759-r6ptx"
Jul 28 10:08:12.154819 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:12.154678    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 10:08:12.154819 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:12.154694    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f9765899f-nlhq4: secret "image-registry-tls" not found
Jul 28 10:08:12.154819 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.154700    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9e06bdeb-9ead-430e-a448-950a1d18cc7a-ca-trust-extracted\") pod \"image-registry-7f9765899f-nlhq4\" (UID: \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\") " pod="openshift-image-registry/image-registry-7f9765899f-nlhq4"
Jul 28 10:08:12.154819 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:12.154727    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9e06bdeb-9ead-430e-a448-950a1d18cc7a-registry-tls podName:9e06bdeb-9ead-430e-a448-950a1d18cc7a nodeName:}" failed. No retries permitted until 2026-07-28 10:08:12.65471577 +0000 UTC m=+36.096257088 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9e06bdeb-9ead-430e-a448-950a1d18cc7a-registry-tls") pod "image-registry-7f9765899f-nlhq4" (UID: "9e06bdeb-9ead-430e-a448-950a1d18cc7a") : secret "image-registry-tls" not found
Jul 28 10:08:12.155285 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.154967    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f8b50686-28d7-4749-9430-54bd0f88f2f2-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-czhcl\" (UID: \"f8b50686-28d7-4749-9430-54bd0f88f2f2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-czhcl"
Jul 28 10:08:12.155558 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.155419    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f6411b5d-e39d-4922-a1fc-67c4952bbb01-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-r6ptx\" (UID: \"f6411b5d-e39d-4922-a1fc-67c4952bbb01\") " pod="openshift-insights/insights-operator-5885fdb759-r6ptx"
Jul 28 10:08:12.155959 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.155913    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bebcadc4-87cd-4b77-997e-bf1daa3bdfb5-config\") pod \"console-operator-584dcb6b54-g7s47\" (UID: \"bebcadc4-87cd-4b77-997e-bf1daa3bdfb5\") " pod="openshift-console-operator/console-operator-584dcb6b54-g7s47"
Jul 28 10:08:12.160346 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.159441    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f8b50686-28d7-4749-9430-54bd0f88f2f2-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-czhcl\" (UID: \"f8b50686-28d7-4749-9430-54bd0f88f2f2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-czhcl"
Jul 28 10:08:12.160346 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.159522    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9e06bdeb-9ead-430e-a448-950a1d18cc7a-installation-pull-secrets\") pod \"image-registry-7f9765899f-nlhq4\" (UID: \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\") " pod="openshift-image-registry/image-registry-7f9765899f-nlhq4"
Jul 28 10:08:12.160346 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.159539    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9e06bdeb-9ead-430e-a448-950a1d18cc7a-image-registry-private-configuration\") pod \"image-registry-7f9765899f-nlhq4\" (UID: \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\") " pod="openshift-image-registry/image-registry-7f9765899f-nlhq4"
Jul 28 10:08:12.160346 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.159768    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bebcadc4-87cd-4b77-997e-bf1daa3bdfb5-serving-cert\") pod \"console-operator-584dcb6b54-g7s47\" (UID: \"bebcadc4-87cd-4b77-997e-bf1daa3bdfb5\") " pod="openshift-console-operator/console-operator-584dcb6b54-g7s47"
Jul 28 10:08:12.160346 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.159894    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f6411b5d-e39d-4922-a1fc-67c4952bbb01-serving-cert\") pod \"insights-operator-5885fdb759-r6ptx\" (UID: \"f6411b5d-e39d-4922-a1fc-67c4952bbb01\") " pod="openshift-insights/insights-operator-5885fdb759-r6ptx"
Jul 28 10:08:12.160346 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.160283    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/70a2f95e-67d1-4353-b4d9-d59a5c23f970-stats-auth\") pod \"router-default-65b54959b5-rhznz\" (UID: \"70a2f95e-67d1-4353-b4d9-d59a5c23f970\") " pod="openshift-ingress/router-default-65b54959b5-rhznz"
Jul 28 10:08:12.161293 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.161265    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/70a2f95e-67d1-4353-b4d9-d59a5c23f970-default-certificate\") pod \"router-default-65b54959b5-rhznz\" (UID: \"70a2f95e-67d1-4353-b4d9-d59a5c23f970\") " pod="openshift-ingress/router-default-65b54959b5-rhznz"
Jul 28 10:08:12.166427 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.166324    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25qt9\" (UniqueName: \"kubernetes.io/projected/9e06bdeb-9ead-430e-a448-950a1d18cc7a-kube-api-access-25qt9\") pod \"image-registry-7f9765899f-nlhq4\" (UID: \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\") " pod="openshift-image-registry/image-registry-7f9765899f-nlhq4"
Jul 28 10:08:12.166559 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.166428    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsvx7\" (UniqueName: \"kubernetes.io/projected/7f4d862e-1588-468a-bfce-d45f2a036b02-kube-api-access-nsvx7\") pod \"volume-data-source-validator-549c6d54cf-nljwq\" (UID: \"7f4d862e-1588-468a-bfce-d45f2a036b02\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-nljwq"
Jul 28 10:08:12.166559 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.166477    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2f8jk\" (UniqueName: \"kubernetes.io/projected/f6411b5d-e39d-4922-a1fc-67c4952bbb01-kube-api-access-2f8jk\") pod \"insights-operator-5885fdb759-r6ptx\" (UID: \"f6411b5d-e39d-4922-a1fc-67c4952bbb01\") " pod="openshift-insights/insights-operator-5885fdb759-r6ptx"
Jul 28 10:08:12.167897 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.167848    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75t4j\" (UniqueName: \"kubernetes.io/projected/70a2f95e-67d1-4353-b4d9-d59a5c23f970-kube-api-access-75t4j\") pod \"router-default-65b54959b5-rhznz\" (UID: \"70a2f95e-67d1-4353-b4d9-d59a5c23f970\") " pod="openshift-ingress/router-default-65b54959b5-rhznz"
Jul 28 10:08:12.168041 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.167920    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkd8s\" (UniqueName: \"kubernetes.io/projected/94b8b045-295f-4194-a022-913c35c0529f-kube-api-access-zkd8s\") pod \"cluster-samples-operator-6748489656-vjpcz\" (UID: \"94b8b045-295f-4194-a022-913c35c0529f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vjpcz"
Jul 28 10:08:12.168191 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.168166    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8m26\" (UniqueName: \"kubernetes.io/projected/f8b50686-28d7-4749-9430-54bd0f88f2f2-kube-api-access-p8m26\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-czhcl\" (UID: \"f8b50686-28d7-4749-9430-54bd0f88f2f2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-czhcl"
Jul 28 10:08:12.168671 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.168629    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzmpf\" (UniqueName: \"kubernetes.io/projected/bebcadc4-87cd-4b77-997e-bf1daa3bdfb5-kube-api-access-kzmpf\") pod \"console-operator-584dcb6b54-g7s47\" (UID: \"bebcadc4-87cd-4b77-997e-bf1daa3bdfb5\") " pod="openshift-console-operator/console-operator-584dcb6b54-g7s47"
Jul 28 10:08:12.168967 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.168943    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9e06bdeb-9ead-430e-a448-950a1d18cc7a-bound-sa-token\") pod \"image-registry-7f9765899f-nlhq4\" (UID: \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\") " pod="openshift-image-registry/image-registry-7f9765899f-nlhq4"
Jul 28 10:08:12.254866 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.254777    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/898e925d-8503-4a54-adef-e90195218e36-serving-cert\") pod \"service-ca-operator-5d6c75f84f-gn75p\" (UID: \"898e925d-8503-4a54-adef-e90195218e36\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gn75p"
Jul 28 10:08:12.254866 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.254857    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3014f727-2c79-465d-ba86-bbea763014e5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-x6bhh\" (UID: \"3014f727-2c79-465d-ba86-bbea763014e5\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-x6bhh"
Jul 28 10:08:12.255071 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.254899    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krwqj\" (UniqueName: \"kubernetes.io/projected/898e925d-8503-4a54-adef-e90195218e36-kube-api-access-krwqj\") pod \"service-ca-operator-5d6c75f84f-gn75p\" (UID: \"898e925d-8503-4a54-adef-e90195218e36\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gn75p"
Jul 28 10:08:12.255071 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.254927    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6813c6f-ee42-4551-af42-d4fd8af7cf98-cert\") pod \"ingress-canary-287lq\" (UID: \"e6813c6f-ee42-4551-af42-d4fd8af7cf98\") " pod="openshift-ingress-canary/ingress-canary-287lq"
Jul 28 10:08:12.255071 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.254953    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqvzc\" (UniqueName: \"kubernetes.io/projected/3014f727-2c79-465d-ba86-bbea763014e5-kube-api-access-qqvzc\") pod \"cluster-monitoring-operator-548b489cb8-x6bhh\" (UID: \"3014f727-2c79-465d-ba86-bbea763014e5\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-x6bhh"
Jul 28 10:08:12.255071 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.254985    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5whz\" (UniqueName: \"kubernetes.io/projected/826225a0-88f8-4113-9f7d-05a75796d40d-kube-api-access-h5whz\") pod \"dns-default-thhvn\" (UID: \"826225a0-88f8-4113-9f7d-05a75796d40d\") " pod="openshift-dns/dns-default-thhvn"
Jul 28 10:08:12.255071 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:12.255017    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 10:08:12.255071 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.255030    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/826225a0-88f8-4113-9f7d-05a75796d40d-metrics-tls\") pod \"dns-default-thhvn\" (UID: \"826225a0-88f8-4113-9f7d-05a75796d40d\") " pod="openshift-dns/dns-default-thhvn"
Jul 28 10:08:12.255280 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.255073    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/826225a0-88f8-4113-9f7d-05a75796d40d-tmp-dir\") pod \"dns-default-thhvn\" (UID: \"826225a0-88f8-4113-9f7d-05a75796d40d\") " pod="openshift-dns/dns-default-thhvn"
Jul 28 10:08:12.255280 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:12.255091    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3014f727-2c79-465d-ba86-bbea763014e5-cluster-monitoring-operator-tls podName:3014f727-2c79-465d-ba86-bbea763014e5 nodeName:}" failed. No retries permitted until 2026-07-28 10:08:12.755074667 +0000 UTC m=+36.196615984 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3014f727-2c79-465d-ba86-bbea763014e5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-x6bhh" (UID: "3014f727-2c79-465d-ba86-bbea763014e5") : secret "cluster-monitoring-operator-tls" not found
Jul 28 10:08:12.255280 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.255170    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwpqk\" (UniqueName: \"kubernetes.io/projected/e6813c6f-ee42-4551-af42-d4fd8af7cf98-kube-api-access-vwpqk\") pod \"ingress-canary-287lq\" (UID: \"e6813c6f-ee42-4551-af42-d4fd8af7cf98\") " pod="openshift-ingress-canary/ingress-canary-287lq"
Jul 28 10:08:12.255280 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.255200    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/898e925d-8503-4a54-adef-e90195218e36-config\") pod \"service-ca-operator-5d6c75f84f-gn75p\" (UID: \"898e925d-8503-4a54-adef-e90195218e36\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gn75p"
Jul 28 10:08:12.255280 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.255228    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3014f727-2c79-465d-ba86-bbea763014e5-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-x6bhh\" (UID: \"3014f727-2c79-465d-ba86-bbea763014e5\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-x6bhh"
Jul 28 10:08:12.255280 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.255249    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/826225a0-88f8-4113-9f7d-05a75796d40d-config-volume\") pod \"dns-default-thhvn\" (UID: \"826225a0-88f8-4113-9f7d-05a75796d40d\") " pod="openshift-dns/dns-default-thhvn"
Jul 28 10:08:12.255596 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.255331    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/826225a0-88f8-4113-9f7d-05a75796d40d-tmp-dir\") pod \"dns-default-thhvn\" (UID: \"826225a0-88f8-4113-9f7d-05a75796d40d\") " pod="openshift-dns/dns-default-thhvn"
Jul 28 10:08:12.255835 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.255803    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/826225a0-88f8-4113-9f7d-05a75796d40d-config-volume\") pod \"dns-default-thhvn\" (UID: \"826225a0-88f8-4113-9f7d-05a75796d40d\") " pod="openshift-dns/dns-default-thhvn"
Jul 28 10:08:12.256857 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.256816    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3014f727-2c79-465d-ba86-bbea763014e5-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-x6bhh\" (UID: \"3014f727-2c79-465d-ba86-bbea763014e5\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-x6bhh"
Jul 28 10:08:12.257117 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:12.257094    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 10:08:12.257216 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:12.257170    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/826225a0-88f8-4113-9f7d-05a75796d40d-metrics-tls podName:826225a0-88f8-4113-9f7d-05a75796d40d nodeName:}" failed. No retries permitted until 2026-07-28 10:08:12.757153733 +0000 UTC m=+36.198695062 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/826225a0-88f8-4113-9f7d-05a75796d40d-metrics-tls") pod "dns-default-thhvn" (UID: "826225a0-88f8-4113-9f7d-05a75796d40d") : secret "dns-default-metrics-tls" not found
Jul 28 10:08:12.257384 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.257330    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/898e925d-8503-4a54-adef-e90195218e36-config\") pod \"service-ca-operator-5d6c75f84f-gn75p\" (UID: \"898e925d-8503-4a54-adef-e90195218e36\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gn75p"
Jul 28 10:08:12.257815 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.257793    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/898e925d-8503-4a54-adef-e90195218e36-serving-cert\") pod \"service-ca-operator-5d6c75f84f-gn75p\" (UID: \"898e925d-8503-4a54-adef-e90195218e36\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gn75p"
Jul 28 10:08:12.258500 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.258478    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6813c6f-ee42-4551-af42-d4fd8af7cf98-cert\") pod \"ingress-canary-287lq\" (UID: \"e6813c6f-ee42-4551-af42-d4fd8af7cf98\") " pod="openshift-ingress-canary/ingress-canary-287lq"
Jul 28 10:08:12.265947 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.265915    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5whz\" (UniqueName: \"kubernetes.io/projected/826225a0-88f8-4113-9f7d-05a75796d40d-kube-api-access-h5whz\") pod \"dns-default-thhvn\" (UID: \"826225a0-88f8-4113-9f7d-05a75796d40d\") " pod="openshift-dns/dns-default-thhvn"
Jul 28 10:08:12.266122 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.265915    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krwqj\" (UniqueName: \"kubernetes.io/projected/898e925d-8503-4a54-adef-e90195218e36-kube-api-access-krwqj\") pod \"service-ca-operator-5d6c75f84f-gn75p\" (UID: \"898e925d-8503-4a54-adef-e90195218e36\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gn75p"
Jul 28 10:08:12.266122 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.266108    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwpqk\" (UniqueName: \"kubernetes.io/projected/e6813c6f-ee42-4551-af42-d4fd8af7cf98-kube-api-access-vwpqk\") pod \"ingress-canary-287lq\" (UID: \"e6813c6f-ee42-4551-af42-d4fd8af7cf98\") " pod="openshift-ingress-canary/ingress-canary-287lq"
Jul 28 10:08:12.266890 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.266865    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqvzc\" (UniqueName: \"kubernetes.io/projected/3014f727-2c79-465d-ba86-bbea763014e5-kube-api-access-qqvzc\") pod \"cluster-monitoring-operator-548b489cb8-x6bhh\" (UID: \"3014f727-2c79-465d-ba86-bbea763014e5\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-x6bhh"
Jul 28 10:08:12.285577 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.285542    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-nljwq"
Jul 28 10:08:12.320182 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.320144    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-g7s47"
Jul 28 10:08:12.328302 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.328271    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-czhcl"
Jul 28 10:08:12.344470 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.344431    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-r6ptx"
Jul 28 10:08:12.359628 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.359591    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gn75p"
Jul 28 10:08:12.384601 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.384562    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-287lq"
Jul 28 10:08:12.658835 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.658791    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9e06bdeb-9ead-430e-a448-950a1d18cc7a-registry-tls\") pod \"image-registry-7f9765899f-nlhq4\" (UID: \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\") " pod="openshift-image-registry/image-registry-7f9765899f-nlhq4"
Jul 28 10:08:12.658990 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.658871    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70a2f95e-67d1-4353-b4d9-d59a5c23f970-service-ca-bundle\") pod \"router-default-65b54959b5-rhznz\" (UID: \"70a2f95e-67d1-4353-b4d9-d59a5c23f970\") " pod="openshift-ingress/router-default-65b54959b5-rhznz"
Jul 28 10:08:12.658990 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:12.658908    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 10:08:12.658990 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:12.658924    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f9765899f-nlhq4: secret "image-registry-tls" not found
Jul 28 10:08:12.658990 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:12.658973    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 10:08:12.658990 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:12.658984    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9e06bdeb-9ead-430e-a448-950a1d18cc7a-registry-tls podName:9e06bdeb-9ead-430e-a448-950a1d18cc7a nodeName:}" failed. No retries permitted until 2026-07-28 10:08:13.658964972 +0000 UTC m=+37.100506294 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9e06bdeb-9ead-430e-a448-950a1d18cc7a-registry-tls") pod "image-registry-7f9765899f-nlhq4" (UID: "9e06bdeb-9ead-430e-a448-950a1d18cc7a") : secret "image-registry-tls" not found
Jul 28 10:08:12.659163 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:12.659011    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70a2f95e-67d1-4353-b4d9-d59a5c23f970-metrics-certs podName:70a2f95e-67d1-4353-b4d9-d59a5c23f970 nodeName:}" failed. No retries permitted until 2026-07-28 10:08:13.659000134 +0000 UTC m=+37.100541456 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/70a2f95e-67d1-4353-b4d9-d59a5c23f970-metrics-certs") pod "router-default-65b54959b5-rhznz" (UID: "70a2f95e-67d1-4353-b4d9-d59a5c23f970") : secret "router-metrics-certs-default" not found
Jul 28 10:08:12.659163 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:12.659071    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/70a2f95e-67d1-4353-b4d9-d59a5c23f970-service-ca-bundle podName:70a2f95e-67d1-4353-b4d9-d59a5c23f970 nodeName:}" failed. No retries permitted until 2026-07-28 10:08:13.659061004 +0000 UTC m=+37.100602326 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/70a2f95e-67d1-4353-b4d9-d59a5c23f970-service-ca-bundle") pod "router-default-65b54959b5-rhznz" (UID: "70a2f95e-67d1-4353-b4d9-d59a5c23f970") : configmap references non-existent config key: service-ca.crt
Jul 28 10:08:12.659163 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.658907    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70a2f95e-67d1-4353-b4d9-d59a5c23f970-metrics-certs\") pod \"router-default-65b54959b5-rhznz\" (UID: \"70a2f95e-67d1-4353-b4d9-d59a5c23f970\") " pod="openshift-ingress/router-default-65b54959b5-rhznz"
Jul 28 10:08:12.659258 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.659158    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/94b8b045-295f-4194-a022-913c35c0529f-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vjpcz\" (UID: \"94b8b045-295f-4194-a022-913c35c0529f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vjpcz"
Jul 28 10:08:12.659288 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:12.659263    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 10:08:12.659319 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:12.659299    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94b8b045-295f-4194-a022-913c35c0529f-samples-operator-tls podName:94b8b045-295f-4194-a022-913c35c0529f nodeName:}" failed. No retries permitted until 2026-07-28 10:08:13.659288558 +0000 UTC m=+37.100829886 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/94b8b045-295f-4194-a022-913c35c0529f-samples-operator-tls") pod "cluster-samples-operator-6748489656-vjpcz" (UID: "94b8b045-295f-4194-a022-913c35c0529f") : secret "samples-operator-tls" not found
Jul 28 10:08:12.760565 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.760528    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/826225a0-88f8-4113-9f7d-05a75796d40d-metrics-tls\") pod \"dns-default-thhvn\" (UID: \"826225a0-88f8-4113-9f7d-05a75796d40d\") " pod="openshift-dns/dns-default-thhvn"
Jul 28 10:08:12.760755 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:12.760653    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3014f727-2c79-465d-ba86-bbea763014e5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-x6bhh\" (UID: \"3014f727-2c79-465d-ba86-bbea763014e5\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-x6bhh"
Jul 28 10:08:12.760755 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:12.760710    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 10:08:12.760868 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:12.760761    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 10:08:12.760868 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:12.760794    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/826225a0-88f8-4113-9f7d-05a75796d40d-metrics-tls podName:826225a0-88f8-4113-9f7d-05a75796d40d nodeName:}" failed. No retries permitted until 2026-07-28 10:08:13.760773101 +0000 UTC m=+37.202314420 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/826225a0-88f8-4113-9f7d-05a75796d40d-metrics-tls") pod "dns-default-thhvn" (UID: "826225a0-88f8-4113-9f7d-05a75796d40d") : secret "dns-default-metrics-tls" not found
Jul 28 10:08:12.760868 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:12.760818    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3014f727-2c79-465d-ba86-bbea763014e5-cluster-monitoring-operator-tls podName:3014f727-2c79-465d-ba86-bbea763014e5 nodeName:}" failed. No retries permitted until 2026-07-28 10:08:13.760805772 +0000 UTC m=+37.202347110 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3014f727-2c79-465d-ba86-bbea763014e5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-x6bhh" (UID: "3014f727-2c79-465d-ba86-bbea763014e5") : secret "cluster-monitoring-operator-tls" not found
Jul 28 10:08:13.128840 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:13.128804    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-299qd"
Jul 28 10:08:13.129585 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:13.128805    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmrjd"
Jul 28 10:08:13.129585 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:13.128816    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nz956"
Jul 28 10:08:13.131725 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:13.131700    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l65kq\""
Jul 28 10:08:13.131725 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:13.131700    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 10:08:13.131963 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:13.131829    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 10:08:13.131963 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:13.131865    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 10:08:13.132106 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:13.131979    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xsq5w\""
Jul 28 10:08:13.132171 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:13.132105    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 10:08:13.651721 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:13.651420    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-287lq"]
Jul 28 10:08:13.652549 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:13.652524    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gn75p"]
Jul 28 10:08:13.664131 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:13.664031    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-nljwq"]
Jul 28 10:08:13.666029 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:13.666005    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-czhcl"]
Jul 28 10:08:13.670442 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:13.670413    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-g7s47"]
Jul 28 10:08:13.670761 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:13.670738    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/94b8b045-295f-4194-a022-913c35c0529f-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vjpcz\" (UID: \"94b8b045-295f-4194-a022-913c35c0529f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vjpcz"
Jul 28 10:08:13.670841 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:13.670826    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9e06bdeb-9ead-430e-a448-950a1d18cc7a-registry-tls\") pod \"image-registry-7f9765899f-nlhq4\" (UID: \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\") " pod="openshift-image-registry/image-registry-7f9765899f-nlhq4"
Jul 28 10:08:13.670908 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:13.670882    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 10:08:13.671064 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:13.670940    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94b8b045-295f-4194-a022-913c35c0529f-samples-operator-tls podName:94b8b045-295f-4194-a022-913c35c0529f nodeName:}" failed. No retries permitted until 2026-07-28 10:08:15.670923323 +0000 UTC m=+39.112464638 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/94b8b045-295f-4194-a022-913c35c0529f-samples-operator-tls") pod "cluster-samples-operator-6748489656-vjpcz" (UID: "94b8b045-295f-4194-a022-913c35c0529f") : secret "samples-operator-tls" not found
Jul 28 10:08:13.671064 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:13.670887    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70a2f95e-67d1-4353-b4d9-d59a5c23f970-service-ca-bundle\") pod \"router-default-65b54959b5-rhznz\" (UID: \"70a2f95e-67d1-4353-b4d9-d59a5c23f970\") " pod="openshift-ingress/router-default-65b54959b5-rhznz"
Jul 28 10:08:13.671064 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:13.670969    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 10:08:13.671064 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:13.670981    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/70a2f95e-67d1-4353-b4d9-d59a5c23f970-service-ca-bundle podName:70a2f95e-67d1-4353-b4d9-d59a5c23f970 nodeName:}" failed. No retries permitted until 2026-07-28 10:08:15.67097374 +0000 UTC m=+39.112515056 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/70a2f95e-67d1-4353-b4d9-d59a5c23f970-service-ca-bundle") pod "router-default-65b54959b5-rhznz" (UID: "70a2f95e-67d1-4353-b4d9-d59a5c23f970") : configmap references non-existent config key: service-ca.crt
Jul 28 10:08:13.671064 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:13.670986    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f9765899f-nlhq4: secret "image-registry-tls" not found
Jul 28 10:08:13.671064 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:13.671004    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70a2f95e-67d1-4353-b4d9-d59a5c23f970-metrics-certs\") pod \"router-default-65b54959b5-rhznz\" (UID: \"70a2f95e-67d1-4353-b4d9-d59a5c23f970\") " pod="openshift-ingress/router-default-65b54959b5-rhznz"
Jul 28 10:08:13.671064 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:13.671030    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9e06bdeb-9ead-430e-a448-950a1d18cc7a-registry-tls podName:9e06bdeb-9ead-430e-a448-950a1d18cc7a nodeName:}" failed. No retries permitted until 2026-07-28 10:08:15.67101566 +0000 UTC m=+39.112556999 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9e06bdeb-9ead-430e-a448-950a1d18cc7a-registry-tls") pod "image-registry-7f9765899f-nlhq4" (UID: "9e06bdeb-9ead-430e-a448-950a1d18cc7a") : secret "image-registry-tls" not found
Jul 28 10:08:13.671064 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:13.671056    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 10:08:13.671603 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:13.671078    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70a2f95e-67d1-4353-b4d9-d59a5c23f970-metrics-certs podName:70a2f95e-67d1-4353-b4d9-d59a5c23f970 nodeName:}" failed. No retries permitted until 2026-07-28 10:08:15.671072152 +0000 UTC m=+39.112613466 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/70a2f95e-67d1-4353-b4d9-d59a5c23f970-metrics-certs") pod "router-default-65b54959b5-rhznz" (UID: "70a2f95e-67d1-4353-b4d9-d59a5c23f970") : secret "router-metrics-certs-default" not found
Jul 28 10:08:13.672908 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:13.672878    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-r6ptx"]
Jul 28 10:08:13.677350 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:08:13.677321    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbebcadc4_87cd_4b77_997e_bf1daa3bdfb5.slice/crio-f268b59faf3a5271f3d0105eec1a9a69c9197173c03601d928571708f618de1c WatchSource:0}: Error finding container f268b59faf3a5271f3d0105eec1a9a69c9197173c03601d928571708f618de1c: Status 404 returned error can't find the container with id f268b59faf3a5271f3d0105eec1a9a69c9197173c03601d928571708f618de1c
Jul 28 10:08:13.678041 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:08:13.678011    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6813c6f_ee42_4551_af42_d4fd8af7cf98.slice/crio-0f13cbb93b7957e35c4f4d894f2f10e6ddbdfc938b00233601025c926a834f03 WatchSource:0}: Error finding container 0f13cbb93b7957e35c4f4d894f2f10e6ddbdfc938b00233601025c926a834f03: Status 404 returned error can't find the container with id 0f13cbb93b7957e35c4f4d894f2f10e6ddbdfc938b00233601025c926a834f03
Jul 28 10:08:13.678813 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:08:13.678773    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8b50686_28d7_4749_9430_54bd0f88f2f2.slice/crio-6f0c69676a2cf61bcae26e06ff86b88b5e39dba874a4af270973cb82159cdecb WatchSource:0}: Error finding container 6f0c69676a2cf61bcae26e06ff86b88b5e39dba874a4af270973cb82159cdecb: Status 404 returned error can't find the container with id 6f0c69676a2cf61bcae26e06ff86b88b5e39dba874a4af270973cb82159cdecb
Jul 28 10:08:13.680049 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:08:13.680025    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod898e925d_8503_4a54_adef_e90195218e36.slice/crio-b1c1883c74f4b0a1336f1680c5971b0f7875c80abaf509f5ab958a9c9c62ea34 WatchSource:0}: Error finding container b1c1883c74f4b0a1336f1680c5971b0f7875c80abaf509f5ab958a9c9c62ea34: Status 404 returned error can't find the container with id b1c1883c74f4b0a1336f1680c5971b0f7875c80abaf509f5ab958a9c9c62ea34
Jul 28 10:08:13.681700 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:08:13.681250    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6411b5d_e39d_4922_a1fc_67c4952bbb01.slice/crio-6fcd1adea4a9f3d35bb8f3dba5796e9a2bad44d8d64a5c1d03fc2f116babe7ff WatchSource:0}: Error finding container 6fcd1adea4a9f3d35bb8f3dba5796e9a2bad44d8d64a5c1d03fc2f116babe7ff: Status 404 returned error can't find the container with id 6fcd1adea4a9f3d35bb8f3dba5796e9a2bad44d8d64a5c1d03fc2f116babe7ff
Jul 28 10:08:13.681700 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:08:13.681517    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f4d862e_1588_468a_bfce_d45f2a036b02.slice/crio-aa7cc53c1d5627af25f3b4c724cef365f8b0198f7cace2216faaba55b0650f25 WatchSource:0}: Error finding container aa7cc53c1d5627af25f3b4c724cef365f8b0198f7cace2216faaba55b0650f25: Status 404 returned error can't find the container with id aa7cc53c1d5627af25f3b4c724cef365f8b0198f7cace2216faaba55b0650f25
Jul 28 10:08:13.771924 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:13.771879    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3014f727-2c79-465d-ba86-bbea763014e5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-x6bhh\" (UID: \"3014f727-2c79-465d-ba86-bbea763014e5\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-x6bhh"
Jul 28 10:08:13.772069 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:13.771991    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/826225a0-88f8-4113-9f7d-05a75796d40d-metrics-tls\") pod \"dns-default-thhvn\" (UID: \"826225a0-88f8-4113-9f7d-05a75796d40d\") " pod="openshift-dns/dns-default-thhvn"
Jul 28 10:08:13.772069 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:13.772014    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 10:08:13.772129 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:13.772080    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3014f727-2c79-465d-ba86-bbea763014e5-cluster-monitoring-operator-tls podName:3014f727-2c79-465d-ba86-bbea763014e5 nodeName:}" failed. No retries permitted until 2026-07-28 10:08:15.772064629 +0000 UTC m=+39.213605947 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3014f727-2c79-465d-ba86-bbea763014e5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-x6bhh" (UID: "3014f727-2c79-465d-ba86-bbea763014e5") : secret "cluster-monitoring-operator-tls" not found
Jul 28 10:08:13.772192 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:13.772131    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 10:08:13.772192 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:13.772186    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/826225a0-88f8-4113-9f7d-05a75796d40d-metrics-tls podName:826225a0-88f8-4113-9f7d-05a75796d40d nodeName:}" failed. No retries permitted until 2026-07-28 10:08:15.77217271 +0000 UTC m=+39.213714026 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/826225a0-88f8-4113-9f7d-05a75796d40d-metrics-tls") pod "dns-default-thhvn" (UID: "826225a0-88f8-4113-9f7d-05a75796d40d") : secret "dns-default-metrics-tls" not found
Jul 28 10:08:14.006616 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:14.006526    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-vf6dm"]
Jul 28 10:08:14.011217 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:14.009563    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-vf6dm"
Jul 28 10:08:14.012248 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:14.012220    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-csqkj\""
Jul 28 10:08:14.019588 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:14.019406    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-vf6dm"]
Jul 28 10:08:14.175679 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:14.175639    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pqbd\" (UniqueName: \"kubernetes.io/projected/b486a7fe-4047-4522-b588-de2eb9aa52aa-kube-api-access-9pqbd\") pod \"network-check-source-7d88cb55d8-vf6dm\" (UID: \"b486a7fe-4047-4522-b588-de2eb9aa52aa\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-vf6dm"
Jul 28 10:08:14.276637 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:14.276469    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pqbd\" (UniqueName: \"kubernetes.io/projected/b486a7fe-4047-4522-b588-de2eb9aa52aa-kube-api-access-9pqbd\") pod \"network-check-source-7d88cb55d8-vf6dm\" (UID: \"b486a7fe-4047-4522-b588-de2eb9aa52aa\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-vf6dm"
Jul 28 10:08:14.291382 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:14.291314    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pqbd\" (UniqueName: \"kubernetes.io/projected/b486a7fe-4047-4522-b588-de2eb9aa52aa-kube-api-access-9pqbd\") pod \"network-check-source-7d88cb55d8-vf6dm\" (UID: \"b486a7fe-4047-4522-b588-de2eb9aa52aa\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-vf6dm"
Jul 28 10:08:14.313566 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:14.313508    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-287lq" event={"ID":"e6813c6f-ee42-4551-af42-d4fd8af7cf98","Type":"ContainerStarted","Data":"0f13cbb93b7957e35c4f4d894f2f10e6ddbdfc938b00233601025c926a834f03"}
Jul 28 10:08:14.316045 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:14.316002    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gn75p" event={"ID":"898e925d-8503-4a54-adef-e90195218e36","Type":"ContainerStarted","Data":"b1c1883c74f4b0a1336f1680c5971b0f7875c80abaf509f5ab958a9c9c62ea34"}
Jul 28 10:08:14.321352 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:14.321315    2563 generic.go:356] "Generic (PLEG): container finished" podID="9e2fbc65-eba0-483e-ad62-d8f17b9bd452" containerID="1c2d34b510bc74da03090cccb428c331de480c2d8d6ced5f18cc1886b43a4b5c" exitCode=0
Jul 28 10:08:14.321516 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:14.321453    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bmkbw" event={"ID":"9e2fbc65-eba0-483e-ad62-d8f17b9bd452","Type":"ContainerDied","Data":"1c2d34b510bc74da03090cccb428c331de480c2d8d6ced5f18cc1886b43a4b5c"}
Jul 28 10:08:14.322733 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:14.322711    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-vf6dm"
Jul 28 10:08:14.323578 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:14.323548    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-g7s47" event={"ID":"bebcadc4-87cd-4b77-997e-bf1daa3bdfb5","Type":"ContainerStarted","Data":"f268b59faf3a5271f3d0105eec1a9a69c9197173c03601d928571708f618de1c"}
Jul 28 10:08:14.330080 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:14.330046    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-r6ptx" event={"ID":"f6411b5d-e39d-4922-a1fc-67c4952bbb01","Type":"ContainerStarted","Data":"6fcd1adea4a9f3d35bb8f3dba5796e9a2bad44d8d64a5c1d03fc2f116babe7ff"}
Jul 28 10:08:14.334334 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:14.334301    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-nljwq" event={"ID":"7f4d862e-1588-468a-bfce-d45f2a036b02","Type":"ContainerStarted","Data":"aa7cc53c1d5627af25f3b4c724cef365f8b0198f7cace2216faaba55b0650f25"}
Jul 28 10:08:14.336530 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:14.336490    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-czhcl" event={"ID":"f8b50686-28d7-4749-9430-54bd0f88f2f2","Type":"ContainerStarted","Data":"6f0c69676a2cf61bcae26e06ff86b88b5e39dba874a4af270973cb82159cdecb"}
Jul 28 10:08:14.540196 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:14.540070    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-vf6dm"]
Jul 28 10:08:14.976376 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:08:14.976320    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb486a7fe_4047_4522_b588_de2eb9aa52aa.slice/crio-84a2434d3d3042a231bc4901018e3f0a896364e4014c56e3a489c10619c57084 WatchSource:0}: Error finding container 84a2434d3d3042a231bc4901018e3f0a896364e4014c56e3a489c10619c57084: Status 404 returned error can't find the container with id 84a2434d3d3042a231bc4901018e3f0a896364e4014c56e3a489c10619c57084
Jul 28 10:08:15.341586 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:15.341545    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-vf6dm" event={"ID":"b486a7fe-4047-4522-b588-de2eb9aa52aa","Type":"ContainerStarted","Data":"84a2434d3d3042a231bc4901018e3f0a896364e4014c56e3a489c10619c57084"}
Jul 28 10:08:15.351996 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:15.347015    2563 generic.go:356] "Generic (PLEG): container finished" podID="9e2fbc65-eba0-483e-ad62-d8f17b9bd452" containerID="5e262f597924298340fd010e69eb55dce68d3d3481151f6c627f8c3d1def5b0a" exitCode=0
Jul 28 10:08:15.351996 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:15.347084    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bmkbw" event={"ID":"9e2fbc65-eba0-483e-ad62-d8f17b9bd452","Type":"ContainerDied","Data":"5e262f597924298340fd010e69eb55dce68d3d3481151f6c627f8c3d1def5b0a"}
Jul 28 10:08:15.695032 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:15.693995    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/94b8b045-295f-4194-a022-913c35c0529f-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vjpcz\" (UID: \"94b8b045-295f-4194-a022-913c35c0529f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vjpcz"
Jul 28 10:08:15.695032 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:15.694088    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9e06bdeb-9ead-430e-a448-950a1d18cc7a-registry-tls\") pod \"image-registry-7f9765899f-nlhq4\" (UID: \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\") " pod="openshift-image-registry/image-registry-7f9765899f-nlhq4"
Jul 28 10:08:15.695032 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:15.694147    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70a2f95e-67d1-4353-b4d9-d59a5c23f970-service-ca-bundle\") pod \"router-default-65b54959b5-rhznz\" (UID: \"70a2f95e-67d1-4353-b4d9-d59a5c23f970\") " pod="openshift-ingress/router-default-65b54959b5-rhznz"
Jul 28 10:08:15.695032 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:15.694179    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70a2f95e-67d1-4353-b4d9-d59a5c23f970-metrics-certs\") pod \"router-default-65b54959b5-rhznz\" (UID: \"70a2f95e-67d1-4353-b4d9-d59a5c23f970\") " pod="openshift-ingress/router-default-65b54959b5-rhznz"
Jul 28 10:08:15.695032 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:15.694332    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 10:08:15.695032 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:15.694415    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70a2f95e-67d1-4353-b4d9-d59a5c23f970-metrics-certs podName:70a2f95e-67d1-4353-b4d9-d59a5c23f970 nodeName:}" failed. No retries permitted until 2026-07-28 10:08:19.6943937 +0000 UTC m=+43.135935021 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/70a2f95e-67d1-4353-b4d9-d59a5c23f970-metrics-certs") pod "router-default-65b54959b5-rhznz" (UID: "70a2f95e-67d1-4353-b4d9-d59a5c23f970") : secret "router-metrics-certs-default" not found
Jul 28 10:08:15.695032 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:15.694765    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 10:08:15.695032 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:15.694789    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f9765899f-nlhq4: secret "image-registry-tls" not found
Jul 28 10:08:15.695032 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:15.694904    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 10:08:15.695707 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:15.695680    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/70a2f95e-67d1-4353-b4d9-d59a5c23f970-service-ca-bundle podName:70a2f95e-67d1-4353-b4d9-d59a5c23f970 nodeName:}" failed. No retries permitted until 2026-07-28 10:08:19.694802751 +0000 UTC m=+43.136344072 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/70a2f95e-67d1-4353-b4d9-d59a5c23f970-service-ca-bundle") pod "router-default-65b54959b5-rhznz" (UID: "70a2f95e-67d1-4353-b4d9-d59a5c23f970") : configmap references non-existent config key: service-ca.crt
Jul 28 10:08:15.695889 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:15.695725    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9e06bdeb-9ead-430e-a448-950a1d18cc7a-registry-tls podName:9e06bdeb-9ead-430e-a448-950a1d18cc7a nodeName:}" failed. No retries permitted until 2026-07-28 10:08:19.695712901 +0000 UTC m=+43.137254222 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9e06bdeb-9ead-430e-a448-950a1d18cc7a-registry-tls") pod "image-registry-7f9765899f-nlhq4" (UID: "9e06bdeb-9ead-430e-a448-950a1d18cc7a") : secret "image-registry-tls" not found
Jul 28 10:08:15.695889 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:15.695741    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94b8b045-295f-4194-a022-913c35c0529f-samples-operator-tls podName:94b8b045-295f-4194-a022-913c35c0529f nodeName:}" failed. No retries permitted until 2026-07-28 10:08:19.695732275 +0000 UTC m=+43.137273593 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/94b8b045-295f-4194-a022-913c35c0529f-samples-operator-tls") pod "cluster-samples-operator-6748489656-vjpcz" (UID: "94b8b045-295f-4194-a022-913c35c0529f") : secret "samples-operator-tls" not found
Jul 28 10:08:15.796336 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:15.795497    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3014f727-2c79-465d-ba86-bbea763014e5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-x6bhh\" (UID: \"3014f727-2c79-465d-ba86-bbea763014e5\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-x6bhh"
Jul 28 10:08:15.796336 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:15.795605    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/826225a0-88f8-4113-9f7d-05a75796d40d-metrics-tls\") pod \"dns-default-thhvn\" (UID: \"826225a0-88f8-4113-9f7d-05a75796d40d\") " pod="openshift-dns/dns-default-thhvn"
Jul 28 10:08:15.796336 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:15.795796    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 10:08:15.796336 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:15.795861    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/826225a0-88f8-4113-9f7d-05a75796d40d-metrics-tls podName:826225a0-88f8-4113-9f7d-05a75796d40d nodeName:}" failed. No retries permitted until 2026-07-28 10:08:19.795842503 +0000 UTC m=+43.237383831 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/826225a0-88f8-4113-9f7d-05a75796d40d-metrics-tls") pod "dns-default-thhvn" (UID: "826225a0-88f8-4113-9f7d-05a75796d40d") : secret "dns-default-metrics-tls" not found
Jul 28 10:08:15.796336 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:15.796254    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 10:08:15.796336 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:15.796300    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3014f727-2c79-465d-ba86-bbea763014e5-cluster-monitoring-operator-tls podName:3014f727-2c79-465d-ba86-bbea763014e5 nodeName:}" failed. No retries permitted until 2026-07-28 10:08:19.796284897 +0000 UTC m=+43.237826218 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3014f727-2c79-465d-ba86-bbea763014e5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-x6bhh" (UID: "3014f727-2c79-465d-ba86-bbea763014e5") : secret "cluster-monitoring-operator-tls" not found
Jul 28 10:08:19.734535 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:19.734472    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9e06bdeb-9ead-430e-a448-950a1d18cc7a-registry-tls\") pod \"image-registry-7f9765899f-nlhq4\" (UID: \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\") " pod="openshift-image-registry/image-registry-7f9765899f-nlhq4"
Jul 28 10:08:19.734998 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:19.734564    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70a2f95e-67d1-4353-b4d9-d59a5c23f970-service-ca-bundle\") pod \"router-default-65b54959b5-rhznz\" (UID: \"70a2f95e-67d1-4353-b4d9-d59a5c23f970\") " pod="openshift-ingress/router-default-65b54959b5-rhznz"
Jul 28 10:08:19.734998 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:19.734657    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 10:08:19.734998 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:19.734681    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f9765899f-nlhq4: secret "image-registry-tls" not found
Jul 28 10:08:19.734998 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:19.734698    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/70a2f95e-67d1-4353-b4d9-d59a5c23f970-service-ca-bundle podName:70a2f95e-67d1-4353-b4d9-d59a5c23f970 nodeName:}" failed. No retries permitted until 2026-07-28 10:08:27.734679305 +0000 UTC m=+51.176220625 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/70a2f95e-67d1-4353-b4d9-d59a5c23f970-service-ca-bundle") pod "router-default-65b54959b5-rhznz" (UID: "70a2f95e-67d1-4353-b4d9-d59a5c23f970") : configmap references non-existent config key: service-ca.crt
Jul 28 10:08:19.734998 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:19.734737    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9e06bdeb-9ead-430e-a448-950a1d18cc7a-registry-tls podName:9e06bdeb-9ead-430e-a448-950a1d18cc7a nodeName:}" failed. No retries permitted until 2026-07-28 10:08:27.734721272 +0000 UTC m=+51.176262591 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9e06bdeb-9ead-430e-a448-950a1d18cc7a-registry-tls") pod "image-registry-7f9765899f-nlhq4" (UID: "9e06bdeb-9ead-430e-a448-950a1d18cc7a") : secret "image-registry-tls" not found
Jul 28 10:08:19.734998 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:19.734751    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70a2f95e-67d1-4353-b4d9-d59a5c23f970-metrics-certs\") pod \"router-default-65b54959b5-rhznz\" (UID: \"70a2f95e-67d1-4353-b4d9-d59a5c23f970\") " pod="openshift-ingress/router-default-65b54959b5-rhznz"
Jul 28 10:08:19.734998 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:19.734855    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/94b8b045-295f-4194-a022-913c35c0529f-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vjpcz\" (UID: \"94b8b045-295f-4194-a022-913c35c0529f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vjpcz"
Jul 28 10:08:19.734998 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:19.734860    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 10:08:19.734998 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:19.734954    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70a2f95e-67d1-4353-b4d9-d59a5c23f970-metrics-certs podName:70a2f95e-67d1-4353-b4d9-d59a5c23f970 nodeName:}" failed. No retries permitted until 2026-07-28 10:08:27.734938079 +0000 UTC m=+51.176479408 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/70a2f95e-67d1-4353-b4d9-d59a5c23f970-metrics-certs") pod "router-default-65b54959b5-rhznz" (UID: "70a2f95e-67d1-4353-b4d9-d59a5c23f970") : secret "router-metrics-certs-default" not found
Jul 28 10:08:19.734998 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:19.734974    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 10:08:19.735402 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:19.735015    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94b8b045-295f-4194-a022-913c35c0529f-samples-operator-tls podName:94b8b045-295f-4194-a022-913c35c0529f nodeName:}" failed. No retries permitted until 2026-07-28 10:08:27.735002697 +0000 UTC m=+51.176544025 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/94b8b045-295f-4194-a022-913c35c0529f-samples-operator-tls") pod "cluster-samples-operator-6748489656-vjpcz" (UID: "94b8b045-295f-4194-a022-913c35c0529f") : secret "samples-operator-tls" not found
Jul 28 10:08:19.835582 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:19.835537    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3014f727-2c79-465d-ba86-bbea763014e5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-x6bhh\" (UID: \"3014f727-2c79-465d-ba86-bbea763014e5\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-x6bhh"
Jul 28 10:08:19.835793 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:19.835641    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/826225a0-88f8-4113-9f7d-05a75796d40d-metrics-tls\") pod \"dns-default-thhvn\" (UID: \"826225a0-88f8-4113-9f7d-05a75796d40d\") " pod="openshift-dns/dns-default-thhvn"
Jul 28 10:08:19.835793 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:19.835729    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 10:08:19.835793 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:19.835752    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 10:08:19.835945 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:19.835823    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3014f727-2c79-465d-ba86-bbea763014e5-cluster-monitoring-operator-tls podName:3014f727-2c79-465d-ba86-bbea763014e5 nodeName:}" failed. No retries permitted until 2026-07-28 10:08:27.835799668 +0000 UTC m=+51.277340986 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3014f727-2c79-465d-ba86-bbea763014e5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-x6bhh" (UID: "3014f727-2c79-465d-ba86-bbea763014e5") : secret "cluster-monitoring-operator-tls" not found
Jul 28 10:08:19.835945 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:19.835844    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/826225a0-88f8-4113-9f7d-05a75796d40d-metrics-tls podName:826225a0-88f8-4113-9f7d-05a75796d40d nodeName:}" failed. No retries permitted until 2026-07-28 10:08:27.835833493 +0000 UTC m=+51.277374813 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/826225a0-88f8-4113-9f7d-05a75796d40d-metrics-tls") pod "dns-default-thhvn" (UID: "826225a0-88f8-4113-9f7d-05a75796d40d") : secret "dns-default-metrics-tls" not found
Jul 28 10:08:21.368898 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:21.368828    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-nljwq" event={"ID":"7f4d862e-1588-468a-bfce-d45f2a036b02","Type":"ContainerStarted","Data":"fb6511346ef3f74f2758beb47aa10c42c376086bbbce5c4d0aaa8c7c57e7d4d1"}
Jul 28 10:08:21.370801 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:21.370741    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-czhcl" event={"ID":"f8b50686-28d7-4749-9430-54bd0f88f2f2","Type":"ContainerStarted","Data":"b7c783114401aaec37250d3f97f822a5adcdd17662b29ba129fbd5cb87daea07"}
Jul 28 10:08:21.373080 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:21.372632    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-287lq" event={"ID":"e6813c6f-ee42-4551-af42-d4fd8af7cf98","Type":"ContainerStarted","Data":"514159f8f92391357acf9cd07f64b18f1c30ea182e44b1f732b82e7512782661"}
Jul 28 10:08:21.375594 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:21.375419    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gn75p" event={"ID":"898e925d-8503-4a54-adef-e90195218e36","Type":"ContainerStarted","Data":"5d0fb9efa0b5815d23bd7864f11ecb400e2ff2805f676293e58a0ccebbfbd2ee"}
Jul 28 10:08:21.377530 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:21.377442    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-vf6dm" event={"ID":"b486a7fe-4047-4522-b588-de2eb9aa52aa","Type":"ContainerStarted","Data":"ea4cb57e0cb33caac8eaff066b8571823056b6bb4c1c7746ed1ad5e08c0c878d"}
Jul 28 10:08:21.380963 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:21.380937    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bmkbw" event={"ID":"9e2fbc65-eba0-483e-ad62-d8f17b9bd452","Type":"ContainerStarted","Data":"ce385e6b7963fb0cd6884fd951387d201dc95b84519efc532843de132429b8df"}
Jul 28 10:08:21.383110 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:21.383090    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/0.log"
Jul 28 10:08:21.383224 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:21.383121    2563 generic.go:356] "Generic (PLEG): container finished" podID="bebcadc4-87cd-4b77-997e-bf1daa3bdfb5" containerID="1b5790004236f9d6de5e83d5b99d5f328e6ceaf5b66018dfe60ac3347e8d1e31" exitCode=255
Jul 28 10:08:21.383224 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:21.383171    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-g7s47" event={"ID":"bebcadc4-87cd-4b77-997e-bf1daa3bdfb5","Type":"ContainerDied","Data":"1b5790004236f9d6de5e83d5b99d5f328e6ceaf5b66018dfe60ac3347e8d1e31"}
Jul 28 10:08:21.383675 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:21.383404    2563 scope.go:117] "RemoveContainer" containerID="1b5790004236f9d6de5e83d5b99d5f328e6ceaf5b66018dfe60ac3347e8d1e31"
Jul 28 10:08:21.385127 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:21.385101    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-r6ptx" event={"ID":"f6411b5d-e39d-4922-a1fc-67c4952bbb01","Type":"ContainerStarted","Data":"ef41c7b4023f77ca2e8c5898ddd6ec9cb959637b0fced76ce067d0148b5f4ed9"}
Jul 28 10:08:21.406752 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:21.406694    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-nljwq" podStartSLOduration=13.455234766 podStartE2EDuration="19.406676271s" totalImagesPullingTime="5.951441505s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:08:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:08:21.404595914 +0000 UTC m=+44.846137252" watchObservedRunningTime="2026-07-28 10:08:21.406676271 +0000 UTC m=+44.848217608"
Jul 28 10:08:21.439707 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:21.439021    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gn75p" podStartSLOduration=12.094096135 podStartE2EDuration="19.438995986s" totalImagesPullingTime="7.344899851s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:08:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:08:21.436615021 +0000 UTC m=+44.878156342" watchObservedRunningTime="2026-07-28 10:08:21.438995986 +0000 UTC m=+44.880537325"
Jul 28 10:08:21.481950 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:21.480434    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-vf6dm" podStartSLOduration=2.398496417 podStartE2EDuration="8.48041008s" totalImagesPullingTime="6.081913663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:08:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:08:21.476951218 +0000 UTC m=+44.918492557" watchObservedRunningTime="2026-07-28 10:08:21.48041008 +0000 UTC m=+44.921951418"
Jul 28 10:08:21.509865 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:21.509810    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-bmkbw" podStartSLOduration=21.864606518 podStartE2EDuration="44.509787583s" totalImagesPullingTime="22.645181065s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 10:07:37 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:08:21.508322266 +0000 UTC m=+44.949863605" watchObservedRunningTime="2026-07-28 10:08:21.509787583 +0000 UTC m=+44.951328921"
Jul 28 10:08:21.560383 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:21.559623    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-287lq" podStartSLOduration=3.190381482 podStartE2EDuration="10.559601117s" totalImagesPullingTime="7.369219635s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:08:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:08:21.558136231 +0000 UTC m=+44.999677579" watchObservedRunningTime="2026-07-28 10:08:21.559601117 +0000 UTC m=+45.001142456"
Jul 28 10:08:21.629645 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:21.629527    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-r6ptx" podStartSLOduration=13.673493888 podStartE2EDuration="19.629501944s" totalImagesPullingTime="5.956008056s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:08:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:08:21.585848336 +0000 UTC m=+45.027389674" watchObservedRunningTime="2026-07-28 10:08:21.629501944 +0000 UTC m=+45.071043285"
Jul 28 10:08:22.320962 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:22.320922    2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-g7s47"
Jul 28 10:08:22.321166 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:22.321011    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-g7s47"
Jul 28 10:08:22.391047 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:22.391016    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 10:08:22.391523 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:22.391507    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/0.log"
Jul 28 10:08:22.391582 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:22.391544    2563 generic.go:356] "Generic (PLEG): container finished" podID="bebcadc4-87cd-4b77-997e-bf1daa3bdfb5" containerID="84bc55e3ec16b9abe5fbab4e83da8b762f5cbdb78489f421e780d89c516d3371" exitCode=255
Jul 28 10:08:22.391666 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:22.391642    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-g7s47" event={"ID":"bebcadc4-87cd-4b77-997e-bf1daa3bdfb5","Type":"ContainerDied","Data":"84bc55e3ec16b9abe5fbab4e83da8b762f5cbdb78489f421e780d89c516d3371"}
Jul 28 10:08:22.391725 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:22.391696    2563 scope.go:117] "RemoveContainer" containerID="1b5790004236f9d6de5e83d5b99d5f328e6ceaf5b66018dfe60ac3347e8d1e31"
Jul 28 10:08:22.391849 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:22.391834    2563 scope.go:117] "RemoveContainer" containerID="84bc55e3ec16b9abe5fbab4e83da8b762f5cbdb78489f421e780d89c516d3371"
Jul 28 10:08:22.392734 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:22.392034    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-g7s47_openshift-console-operator(bebcadc4-87cd-4b77-997e-bf1daa3bdfb5)\"" pod="openshift-console-operator/console-operator-584dcb6b54-g7s47" podUID="bebcadc4-87cd-4b77-997e-bf1daa3bdfb5"
Jul 28 10:08:22.416699 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:22.416642    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-czhcl" podStartSLOduration=14.461372961 podStartE2EDuration="20.416618386s" totalImagesPullingTime="5.955245425s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:08:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:08:21.641276647 +0000 UTC m=+45.082817985" watchObservedRunningTime="2026-07-28 10:08:22.416618386 +0000 UTC m=+45.858159723"
Jul 28 10:08:22.731756 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:22.731666    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-vj25w"]
Jul 28 10:08:22.760406 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:22.760347    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vj25w"
Jul 28 10:08:22.767723 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:22.767700    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-q8lkz\""
Jul 28 10:08:22.767918 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:22.767899    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 28 10:08:22.767977 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:22.767951    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 28 10:08:22.770511 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:22.770486    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vj25w"]
Jul 28 10:08:22.867918 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:22.867869    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/562ec3c6-ef0d-41f0-9279-498b868331e8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vj25w\" (UID: \"562ec3c6-ef0d-41f0-9279-498b868331e8\") " pod="openshift-insights/insights-runtime-extractor-vj25w"
Jul 28 10:08:22.868112 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:22.868038    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/562ec3c6-ef0d-41f0-9279-498b868331e8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vj25w\" (UID: \"562ec3c6-ef0d-41f0-9279-498b868331e8\") " pod="openshift-insights/insights-runtime-extractor-vj25w"
Jul 28 10:08:22.868112 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:22.868086    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcclv\" (UniqueName: \"kubernetes.io/projected/562ec3c6-ef0d-41f0-9279-498b868331e8-kube-api-access-gcclv\") pod \"insights-runtime-extractor-vj25w\" (UID: \"562ec3c6-ef0d-41f0-9279-498b868331e8\") " pod="openshift-insights/insights-runtime-extractor-vj25w"
Jul 28 10:08:22.868208 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:22.868150    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/562ec3c6-ef0d-41f0-9279-498b868331e8-crio-socket\") pod \"insights-runtime-extractor-vj25w\" (UID: \"562ec3c6-ef0d-41f0-9279-498b868331e8\") " pod="openshift-insights/insights-runtime-extractor-vj25w"
Jul 28 10:08:22.868208 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:22.868168    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/562ec3c6-ef0d-41f0-9279-498b868331e8-data-volume\") pod \"insights-runtime-extractor-vj25w\" (UID: \"562ec3c6-ef0d-41f0-9279-498b868331e8\") " pod="openshift-insights/insights-runtime-extractor-vj25w"
Jul 28 10:08:22.968835 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:22.968790    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/562ec3c6-ef0d-41f0-9279-498b868331e8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vj25w\" (UID: \"562ec3c6-ef0d-41f0-9279-498b868331e8\") " pod="openshift-insights/insights-runtime-extractor-vj25w"
Jul 28 10:08:22.968835 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:22.968843    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcclv\" (UniqueName: \"kubernetes.io/projected/562ec3c6-ef0d-41f0-9279-498b868331e8-kube-api-access-gcclv\") pod \"insights-runtime-extractor-vj25w\" (UID: \"562ec3c6-ef0d-41f0-9279-498b868331e8\") " pod="openshift-insights/insights-runtime-extractor-vj25w"
Jul 28 10:08:22.969097 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:22.968890    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/562ec3c6-ef0d-41f0-9279-498b868331e8-crio-socket\") pod \"insights-runtime-extractor-vj25w\" (UID: \"562ec3c6-ef0d-41f0-9279-498b868331e8\") " pod="openshift-insights/insights-runtime-extractor-vj25w"
Jul 28 10:08:22.969097 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:22.968914    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/562ec3c6-ef0d-41f0-9279-498b868331e8-data-volume\") pod \"insights-runtime-extractor-vj25w\" (UID: \"562ec3c6-ef0d-41f0-9279-498b868331e8\") " pod="openshift-insights/insights-runtime-extractor-vj25w"
Jul 28 10:08:22.969097 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:22.968958    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 10:08:22.969097 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:22.969012    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/562ec3c6-ef0d-41f0-9279-498b868331e8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vj25w\" (UID: \"562ec3c6-ef0d-41f0-9279-498b868331e8\") " pod="openshift-insights/insights-runtime-extractor-vj25w"
Jul 28 10:08:22.969097 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:22.969038    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/562ec3c6-ef0d-41f0-9279-498b868331e8-insights-runtime-extractor-tls podName:562ec3c6-ef0d-41f0-9279-498b868331e8 nodeName:}" failed. No retries permitted until 2026-07-28 10:08:23.469018201 +0000 UTC m=+46.910559522 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/562ec3c6-ef0d-41f0-9279-498b868331e8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vj25w" (UID: "562ec3c6-ef0d-41f0-9279-498b868331e8") : secret "insights-runtime-extractor-tls" not found
Jul 28 10:08:22.969097 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:22.969065    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 10:08:22.969097 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:22.969071    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/562ec3c6-ef0d-41f0-9279-498b868331e8-crio-socket\") pod \"insights-runtime-extractor-vj25w\" (UID: \"562ec3c6-ef0d-41f0-9279-498b868331e8\") " pod="openshift-insights/insights-runtime-extractor-vj25w"
Jul 28 10:08:22.969457 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:22.969123    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/562ec3c6-ef0d-41f0-9279-498b868331e8-kube-rbac-proxy-cm podName:562ec3c6-ef0d-41f0-9279-498b868331e8 nodeName:}" failed. No retries permitted until 2026-07-28 10:08:23.469107565 +0000 UTC m=+46.910648892 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/562ec3c6-ef0d-41f0-9279-498b868331e8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vj25w" (UID: "562ec3c6-ef0d-41f0-9279-498b868331e8") : configmap "kube-rbac-proxy" not found
Jul 28 10:08:22.969457 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:22.969316    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/562ec3c6-ef0d-41f0-9279-498b868331e8-data-volume\") pod \"insights-runtime-extractor-vj25w\" (UID: \"562ec3c6-ef0d-41f0-9279-498b868331e8\") " pod="openshift-insights/insights-runtime-extractor-vj25w"
Jul 28 10:08:23.008026 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:23.007944    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcclv\" (UniqueName: \"kubernetes.io/projected/562ec3c6-ef0d-41f0-9279-498b868331e8-kube-api-access-gcclv\") pod \"insights-runtime-extractor-vj25w\" (UID: \"562ec3c6-ef0d-41f0-9279-498b868331e8\") " pod="openshift-insights/insights-runtime-extractor-vj25w"
Jul 28 10:08:23.396446 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:23.396418    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 10:08:23.398057 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:23.397299    2563 scope.go:117] "RemoveContainer" containerID="84bc55e3ec16b9abe5fbab4e83da8b762f5cbdb78489f421e780d89c516d3371"
Jul 28 10:08:23.400462 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:23.398258    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-g7s47_openshift-console-operator(bebcadc4-87cd-4b77-997e-bf1daa3bdfb5)\"" pod="openshift-console-operator/console-operator-584dcb6b54-g7s47" podUID="bebcadc4-87cd-4b77-997e-bf1daa3bdfb5"
Jul 28 10:08:23.474570 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:23.474528    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/562ec3c6-ef0d-41f0-9279-498b868331e8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vj25w\" (UID: \"562ec3c6-ef0d-41f0-9279-498b868331e8\") " pod="openshift-insights/insights-runtime-extractor-vj25w"
Jul 28 10:08:23.474757 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:23.474669    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 10:08:23.474757 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:23.474703    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/562ec3c6-ef0d-41f0-9279-498b868331e8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vj25w\" (UID: \"562ec3c6-ef0d-41f0-9279-498b868331e8\") " pod="openshift-insights/insights-runtime-extractor-vj25w"
Jul 28 10:08:23.474757 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:23.474747    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/562ec3c6-ef0d-41f0-9279-498b868331e8-kube-rbac-proxy-cm podName:562ec3c6-ef0d-41f0-9279-498b868331e8 nodeName:}" failed. No retries permitted until 2026-07-28 10:08:24.474723768 +0000 UTC m=+47.916265087 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/562ec3c6-ef0d-41f0-9279-498b868331e8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vj25w" (UID: "562ec3c6-ef0d-41f0-9279-498b868331e8") : configmap "kube-rbac-proxy" not found
Jul 28 10:08:23.474924 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:23.474809    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 10:08:23.474924 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:23.474879    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/562ec3c6-ef0d-41f0-9279-498b868331e8-insights-runtime-extractor-tls podName:562ec3c6-ef0d-41f0-9279-498b868331e8 nodeName:}" failed. No retries permitted until 2026-07-28 10:08:24.474857272 +0000 UTC m=+47.916398595 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/562ec3c6-ef0d-41f0-9279-498b868331e8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vj25w" (UID: "562ec3c6-ef0d-41f0-9279-498b868331e8") : secret "insights-runtime-extractor-tls" not found
Jul 28 10:08:24.484467 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:24.484433    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/562ec3c6-ef0d-41f0-9279-498b868331e8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vj25w\" (UID: \"562ec3c6-ef0d-41f0-9279-498b868331e8\") " pod="openshift-insights/insights-runtime-extractor-vj25w"
Jul 28 10:08:24.484932 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:24.484570    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 10:08:24.484932 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:24.484647    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/562ec3c6-ef0d-41f0-9279-498b868331e8-kube-rbac-proxy-cm podName:562ec3c6-ef0d-41f0-9279-498b868331e8 nodeName:}" failed. No retries permitted until 2026-07-28 10:08:26.484630297 +0000 UTC m=+49.926171626 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/562ec3c6-ef0d-41f0-9279-498b868331e8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vj25w" (UID: "562ec3c6-ef0d-41f0-9279-498b868331e8") : configmap "kube-rbac-proxy" not found
Jul 28 10:08:24.484932 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:24.484684    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 10:08:24.484932 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:24.484575    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/562ec3c6-ef0d-41f0-9279-498b868331e8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vj25w\" (UID: \"562ec3c6-ef0d-41f0-9279-498b868331e8\") " pod="openshift-insights/insights-runtime-extractor-vj25w"
Jul 28 10:08:24.484932 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:24.484739    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/562ec3c6-ef0d-41f0-9279-498b868331e8-insights-runtime-extractor-tls podName:562ec3c6-ef0d-41f0-9279-498b868331e8 nodeName:}" failed. No retries permitted until 2026-07-28 10:08:26.484721975 +0000 UTC m=+49.926263311 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/562ec3c6-ef0d-41f0-9279-498b868331e8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vj25w" (UID: "562ec3c6-ef0d-41f0-9279-498b868331e8") : secret "insights-runtime-extractor-tls" not found
Jul 28 10:08:25.664738 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:25.664705    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hpntr_0f2a806a-05a5-4fd1-be4f-be8b64a5013b/dns-node-resolver/0.log"
Jul 28 10:08:26.268973 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:26.268938    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-d48vp_4b1075b5-7b36-420f-aa9f-90022e20d840/node-ca/0.log"
Jul 28 10:08:26.504145 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:26.504106    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/562ec3c6-ef0d-41f0-9279-498b868331e8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vj25w\" (UID: \"562ec3c6-ef0d-41f0-9279-498b868331e8\") " pod="openshift-insights/insights-runtime-extractor-vj25w"
Jul 28 10:08:26.504324 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:26.504191    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/562ec3c6-ef0d-41f0-9279-498b868331e8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vj25w\" (UID: \"562ec3c6-ef0d-41f0-9279-498b868331e8\") " pod="openshift-insights/insights-runtime-extractor-vj25w"
Jul 28 10:08:26.504324 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:26.504271    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 10:08:26.504324 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:26.504272    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 10:08:26.504324 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:26.504325    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/562ec3c6-ef0d-41f0-9279-498b868331e8-kube-rbac-proxy-cm podName:562ec3c6-ef0d-41f0-9279-498b868331e8 nodeName:}" failed. No retries permitted until 2026-07-28 10:08:30.504311302 +0000 UTC m=+53.945852617 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/562ec3c6-ef0d-41f0-9279-498b868331e8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vj25w" (UID: "562ec3c6-ef0d-41f0-9279-498b868331e8") : configmap "kube-rbac-proxy" not found
Jul 28 10:08:26.504491 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:26.504339    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/562ec3c6-ef0d-41f0-9279-498b868331e8-insights-runtime-extractor-tls podName:562ec3c6-ef0d-41f0-9279-498b868331e8 nodeName:}" failed. No retries permitted until 2026-07-28 10:08:30.504332606 +0000 UTC m=+53.945873921 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/562ec3c6-ef0d-41f0-9279-498b868331e8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vj25w" (UID: "562ec3c6-ef0d-41f0-9279-498b868331e8") : secret "insights-runtime-extractor-tls" not found
Jul 28 10:08:27.063751 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:27.063721    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-287lq_e6813c6f-ee42-4551-af42-d4fd8af7cf98/serve-healthcheck-canary/0.log"
Jul 28 10:08:27.815881 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:27.815838    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/94b8b045-295f-4194-a022-913c35c0529f-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vjpcz\" (UID: \"94b8b045-295f-4194-a022-913c35c0529f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vjpcz"
Jul 28 10:08:27.816048 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:27.815914    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9e06bdeb-9ead-430e-a448-950a1d18cc7a-registry-tls\") pod \"image-registry-7f9765899f-nlhq4\" (UID: \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\") " pod="openshift-image-registry/image-registry-7f9765899f-nlhq4"
Jul 28 10:08:27.816048 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:27.815963    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70a2f95e-67d1-4353-b4d9-d59a5c23f970-service-ca-bundle\") pod \"router-default-65b54959b5-rhznz\" (UID: \"70a2f95e-67d1-4353-b4d9-d59a5c23f970\") " pod="openshift-ingress/router-default-65b54959b5-rhznz"
Jul 28 10:08:27.816048 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:27.815989    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70a2f95e-67d1-4353-b4d9-d59a5c23f970-metrics-certs\") pod \"router-default-65b54959b5-rhznz\" (UID: \"70a2f95e-67d1-4353-b4d9-d59a5c23f970\") " pod="openshift-ingress/router-default-65b54959b5-rhznz"
Jul 28 10:08:27.816048 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:27.815993    2563 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 10:08:27.816048 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:27.816044    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 10:08:27.816261 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:27.816059    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f9765899f-nlhq4: secret "image-registry-tls" not found
Jul 28 10:08:27.816261 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:27.816074    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94b8b045-295f-4194-a022-913c35c0529f-samples-operator-tls podName:94b8b045-295f-4194-a022-913c35c0529f nodeName:}" failed. No retries permitted until 2026-07-28 10:08:43.816052379 +0000 UTC m=+67.257593695 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/94b8b045-295f-4194-a022-913c35c0529f-samples-operator-tls") pod "cluster-samples-operator-6748489656-vjpcz" (UID: "94b8b045-295f-4194-a022-913c35c0529f") : secret "samples-operator-tls" not found
Jul 28 10:08:27.816261 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:27.816106    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9e06bdeb-9ead-430e-a448-950a1d18cc7a-registry-tls podName:9e06bdeb-9ead-430e-a448-950a1d18cc7a nodeName:}" failed. No retries permitted until 2026-07-28 10:08:43.81609578 +0000 UTC m=+67.257637095 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9e06bdeb-9ead-430e-a448-950a1d18cc7a-registry-tls") pod "image-registry-7f9765899f-nlhq4" (UID: "9e06bdeb-9ead-430e-a448-950a1d18cc7a") : secret "image-registry-tls" not found
Jul 28 10:08:27.816261 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:27.816111    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 10:08:27.816261 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:27.816121    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/70a2f95e-67d1-4353-b4d9-d59a5c23f970-service-ca-bundle podName:70a2f95e-67d1-4353-b4d9-d59a5c23f970 nodeName:}" failed. No retries permitted until 2026-07-28 10:08:43.81611347 +0000 UTC m=+67.257654785 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/70a2f95e-67d1-4353-b4d9-d59a5c23f970-service-ca-bundle") pod "router-default-65b54959b5-rhznz" (UID: "70a2f95e-67d1-4353-b4d9-d59a5c23f970") : configmap references non-existent config key: service-ca.crt
Jul 28 10:08:27.816261 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:27.816156    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70a2f95e-67d1-4353-b4d9-d59a5c23f970-metrics-certs podName:70a2f95e-67d1-4353-b4d9-d59a5c23f970 nodeName:}" failed. No retries permitted until 2026-07-28 10:08:43.816142487 +0000 UTC m=+67.257683814 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/70a2f95e-67d1-4353-b4d9-d59a5c23f970-metrics-certs") pod "router-default-65b54959b5-rhznz" (UID: "70a2f95e-67d1-4353-b4d9-d59a5c23f970") : secret "router-metrics-certs-default" not found
Jul 28 10:08:27.917182 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:27.917137    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3014f727-2c79-465d-ba86-bbea763014e5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-x6bhh\" (UID: \"3014f727-2c79-465d-ba86-bbea763014e5\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-x6bhh"
Jul 28 10:08:27.917392 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:27.917210    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/826225a0-88f8-4113-9f7d-05a75796d40d-metrics-tls\") pod \"dns-default-thhvn\" (UID: \"826225a0-88f8-4113-9f7d-05a75796d40d\") " pod="openshift-dns/dns-default-thhvn"
Jul 28 10:08:27.917392 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:27.917296    2563 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 10:08:27.917392 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:27.917304    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 10:08:27.917392 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:27.917387    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3014f727-2c79-465d-ba86-bbea763014e5-cluster-monitoring-operator-tls podName:3014f727-2c79-465d-ba86-bbea763014e5 nodeName:}" failed. No retries permitted until 2026-07-28 10:08:43.917345063 +0000 UTC m=+67.358886378 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3014f727-2c79-465d-ba86-bbea763014e5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-x6bhh" (UID: "3014f727-2c79-465d-ba86-bbea763014e5") : secret "cluster-monitoring-operator-tls" not found
Jul 28 10:08:27.917539 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:27.917407    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/826225a0-88f8-4113-9f7d-05a75796d40d-metrics-tls podName:826225a0-88f8-4113-9f7d-05a75796d40d nodeName:}" failed. No retries permitted until 2026-07-28 10:08:43.917399847 +0000 UTC m=+67.358941162 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/826225a0-88f8-4113-9f7d-05a75796d40d-metrics-tls") pod "dns-default-thhvn" (UID: "826225a0-88f8-4113-9f7d-05a75796d40d") : secret "dns-default-metrics-tls" not found
Jul 28 10:08:28.065621 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:28.065590    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-czhcl_f8b50686-28d7-4749-9430-54bd0f88f2f2/kube-storage-version-migrator-operator/0.log"
Jul 28 10:08:30.540933 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:30.540889    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/562ec3c6-ef0d-41f0-9279-498b868331e8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vj25w\" (UID: \"562ec3c6-ef0d-41f0-9279-498b868331e8\") " pod="openshift-insights/insights-runtime-extractor-vj25w"
Jul 28 10:08:30.541462 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:30.541023    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/562ec3c6-ef0d-41f0-9279-498b868331e8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vj25w\" (UID: \"562ec3c6-ef0d-41f0-9279-498b868331e8\") " pod="openshift-insights/insights-runtime-extractor-vj25w"
Jul 28 10:08:30.541462 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:30.541181    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 10:08:30.541462 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:30.541265    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/562ec3c6-ef0d-41f0-9279-498b868331e8-insights-runtime-extractor-tls podName:562ec3c6-ef0d-41f0-9279-498b868331e8 nodeName:}" failed. No retries permitted until 2026-07-28 10:08:38.541243629 +0000 UTC m=+61.982784944 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/562ec3c6-ef0d-41f0-9279-498b868331e8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vj25w" (UID: "562ec3c6-ef0d-41f0-9279-498b868331e8") : secret "insights-runtime-extractor-tls" not found
Jul 28 10:08:30.541618 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:30.541597    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/562ec3c6-ef0d-41f0-9279-498b868331e8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vj25w\" (UID: \"562ec3c6-ef0d-41f0-9279-498b868331e8\") " pod="openshift-insights/insights-runtime-extractor-vj25w"
Jul 28 10:08:32.320896 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:32.320843    2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-g7s47"
Jul 28 10:08:32.321443 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:32.320925    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-g7s47"
Jul 28 10:08:32.321443 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:32.321270    2563 scope.go:117] "RemoveContainer" containerID="84bc55e3ec16b9abe5fbab4e83da8b762f5cbdb78489f421e780d89c516d3371"
Jul 28 10:08:32.430973 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:32.430940    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 10:08:32.431159 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:32.431050    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-g7s47" event={"ID":"bebcadc4-87cd-4b77-997e-bf1daa3bdfb5","Type":"ContainerStarted","Data":"9ec49e0d3b3304c183660747fc8ab71e98f5751d0fa005853f5f7de0fdf0badf"}
Jul 28 10:08:32.431420 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:32.431391    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-g7s47"
Jul 28 10:08:32.433908 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:32.433864    2563 patch_prober.go:28] interesting pod/console-operator-584dcb6b54-g7s47 container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.13:8443/readyz\": dial tcp 10.132.0.13:8443: connect: connection refused" start-of-body=
Jul 28 10:08:32.434050 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:32.433930    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-584dcb6b54-g7s47" podUID="bebcadc4-87cd-4b77-997e-bf1daa3bdfb5" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.13:8443/readyz\": dial tcp 10.132.0.13:8443: connect: connection refused"
Jul 28 10:08:32.453328 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:32.453267    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-g7s47" podStartSLOduration=23.257956419 podStartE2EDuration="30.453210436s" totalImagesPullingTime="7.195254017s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:08:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:08:32.452864663 +0000 UTC m=+55.894406000" watchObservedRunningTime="2026-07-28 10:08:32.453210436 +0000 UTC m=+55.894751774"
Jul 28 10:08:33.439501 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:33.439467    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-g7s47"
Jul 28 10:08:37.313495 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:37.313461    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4sg48"
Jul 28 10:08:38.618761 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:38.618721    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/562ec3c6-ef0d-41f0-9279-498b868331e8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vj25w\" (UID: \"562ec3c6-ef0d-41f0-9279-498b868331e8\") " pod="openshift-insights/insights-runtime-extractor-vj25w"
Jul 28 10:08:38.621221 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:38.621195    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/562ec3c6-ef0d-41f0-9279-498b868331e8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vj25w\" (UID: \"562ec3c6-ef0d-41f0-9279-498b868331e8\") " pod="openshift-insights/insights-runtime-extractor-vj25w"
Jul 28 10:08:38.673444 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:38.673414    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-q8lkz\""
Jul 28 10:08:38.681511 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:38.681483    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vj25w"
Jul 28 10:08:38.817821 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:38.817791    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vj25w"]
Jul 28 10:08:38.820723 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:08:38.820700    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod562ec3c6_ef0d_41f0_9279_498b868331e8.slice/crio-30eb619f8e851a1b38ff5607f211ac047296a2aaf6f71966f2bcb6a127291257 WatchSource:0}: Error finding container 30eb619f8e851a1b38ff5607f211ac047296a2aaf6f71966f2bcb6a127291257: Status 404 returned error can't find the container with id 30eb619f8e851a1b38ff5607f211ac047296a2aaf6f71966f2bcb6a127291257
Jul 28 10:08:39.457907 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:39.457807    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vj25w" event={"ID":"562ec3c6-ef0d-41f0-9279-498b868331e8","Type":"ContainerStarted","Data":"30eb619f8e851a1b38ff5607f211ac047296a2aaf6f71966f2bcb6a127291257"}
Jul 28 10:08:42.757585 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:42.757545    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f9bc3b4-d2b8-40de-8c04-846085757396-original-pull-secret\") pod \"global-pull-secret-syncer-nz956\" (UID: \"5f9bc3b4-d2b8-40de-8c04-846085757396\") " pod="kube-system/global-pull-secret-syncer-nz956"
Jul 28 10:08:42.758076 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:42.757607    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfe1e00b-5d1b-4127-b270-abe1d44d3dd0-metrics-certs\") pod \"network-metrics-daemon-299qd\" (UID: \"bfe1e00b-5d1b-4127-b270-abe1d44d3dd0\") " pod="openshift-multus/network-metrics-daemon-299qd"
Jul 28 10:08:42.759991 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:42.759963    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 10:08:42.760113 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:42.759967    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 10:08:42.770600 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:42.770558    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfe1e00b-5d1b-4127-b270-abe1d44d3dd0-metrics-certs\") pod \"network-metrics-daemon-299qd\" (UID: \"bfe1e00b-5d1b-4127-b270-abe1d44d3dd0\") " pod="openshift-multus/network-metrics-daemon-299qd"
Jul 28 10:08:42.770768 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:42.770676    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f9bc3b4-d2b8-40de-8c04-846085757396-original-pull-secret\") pod \"global-pull-secret-syncer-nz956\" (UID: \"5f9bc3b4-d2b8-40de-8c04-846085757396\") " pod="kube-system/global-pull-secret-syncer-nz956"
Jul 28 10:08:42.843402 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:42.843348    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l65kq\""
Jul 28 10:08:42.851649 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:42.851622    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-299qd"
Jul 28 10:08:42.854958 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:42.854924    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nz956"
Jul 28 10:08:42.959281 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:42.959242    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxmcx\" (UniqueName: \"kubernetes.io/projected/598884b3-5fd8-42ae-87ee-4f24d6c4bdb3-kube-api-access-cxmcx\") pod \"network-check-target-cmrjd\" (UID: \"598884b3-5fd8-42ae-87ee-4f24d6c4bdb3\") " pod="openshift-network-diagnostics/network-check-target-cmrjd"
Jul 28 10:08:42.962202 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:42.962165    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxmcx\" (UniqueName: \"kubernetes.io/projected/598884b3-5fd8-42ae-87ee-4f24d6c4bdb3-kube-api-access-cxmcx\") pod \"network-check-target-cmrjd\" (UID: \"598884b3-5fd8-42ae-87ee-4f24d6c4bdb3\") " pod="openshift-network-diagnostics/network-check-target-cmrjd"
Jul 28 10:08:43.150371 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:43.150330    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xsq5w\""
Jul 28 10:08:43.158227 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:43.158195    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cmrjd"
Jul 28 10:08:43.869163 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:43.868020    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9e06bdeb-9ead-430e-a448-950a1d18cc7a-registry-tls\") pod \"image-registry-7f9765899f-nlhq4\" (UID: \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\") " pod="openshift-image-registry/image-registry-7f9765899f-nlhq4"
Jul 28 10:08:43.869163 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:43.868081    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70a2f95e-67d1-4353-b4d9-d59a5c23f970-service-ca-bundle\") pod \"router-default-65b54959b5-rhznz\" (UID: \"70a2f95e-67d1-4353-b4d9-d59a5c23f970\") " pod="openshift-ingress/router-default-65b54959b5-rhznz"
Jul 28 10:08:43.869163 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:43.868115    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70a2f95e-67d1-4353-b4d9-d59a5c23f970-metrics-certs\") pod \"router-default-65b54959b5-rhznz\" (UID: \"70a2f95e-67d1-4353-b4d9-d59a5c23f970\") " pod="openshift-ingress/router-default-65b54959b5-rhznz"
Jul 28 10:08:43.869163 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:43.868181    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/94b8b045-295f-4194-a022-913c35c0529f-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vjpcz\" (UID: \"94b8b045-295f-4194-a022-913c35c0529f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vjpcz"
Jul 28 10:08:43.871144 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:43.871086    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70a2f95e-67d1-4353-b4d9-d59a5c23f970-service-ca-bundle\") pod \"router-default-65b54959b5-rhznz\" (UID: \"70a2f95e-67d1-4353-b4d9-d59a5c23f970\") " pod="openshift-ingress/router-default-65b54959b5-rhznz"
Jul 28 10:08:43.871801 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:43.871573    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9e06bdeb-9ead-430e-a448-950a1d18cc7a-registry-tls\") pod \"image-registry-7f9765899f-nlhq4\" (UID: \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\") " pod="openshift-image-registry/image-registry-7f9765899f-nlhq4"
Jul 28 10:08:43.873417 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:43.872952    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/94b8b045-295f-4194-a022-913c35c0529f-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-vjpcz\" (UID: \"94b8b045-295f-4194-a022-913c35c0529f\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vjpcz"
Jul 28 10:08:43.873417 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:43.873379    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70a2f95e-67d1-4353-b4d9-d59a5c23f970-metrics-certs\") pod \"router-default-65b54959b5-rhznz\" (UID: \"70a2f95e-67d1-4353-b4d9-d59a5c23f970\") " pod="openshift-ingress/router-default-65b54959b5-rhznz"
Jul 28 10:08:43.969494 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:43.968832    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/826225a0-88f8-4113-9f7d-05a75796d40d-metrics-tls\") pod \"dns-default-thhvn\" (UID: \"826225a0-88f8-4113-9f7d-05a75796d40d\") " pod="openshift-dns/dns-default-thhvn"
Jul 28 10:08:43.969494 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:43.968939    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3014f727-2c79-465d-ba86-bbea763014e5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-x6bhh\" (UID: \"3014f727-2c79-465d-ba86-bbea763014e5\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-x6bhh"
Jul 28 10:08:43.973034 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:43.972989    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3014f727-2c79-465d-ba86-bbea763014e5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-x6bhh\" (UID: \"3014f727-2c79-465d-ba86-bbea763014e5\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-x6bhh"
Jul 28 10:08:43.973761 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:43.973719    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/826225a0-88f8-4113-9f7d-05a75796d40d-metrics-tls\") pod \"dns-default-thhvn\" (UID: \"826225a0-88f8-4113-9f7d-05a75796d40d\") " pod="openshift-dns/dns-default-thhvn"
Jul 28 10:08:44.011179 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:44.011112    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cmrjd"]
Jul 28 10:08:44.015516 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:08:44.014685    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod598884b3_5fd8_42ae_87ee_4f24d6c4bdb3.slice/crio-dbfd3e3fcf29ab6fdc8e5da4667201e940ff7aeac63393933fe2fa64ddb0a557 WatchSource:0}: Error finding container dbfd3e3fcf29ab6fdc8e5da4667201e940ff7aeac63393933fe2fa64ddb0a557: Status 404 returned error can't find the container with id dbfd3e3fcf29ab6fdc8e5da4667201e940ff7aeac63393933fe2fa64ddb0a557
Jul 28 10:08:44.032123 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:44.032087    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nz956"]
Jul 28 10:08:44.035208 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:08:44.035177    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f9bc3b4_d2b8_40de_8c04_846085757396.slice/crio-92d481be3715d4e06a5d88f307fcbdeef12e6a4daa1ba0410c5f9b953c4b5362 WatchSource:0}: Error finding container 92d481be3715d4e06a5d88f307fcbdeef12e6a4daa1ba0410c5f9b953c4b5362: Status 404 returned error can't find the container with id 92d481be3715d4e06a5d88f307fcbdeef12e6a4daa1ba0410c5f9b953c4b5362
Jul 28 10:08:44.037050 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:44.037028    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-299qd"]
Jul 28 10:08:44.043502 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:08:44.043462    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfe1e00b_5d1b_4127_b270_abe1d44d3dd0.slice/crio-0c7ad16398212bd8df6689705d382f4bd7b2005f5c8324e315add233f18b185b WatchSource:0}: Error finding container 0c7ad16398212bd8df6689705d382f4bd7b2005f5c8324e315add233f18b185b: Status 404 returned error can't find the container with id 0c7ad16398212bd8df6689705d382f4bd7b2005f5c8324e315add233f18b185b
Jul 28 10:08:44.102413 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:44.102300    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-m29b4\""
Jul 28 10:08:44.110289 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:44.110269    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vjpcz"
Jul 28 10:08:44.117411 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:44.117386    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-f68cp\""
Jul 28 10:08:44.125779 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:44.125750    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-65b54959b5-rhznz"
Jul 28 10:08:44.139793 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:44.139767    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-fn5hp\""
Jul 28 10:08:44.147110 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:44.147080    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f9765899f-nlhq4"
Jul 28 10:08:44.154003 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:44.153974    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-6pzkk\""
Jul 28 10:08:44.161862 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:44.161647    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-x6bhh"
Jul 28 10:08:44.199213 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:44.199185    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dv6fr\""
Jul 28 10:08:44.206190 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:44.206123    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-thhvn"
Jul 28 10:08:44.290710 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:44.290652    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vjpcz"]
Jul 28 10:08:44.319803 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:44.319770    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-65b54959b5-rhznz"]
Jul 28 10:08:44.328066 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:08:44.328029    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70a2f95e_67d1_4353_b4d9_d59a5c23f970.slice/crio-78475de64b54b05e6f86a53325db02a98c8d948b1a8891d67987a12ee695184d WatchSource:0}: Error finding container 78475de64b54b05e6f86a53325db02a98c8d948b1a8891d67987a12ee695184d: Status 404 returned error can't find the container with id 78475de64b54b05e6f86a53325db02a98c8d948b1a8891d67987a12ee695184d
Jul 28 10:08:44.344986 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:44.344929    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7f9765899f-nlhq4"]
Jul 28 10:08:44.348124 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:08:44.348062    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e06bdeb_9ead_430e_a448_950a1d18cc7a.slice/crio-db02a1558335d4b87d5eb071a6846a793baab88eaa64df7bab4472ac3c4b3b1c WatchSource:0}: Error finding container db02a1558335d4b87d5eb071a6846a793baab88eaa64df7bab4472ac3c4b3b1c: Status 404 returned error can't find the container with id db02a1558335d4b87d5eb071a6846a793baab88eaa64df7bab4472ac3c4b3b1c
Jul 28 10:08:44.361492 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:44.361399    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-x6bhh"]
Jul 28 10:08:44.368511 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:08:44.368483    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3014f727_2c79_465d_ba86_bbea763014e5.slice/crio-a476bf26cdc98c73dd138ba710c23a9da36e97b6794938d3a6c7acf3b058c4b3 WatchSource:0}: Error finding container a476bf26cdc98c73dd138ba710c23a9da36e97b6794938d3a6c7acf3b058c4b3: Status 404 returned error can't find the container with id a476bf26cdc98c73dd138ba710c23a9da36e97b6794938d3a6c7acf3b058c4b3
Jul 28 10:08:44.400845 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:44.400816    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-thhvn"]
Jul 28 10:08:44.404530 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:08:44.404504    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod826225a0_88f8_4113_9f7d_05a75796d40d.slice/crio-bf1d17b391319cb97ee7799a155df05a7225705583319451f91f4222da6701c2 WatchSource:0}: Error finding container bf1d17b391319cb97ee7799a155df05a7225705583319451f91f4222da6701c2: Status 404 returned error can't find the container with id bf1d17b391319cb97ee7799a155df05a7225705583319451f91f4222da6701c2
Jul 28 10:08:44.478467 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:44.478435    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vj25w" event={"ID":"562ec3c6-ef0d-41f0-9279-498b868331e8","Type":"ContainerStarted","Data":"bacced94e32ec3a25d73f3e18747acf5a3714006db9cca5dd06c2160769e54e8"}
Jul 28 10:08:44.479507 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:44.479466    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vjpcz" event={"ID":"94b8b045-295f-4194-a022-913c35c0529f","Type":"ContainerStarted","Data":"d2022278f52cd440c760874febe7ef97fc6d23c20db5cb9f08f21e55d06f1160"}
Jul 28 10:08:44.480391 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:44.480369    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nz956" event={"ID":"5f9bc3b4-d2b8-40de-8c04-846085757396","Type":"ContainerStarted","Data":"92d481be3715d4e06a5d88f307fcbdeef12e6a4daa1ba0410c5f9b953c4b5362"}
Jul 28 10:08:44.481384 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:44.481339    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-thhvn" event={"ID":"826225a0-88f8-4113-9f7d-05a75796d40d","Type":"ContainerStarted","Data":"bf1d17b391319cb97ee7799a155df05a7225705583319451f91f4222da6701c2"}
Jul 28 10:08:44.482800 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:44.482611    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-299qd" event={"ID":"bfe1e00b-5d1b-4127-b270-abe1d44d3dd0","Type":"ContainerStarted","Data":"0c7ad16398212bd8df6689705d382f4bd7b2005f5c8324e315add233f18b185b"}
Jul 28 10:08:44.485847 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:44.485821    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f9765899f-nlhq4" event={"ID":"9e06bdeb-9ead-430e-a448-950a1d18cc7a","Type":"ContainerStarted","Data":"d81d69f996b10ea4de52a4e32b636d7a9a4c6231b309a08ceb562cf9ed1d88f8"}
Jul 28 10:08:44.485944 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:44.485864    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f9765899f-nlhq4" event={"ID":"9e06bdeb-9ead-430e-a448-950a1d18cc7a","Type":"ContainerStarted","Data":"db02a1558335d4b87d5eb071a6846a793baab88eaa64df7bab4472ac3c4b3b1c"}
Jul 28 10:08:44.486086 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:44.486071    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7f9765899f-nlhq4"
Jul 28 10:08:44.487404 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:44.487322    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cmrjd" event={"ID":"598884b3-5fd8-42ae-87ee-4f24d6c4bdb3","Type":"ContainerStarted","Data":"62efacb1952909b3ff0330e4fe2fe0feefff2c0f92445506fe8c5b4fe527e11f"}
Jul 28 10:08:44.487404 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:44.487390    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cmrjd" event={"ID":"598884b3-5fd8-42ae-87ee-4f24d6c4bdb3","Type":"ContainerStarted","Data":"dbfd3e3fcf29ab6fdc8e5da4667201e940ff7aeac63393933fe2fa64ddb0a557"}
Jul 28 10:08:44.487589 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:44.487421    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-cmrjd"
Jul 28 10:08:44.488781 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:44.488758    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-x6bhh" event={"ID":"3014f727-2c79-465d-ba86-bbea763014e5","Type":"ContainerStarted","Data":"a476bf26cdc98c73dd138ba710c23a9da36e97b6794938d3a6c7acf3b058c4b3"}
Jul 28 10:08:44.490289 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:44.490268    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-65b54959b5-rhznz" event={"ID":"70a2f95e-67d1-4353-b4d9-d59a5c23f970","Type":"ContainerStarted","Data":"99d49ecc7785908fe9a902de1e31523da618528c5a497268451091f35814441a"}
Jul 28 10:08:44.490419 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:44.490299    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-65b54959b5-rhznz" event={"ID":"70a2f95e-67d1-4353-b4d9-d59a5c23f970","Type":"ContainerStarted","Data":"78475de64b54b05e6f86a53325db02a98c8d948b1a8891d67987a12ee695184d"}
Jul 28 10:08:44.515157 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:44.515088    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7f9765899f-nlhq4" podStartSLOduration=66.515065425 podStartE2EDuration="1m6.515065425s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:07:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:08:44.513316839 +0000 UTC m=+67.954858177" watchObservedRunningTime="2026-07-28 10:08:44.515065425 +0000 UTC m=+67.956606763"
Jul 28 10:08:44.534704 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:44.534650    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-65b54959b5-rhznz" podStartSLOduration=42.534631967 podStartE2EDuration="42.534631967s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:08:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:08:44.533576675 +0000 UTC m=+67.975118033" watchObservedRunningTime="2026-07-28 10:08:44.534631967 +0000 UTC m=+67.976173307"
Jul 28 10:08:45.134148 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:45.133804    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-65b54959b5-rhznz"
Jul 28 10:08:45.134148 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:45.133938    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-65b54959b5-rhznz"
Jul 28 10:08:45.160314 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:45.160246    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-cmrjd" podStartSLOduration=68.160221524 podStartE2EDuration="1m8.160221524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:07:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:08:44.549492594 +0000 UTC m=+67.991033930" watchObservedRunningTime="2026-07-28 10:08:45.160221524 +0000 UTC m=+68.601762859"
Jul 28 10:08:45.496249 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:45.496132    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-65b54959b5-rhznz"
Jul 28 10:08:45.497993 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:45.497965    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-65b54959b5-rhznz"
Jul 28 10:08:53.511927 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:53.511894    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9pwjg"]
Jul 28 10:08:53.514079 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:53.514056    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9pwjg"
Jul 28 10:08:53.516271 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:53.516247    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 28 10:08:53.516404 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:53.516292    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-98k9g\""
Jul 28 10:08:53.529014 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:53.528983    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9pwjg"]
Jul 28 10:08:53.530275 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:53.530246    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-x6bhh" event={"ID":"3014f727-2c79-465d-ba86-bbea763014e5","Type":"ContainerStarted","Data":"c8bdcdc6a1cfd4eebded0459b2d927972547ce51a886397965f50eb9e75af6e5"}
Jul 28 10:08:53.532202 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:53.532178    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vj25w" event={"ID":"562ec3c6-ef0d-41f0-9279-498b868331e8","Type":"ContainerStarted","Data":"0f82e3b27309818ad14aa555d5103e5b6376373200ebe79e3a63a8329b53efb4"}
Jul 28 10:08:53.534191 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:53.534157    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vjpcz" event={"ID":"94b8b045-295f-4194-a022-913c35c0529f","Type":"ContainerStarted","Data":"ba373e32cc164a272f66bb41eaf97e1d2ffd0036d2c5d758d21a2ec71bdfaedf"}
Jul 28 10:08:53.534300 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:53.534201    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vjpcz" event={"ID":"94b8b045-295f-4194-a022-913c35c0529f","Type":"ContainerStarted","Data":"3b29fc501efec50e5ee277021047672b27b4f1155c8d1efd496fffbebdb1d4fb"}
Jul 28 10:08:53.535747 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:53.535725    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nz956" event={"ID":"5f9bc3b4-d2b8-40de-8c04-846085757396","Type":"ContainerStarted","Data":"7b3281490c8bd4c9938ff9c651bd8e72f7f3b2964d2086f4119fa6fea3b5564a"}
Jul 28 10:08:53.537759 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:53.537737    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-thhvn" event={"ID":"826225a0-88f8-4113-9f7d-05a75796d40d","Type":"ContainerStarted","Data":"e939e4cacd4eee5996978ec8fe230e8e00909cee2430678f144f260db268a743"}
Jul 28 10:08:53.537867 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:53.537783    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-thhvn" event={"ID":"826225a0-88f8-4113-9f7d-05a75796d40d","Type":"ContainerStarted","Data":"659b5922a36346261bd13ae44904dec310a2b6b9e6d7f37f273b56e0a0468c0d"}
Jul 28 10:08:53.537924 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:53.537861    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-thhvn"
Jul 28 10:08:53.540265 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:53.540242    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-299qd" event={"ID":"bfe1e00b-5d1b-4127-b270-abe1d44d3dd0","Type":"ContainerStarted","Data":"977ab22e1823ce73b0a67c22233b120710f1412d3b758bb5a8dadeb41cee50bc"}
Jul 28 10:08:53.540398 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:53.540290    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-299qd" event={"ID":"bfe1e00b-5d1b-4127-b270-abe1d44d3dd0","Type":"ContainerStarted","Data":"678d3d840274ddb9345b17067347f0c601b32c91e0963ebed964c041c77c1ea0"}
Jul 28 10:08:53.562410 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:53.562328    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-thhvn" podStartSLOduration=34.135969637 podStartE2EDuration="42.562309617s" totalImagesPullingTime="8.42633998s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:08:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:08:53.560061704 +0000 UTC m=+77.001603066" watchObservedRunningTime="2026-07-28 10:08:53.562309617 +0000 UTC m=+77.003850957"
Jul 28 10:08:53.578476 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:53.578422    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-vjpcz" podStartSLOduration=42.432918372 podStartE2EDuration="50.578402215s" totalImagesPullingTime="8.145483843s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:08:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:08:53.57775126 +0000 UTC m=+77.019292618" watchObservedRunningTime="2026-07-28 10:08:53.578402215 +0000 UTC m=+77.019943553"
Jul 28 10:08:53.600586 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:53.600478    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-x6bhh" podStartSLOduration=43.46223316 podStartE2EDuration="51.600460889s" totalImagesPullingTime="8.138227729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:08:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:08:53.599769143 +0000 UTC m=+77.041310481" watchObservedRunningTime="2026-07-28 10:08:53.600460889 +0000 UTC m=+77.042002225"
Jul 28 10:08:53.617096 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:53.617043    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-nz956" podStartSLOduration=67.774876401 podStartE2EDuration="1m16.617026009s" totalImagesPullingTime="8.842149608s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 10:07:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:08:53.616482096 +0000 UTC m=+77.058023442" watchObservedRunningTime="2026-07-28 10:08:53.617026009 +0000 UTC m=+77.058567346"
Jul 28 10:08:53.642682 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:53.642617    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-299qd" podStartSLOduration=68.888274825 podStartE2EDuration="1m16.642594156s" totalImagesPullingTime="7.754319331s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:07:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:08:53.639107177 +0000 UTC m=+77.080648533" watchObservedRunningTime="2026-07-28 10:08:53.642594156 +0000 UTC m=+77.084135493"
Jul 28 10:08:53.654727 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:53.654688    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/82569a5b-837e-48eb-8f0e-efd6e2538b19-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-9pwjg\" (UID: \"82569a5b-837e-48eb-8f0e-efd6e2538b19\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9pwjg"
Jul 28 10:08:53.756041 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:53.755996    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/82569a5b-837e-48eb-8f0e-efd6e2538b19-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-9pwjg\" (UID: \"82569a5b-837e-48eb-8f0e-efd6e2538b19\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9pwjg"
Jul 28 10:08:53.756229 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:53.756151    2563 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 28 10:08:53.756229 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:08:53.756226    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82569a5b-837e-48eb-8f0e-efd6e2538b19-tls-certificates podName:82569a5b-837e-48eb-8f0e-efd6e2538b19 nodeName:}" failed. No retries permitted until 2026-07-28 10:08:54.256208473 +0000 UTC m=+77.697749791 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/82569a5b-837e-48eb-8f0e-efd6e2538b19-tls-certificates") pod "prometheus-operator-admission-webhook-7f8d78f54b-9pwjg" (UID: "82569a5b-837e-48eb-8f0e-efd6e2538b19") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 28 10:08:54.260952 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:54.260917    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/82569a5b-837e-48eb-8f0e-efd6e2538b19-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-9pwjg\" (UID: \"82569a5b-837e-48eb-8f0e-efd6e2538b19\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9pwjg"
Jul 28 10:08:54.263718 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:54.263688    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/82569a5b-837e-48eb-8f0e-efd6e2538b19-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-9pwjg\" (UID: \"82569a5b-837e-48eb-8f0e-efd6e2538b19\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9pwjg"
Jul 28 10:08:54.423722 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:54.423689    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9pwjg"
Jul 28 10:08:54.582255 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:54.582218    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9pwjg"]
Jul 28 10:08:54.586020 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:08:54.585980    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82569a5b_837e_48eb_8f0e_efd6e2538b19.slice/crio-093379ed58e74c951f4de6496ef5817686c8fd69c6666ec9887e03ec242b6e8e WatchSource:0}: Error finding container 093379ed58e74c951f4de6496ef5817686c8fd69c6666ec9887e03ec242b6e8e: Status 404 returned error can't find the container with id 093379ed58e74c951f4de6496ef5817686c8fd69c6666ec9887e03ec242b6e8e
Jul 28 10:08:55.553051 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:55.552999    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9pwjg" event={"ID":"82569a5b-837e-48eb-8f0e-efd6e2538b19","Type":"ContainerStarted","Data":"093379ed58e74c951f4de6496ef5817686c8fd69c6666ec9887e03ec242b6e8e"}
Jul 28 10:08:57.564117 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:57.564029    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vj25w" event={"ID":"562ec3c6-ef0d-41f0-9279-498b868331e8","Type":"ContainerStarted","Data":"8c920131e89d2ac0de8452c3ad3e0ac0c481dd6fa1298f21da89063b36023ccc"}
Jul 28 10:08:57.565638 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:57.565614    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9pwjg" event={"ID":"82569a5b-837e-48eb-8f0e-efd6e2538b19","Type":"ContainerStarted","Data":"7e803c473474b2ec2263c78a2049a580fd30e70c12c8aa5fed215445d6610dce"}
Jul 28 10:08:57.565848 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:57.565823    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9pwjg"
Jul 28 10:08:57.570759 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:57.570734    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9pwjg"
Jul 28 10:08:57.591628 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:57.591575    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-vj25w" podStartSLOduration=17.401805081 podStartE2EDuration="35.591559684s" totalImagesPullingTime="18.189754603s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 10:08:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:08:57.591133279 +0000 UTC m=+81.032674638" watchObservedRunningTime="2026-07-28 10:08:57.591559684 +0000 UTC m=+81.033101021"
Jul 28 10:08:57.612261 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:08:57.612211    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-9pwjg" podStartSLOduration=1.9189909 podStartE2EDuration="4.61219387s" totalImagesPullingTime="2.69320297s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:08:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:08:57.611971819 +0000 UTC m=+81.053513156" watchObservedRunningTime="2026-07-28 10:08:57.61219387 +0000 UTC m=+81.053735205"
Jul 28 10:09:03.549676 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:03.549643    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-thhvn"
Jul 28 10:09:05.499655 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:05.499625    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7f9765899f-nlhq4"
Jul 28 10:09:09.960739 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:09.960695    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7f9765899f-nlhq4"]
Jul 28 10:09:15.036185 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.036145    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-g2lz6"]
Jul 28 10:09:15.065644 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.065611    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-gxdc6"]
Jul 28 10:09:15.065820 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.065760    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-g2lz6"
Jul 28 10:09:15.070789 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.070759    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-g2lz6"]
Jul 28 10:09:15.070975 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.070861    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gxdc6"
Jul 28 10:09:15.071534 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.071507    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 28 10:09:15.071673 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.071515    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 28 10:09:15.071673 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.071565    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 28 10:09:15.071673 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.071638    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-zdrbl\""
Jul 28 10:09:15.075927 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.075900    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 28 10:09:15.076211 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.075927    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 28 10:09:15.076309 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.076225    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-xqkxz\""
Jul 28 10:09:15.076309 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.075933    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 28 10:09:15.124920 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.124888    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/995d66bd-d908-4743-90f3-e5db8d8f5fcb-node-exporter-tls\") pod \"node-exporter-gxdc6\" (UID: \"995d66bd-d908-4743-90f3-e5db8d8f5fcb\") " pod="openshift-monitoring/node-exporter-gxdc6"
Jul 28 10:09:15.124920 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.124926    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/995d66bd-d908-4743-90f3-e5db8d8f5fcb-node-exporter-textfile\") pod \"node-exporter-gxdc6\" (UID: \"995d66bd-d908-4743-90f3-e5db8d8f5fcb\") " pod="openshift-monitoring/node-exporter-gxdc6"
Jul 28 10:09:15.125210 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.124946    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/995d66bd-d908-4743-90f3-e5db8d8f5fcb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gxdc6\" (UID: \"995d66bd-d908-4743-90f3-e5db8d8f5fcb\") " pod="openshift-monitoring/node-exporter-gxdc6"
Jul 28 10:09:15.125210 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.124963    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8s54\" (UniqueName: \"kubernetes.io/projected/995d66bd-d908-4743-90f3-e5db8d8f5fcb-kube-api-access-h8s54\") pod \"node-exporter-gxdc6\" (UID: \"995d66bd-d908-4743-90f3-e5db8d8f5fcb\") " pod="openshift-monitoring/node-exporter-gxdc6"
Jul 28 10:09:15.125210 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.124981    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/995d66bd-d908-4743-90f3-e5db8d8f5fcb-node-exporter-wtmp\") pod \"node-exporter-gxdc6\" (UID: \"995d66bd-d908-4743-90f3-e5db8d8f5fcb\") " pod="openshift-monitoring/node-exporter-gxdc6"
Jul 28 10:09:15.125210 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.125077    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksjg6\" (UniqueName: \"kubernetes.io/projected/a33c1bbf-f7c5-4302-93f0-3bf51a4b1f7f-kube-api-access-ksjg6\") pod \"openshift-state-metrics-b8f49f5c6-g2lz6\" (UID: \"a33c1bbf-f7c5-4302-93f0-3bf51a4b1f7f\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-g2lz6"
Jul 28 10:09:15.125210 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.125120    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/995d66bd-d908-4743-90f3-e5db8d8f5fcb-root\") pod \"node-exporter-gxdc6\" (UID: \"995d66bd-d908-4743-90f3-e5db8d8f5fcb\") " pod="openshift-monitoring/node-exporter-gxdc6"
Jul 28 10:09:15.125210 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.125154    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/995d66bd-d908-4743-90f3-e5db8d8f5fcb-sys\") pod \"node-exporter-gxdc6\" (UID: \"995d66bd-d908-4743-90f3-e5db8d8f5fcb\") " pod="openshift-monitoring/node-exporter-gxdc6"
Jul 28 10:09:15.125210 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.125185    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/995d66bd-d908-4743-90f3-e5db8d8f5fcb-metrics-client-ca\") pod \"node-exporter-gxdc6\" (UID: \"995d66bd-d908-4743-90f3-e5db8d8f5fcb\") " pod="openshift-monitoring/node-exporter-gxdc6"
Jul 28 10:09:15.125210 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.125212    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a33c1bbf-f7c5-4302-93f0-3bf51a4b1f7f-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-g2lz6\" (UID: \"a33c1bbf-f7c5-4302-93f0-3bf51a4b1f7f\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-g2lz6"
Jul 28 10:09:15.125567 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.125245    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a33c1bbf-f7c5-4302-93f0-3bf51a4b1f7f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-g2lz6\" (UID: \"a33c1bbf-f7c5-4302-93f0-3bf51a4b1f7f\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-g2lz6"
Jul 28 10:09:15.125567 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.125279    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a33c1bbf-f7c5-4302-93f0-3bf51a4b1f7f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-g2lz6\" (UID: \"a33c1bbf-f7c5-4302-93f0-3bf51a4b1f7f\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-g2lz6"
Jul 28 10:09:15.125567 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.125331    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/995d66bd-d908-4743-90f3-e5db8d8f5fcb-node-exporter-accelerators-collector-config\") pod \"node-exporter-gxdc6\" (UID: \"995d66bd-d908-4743-90f3-e5db8d8f5fcb\") " pod="openshift-monitoring/node-exporter-gxdc6"
Jul 28 10:09:15.225729 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.225689    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/995d66bd-d908-4743-90f3-e5db8d8f5fcb-node-exporter-accelerators-collector-config\") pod \"node-exporter-gxdc6\" (UID: \"995d66bd-d908-4743-90f3-e5db8d8f5fcb\") " pod="openshift-monitoring/node-exporter-gxdc6"
Jul 28 10:09:15.225729 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.225732    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/995d66bd-d908-4743-90f3-e5db8d8f5fcb-node-exporter-tls\") pod \"node-exporter-gxdc6\" (UID: \"995d66bd-d908-4743-90f3-e5db8d8f5fcb\") " pod="openshift-monitoring/node-exporter-gxdc6"
Jul 28 10:09:15.225972 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.225759    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/995d66bd-d908-4743-90f3-e5db8d8f5fcb-node-exporter-textfile\") pod \"node-exporter-gxdc6\" (UID: \"995d66bd-d908-4743-90f3-e5db8d8f5fcb\") " pod="openshift-monitoring/node-exporter-gxdc6"
Jul 28 10:09:15.225972 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.225776    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/995d66bd-d908-4743-90f3-e5db8d8f5fcb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gxdc6\" (UID: \"995d66bd-d908-4743-90f3-e5db8d8f5fcb\") " pod="openshift-monitoring/node-exporter-gxdc6"
Jul 28 10:09:15.225972 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.225793    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8s54\" (UniqueName: \"kubernetes.io/projected/995d66bd-d908-4743-90f3-e5db8d8f5fcb-kube-api-access-h8s54\") pod \"node-exporter-gxdc6\" (UID: \"995d66bd-d908-4743-90f3-e5db8d8f5fcb\") " pod="openshift-monitoring/node-exporter-gxdc6"
Jul 28 10:09:15.225972 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.225810    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/995d66bd-d908-4743-90f3-e5db8d8f5fcb-node-exporter-wtmp\") pod \"node-exporter-gxdc6\" (UID: \"995d66bd-d908-4743-90f3-e5db8d8f5fcb\") " pod="openshift-monitoring/node-exporter-gxdc6"
Jul 28 10:09:15.225972 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.225833    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksjg6\" (UniqueName: \"kubernetes.io/projected/a33c1bbf-f7c5-4302-93f0-3bf51a4b1f7f-kube-api-access-ksjg6\") pod \"openshift-state-metrics-b8f49f5c6-g2lz6\" (UID: \"a33c1bbf-f7c5-4302-93f0-3bf51a4b1f7f\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-g2lz6"
Jul 28 10:09:15.225972 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.225858    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/995d66bd-d908-4743-90f3-e5db8d8f5fcb-root\") pod \"node-exporter-gxdc6\" (UID: \"995d66bd-d908-4743-90f3-e5db8d8f5fcb\") " pod="openshift-monitoring/node-exporter-gxdc6"
Jul 28 10:09:15.225972 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.225888    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/995d66bd-d908-4743-90f3-e5db8d8f5fcb-sys\") pod \"node-exporter-gxdc6\" (UID: \"995d66bd-d908-4743-90f3-e5db8d8f5fcb\") " pod="openshift-monitoring/node-exporter-gxdc6"
Jul 28 10:09:15.225972 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.225917    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/995d66bd-d908-4743-90f3-e5db8d8f5fcb-metrics-client-ca\") pod \"node-exporter-gxdc6\" (UID: \"995d66bd-d908-4743-90f3-e5db8d8f5fcb\") " pod="openshift-monitoring/node-exporter-gxdc6"
Jul 28 10:09:15.225972 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.225945    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a33c1bbf-f7c5-4302-93f0-3bf51a4b1f7f-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-g2lz6\" (UID: \"a33c1bbf-f7c5-4302-93f0-3bf51a4b1f7f\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-g2lz6"
Jul 28 10:09:15.225972 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:09:15.225949    2563 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 28 10:09:15.225972 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.225974    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a33c1bbf-f7c5-4302-93f0-3bf51a4b1f7f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-g2lz6\" (UID: \"a33c1bbf-f7c5-4302-93f0-3bf51a4b1f7f\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-g2lz6"
Jul 28 10:09:15.226532 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.226004    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a33c1bbf-f7c5-4302-93f0-3bf51a4b1f7f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-g2lz6\" (UID: \"a33c1bbf-f7c5-4302-93f0-3bf51a4b1f7f\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-g2lz6"
Jul 28 10:09:15.226532 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:09:15.226024    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/995d66bd-d908-4743-90f3-e5db8d8f5fcb-node-exporter-tls podName:995d66bd-d908-4743-90f3-e5db8d8f5fcb nodeName:}" failed. No retries permitted until 2026-07-28 10:09:15.726001189 +0000 UTC m=+99.167542525 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/995d66bd-d908-4743-90f3-e5db8d8f5fcb-node-exporter-tls") pod "node-exporter-gxdc6" (UID: "995d66bd-d908-4743-90f3-e5db8d8f5fcb") : secret "node-exporter-tls" not found
Jul 28 10:09:15.226532 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.226032    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/995d66bd-d908-4743-90f3-e5db8d8f5fcb-node-exporter-wtmp\") pod \"node-exporter-gxdc6\" (UID: \"995d66bd-d908-4743-90f3-e5db8d8f5fcb\") " pod="openshift-monitoring/node-exporter-gxdc6"
Jul 28 10:09:15.226532 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.226514    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/995d66bd-d908-4743-90f3-e5db8d8f5fcb-node-exporter-accelerators-collector-config\") pod \"node-exporter-gxdc6\" (UID: \"995d66bd-d908-4743-90f3-e5db8d8f5fcb\") " pod="openshift-monitoring/node-exporter-gxdc6"
Jul 28 10:09:15.226754 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.226594    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/995d66bd-d908-4743-90f3-e5db8d8f5fcb-node-exporter-textfile\") pod \"node-exporter-gxdc6\" (UID: \"995d66bd-d908-4743-90f3-e5db8d8f5fcb\") " pod="openshift-monitoring/node-exporter-gxdc6"
Jul 28 10:09:15.226754 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.226597    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/995d66bd-d908-4743-90f3-e5db8d8f5fcb-root\") pod \"node-exporter-gxdc6\" (UID: \"995d66bd-d908-4743-90f3-e5db8d8f5fcb\") " pod="openshift-monitoring/node-exporter-gxdc6"
Jul 28 10:09:15.226754 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.226643    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/995d66bd-d908-4743-90f3-e5db8d8f5fcb-sys\") pod \"node-exporter-gxdc6\" (UID: \"995d66bd-d908-4743-90f3-e5db8d8f5fcb\") " pod="openshift-monitoring/node-exporter-gxdc6"
Jul 28 10:09:15.226754 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.226648    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/995d66bd-d908-4743-90f3-e5db8d8f5fcb-metrics-client-ca\") pod \"node-exporter-gxdc6\" (UID: \"995d66bd-d908-4743-90f3-e5db8d8f5fcb\") " pod="openshift-monitoring/node-exporter-gxdc6"
Jul 28 10:09:15.226987 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.226965    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a33c1bbf-f7c5-4302-93f0-3bf51a4b1f7f-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-g2lz6\" (UID: \"a33c1bbf-f7c5-4302-93f0-3bf51a4b1f7f\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-g2lz6"
Jul 28 10:09:15.228849 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.228818    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/995d66bd-d908-4743-90f3-e5db8d8f5fcb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-gxdc6\" (UID: \"995d66bd-d908-4743-90f3-e5db8d8f5fcb\") " pod="openshift-monitoring/node-exporter-gxdc6"
Jul 28 10:09:15.229102 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.229083    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a33c1bbf-f7c5-4302-93f0-3bf51a4b1f7f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-g2lz6\" (UID: \"a33c1bbf-f7c5-4302-93f0-3bf51a4b1f7f\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-g2lz6"
Jul 28 10:09:15.229276 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.229254    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a33c1bbf-f7c5-4302-93f0-3bf51a4b1f7f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-g2lz6\" (UID: \"a33c1bbf-f7c5-4302-93f0-3bf51a4b1f7f\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-g2lz6"
Jul 28 10:09:15.255716 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.255682    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksjg6\" (UniqueName: \"kubernetes.io/projected/a33c1bbf-f7c5-4302-93f0-3bf51a4b1f7f-kube-api-access-ksjg6\") pod \"openshift-state-metrics-b8f49f5c6-g2lz6\" (UID: \"a33c1bbf-f7c5-4302-93f0-3bf51a4b1f7f\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-g2lz6"
Jul 28 10:09:15.256787 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.256763    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8s54\" (UniqueName: \"kubernetes.io/projected/995d66bd-d908-4743-90f3-e5db8d8f5fcb-kube-api-access-h8s54\") pod \"node-exporter-gxdc6\" (UID: \"995d66bd-d908-4743-90f3-e5db8d8f5fcb\") " pod="openshift-monitoring/node-exporter-gxdc6"
Jul 28 10:09:15.379315 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.379277    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-g2lz6"
Jul 28 10:09:15.498698 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.498666    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-cmrjd"
Jul 28 10:09:15.579289 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.579254    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-g2lz6"]
Jul 28 10:09:15.582276 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:09:15.582243    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda33c1bbf_f7c5_4302_93f0_3bf51a4b1f7f.slice/crio-467402ccada57be34f088f9625d9ada183a333ee7d9bda2b9fbd0f109f429278 WatchSource:0}: Error finding container 467402ccada57be34f088f9625d9ada183a333ee7d9bda2b9fbd0f109f429278: Status 404 returned error can't find the container with id 467402ccada57be34f088f9625d9ada183a333ee7d9bda2b9fbd0f109f429278
Jul 28 10:09:15.633230 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.633142    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-g2lz6" event={"ID":"a33c1bbf-f7c5-4302-93f0-3bf51a4b1f7f","Type":"ContainerStarted","Data":"467402ccada57be34f088f9625d9ada183a333ee7d9bda2b9fbd0f109f429278"}
Jul 28 10:09:15.729495 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:15.729465    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/995d66bd-d908-4743-90f3-e5db8d8f5fcb-node-exporter-tls\") pod \"node-exporter-gxdc6\" (UID: \"995d66bd-d908-4743-90f3-e5db8d8f5fcb\") " pod="openshift-monitoring/node-exporter-gxdc6"
Jul 28 10:09:15.729668 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:09:15.729645    2563 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 28 10:09:15.729747 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:09:15.729736    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/995d66bd-d908-4743-90f3-e5db8d8f5fcb-node-exporter-tls podName:995d66bd-d908-4743-90f3-e5db8d8f5fcb nodeName:}" failed. No retries permitted until 2026-07-28 10:09:16.729715933 +0000 UTC m=+100.171257261 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/995d66bd-d908-4743-90f3-e5db8d8f5fcb-node-exporter-tls") pod "node-exporter-gxdc6" (UID: "995d66bd-d908-4743-90f3-e5db8d8f5fcb") : secret "node-exporter-tls" not found
Jul 28 10:09:16.094702 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.094666    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 10:09:16.119347 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.119313    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 10:09:16.119554 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.119502    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:16.123308 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.123269    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 28 10:09:16.123542 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.123318    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 28 10:09:16.123542 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.123269    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 28 10:09:16.123542 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.123269    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 28 10:09:16.123811 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.123796    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-r9rcw\""
Jul 28 10:09:16.123878 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.123852    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 28 10:09:16.124130 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.124107    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 28 10:09:16.124316 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.124278    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 28 10:09:16.125894 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.125872    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 28 10:09:16.129954 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.129929    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 28 10:09:16.132472 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.132451    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:16.132577 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.132480    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:16.132577 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.132498    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:16.132577 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.132530    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-config-out\") pod \"alertmanager-main-0\" (UID: \"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:16.132577 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.132566    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:16.132713 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.132612    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:16.132713 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.132646    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-web-config\") pod \"alertmanager-main-0\" (UID: \"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:16.132713 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.132682    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:16.132815 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.132713    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-config-volume\") pod \"alertmanager-main-0\" (UID: \"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:16.132815 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.132789    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:16.132891 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.132822    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:16.132891 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.132858    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whvtg\" (UniqueName: \"kubernetes.io/projected/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-kube-api-access-whvtg\") pod \"alertmanager-main-0\" (UID: \"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:16.132965 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.132891    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:16.233949 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.233908    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:16.233949 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.233947    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:16.234206 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.233976    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:16.234206 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.234015    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-config-out\") pod \"alertmanager-main-0\" (UID: \"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:16.234206 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.234038    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:16.234206 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.234065    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:16.234661 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.234332    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-web-config\") pod \"alertmanager-main-0\" (UID: \"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:16.234736 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:09:16.234661    2563 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 28 10:09:16.234736 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.234692    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:16.234736 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:09:16.234733    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-secret-alertmanager-main-tls podName:bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e nodeName:}" failed. No retries permitted until 2026-07-28 10:09:16.734710885 +0000 UTC m=+100.176252216 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e") : secret "alertmanager-main-tls" not found
Jul 28 10:09:16.235066 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.234769    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-config-volume\") pod \"alertmanager-main-0\" (UID: \"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:16.235066 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.234843    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:16.235066 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.234873    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:16.235066 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.234906    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whvtg\" (UniqueName: \"kubernetes.io/projected/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-kube-api-access-whvtg\") pod \"alertmanager-main-0\" (UID: \"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:16.235066 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.234941    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:16.235306 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.235248    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:16.235666 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.235613    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:16.236022 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.235971    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:16.238115 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.238075    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:16.238784 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.238748    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:16.239220 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.238962    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:16.239220 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.239106    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:16.239465 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.239438    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:16.239778 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.239732    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-web-config\") pod \"alertmanager-main-0\" (UID: \"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:16.240504 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.240478    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-config-volume\") pod \"alertmanager-main-0\" (UID: \"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:16.240842 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.240822    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-config-out\") pod \"alertmanager-main-0\" (UID: \"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:16.244416 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.244388    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whvtg\" (UniqueName: \"kubernetes.io/projected/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-kube-api-access-whvtg\") pod \"alertmanager-main-0\" (UID: \"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:16.641040 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.640996    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-g2lz6" event={"ID":"a33c1bbf-f7c5-4302-93f0-3bf51a4b1f7f","Type":"ContainerStarted","Data":"6a45d64b8c8cadb6753b5bb3430bd30f2dcd3aafa30e119799da5be7724b556b"}
Jul 28 10:09:16.641270 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.641107    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-g2lz6" event={"ID":"a33c1bbf-f7c5-4302-93f0-3bf51a4b1f7f","Type":"ContainerStarted","Data":"ecf0ca951078420fd9b1b4e3a447eadb9bf6fa358d266e081548d20896fc59e4"}
Jul 28 10:09:16.739600 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.739553    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/995d66bd-d908-4743-90f3-e5db8d8f5fcb-node-exporter-tls\") pod \"node-exporter-gxdc6\" (UID: \"995d66bd-d908-4743-90f3-e5db8d8f5fcb\") " pod="openshift-monitoring/node-exporter-gxdc6"
Jul 28 10:09:16.739818 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.739650    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:16.742220 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.742186    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/995d66bd-d908-4743-90f3-e5db8d8f5fcb-node-exporter-tls\") pod \"node-exporter-gxdc6\" (UID: \"995d66bd-d908-4743-90f3-e5db8d8f5fcb\") " pod="openshift-monitoring/node-exporter-gxdc6"
Jul 28 10:09:16.742220 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.742217    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:16.883588 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:16.883504    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-gxdc6"
Jul 28 10:09:17.002154 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:09:17.002120    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod995d66bd_d908_4743_90f3_e5db8d8f5fcb.slice/crio-60f20083834d466cb41867ae75ed664023ee21b1656e96acf9d0517848ef714c WatchSource:0}: Error finding container 60f20083834d466cb41867ae75ed664023ee21b1656e96acf9d0517848ef714c: Status 404 returned error can't find the container with id 60f20083834d466cb41867ae75ed664023ee21b1656e96acf9d0517848ef714c
Jul 28 10:09:17.032412 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.032371    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 28 10:09:17.179084 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.178999    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7d57b56d8-ws28t"]
Jul 28 10:09:17.203706 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.203669    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7d57b56d8-ws28t"
Jul 28 10:09:17.228599 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.228563    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 28 10:09:17.228859 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.228838    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 28 10:09:17.229086 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.229066    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 28 10:09:17.229512 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.229495    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 28 10:09:17.230030 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.229968    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-ac6s7b89fhkbo\""
Jul 28 10:09:17.230441 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.230405    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-xf4vt\""
Jul 28 10:09:17.231536 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.231515    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7d57b56d8-ws28t"]
Jul 28 10:09:17.243663 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.243629    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5ab933ad-8909-4f95-8f91-bf3ddeeb6876-secret-grpc-tls\") pod \"thanos-querier-7d57b56d8-ws28t\" (UID: \"5ab933ad-8909-4f95-8f91-bf3ddeeb6876\") " pod="openshift-monitoring/thanos-querier-7d57b56d8-ws28t"
Jul 28 10:09:17.243830 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.243668    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5ab933ad-8909-4f95-8f91-bf3ddeeb6876-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d57b56d8-ws28t\" (UID: \"5ab933ad-8909-4f95-8f91-bf3ddeeb6876\") " pod="openshift-monitoring/thanos-querier-7d57b56d8-ws28t"
Jul 28 10:09:17.243830 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.243712    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5ab933ad-8909-4f95-8f91-bf3ddeeb6876-secret-thanos-querier-tls\") pod \"thanos-querier-7d57b56d8-ws28t\" (UID: \"5ab933ad-8909-4f95-8f91-bf3ddeeb6876\") " pod="openshift-monitoring/thanos-querier-7d57b56d8-ws28t"
Jul 28 10:09:17.243830 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.243735    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5ab933ad-8909-4f95-8f91-bf3ddeeb6876-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d57b56d8-ws28t\" (UID: \"5ab933ad-8909-4f95-8f91-bf3ddeeb6876\") " pod="openshift-monitoring/thanos-querier-7d57b56d8-ws28t"
Jul 28 10:09:17.243830 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.243763    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5ab933ad-8909-4f95-8f91-bf3ddeeb6876-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d57b56d8-ws28t\" (UID: \"5ab933ad-8909-4f95-8f91-bf3ddeeb6876\") " pod="openshift-monitoring/thanos-querier-7d57b56d8-ws28t"
Jul 28 10:09:17.243830 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.243820    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5ab933ad-8909-4f95-8f91-bf3ddeeb6876-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d57b56d8-ws28t\" (UID: \"5ab933ad-8909-4f95-8f91-bf3ddeeb6876\") " pod="openshift-monitoring/thanos-querier-7d57b56d8-ws28t"
Jul 28 10:09:17.244020 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.243851    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvsth\" (UniqueName: \"kubernetes.io/projected/5ab933ad-8909-4f95-8f91-bf3ddeeb6876-kube-api-access-vvsth\") pod \"thanos-querier-7d57b56d8-ws28t\" (UID: \"5ab933ad-8909-4f95-8f91-bf3ddeeb6876\") " pod="openshift-monitoring/thanos-querier-7d57b56d8-ws28t"
Jul 28 10:09:17.244020 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.243872    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ab933ad-8909-4f95-8f91-bf3ddeeb6876-metrics-client-ca\") pod \"thanos-querier-7d57b56d8-ws28t\" (UID: \"5ab933ad-8909-4f95-8f91-bf3ddeeb6876\") " pod="openshift-monitoring/thanos-querier-7d57b56d8-ws28t"
Jul 28 10:09:17.247146 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.247125    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 28 10:09:17.270926 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.270876    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 28 10:09:17.344988 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.344948    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5ab933ad-8909-4f95-8f91-bf3ddeeb6876-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d57b56d8-ws28t\" (UID: \"5ab933ad-8909-4f95-8f91-bf3ddeeb6876\") " pod="openshift-monitoring/thanos-querier-7d57b56d8-ws28t"
Jul 28 10:09:17.344988 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.344991    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5ab933ad-8909-4f95-8f91-bf3ddeeb6876-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d57b56d8-ws28t\" (UID: \"5ab933ad-8909-4f95-8f91-bf3ddeeb6876\") " pod="openshift-monitoring/thanos-querier-7d57b56d8-ws28t"
Jul 28 10:09:17.345258 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.345013    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvsth\" (UniqueName: \"kubernetes.io/projected/5ab933ad-8909-4f95-8f91-bf3ddeeb6876-kube-api-access-vvsth\") pod \"thanos-querier-7d57b56d8-ws28t\" (UID: \"5ab933ad-8909-4f95-8f91-bf3ddeeb6876\") " pod="openshift-monitoring/thanos-querier-7d57b56d8-ws28t"
Jul 28 10:09:17.345258 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.345035    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ab933ad-8909-4f95-8f91-bf3ddeeb6876-metrics-client-ca\") pod \"thanos-querier-7d57b56d8-ws28t\" (UID: \"5ab933ad-8909-4f95-8f91-bf3ddeeb6876\") " pod="openshift-monitoring/thanos-querier-7d57b56d8-ws28t"
Jul 28 10:09:17.345258 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.345058    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5ab933ad-8909-4f95-8f91-bf3ddeeb6876-secret-grpc-tls\") pod \"thanos-querier-7d57b56d8-ws28t\" (UID: \"5ab933ad-8909-4f95-8f91-bf3ddeeb6876\") " pod="openshift-monitoring/thanos-querier-7d57b56d8-ws28t"
Jul 28 10:09:17.345258 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.345080    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5ab933ad-8909-4f95-8f91-bf3ddeeb6876-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d57b56d8-ws28t\" (UID: \"5ab933ad-8909-4f95-8f91-bf3ddeeb6876\") " pod="openshift-monitoring/thanos-querier-7d57b56d8-ws28t"
Jul 28 10:09:17.345258 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.345150    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5ab933ad-8909-4f95-8f91-bf3ddeeb6876-secret-thanos-querier-tls\") pod \"thanos-querier-7d57b56d8-ws28t\" (UID: \"5ab933ad-8909-4f95-8f91-bf3ddeeb6876\") " pod="openshift-monitoring/thanos-querier-7d57b56d8-ws28t"
Jul 28 10:09:17.345258 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.345196    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5ab933ad-8909-4f95-8f91-bf3ddeeb6876-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d57b56d8-ws28t\" (UID: \"5ab933ad-8909-4f95-8f91-bf3ddeeb6876\") " pod="openshift-monitoring/thanos-querier-7d57b56d8-ws28t"
Jul 28 10:09:17.346071 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.345866    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ab933ad-8909-4f95-8f91-bf3ddeeb6876-metrics-client-ca\") pod \"thanos-querier-7d57b56d8-ws28t\" (UID: \"5ab933ad-8909-4f95-8f91-bf3ddeeb6876\") " pod="openshift-monitoring/thanos-querier-7d57b56d8-ws28t"
Jul 28 10:09:17.348021 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.347926    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5ab933ad-8909-4f95-8f91-bf3ddeeb6876-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d57b56d8-ws28t\" (UID: \"5ab933ad-8909-4f95-8f91-bf3ddeeb6876\") " pod="openshift-monitoring/thanos-querier-7d57b56d8-ws28t"
Jul 28 10:09:17.348150 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.348021    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5ab933ad-8909-4f95-8f91-bf3ddeeb6876-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d57b56d8-ws28t\" (UID: \"5ab933ad-8909-4f95-8f91-bf3ddeeb6876\") " pod="openshift-monitoring/thanos-querier-7d57b56d8-ws28t"
Jul 28 10:09:17.348513 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.348492    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5ab933ad-8909-4f95-8f91-bf3ddeeb6876-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d57b56d8-ws28t\" (UID: \"5ab933ad-8909-4f95-8f91-bf3ddeeb6876\") " pod="openshift-monitoring/thanos-querier-7d57b56d8-ws28t"
Jul 28 10:09:17.348562 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.348512    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5ab933ad-8909-4f95-8f91-bf3ddeeb6876-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d57b56d8-ws28t\" (UID: \"5ab933ad-8909-4f95-8f91-bf3ddeeb6876\") " pod="openshift-monitoring/thanos-querier-7d57b56d8-ws28t"
Jul 28 10:09:17.348659 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.348643    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5ab933ad-8909-4f95-8f91-bf3ddeeb6876-secret-grpc-tls\") pod \"thanos-querier-7d57b56d8-ws28t\" (UID: \"5ab933ad-8909-4f95-8f91-bf3ddeeb6876\") " pod="openshift-monitoring/thanos-querier-7d57b56d8-ws28t"
Jul 28 10:09:17.348904 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.348886    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5ab933ad-8909-4f95-8f91-bf3ddeeb6876-secret-thanos-querier-tls\") pod \"thanos-querier-7d57b56d8-ws28t\" (UID: \"5ab933ad-8909-4f95-8f91-bf3ddeeb6876\") " pod="openshift-monitoring/thanos-querier-7d57b56d8-ws28t"
Jul 28 10:09:17.351268 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:09:17.351244    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb5b0c00_bbb6_4377_acd1_2f4a3ed8714e.slice/crio-f824260bbe2589849091f7ae8f1a2e8d02949105da3c1c1ebc7229f6bbfee9fa WatchSource:0}: Error finding container f824260bbe2589849091f7ae8f1a2e8d02949105da3c1c1ebc7229f6bbfee9fa: Status 404 returned error can't find the container with id f824260bbe2589849091f7ae8f1a2e8d02949105da3c1c1ebc7229f6bbfee9fa
Jul 28 10:09:17.358908 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.358881    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvsth\" (UniqueName: \"kubernetes.io/projected/5ab933ad-8909-4f95-8f91-bf3ddeeb6876-kube-api-access-vvsth\") pod \"thanos-querier-7d57b56d8-ws28t\" (UID: \"5ab933ad-8909-4f95-8f91-bf3ddeeb6876\") " pod="openshift-monitoring/thanos-querier-7d57b56d8-ws28t"
Jul 28 10:09:17.514439 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.514391    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7d57b56d8-ws28t"
Jul 28 10:09:17.647045 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.647008    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e","Type":"ContainerStarted","Data":"f824260bbe2589849091f7ae8f1a2e8d02949105da3c1c1ebc7229f6bbfee9fa"}
Jul 28 10:09:17.652146 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.650255    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-g2lz6" event={"ID":"a33c1bbf-f7c5-4302-93f0-3bf51a4b1f7f","Type":"ContainerStarted","Data":"b412f7067f140f419cae651ab3dacd8db5d0b18784a8ddf7143c775ed9610b22"}
Jul 28 10:09:17.653113 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.652484    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gxdc6" event={"ID":"995d66bd-d908-4743-90f3-e5db8d8f5fcb","Type":"ContainerStarted","Data":"60f20083834d466cb41867ae75ed664023ee21b1656e96acf9d0517848ef714c"}
Jul 28 10:09:17.677651 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.677551    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-g2lz6" podStartSLOduration=1.1588291530000001 podStartE2EDuration="2.677529529s" totalImagesPullingTime="1.518700376s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:09:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:09:17.67585692 +0000 UTC m=+101.117398254" watchObservedRunningTime="2026-07-28 10:09:17.677529529 +0000 UTC m=+101.119070870"
Jul 28 10:09:17.735451 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:17.735323    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7d57b56d8-ws28t"]
Jul 28 10:09:17.739641 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:09:17.739597    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ab933ad_8909_4f95_8f91_bf3ddeeb6876.slice/crio-71d3d09f8b8e42e4ffc3d5a2c8b1ffc5745464845c4e912e84189dafe68c8841 WatchSource:0}: Error finding container 71d3d09f8b8e42e4ffc3d5a2c8b1ffc5745464845c4e912e84189dafe68c8841: Status 404 returned error can't find the container with id 71d3d09f8b8e42e4ffc3d5a2c8b1ffc5745464845c4e912e84189dafe68c8841
Jul 28 10:09:18.660859 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:18.660816    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gxdc6" event={"ID":"995d66bd-d908-4743-90f3-e5db8d8f5fcb","Type":"ContainerStarted","Data":"16ca1b7103640b5650410abdcc3bca6e0a23ef935822d8b5a229b40ec31f04e3"}
Jul 28 10:09:18.662128 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:18.662100    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d57b56d8-ws28t" event={"ID":"5ab933ad-8909-4f95-8f91-bf3ddeeb6876","Type":"ContainerStarted","Data":"71d3d09f8b8e42e4ffc3d5a2c8b1ffc5745464845c4e912e84189dafe68c8841"}
Jul 28 10:09:19.668606 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:19.668567    2563 generic.go:356] "Generic (PLEG): container finished" podID="995d66bd-d908-4743-90f3-e5db8d8f5fcb" containerID="16ca1b7103640b5650410abdcc3bca6e0a23ef935822d8b5a229b40ec31f04e3" exitCode=0
Jul 28 10:09:19.669061 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:19.668707    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gxdc6" event={"ID":"995d66bd-d908-4743-90f3-e5db8d8f5fcb","Type":"ContainerDied","Data":"16ca1b7103640b5650410abdcc3bca6e0a23ef935822d8b5a229b40ec31f04e3"}
Jul 28 10:09:19.783503 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:19.783468    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-blv5n"]
Jul 28 10:09:19.806034 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:19.805985    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-blv5n"]
Jul 28 10:09:19.806225 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:19.806166    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-blv5n"
Jul 28 10:09:19.809870 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:19.808589    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 28 10:09:19.809870 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:19.808595    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-kfpwn\""
Jul 28 10:09:19.865320 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:19.865275    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fae66faf-70d8-4771-a93b-81c86a4dbb89-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-blv5n\" (UID: \"fae66faf-70d8-4771-a93b-81c86a4dbb89\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-blv5n"
Jul 28 10:09:19.865320 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:19.865325    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjzc7\" (UniqueName: \"kubernetes.io/projected/fae66faf-70d8-4771-a93b-81c86a4dbb89-kube-api-access-fjzc7\") pod \"monitoring-plugin-68b7fc9574-blv5n\" (UID: \"fae66faf-70d8-4771-a93b-81c86a4dbb89\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-blv5n"
Jul 28 10:09:19.966806 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:19.966711    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fae66faf-70d8-4771-a93b-81c86a4dbb89-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-blv5n\" (UID: \"fae66faf-70d8-4771-a93b-81c86a4dbb89\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-blv5n"
Jul 28 10:09:19.966806 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:19.966785    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjzc7\" (UniqueName: \"kubernetes.io/projected/fae66faf-70d8-4771-a93b-81c86a4dbb89-kube-api-access-fjzc7\") pod \"monitoring-plugin-68b7fc9574-blv5n\" (UID: \"fae66faf-70d8-4771-a93b-81c86a4dbb89\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-blv5n"
Jul 28 10:09:19.967050 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:09:19.966887    2563 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 28 10:09:19.967050 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:09:19.966974    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fae66faf-70d8-4771-a93b-81c86a4dbb89-monitoring-plugin-cert podName:fae66faf-70d8-4771-a93b-81c86a4dbb89 nodeName:}" failed. No retries permitted until 2026-07-28 10:09:20.466950926 +0000 UTC m=+103.908492248 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/fae66faf-70d8-4771-a93b-81c86a4dbb89-monitoring-plugin-cert") pod "monitoring-plugin-68b7fc9574-blv5n" (UID: "fae66faf-70d8-4771-a93b-81c86a4dbb89") : secret "monitoring-plugin-cert" not found
Jul 28 10:09:19.976665 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:19.976631    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjzc7\" (UniqueName: \"kubernetes.io/projected/fae66faf-70d8-4771-a93b-81c86a4dbb89-kube-api-access-fjzc7\") pod \"monitoring-plugin-68b7fc9574-blv5n\" (UID: \"fae66faf-70d8-4771-a93b-81c86a4dbb89\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-blv5n"
Jul 28 10:09:20.470502 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:20.470482    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fae66faf-70d8-4771-a93b-81c86a4dbb89-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-blv5n\" (UID: \"fae66faf-70d8-4771-a93b-81c86a4dbb89\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-blv5n"
Jul 28 10:09:20.472840 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:20.472817    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fae66faf-70d8-4771-a93b-81c86a4dbb89-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-blv5n\" (UID: \"fae66faf-70d8-4771-a93b-81c86a4dbb89\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-blv5n"
Jul 28 10:09:20.674202 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:20.674165    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gxdc6" event={"ID":"995d66bd-d908-4743-90f3-e5db8d8f5fcb","Type":"ContainerStarted","Data":"a68b2fc45352952294104fd17faa90c46de93525295b4a41b55a953c92d104ed"}
Jul 28 10:09:20.675794 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:20.675767    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d57b56d8-ws28t" event={"ID":"5ab933ad-8909-4f95-8f91-bf3ddeeb6876","Type":"ContainerStarted","Data":"7ade0ebb195968d4d7582c0001a9fcb02a8dc404102aed10e5f85f41efe5cddc"}
Jul 28 10:09:20.677241 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:20.677217    2563 generic.go:356] "Generic (PLEG): container finished" podID="bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e" containerID="cb37a88cdff2cc093681991e0c4e237cb462691957064affd966023c465df09d" exitCode=0
Jul 28 10:09:20.677343 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:20.677270    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e","Type":"ContainerDied","Data":"cb37a88cdff2cc093681991e0c4e237cb462691957064affd966023c465df09d"}
Jul 28 10:09:20.718070 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:20.718030    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-blv5n"
Jul 28 10:09:20.880875 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:20.880831    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-blv5n"]
Jul 28 10:09:20.884007 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:09:20.883968    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfae66faf_70d8_4771_a93b_81c86a4dbb89.slice/crio-205af1ae77e1309324c4ffaef5d265dae85c941af49fea3e6b9388f5e6f86eb6 WatchSource:0}: Error finding container 205af1ae77e1309324c4ffaef5d265dae85c941af49fea3e6b9388f5e6f86eb6: Status 404 returned error can't find the container with id 205af1ae77e1309324c4ffaef5d265dae85c941af49fea3e6b9388f5e6f86eb6
Jul 28 10:09:21.686824 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:21.686782    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-gxdc6" event={"ID":"995d66bd-d908-4743-90f3-e5db8d8f5fcb","Type":"ContainerStarted","Data":"f79f6d84c538f11e84aed5360481df70c9b070656822a4aa6d994273b4f82255"}
Jul 28 10:09:21.689986 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:21.689868    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d57b56d8-ws28t" event={"ID":"5ab933ad-8909-4f95-8f91-bf3ddeeb6876","Type":"ContainerStarted","Data":"99f4efbeb91260da8faf05d4bc63316432f2cea3eb1d62469104f5d97b716040"}
Jul 28 10:09:21.689986 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:21.689949    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d57b56d8-ws28t" event={"ID":"5ab933ad-8909-4f95-8f91-bf3ddeeb6876","Type":"ContainerStarted","Data":"95569d251141768604b7207b535f4bf620f0b6d08d78667f0bfe4ddbb0f65eaf"}
Jul 28 10:09:21.692086 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:21.692054    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-blv5n" event={"ID":"fae66faf-70d8-4771-a93b-81c86a4dbb89","Type":"ContainerStarted","Data":"205af1ae77e1309324c4ffaef5d265dae85c941af49fea3e6b9388f5e6f86eb6"}
Jul 28 10:09:21.709303 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:21.708714    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-gxdc6" podStartSLOduration=5.117177294 podStartE2EDuration="6.708692618s" totalImagesPullingTime="1.447237505s" totalInitContainerRuntime="144.277819ms" isStatefulPod=true podCreationTimestamp="2026-07-28 10:09:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:09:21.708175759 +0000 UTC m=+105.149717108" watchObservedRunningTime="2026-07-28 10:09:21.708692618 +0000 UTC m=+105.150233951"
Jul 28 10:09:23.703647 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:23.703612    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e","Type":"ContainerStarted","Data":"9fbbe8639a6ed02c543d7a34770cacaa9d5fb3ca5c66dd6bbf6d39fdf9c52622"}
Jul 28 10:09:23.704062 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:23.703736    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e","Type":"ContainerStarted","Data":"55a49bf5c26d4126e6ebcd54d0474e180f7a701ae8c51f101d38159d103ff877"}
Jul 28 10:09:23.704062 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:23.703787    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e","Type":"ContainerStarted","Data":"314390fa7587a73c64c1a0e78ef565f95d4d9ddb575aafd216b221b6d64a0f14"}
Jul 28 10:09:23.706788 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:23.706753    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d57b56d8-ws28t" event={"ID":"5ab933ad-8909-4f95-8f91-bf3ddeeb6876","Type":"ContainerStarted","Data":"3f1ed0fb2ad34bdd6410161ed50d48f9bd1c78232d0abed6908f405202538049"}
Jul 28 10:09:23.706950 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:23.706856    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d57b56d8-ws28t" event={"ID":"5ab933ad-8909-4f95-8f91-bf3ddeeb6876","Type":"ContainerStarted","Data":"4210b00948dbb3cd406bbd456c94cf055ff999466055bdad6ae7f61f4e8c5b62"}
Jul 28 10:09:23.706950 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:23.706902    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d57b56d8-ws28t" event={"ID":"5ab933ad-8909-4f95-8f91-bf3ddeeb6876","Type":"ContainerStarted","Data":"d20814125c92f19bc3ad219507e8b6ed4a1ed7d113297f6dcd4362a2ffd9601f"}
Jul 28 10:09:23.706950 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:23.706946    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7d57b56d8-ws28t"
Jul 28 10:09:23.708338 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:23.708313    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-blv5n" event={"ID":"fae66faf-70d8-4771-a93b-81c86a4dbb89","Type":"ContainerStarted","Data":"069a8c9f00e305145fc4fcb8b25db4212049f69e433b9913200bc39aecdd37b1"}
Jul 28 10:09:23.708561 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:23.708546    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-blv5n"
Jul 28 10:09:23.714457 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:23.714427    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-blv5n"
Jul 28 10:09:23.735193 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:23.735145    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7d57b56d8-ws28t" podStartSLOduration=2.732450961 podStartE2EDuration="6.735125529s" totalImagesPullingTime="4.002674568s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:09:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:09:23.732586285 +0000 UTC m=+107.174127654" watchObservedRunningTime="2026-07-28 10:09:23.735125529 +0000 UTC m=+107.176666865"
Jul 28 10:09:23.750885 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:23.750815    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-blv5n" podStartSLOduration=2.336792014 podStartE2EDuration="4.750799663s" totalImagesPullingTime="2.414007649s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:09:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:09:23.749324366 +0000 UTC m=+107.190865737" watchObservedRunningTime="2026-07-28 10:09:23.750799663 +0000 UTC m=+107.192341029"
Jul 28 10:09:24.716329 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:24.716286    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e","Type":"ContainerStarted","Data":"a2480c07f40362c49ec663384e7d365e6097291c961012eee1871caef9108b4c"}
Jul 28 10:09:24.716846 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:24.716414    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e","Type":"ContainerStarted","Data":"98e64b859c565987d7c481b413f3823d757652c00a055fa412a85e6e198dc24f"}
Jul 28 10:09:24.716846 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:24.716453    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e","Type":"ContainerStarted","Data":"ac9e9db705c7210d10a93cb04f0eb4dba035cf51633d063a3cb47a75560abe1c"}
Jul 28 10:09:24.745581 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:24.745526    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.001259865 podStartE2EDuration="8.745509009s" totalImagesPullingTime="5.703070168s" totalInitContainerRuntime="41.178976ms" isStatefulPod=false podCreationTimestamp="2026-07-28 10:09:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:09:24.743806002 +0000 UTC m=+108.185347340" watchObservedRunningTime="2026-07-28 10:09:24.745509009 +0000 UTC m=+108.187050345"
Jul 28 10:09:29.723305 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:29.723275    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7d57b56d8-ws28t"
Jul 28 10:09:34.980687 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:34.980623    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7f9765899f-nlhq4" podUID="9e06bdeb-9ead-430e-a448-950a1d18cc7a" containerName="registry" containerID="cri-o://d81d69f996b10ea4de52a4e32b636d7a9a4c6231b309a08ceb562cf9ed1d88f8" gracePeriod=30
Jul 28 10:09:35.233024 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:35.232958    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f9765899f-nlhq4"
Jul 28 10:09:35.298523 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:35.298493    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9e06bdeb-9ead-430e-a448-950a1d18cc7a-bound-sa-token\") pod \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\" (UID: \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\") "
Jul 28 10:09:35.298678 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:35.298532    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9e06bdeb-9ead-430e-a448-950a1d18cc7a-ca-trust-extracted\") pod \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\" (UID: \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\") "
Jul 28 10:09:35.298678 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:35.298572    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9e06bdeb-9ead-430e-a448-950a1d18cc7a-image-registry-private-configuration\") pod \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\" (UID: \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\") "
Jul 28 10:09:35.298678 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:35.298597    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-25qt9\" (UniqueName: \"kubernetes.io/projected/9e06bdeb-9ead-430e-a448-950a1d18cc7a-kube-api-access-25qt9\") pod \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\" (UID: \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\") "
Jul 28 10:09:35.298678 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:35.298619    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9e06bdeb-9ead-430e-a448-950a1d18cc7a-registry-tls\") pod \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\" (UID: \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\") "
Jul 28 10:09:35.298678 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:35.298655    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9e06bdeb-9ead-430e-a448-950a1d18cc7a-installation-pull-secrets\") pod \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\" (UID: \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\") "
Jul 28 10:09:35.298930 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:35.298714    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9e06bdeb-9ead-430e-a448-950a1d18cc7a-trusted-ca\") pod \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\" (UID: \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\") "
Jul 28 10:09:35.298930 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:35.298744    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9e06bdeb-9ead-430e-a448-950a1d18cc7a-registry-certificates\") pod \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\" (UID: \"9e06bdeb-9ead-430e-a448-950a1d18cc7a\") "
Jul 28 10:09:35.299720 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:35.299631    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e06bdeb-9ead-430e-a448-950a1d18cc7a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "9e06bdeb-9ead-430e-a448-950a1d18cc7a" (UID: "9e06bdeb-9ead-430e-a448-950a1d18cc7a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 10:09:35.299881 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:35.299850    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e06bdeb-9ead-430e-a448-950a1d18cc7a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "9e06bdeb-9ead-430e-a448-950a1d18cc7a" (UID: "9e06bdeb-9ead-430e-a448-950a1d18cc7a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 10:09:35.301346 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:35.301309    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e06bdeb-9ead-430e-a448-950a1d18cc7a-kube-api-access-25qt9" (OuterVolumeSpecName: "kube-api-access-25qt9") pod "9e06bdeb-9ead-430e-a448-950a1d18cc7a" (UID: "9e06bdeb-9ead-430e-a448-950a1d18cc7a"). InnerVolumeSpecName "kube-api-access-25qt9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 10:09:35.301585 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:35.301549    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e06bdeb-9ead-430e-a448-950a1d18cc7a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "9e06bdeb-9ead-430e-a448-950a1d18cc7a" (UID: "9e06bdeb-9ead-430e-a448-950a1d18cc7a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 10:09:35.301689 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:35.301582    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e06bdeb-9ead-430e-a448-950a1d18cc7a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "9e06bdeb-9ead-430e-a448-950a1d18cc7a" (UID: "9e06bdeb-9ead-430e-a448-950a1d18cc7a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 10:09:35.301755 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:35.301736    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e06bdeb-9ead-430e-a448-950a1d18cc7a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "9e06bdeb-9ead-430e-a448-950a1d18cc7a" (UID: "9e06bdeb-9ead-430e-a448-950a1d18cc7a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 10:09:35.301808 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:35.301753    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e06bdeb-9ead-430e-a448-950a1d18cc7a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "9e06bdeb-9ead-430e-a448-950a1d18cc7a" (UID: "9e06bdeb-9ead-430e-a448-950a1d18cc7a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 10:09:35.310921 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:35.310877    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e06bdeb-9ead-430e-a448-950a1d18cc7a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "9e06bdeb-9ead-430e-a448-950a1d18cc7a" (UID: "9e06bdeb-9ead-430e-a448-950a1d18cc7a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 10:09:35.399549 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:35.399515    2563 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9e06bdeb-9ead-430e-a448-950a1d18cc7a-bound-sa-token\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 10:09:35.399549 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:35.399544    2563 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9e06bdeb-9ead-430e-a448-950a1d18cc7a-ca-trust-extracted\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 10:09:35.399549 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:35.399556    2563 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9e06bdeb-9ead-430e-a448-950a1d18cc7a-image-registry-private-configuration\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 10:09:35.399783 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:35.399566    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-25qt9\" (UniqueName: \"kubernetes.io/projected/9e06bdeb-9ead-430e-a448-950a1d18cc7a-kube-api-access-25qt9\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 10:09:35.399783 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:35.399577    2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9e06bdeb-9ead-430e-a448-950a1d18cc7a-registry-tls\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 10:09:35.399783 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:35.399585    2563 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9e06bdeb-9ead-430e-a448-950a1d18cc7a-installation-pull-secrets\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 10:09:35.399783 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:35.399594    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9e06bdeb-9ead-430e-a448-950a1d18cc7a-trusted-ca\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 10:09:35.399783 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:35.399601    2563 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9e06bdeb-9ead-430e-a448-950a1d18cc7a-registry-certificates\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 10:09:35.767615 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:35.767584    2563 generic.go:356] "Generic (PLEG): container finished" podID="9e06bdeb-9ead-430e-a448-950a1d18cc7a" containerID="d81d69f996b10ea4de52a4e32b636d7a9a4c6231b309a08ceb562cf9ed1d88f8" exitCode=0
Jul 28 10:09:35.767827 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:35.767633    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f9765899f-nlhq4"
Jul 28 10:09:35.767827 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:35.767682    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f9765899f-nlhq4" event={"ID":"9e06bdeb-9ead-430e-a448-950a1d18cc7a","Type":"ContainerDied","Data":"d81d69f996b10ea4de52a4e32b636d7a9a4c6231b309a08ceb562cf9ed1d88f8"}
Jul 28 10:09:35.767827 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:35.767743    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f9765899f-nlhq4" event={"ID":"9e06bdeb-9ead-430e-a448-950a1d18cc7a","Type":"ContainerDied","Data":"db02a1558335d4b87d5eb071a6846a793baab88eaa64df7bab4472ac3c4b3b1c"}
Jul 28 10:09:35.767827 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:35.767771    2563 scope.go:117] "RemoveContainer" containerID="d81d69f996b10ea4de52a4e32b636d7a9a4c6231b309a08ceb562cf9ed1d88f8"
Jul 28 10:09:35.777761 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:35.777742    2563 scope.go:117] "RemoveContainer" containerID="d81d69f996b10ea4de52a4e32b636d7a9a4c6231b309a08ceb562cf9ed1d88f8"
Jul 28 10:09:35.778127 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:09:35.778094    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d81d69f996b10ea4de52a4e32b636d7a9a4c6231b309a08ceb562cf9ed1d88f8\": container with ID starting with d81d69f996b10ea4de52a4e32b636d7a9a4c6231b309a08ceb562cf9ed1d88f8 not found: ID does not exist" containerID="d81d69f996b10ea4de52a4e32b636d7a9a4c6231b309a08ceb562cf9ed1d88f8"
Jul 28 10:09:35.778191 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:35.778137    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d81d69f996b10ea4de52a4e32b636d7a9a4c6231b309a08ceb562cf9ed1d88f8"} err="failed to get container status \"d81d69f996b10ea4de52a4e32b636d7a9a4c6231b309a08ceb562cf9ed1d88f8\": rpc error: code = NotFound desc = could not find container \"d81d69f996b10ea4de52a4e32b636d7a9a4c6231b309a08ceb562cf9ed1d88f8\": container with ID starting with d81d69f996b10ea4de52a4e32b636d7a9a4c6231b309a08ceb562cf9ed1d88f8 not found: ID does not exist"
Jul 28 10:09:35.789542 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:35.789502    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7f9765899f-nlhq4"]
Jul 28 10:09:35.792594 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:35.792566    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7f9765899f-nlhq4"]
Jul 28 10:09:37.129968 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:37.129932    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e06bdeb-9ead-430e-a448-950a1d18cc7a" path="/var/lib/kubelet/pods/9e06bdeb-9ead-430e-a448-950a1d18cc7a/volumes"
Jul 28 10:09:41.799256 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:41.799220    2563 generic.go:356] "Generic (PLEG): container finished" podID="f6411b5d-e39d-4922-a1fc-67c4952bbb01" containerID="ef41c7b4023f77ca2e8c5898ddd6ec9cb959637b0fced76ce067d0148b5f4ed9" exitCode=0
Jul 28 10:09:41.799655 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:41.799288    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-r6ptx" event={"ID":"f6411b5d-e39d-4922-a1fc-67c4952bbb01","Type":"ContainerDied","Data":"ef41c7b4023f77ca2e8c5898ddd6ec9cb959637b0fced76ce067d0148b5f4ed9"}
Jul 28 10:09:41.799704 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:41.799672    2563 scope.go:117] "RemoveContainer" containerID="ef41c7b4023f77ca2e8c5898ddd6ec9cb959637b0fced76ce067d0148b5f4ed9"
Jul 28 10:09:42.805691 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:42.805660    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-r6ptx" event={"ID":"f6411b5d-e39d-4922-a1fc-67c4952bbb01","Type":"ContainerStarted","Data":"0f1ba116c2070b425b0eb6b71e3af26d65542b70f3ffd8c4e3cd603aea2fedf5"}
Jul 28 10:09:52.851565 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:52.851527    2563 generic.go:356] "Generic (PLEG): container finished" podID="f8b50686-28d7-4749-9430-54bd0f88f2f2" containerID="b7c783114401aaec37250d3f97f822a5adcdd17662b29ba129fbd5cb87daea07" exitCode=0
Jul 28 10:09:52.851956 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:52.851607    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-czhcl" event={"ID":"f8b50686-28d7-4749-9430-54bd0f88f2f2","Type":"ContainerDied","Data":"b7c783114401aaec37250d3f97f822a5adcdd17662b29ba129fbd5cb87daea07"}
Jul 28 10:09:52.852006 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:52.851982    2563 scope.go:117] "RemoveContainer" containerID="b7c783114401aaec37250d3f97f822a5adcdd17662b29ba129fbd5cb87daea07"
Jul 28 10:09:53.858418 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:53.858330    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-czhcl" event={"ID":"f8b50686-28d7-4749-9430-54bd0f88f2f2","Type":"ContainerStarted","Data":"622a7d0bcd77f3022da0401630a28b5b25a37576aff0c6cada2bec2cf19e9430"}
Jul 28 10:09:57.883128 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:57.883094    2563 generic.go:356] "Generic (PLEG): container finished" podID="898e925d-8503-4a54-adef-e90195218e36" containerID="5d0fb9efa0b5815d23bd7864f11ecb400e2ff2805f676293e58a0ccebbfbd2ee" exitCode=0
Jul 28 10:09:57.883608 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:57.883176    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gn75p" event={"ID":"898e925d-8503-4a54-adef-e90195218e36","Type":"ContainerDied","Data":"5d0fb9efa0b5815d23bd7864f11ecb400e2ff2805f676293e58a0ccebbfbd2ee"}
Jul 28 10:09:57.883608 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:57.883571    2563 scope.go:117] "RemoveContainer" containerID="5d0fb9efa0b5815d23bd7864f11ecb400e2ff2805f676293e58a0ccebbfbd2ee"
Jul 28 10:09:58.890132 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:09:58.890095    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gn75p" event={"ID":"898e925d-8503-4a54-adef-e90195218e36","Type":"ContainerStarted","Data":"c6eb9183083a7685ab85ade5d66456ebd074a6e7f08cb3d856f43b3e6d7de691"}
Jul 28 10:10:39.441209 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:39.441070    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5fcc6cf86-n5c9g"]
Jul 28 10:10:39.441886 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:39.441725    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e06bdeb-9ead-430e-a448-950a1d18cc7a" containerName="registry"
Jul 28 10:10:39.441886 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:39.441745    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e06bdeb-9ead-430e-a448-950a1d18cc7a" containerName="registry"
Jul 28 10:10:39.441886 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:39.441827    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e06bdeb-9ead-430e-a448-950a1d18cc7a" containerName="registry"
Jul 28 10:10:39.445172 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:39.445148    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5fcc6cf86-n5c9g"
Jul 28 10:10:39.447505 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:39.447478    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 28 10:10:39.447505 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:39.447494    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-6jzjm\""
Jul 28 10:10:39.447505 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:39.447477    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 28 10:10:39.448018 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:39.447484    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 28 10:10:39.448018 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:39.447641    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 28 10:10:39.448018 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:39.447898    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 28 10:10:39.452743 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:39.452716    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 28 10:10:39.456712 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:39.456686    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5fcc6cf86-n5c9g"]
Jul 28 10:10:39.564163 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:39.564117    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8a0366a2-b544-4511-b5b7-ff44f5c85b80-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5fcc6cf86-n5c9g\" (UID: \"8a0366a2-b544-4511-b5b7-ff44f5c85b80\") " pod="openshift-monitoring/telemeter-client-5fcc6cf86-n5c9g"
Jul 28 10:10:39.564346 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:39.564179    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8a0366a2-b544-4511-b5b7-ff44f5c85b80-telemeter-client-tls\") pod \"telemeter-client-5fcc6cf86-n5c9g\" (UID: \"8a0366a2-b544-4511-b5b7-ff44f5c85b80\") " pod="openshift-monitoring/telemeter-client-5fcc6cf86-n5c9g"
Jul 28 10:10:39.564346 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:39.564249    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a0366a2-b544-4511-b5b7-ff44f5c85b80-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5fcc6cf86-n5c9g\" (UID: \"8a0366a2-b544-4511-b5b7-ff44f5c85b80\") " pod="openshift-monitoring/telemeter-client-5fcc6cf86-n5c9g"
Jul 28 10:10:39.564346 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:39.564297    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8a0366a2-b544-4511-b5b7-ff44f5c85b80-federate-client-tls\") pod \"telemeter-client-5fcc6cf86-n5c9g\" (UID: \"8a0366a2-b544-4511-b5b7-ff44f5c85b80\") " pod="openshift-monitoring/telemeter-client-5fcc6cf86-n5c9g"
Jul 28 10:10:39.564346 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:39.564322    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8a0366a2-b544-4511-b5b7-ff44f5c85b80-secret-telemeter-client\") pod \"telemeter-client-5fcc6cf86-n5c9g\" (UID: \"8a0366a2-b544-4511-b5b7-ff44f5c85b80\") " pod="openshift-monitoring/telemeter-client-5fcc6cf86-n5c9g"
Jul 28 10:10:39.564511 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:39.564428    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a0366a2-b544-4511-b5b7-ff44f5c85b80-serving-certs-ca-bundle\") pod \"telemeter-client-5fcc6cf86-n5c9g\" (UID: \"8a0366a2-b544-4511-b5b7-ff44f5c85b80\") " pod="openshift-monitoring/telemeter-client-5fcc6cf86-n5c9g"
Jul 28 10:10:39.564511 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:39.564450    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a0366a2-b544-4511-b5b7-ff44f5c85b80-metrics-client-ca\") pod \"telemeter-client-5fcc6cf86-n5c9g\" (UID: \"8a0366a2-b544-4511-b5b7-ff44f5c85b80\") " pod="openshift-monitoring/telemeter-client-5fcc6cf86-n5c9g"
Jul 28 10:10:39.564511 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:39.564493    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbh9f\" (UniqueName: \"kubernetes.io/projected/8a0366a2-b544-4511-b5b7-ff44f5c85b80-kube-api-access-vbh9f\") pod \"telemeter-client-5fcc6cf86-n5c9g\" (UID: \"8a0366a2-b544-4511-b5b7-ff44f5c85b80\") " pod="openshift-monitoring/telemeter-client-5fcc6cf86-n5c9g"
Jul 28 10:10:39.665432 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:39.665401    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8a0366a2-b544-4511-b5b7-ff44f5c85b80-federate-client-tls\") pod \"telemeter-client-5fcc6cf86-n5c9g\" (UID: \"8a0366a2-b544-4511-b5b7-ff44f5c85b80\") " pod="openshift-monitoring/telemeter-client-5fcc6cf86-n5c9g"
Jul 28 10:10:39.665432 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:39.665436    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8a0366a2-b544-4511-b5b7-ff44f5c85b80-secret-telemeter-client\") pod \"telemeter-client-5fcc6cf86-n5c9g\" (UID: \"8a0366a2-b544-4511-b5b7-ff44f5c85b80\") " pod="openshift-monitoring/telemeter-client-5fcc6cf86-n5c9g"
Jul 28 10:10:39.665637 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:39.665472    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a0366a2-b544-4511-b5b7-ff44f5c85b80-serving-certs-ca-bundle\") pod \"telemeter-client-5fcc6cf86-n5c9g\" (UID: \"8a0366a2-b544-4511-b5b7-ff44f5c85b80\") " pod="openshift-monitoring/telemeter-client-5fcc6cf86-n5c9g"
Jul 28 10:10:39.665637 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:39.665587    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a0366a2-b544-4511-b5b7-ff44f5c85b80-metrics-client-ca\") pod \"telemeter-client-5fcc6cf86-n5c9g\" (UID: \"8a0366a2-b544-4511-b5b7-ff44f5c85b80\") " pod="openshift-monitoring/telemeter-client-5fcc6cf86-n5c9g"
Jul 28 10:10:39.665637 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:39.665627    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbh9f\" (UniqueName: \"kubernetes.io/projected/8a0366a2-b544-4511-b5b7-ff44f5c85b80-kube-api-access-vbh9f\") pod \"telemeter-client-5fcc6cf86-n5c9g\" (UID: \"8a0366a2-b544-4511-b5b7-ff44f5c85b80\") " pod="openshift-monitoring/telemeter-client-5fcc6cf86-n5c9g"
Jul 28 10:10:39.665750 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:39.665697    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8a0366a2-b544-4511-b5b7-ff44f5c85b80-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5fcc6cf86-n5c9g\" (UID: \"8a0366a2-b544-4511-b5b7-ff44f5c85b80\") " pod="openshift-monitoring/telemeter-client-5fcc6cf86-n5c9g"
Jul 28 10:10:39.665750 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:39.665743    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8a0366a2-b544-4511-b5b7-ff44f5c85b80-telemeter-client-tls\") pod \"telemeter-client-5fcc6cf86-n5c9g\" (UID: \"8a0366a2-b544-4511-b5b7-ff44f5c85b80\") " pod="openshift-monitoring/telemeter-client-5fcc6cf86-n5c9g"
Jul 28 10:10:39.665849 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:39.665772    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a0366a2-b544-4511-b5b7-ff44f5c85b80-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5fcc6cf86-n5c9g\" (UID: \"8a0366a2-b544-4511-b5b7-ff44f5c85b80\") " pod="openshift-monitoring/telemeter-client-5fcc6cf86-n5c9g"
Jul 28 10:10:39.666278 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:39.666253    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a0366a2-b544-4511-b5b7-ff44f5c85b80-serving-certs-ca-bundle\") pod \"telemeter-client-5fcc6cf86-n5c9g\" (UID: \"8a0366a2-b544-4511-b5b7-ff44f5c85b80\") " pod="openshift-monitoring/telemeter-client-5fcc6cf86-n5c9g"
Jul 28 10:10:39.666415 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:39.666253    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8a0366a2-b544-4511-b5b7-ff44f5c85b80-metrics-client-ca\") pod \"telemeter-client-5fcc6cf86-n5c9g\" (UID: \"8a0366a2-b544-4511-b5b7-ff44f5c85b80\") " pod="openshift-monitoring/telemeter-client-5fcc6cf86-n5c9g"
Jul 28 10:10:39.666850 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:39.666827    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8a0366a2-b544-4511-b5b7-ff44f5c85b80-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5fcc6cf86-n5c9g\" (UID: \"8a0366a2-b544-4511-b5b7-ff44f5c85b80\") " pod="openshift-monitoring/telemeter-client-5fcc6cf86-n5c9g"
Jul 28 10:10:39.668351 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:39.668319    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8a0366a2-b544-4511-b5b7-ff44f5c85b80-federate-client-tls\") pod \"telemeter-client-5fcc6cf86-n5c9g\" (UID: \"8a0366a2-b544-4511-b5b7-ff44f5c85b80\") " pod="openshift-monitoring/telemeter-client-5fcc6cf86-n5c9g"
Jul 28 10:10:39.668470 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:39.668377    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8a0366a2-b544-4511-b5b7-ff44f5c85b80-telemeter-client-tls\") pod \"telemeter-client-5fcc6cf86-n5c9g\" (UID: \"8a0366a2-b544-4511-b5b7-ff44f5c85b80\") " pod="openshift-monitoring/telemeter-client-5fcc6cf86-n5c9g"
Jul 28 10:10:39.668653 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:39.668637    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8a0366a2-b544-4511-b5b7-ff44f5c85b80-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5fcc6cf86-n5c9g\" (UID: \"8a0366a2-b544-4511-b5b7-ff44f5c85b80\") " pod="openshift-monitoring/telemeter-client-5fcc6cf86-n5c9g"
Jul 28 10:10:39.668724 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:39.668708    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8a0366a2-b544-4511-b5b7-ff44f5c85b80-secret-telemeter-client\") pod \"telemeter-client-5fcc6cf86-n5c9g\" (UID: \"8a0366a2-b544-4511-b5b7-ff44f5c85b80\") " pod="openshift-monitoring/telemeter-client-5fcc6cf86-n5c9g"
Jul 28 10:10:39.674259 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:39.674231    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbh9f\" (UniqueName: \"kubernetes.io/projected/8a0366a2-b544-4511-b5b7-ff44f5c85b80-kube-api-access-vbh9f\") pod \"telemeter-client-5fcc6cf86-n5c9g\" (UID: \"8a0366a2-b544-4511-b5b7-ff44f5c85b80\") " pod="openshift-monitoring/telemeter-client-5fcc6cf86-n5c9g"
Jul 28 10:10:39.756973 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:39.756879    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5fcc6cf86-n5c9g"
Jul 28 10:10:39.912796 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:39.912758    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5fcc6cf86-n5c9g"]
Jul 28 10:10:39.916552 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:10:39.916520    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a0366a2_b544_4511_b5b7_ff44f5c85b80.slice/crio-02132907e02b59057742a92226bfb3c898e73966f09996a5f67b63dc9df09a8e WatchSource:0}: Error finding container 02132907e02b59057742a92226bfb3c898e73966f09996a5f67b63dc9df09a8e: Status 404 returned error can't find the container with id 02132907e02b59057742a92226bfb3c898e73966f09996a5f67b63dc9df09a8e
Jul 28 10:10:40.059545 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:40.059446    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5fcc6cf86-n5c9g" event={"ID":"8a0366a2-b544-4511-b5b7-ff44f5c85b80","Type":"ContainerStarted","Data":"02132907e02b59057742a92226bfb3c898e73966f09996a5f67b63dc9df09a8e"}
Jul 28 10:10:42.071333 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:42.071292    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5fcc6cf86-n5c9g" event={"ID":"8a0366a2-b544-4511-b5b7-ff44f5c85b80","Type":"ContainerStarted","Data":"ee92f0764d3451a2e33382a73aad9ce12ca23ffc0f3744db9e1a8e4e6a100055"}
Jul 28 10:10:42.071878 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:42.071412    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5fcc6cf86-n5c9g" event={"ID":"8a0366a2-b544-4511-b5b7-ff44f5c85b80","Type":"ContainerStarted","Data":"a29d80d0f25eb59e1ca2b00a1b8b4429bf73e9f7f4eb515984e38383bae17dd8"}
Jul 28 10:10:42.071878 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:42.071436    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5fcc6cf86-n5c9g" event={"ID":"8a0366a2-b544-4511-b5b7-ff44f5c85b80","Type":"ContainerStarted","Data":"6948d6903a8ce9f2ff080f56be5b75d27d0cb03afe003fa422c7010982af81d5"}
Jul 28 10:10:42.093429 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:10:42.093346    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5fcc6cf86-n5c9g" podStartSLOduration=1.364056185 podStartE2EDuration="3.093329179s" totalImagesPullingTime="1.729272994s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:10:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:10:42.091870278 +0000 UTC m=+185.533411627" watchObservedRunningTime="2026-07-28 10:10:42.093329179 +0000 UTC m=+185.534870515"
Jul 28 10:12:36.983226 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:12:36.983197    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 10:12:36.985751 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:12:36.985722    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 10:12:36.995863 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:12:36.995835    2563 kubelet.go:1648] "Image garbage collection succeeded"
Jul 28 10:14:41.451939 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:41.451905    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-gwrbl"]
Jul 28 10:14:41.455103 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:41.455086    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-gwrbl"
Jul 28 10:14:41.458627 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:41.458606    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Jul 28 10:14:41.458945 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:41.458923    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-vg9hd\""
Jul 28 10:14:41.459009 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:41.458942    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Jul 28 10:14:41.459430 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:41.459416    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Jul 28 10:14:41.506678 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:41.506642    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-gwrbl"]
Jul 28 10:14:41.608274 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:41.608216    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jh4f2\" (UniqueName: \"kubernetes.io/projected/afd0d132-9564-4061-afec-662d304a63fa-kube-api-access-jh4f2\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-gwrbl\" (UID: \"afd0d132-9564-4061-afec-662d304a63fa\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-gwrbl"
Jul 28 10:14:41.608486 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:41.608330    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/afd0d132-9564-4061-afec-662d304a63fa-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-gwrbl\" (UID: \"afd0d132-9564-4061-afec-662d304a63fa\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-gwrbl"
Jul 28 10:14:41.708928 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:41.708839    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/afd0d132-9564-4061-afec-662d304a63fa-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-gwrbl\" (UID: \"afd0d132-9564-4061-afec-662d304a63fa\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-gwrbl"
Jul 28 10:14:41.709081 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:41.708916    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jh4f2\" (UniqueName: \"kubernetes.io/projected/afd0d132-9564-4061-afec-662d304a63fa-kube-api-access-jh4f2\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-gwrbl\" (UID: \"afd0d132-9564-4061-afec-662d304a63fa\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-gwrbl"
Jul 28 10:14:41.711379 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:41.711338    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/afd0d132-9564-4061-afec-662d304a63fa-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-gwrbl\" (UID: \"afd0d132-9564-4061-afec-662d304a63fa\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-gwrbl"
Jul 28 10:14:41.718454 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:41.718415    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jh4f2\" (UniqueName: \"kubernetes.io/projected/afd0d132-9564-4061-afec-662d304a63fa-kube-api-access-jh4f2\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-gwrbl\" (UID: \"afd0d132-9564-4061-afec-662d304a63fa\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-gwrbl"
Jul 28 10:14:41.765237 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:41.765198    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-gwrbl"
Jul 28 10:14:41.917014 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:41.916989    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-gwrbl"]
Jul 28 10:14:41.919424 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:14:41.919394    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafd0d132_9564_4061_afec_662d304a63fa.slice/crio-575c9cd0958c0f8b90a1b5c622f2084301de6e16af62671e90aa12b0766e9a1b WatchSource:0}: Error finding container 575c9cd0958c0f8b90a1b5c622f2084301de6e16af62671e90aa12b0766e9a1b: Status 404 returned error can't find the container with id 575c9cd0958c0f8b90a1b5c622f2084301de6e16af62671e90aa12b0766e9a1b
Jul 28 10:14:41.921333 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:41.921309    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 10:14:42.072075 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:42.072045    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-gwrbl" event={"ID":"afd0d132-9564-4061-afec-662d304a63fa","Type":"ContainerStarted","Data":"575c9cd0958c0f8b90a1b5c622f2084301de6e16af62671e90aa12b0766e9a1b"}
Jul 28 10:14:45.956657 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:45.956623    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-77f648d694-c9zcq"]
Jul 28 10:14:45.960156 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:45.960139    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-77f648d694-c9zcq"
Jul 28 10:14:45.963591 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:45.963571    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Jul 28 10:14:45.963720 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:45.963572    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\""
Jul 28 10:14:45.963720 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:45.963632    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-ghrbj\""
Jul 28 10:14:45.971778 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:45.971752    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-77f648d694-c9zcq"]
Jul 28 10:14:46.045921 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:46.045879    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/9cf3d8b0-1d36-41cc-a3e6-0cf0386019ba-cabundle0\") pod \"keda-operator-77f648d694-c9zcq\" (UID: \"9cf3d8b0-1d36-41cc-a3e6-0cf0386019ba\") " pod="openshift-keda/keda-operator-77f648d694-c9zcq"
Jul 28 10:14:46.046124 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:46.045962    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6ww5\" (UniqueName: \"kubernetes.io/projected/9cf3d8b0-1d36-41cc-a3e6-0cf0386019ba-kube-api-access-c6ww5\") pod \"keda-operator-77f648d694-c9zcq\" (UID: \"9cf3d8b0-1d36-41cc-a3e6-0cf0386019ba\") " pod="openshift-keda/keda-operator-77f648d694-c9zcq"
Jul 28 10:14:46.046124 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:46.046008    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/9cf3d8b0-1d36-41cc-a3e6-0cf0386019ba-certificates\") pod \"keda-operator-77f648d694-c9zcq\" (UID: \"9cf3d8b0-1d36-41cc-a3e6-0cf0386019ba\") " pod="openshift-keda/keda-operator-77f648d694-c9zcq"
Jul 28 10:14:46.093743 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:46.093710    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-gwrbl" event={"ID":"afd0d132-9564-4061-afec-662d304a63fa","Type":"ContainerStarted","Data":"fd72b355901511650db29ae5f3c3f098845a87afa6347b87e6f4cb6fca11ff57"}
Jul 28 10:14:46.093925 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:46.093800    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-gwrbl"
Jul 28 10:14:46.123718 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:46.123663    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-gwrbl" podStartSLOduration=1.9432066639999999 podStartE2EDuration="5.123640958s" totalImagesPullingTime="3.180434294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:14:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:14:46.122352734 +0000 UTC m=+429.563894076" watchObservedRunningTime="2026-07-28 10:14:46.123640958 +0000 UTC m=+429.565182297"
Jul 28 10:14:46.147413 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:46.147342    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6ww5\" (UniqueName: \"kubernetes.io/projected/9cf3d8b0-1d36-41cc-a3e6-0cf0386019ba-kube-api-access-c6ww5\") pod \"keda-operator-77f648d694-c9zcq\" (UID: \"9cf3d8b0-1d36-41cc-a3e6-0cf0386019ba\") " pod="openshift-keda/keda-operator-77f648d694-c9zcq"
Jul 28 10:14:46.147584 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:46.147443    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/9cf3d8b0-1d36-41cc-a3e6-0cf0386019ba-certificates\") pod \"keda-operator-77f648d694-c9zcq\" (UID: \"9cf3d8b0-1d36-41cc-a3e6-0cf0386019ba\") " pod="openshift-keda/keda-operator-77f648d694-c9zcq"
Jul 28 10:14:46.147584 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:46.147488    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/9cf3d8b0-1d36-41cc-a3e6-0cf0386019ba-cabundle0\") pod \"keda-operator-77f648d694-c9zcq\" (UID: \"9cf3d8b0-1d36-41cc-a3e6-0cf0386019ba\") " pod="openshift-keda/keda-operator-77f648d694-c9zcq"
Jul 28 10:14:46.147725 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:14:46.147587    2563 projected.go:266] Couldn't get secret openshift-keda/keda-operator-certs: secret "keda-operator-certs" not found
Jul 28 10:14:46.147725 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:14:46.147613    2563 secret.go:281] references non-existent secret key: ca.crt
Jul 28 10:14:46.147725 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:14:46.147623    2563 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Jul 28 10:14:46.147725 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:14:46.147638    2563 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-77f648d694-c9zcq: [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt]
Jul 28 10:14:46.147725 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:14:46.147705    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9cf3d8b0-1d36-41cc-a3e6-0cf0386019ba-certificates podName:9cf3d8b0-1d36-41cc-a3e6-0cf0386019ba nodeName:}" failed. No retries permitted until 2026-07-28 10:14:46.647683127 +0000 UTC m=+430.089224458 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/9cf3d8b0-1d36-41cc-a3e6-0cf0386019ba-certificates") pod "keda-operator-77f648d694-c9zcq" (UID: "9cf3d8b0-1d36-41cc-a3e6-0cf0386019ba") : [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt]
Jul 28 10:14:46.148106 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:46.148087    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/9cf3d8b0-1d36-41cc-a3e6-0cf0386019ba-cabundle0\") pod \"keda-operator-77f648d694-c9zcq\" (UID: \"9cf3d8b0-1d36-41cc-a3e6-0cf0386019ba\") " pod="openshift-keda/keda-operator-77f648d694-c9zcq"
Jul 28 10:14:46.171923 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:46.171894    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6ww5\" (UniqueName: \"kubernetes.io/projected/9cf3d8b0-1d36-41cc-a3e6-0cf0386019ba-kube-api-access-c6ww5\") pod \"keda-operator-77f648d694-c9zcq\" (UID: \"9cf3d8b0-1d36-41cc-a3e6-0cf0386019ba\") " pod="openshift-keda/keda-operator-77f648d694-c9zcq"
Jul 28 10:14:46.652565 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:46.652531    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/9cf3d8b0-1d36-41cc-a3e6-0cf0386019ba-certificates\") pod \"keda-operator-77f648d694-c9zcq\" (UID: \"9cf3d8b0-1d36-41cc-a3e6-0cf0386019ba\") " pod="openshift-keda/keda-operator-77f648d694-c9zcq"
Jul 28 10:14:46.652753 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:14:46.652684    2563 secret.go:281] references non-existent secret key: ca.crt
Jul 28 10:14:46.652753 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:14:46.652703    2563 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Jul 28 10:14:46.652753 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:14:46.652713    2563 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-77f648d694-c9zcq: references non-existent secret key: ca.crt
Jul 28 10:14:46.652882 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:14:46.652763    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9cf3d8b0-1d36-41cc-a3e6-0cf0386019ba-certificates podName:9cf3d8b0-1d36-41cc-a3e6-0cf0386019ba nodeName:}" failed. No retries permitted until 2026-07-28 10:14:47.652749192 +0000 UTC m=+431.094290506 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/9cf3d8b0-1d36-41cc-a3e6-0cf0386019ba-certificates") pod "keda-operator-77f648d694-c9zcq" (UID: "9cf3d8b0-1d36-41cc-a3e6-0cf0386019ba") : references non-existent secret key: ca.crt
Jul 28 10:14:47.661012 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:47.660971    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/9cf3d8b0-1d36-41cc-a3e6-0cf0386019ba-certificates\") pod \"keda-operator-77f648d694-c9zcq\" (UID: \"9cf3d8b0-1d36-41cc-a3e6-0cf0386019ba\") " pod="openshift-keda/keda-operator-77f648d694-c9zcq"
Jul 28 10:14:47.663814 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:47.663780    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/9cf3d8b0-1d36-41cc-a3e6-0cf0386019ba-certificates\") pod \"keda-operator-77f648d694-c9zcq\" (UID: \"9cf3d8b0-1d36-41cc-a3e6-0cf0386019ba\") " pod="openshift-keda/keda-operator-77f648d694-c9zcq"
Jul 28 10:14:47.770921 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:47.770885    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-77f648d694-c9zcq"
Jul 28 10:14:47.934039 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:47.933956    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-77f648d694-c9zcq"]
Jul 28 10:14:47.937515 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:14:47.937486    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cf3d8b0_1d36_41cc_a3e6_0cf0386019ba.slice/crio-750bc269ad8f1abe41c34938e9541652dfac6a3c43dcc2b55b7f4667c64ed82f WatchSource:0}: Error finding container 750bc269ad8f1abe41c34938e9541652dfac6a3c43dcc2b55b7f4667c64ed82f: Status 404 returned error can't find the container with id 750bc269ad8f1abe41c34938e9541652dfac6a3c43dcc2b55b7f4667c64ed82f
Jul 28 10:14:48.104671 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:48.104637    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-77f648d694-c9zcq" event={"ID":"9cf3d8b0-1d36-41cc-a3e6-0cf0386019ba","Type":"ContainerStarted","Data":"750bc269ad8f1abe41c34938e9541652dfac6a3c43dcc2b55b7f4667c64ed82f"}
Jul 28 10:14:52.128523 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:52.128486    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-77f648d694-c9zcq" event={"ID":"9cf3d8b0-1d36-41cc-a3e6-0cf0386019ba","Type":"ContainerStarted","Data":"c0dcab05637324de9e6dd1e90f58c5371bfa5982ec535f946b24b95b2cd766e9"}
Jul 28 10:14:52.128925 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:52.128557    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-77f648d694-c9zcq"
Jul 28 10:14:52.146071 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:14:52.146020    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-77f648d694-c9zcq" podStartSLOduration=3.704327277 podStartE2EDuration="7.146002548s" totalImagesPullingTime="3.441675271s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:14:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:14:52.143998158 +0000 UTC m=+435.585539495" watchObservedRunningTime="2026-07-28 10:14:52.146002548 +0000 UTC m=+435.587543884"
Jul 28 10:15:07.101244 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:15:07.101164    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-gwrbl"
Jul 28 10:15:13.134274 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:15:13.134236    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-77f648d694-c9zcq"
Jul 28 10:16:03.242405 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:16:03.242339    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-9547d8dfc-d5x59"]
Jul 28 10:16:03.244677 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:16:03.244660    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-9547d8dfc-d5x59"
Jul 28 10:16:03.247375 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:16:03.247338    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 28 10:16:03.247606 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:16:03.247589    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 28 10:16:03.247756 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:16:03.247731    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\""
Jul 28 10:16:03.248215 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:16:03.248193    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-hp4jb\""
Jul 28 10:16:03.257786 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:16:03.257762    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-9547d8dfc-d5x59"]
Jul 28 10:16:03.310204 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:16:03.310166    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5288\" (UniqueName: \"kubernetes.io/projected/14da0de4-5f97-4f12-9577-1778aab5fd4b-kube-api-access-w5288\") pod \"llmisvc-controller-manager-9547d8dfc-d5x59\" (UID: \"14da0de4-5f97-4f12-9577-1778aab5fd4b\") " pod="kserve/llmisvc-controller-manager-9547d8dfc-d5x59"
Jul 28 10:16:03.310422 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:16:03.310227    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/14da0de4-5f97-4f12-9577-1778aab5fd4b-cert\") pod \"llmisvc-controller-manager-9547d8dfc-d5x59\" (UID: \"14da0de4-5f97-4f12-9577-1778aab5fd4b\") " pod="kserve/llmisvc-controller-manager-9547d8dfc-d5x59"
Jul 28 10:16:03.411539 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:16:03.411493    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/14da0de4-5f97-4f12-9577-1778aab5fd4b-cert\") pod \"llmisvc-controller-manager-9547d8dfc-d5x59\" (UID: \"14da0de4-5f97-4f12-9577-1778aab5fd4b\") " pod="kserve/llmisvc-controller-manager-9547d8dfc-d5x59"
Jul 28 10:16:03.411733 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:16:03.411569    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5288\" (UniqueName: \"kubernetes.io/projected/14da0de4-5f97-4f12-9577-1778aab5fd4b-kube-api-access-w5288\") pod \"llmisvc-controller-manager-9547d8dfc-d5x59\" (UID: \"14da0de4-5f97-4f12-9577-1778aab5fd4b\") " pod="kserve/llmisvc-controller-manager-9547d8dfc-d5x59"
Jul 28 10:16:03.414224 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:16:03.414193    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/14da0de4-5f97-4f12-9577-1778aab5fd4b-cert\") pod \"llmisvc-controller-manager-9547d8dfc-d5x59\" (UID: \"14da0de4-5f97-4f12-9577-1778aab5fd4b\") " pod="kserve/llmisvc-controller-manager-9547d8dfc-d5x59"
Jul 28 10:16:03.423645 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:16:03.423615    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5288\" (UniqueName: \"kubernetes.io/projected/14da0de4-5f97-4f12-9577-1778aab5fd4b-kube-api-access-w5288\") pod \"llmisvc-controller-manager-9547d8dfc-d5x59\" (UID: \"14da0de4-5f97-4f12-9577-1778aab5fd4b\") " pod="kserve/llmisvc-controller-manager-9547d8dfc-d5x59"
Jul 28 10:16:03.555622 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:16:03.555528    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-9547d8dfc-d5x59"
Jul 28 10:16:03.713005 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:16:03.712970    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-9547d8dfc-d5x59"]
Jul 28 10:16:03.715125 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:16:03.715094    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod14da0de4_5f97_4f12_9577_1778aab5fd4b.slice/crio-ff98ddcbbf74ad51e0b7ae19056f2f6f9ca83cc5807f313be1bf996d6d43edc3 WatchSource:0}: Error finding container ff98ddcbbf74ad51e0b7ae19056f2f6f9ca83cc5807f313be1bf996d6d43edc3: Status 404 returned error can't find the container with id ff98ddcbbf74ad51e0b7ae19056f2f6f9ca83cc5807f313be1bf996d6d43edc3
Jul 28 10:16:04.451662 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:16:04.451630    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-9547d8dfc-d5x59" event={"ID":"14da0de4-5f97-4f12-9577-1778aab5fd4b","Type":"ContainerStarted","Data":"ff98ddcbbf74ad51e0b7ae19056f2f6f9ca83cc5807f313be1bf996d6d43edc3"}
Jul 28 10:16:06.463518 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:16:06.463482    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-9547d8dfc-d5x59" event={"ID":"14da0de4-5f97-4f12-9577-1778aab5fd4b","Type":"ContainerStarted","Data":"ab5900ed3464d936741e6fbf3ec946e511ca0d62e47d8156f073548e2d07f7f7"}
Jul 28 10:16:06.463904 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:16:06.463547    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-9547d8dfc-d5x59"
Jul 28 10:16:06.479959 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:16:06.479904    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-9547d8dfc-d5x59" podStartSLOduration=0.869242613 podStartE2EDuration="3.479887497s" totalImagesPullingTime="2.610644884s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:16:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:16:06.478278647 +0000 UTC m=+509.919819997" watchObservedRunningTime="2026-07-28 10:16:06.479887497 +0000 UTC m=+509.921428833"
Jul 28 10:16:37.469949 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:16:37.469918    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-9547d8dfc-d5x59"
Jul 28 10:17:03.468447 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:03.468409    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-fcn4n"]
Jul 28 10:17:03.471765 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:03.471745    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-fcn4n"
Jul 28 10:17:03.474092 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:03.474060    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-7dhjf\""
Jul 28 10:17:03.474496 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:03.474479    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 28 10:17:03.474615 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:03.474599    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 28 10:17:03.487090 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:03.487058    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-fcn4n"]
Jul 28 10:17:03.545064 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:03.545029    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad134eea-8885-4a29-a232-acd5c9c7edc9-tmp\") pod \"cert-manager-operator-controller-manager-684f79c9b9-fcn4n\" (UID: \"ad134eea-8885-4a29-a232-acd5c9c7edc9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-fcn4n"
Jul 28 10:17:03.545257 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:03.545081    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pl9qc\" (UniqueName: \"kubernetes.io/projected/ad134eea-8885-4a29-a232-acd5c9c7edc9-kube-api-access-pl9qc\") pod \"cert-manager-operator-controller-manager-684f79c9b9-fcn4n\" (UID: \"ad134eea-8885-4a29-a232-acd5c9c7edc9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-fcn4n"
Jul 28 10:17:03.645812 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:03.645768    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl9qc\" (UniqueName: \"kubernetes.io/projected/ad134eea-8885-4a29-a232-acd5c9c7edc9-kube-api-access-pl9qc\") pod \"cert-manager-operator-controller-manager-684f79c9b9-fcn4n\" (UID: \"ad134eea-8885-4a29-a232-acd5c9c7edc9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-fcn4n"
Jul 28 10:17:03.646010 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:03.645867    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad134eea-8885-4a29-a232-acd5c9c7edc9-tmp\") pod \"cert-manager-operator-controller-manager-684f79c9b9-fcn4n\" (UID: \"ad134eea-8885-4a29-a232-acd5c9c7edc9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-fcn4n"
Jul 28 10:17:03.646262 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:03.646245    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad134eea-8885-4a29-a232-acd5c9c7edc9-tmp\") pod \"cert-manager-operator-controller-manager-684f79c9b9-fcn4n\" (UID: \"ad134eea-8885-4a29-a232-acd5c9c7edc9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-fcn4n"
Jul 28 10:17:03.657585 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:03.657545    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pl9qc\" (UniqueName: \"kubernetes.io/projected/ad134eea-8885-4a29-a232-acd5c9c7edc9-kube-api-access-pl9qc\") pod \"cert-manager-operator-controller-manager-684f79c9b9-fcn4n\" (UID: \"ad134eea-8885-4a29-a232-acd5c9c7edc9\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-fcn4n"
Jul 28 10:17:03.788544 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:03.788438    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-fcn4n"
Jul 28 10:17:03.940873 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:03.940842    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-fcn4n"]
Jul 28 10:17:03.946580 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:17:03.946545    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad134eea_8885_4a29_a232_acd5c9c7edc9.slice/crio-93a7f0fb029d17eba2fdf9bbec5acf21f13eaf77b4356193e13e6a485f1abcca WatchSource:0}: Error finding container 93a7f0fb029d17eba2fdf9bbec5acf21f13eaf77b4356193e13e6a485f1abcca: Status 404 returned error can't find the container with id 93a7f0fb029d17eba2fdf9bbec5acf21f13eaf77b4356193e13e6a485f1abcca
Jul 28 10:17:04.732556 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:04.732516    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-fcn4n" event={"ID":"ad134eea-8885-4a29-a232-acd5c9c7edc9","Type":"ContainerStarted","Data":"93a7f0fb029d17eba2fdf9bbec5acf21f13eaf77b4356193e13e6a485f1abcca"}
Jul 28 10:17:06.747025 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:06.746970    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-fcn4n" event={"ID":"ad134eea-8885-4a29-a232-acd5c9c7edc9","Type":"ContainerStarted","Data":"9f2806995c1a4109ee24ec312c87d3da76dc18eb6939ce99fb339e2576b6bafc"}
Jul 28 10:17:06.770792 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:06.770719    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-684f79c9b9-fcn4n" podStartSLOduration=1.580532979 podStartE2EDuration="3.770696953s" totalImagesPullingTime="2.190163974s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:17:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:17:06.768550809 +0000 UTC m=+570.210092158" watchObservedRunningTime="2026-07-28 10:17:06.770696953 +0000 UTC m=+570.212238292"
Jul 28 10:17:25.963168 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:25.963121    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mzlfv"]
Jul 28 10:17:25.966838 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:25.966815    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mzlfv"
Jul 28 10:17:25.968903 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:25.968881    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 28 10:17:25.969444 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:25.969427    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 28 10:17:25.969547 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:25.969507    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-s5w2p\""
Jul 28 10:17:25.973684 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:25.973660    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mzlfv"]
Jul 28 10:17:26.158518 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:26.158484    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2d727c1-a530-483a-b99b-b3ab2e3b71da-tmp\") pod \"openshift-lws-operator-fd8ccff4c-mzlfv\" (UID: \"f2d727c1-a530-483a-b99b-b3ab2e3b71da\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mzlfv"
Jul 28 10:17:26.158691 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:26.158528    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sqk4\" (UniqueName: \"kubernetes.io/projected/f2d727c1-a530-483a-b99b-b3ab2e3b71da-kube-api-access-2sqk4\") pod \"openshift-lws-operator-fd8ccff4c-mzlfv\" (UID: \"f2d727c1-a530-483a-b99b-b3ab2e3b71da\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mzlfv"
Jul 28 10:17:26.259615 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:26.259498    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2d727c1-a530-483a-b99b-b3ab2e3b71da-tmp\") pod \"openshift-lws-operator-fd8ccff4c-mzlfv\" (UID: \"f2d727c1-a530-483a-b99b-b3ab2e3b71da\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mzlfv"
Jul 28 10:17:26.259615 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:26.259573    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sqk4\" (UniqueName: \"kubernetes.io/projected/f2d727c1-a530-483a-b99b-b3ab2e3b71da-kube-api-access-2sqk4\") pod \"openshift-lws-operator-fd8ccff4c-mzlfv\" (UID: \"f2d727c1-a530-483a-b99b-b3ab2e3b71da\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mzlfv"
Jul 28 10:17:26.259899 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:26.259877    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2d727c1-a530-483a-b99b-b3ab2e3b71da-tmp\") pod \"openshift-lws-operator-fd8ccff4c-mzlfv\" (UID: \"f2d727c1-a530-483a-b99b-b3ab2e3b71da\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mzlfv"
Jul 28 10:17:26.268316 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:26.268294    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sqk4\" (UniqueName: \"kubernetes.io/projected/f2d727c1-a530-483a-b99b-b3ab2e3b71da-kube-api-access-2sqk4\") pod \"openshift-lws-operator-fd8ccff4c-mzlfv\" (UID: \"f2d727c1-a530-483a-b99b-b3ab2e3b71da\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mzlfv"
Jul 28 10:17:26.289148 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:26.289121    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mzlfv"
Jul 28 10:17:26.433352 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:26.433324    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mzlfv"]
Jul 28 10:17:26.435732 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:17:26.435709    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf2d727c1_a530_483a_b99b_b3ab2e3b71da.slice/crio-a7c3669762efbb236bb66da79c1b660b6eddc53a647388b7f0bd56c13ab758a1 WatchSource:0}: Error finding container a7c3669762efbb236bb66da79c1b660b6eddc53a647388b7f0bd56c13ab758a1: Status 404 returned error can't find the container with id a7c3669762efbb236bb66da79c1b660b6eddc53a647388b7f0bd56c13ab758a1
Jul 28 10:17:26.842536 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:26.842500    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mzlfv" event={"ID":"f2d727c1-a530-483a-b99b-b3ab2e3b71da","Type":"ContainerStarted","Data":"a7c3669762efbb236bb66da79c1b660b6eddc53a647388b7f0bd56c13ab758a1"}
Jul 28 10:17:29.859613 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:29.859574    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mzlfv" event={"ID":"f2d727c1-a530-483a-b99b-b3ab2e3b71da","Type":"ContainerStarted","Data":"e40c60ee21e0b3582d037256c0b7aaa15ab6ac2d5325a467b63c307bde4acc23"}
Jul 28 10:17:29.876963 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:29.876906    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-mzlfv" podStartSLOduration=2.499435054 podStartE2EDuration="4.876888939s" totalImagesPullingTime="2.377453885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:17:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:17:29.874734139 +0000 UTC m=+593.316275476" watchObservedRunningTime="2026-07-28 10:17:29.876888939 +0000 UTC m=+593.318430276"
Jul 28 10:17:37.016008 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:37.015976    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 10:17:37.016599 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:37.016155    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 10:17:46.772210 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:46.772173    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-68cc75d5fc-895fk"]
Jul 28 10:17:46.777966 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:46.777941    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-68cc75d5fc-895fk"
Jul 28 10:17:46.785525 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:46.785497    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 28 10:17:46.786082 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:46.786059    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 28 10:17:46.786170 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:46.786066    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 28 10:17:46.790847 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:46.790822    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-n2wjv\""
Jul 28 10:17:46.801380 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:46.801314    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-68cc75d5fc-895fk"]
Jul 28 10:17:46.843785 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:46.843739    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prlk7\" (UniqueName: \"kubernetes.io/projected/4f952314-6d4c-485f-8af8-2693950462e2-kube-api-access-prlk7\") pod \"lws-controller-manager-68cc75d5fc-895fk\" (UID: \"4f952314-6d4c-485f-8af8-2693950462e2\") " pod="openshift-lws-operator/lws-controller-manager-68cc75d5fc-895fk"
Jul 28 10:17:46.843785 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:46.843786    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4f952314-6d4c-485f-8af8-2693950462e2-metrics-cert\") pod \"lws-controller-manager-68cc75d5fc-895fk\" (UID: \"4f952314-6d4c-485f-8af8-2693950462e2\") " pod="openshift-lws-operator/lws-controller-manager-68cc75d5fc-895fk"
Jul 28 10:17:46.844007 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:46.843925    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4f952314-6d4c-485f-8af8-2693950462e2-manager-config\") pod \"lws-controller-manager-68cc75d5fc-895fk\" (UID: \"4f952314-6d4c-485f-8af8-2693950462e2\") " pod="openshift-lws-operator/lws-controller-manager-68cc75d5fc-895fk"
Jul 28 10:17:46.844007 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:46.843963    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4f952314-6d4c-485f-8af8-2693950462e2-cert\") pod \"lws-controller-manager-68cc75d5fc-895fk\" (UID: \"4f952314-6d4c-485f-8af8-2693950462e2\") " pod="openshift-lws-operator/lws-controller-manager-68cc75d5fc-895fk"
Jul 28 10:17:46.945003 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:46.944966    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prlk7\" (UniqueName: \"kubernetes.io/projected/4f952314-6d4c-485f-8af8-2693950462e2-kube-api-access-prlk7\") pod \"lws-controller-manager-68cc75d5fc-895fk\" (UID: \"4f952314-6d4c-485f-8af8-2693950462e2\") " pod="openshift-lws-operator/lws-controller-manager-68cc75d5fc-895fk"
Jul 28 10:17:46.945003 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:46.945007    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4f952314-6d4c-485f-8af8-2693950462e2-metrics-cert\") pod \"lws-controller-manager-68cc75d5fc-895fk\" (UID: \"4f952314-6d4c-485f-8af8-2693950462e2\") " pod="openshift-lws-operator/lws-controller-manager-68cc75d5fc-895fk"
Jul 28 10:17:46.945240 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:46.945071    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4f952314-6d4c-485f-8af8-2693950462e2-manager-config\") pod \"lws-controller-manager-68cc75d5fc-895fk\" (UID: \"4f952314-6d4c-485f-8af8-2693950462e2\") " pod="openshift-lws-operator/lws-controller-manager-68cc75d5fc-895fk"
Jul 28 10:17:46.945240 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:46.945091    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4f952314-6d4c-485f-8af8-2693950462e2-cert\") pod \"lws-controller-manager-68cc75d5fc-895fk\" (UID: \"4f952314-6d4c-485f-8af8-2693950462e2\") " pod="openshift-lws-operator/lws-controller-manager-68cc75d5fc-895fk"
Jul 28 10:17:46.945904 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:46.945875    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4f952314-6d4c-485f-8af8-2693950462e2-manager-config\") pod \"lws-controller-manager-68cc75d5fc-895fk\" (UID: \"4f952314-6d4c-485f-8af8-2693950462e2\") " pod="openshift-lws-operator/lws-controller-manager-68cc75d5fc-895fk"
Jul 28 10:17:46.947909 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:46.947879    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4f952314-6d4c-485f-8af8-2693950462e2-metrics-cert\") pod \"lws-controller-manager-68cc75d5fc-895fk\" (UID: \"4f952314-6d4c-485f-8af8-2693950462e2\") " pod="openshift-lws-operator/lws-controller-manager-68cc75d5fc-895fk"
Jul 28 10:17:46.948019 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:46.947916    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4f952314-6d4c-485f-8af8-2693950462e2-cert\") pod \"lws-controller-manager-68cc75d5fc-895fk\" (UID: \"4f952314-6d4c-485f-8af8-2693950462e2\") " pod="openshift-lws-operator/lws-controller-manager-68cc75d5fc-895fk"
Jul 28 10:17:46.953974 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:46.953941    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prlk7\" (UniqueName: \"kubernetes.io/projected/4f952314-6d4c-485f-8af8-2693950462e2-kube-api-access-prlk7\") pod \"lws-controller-manager-68cc75d5fc-895fk\" (UID: \"4f952314-6d4c-485f-8af8-2693950462e2\") " pod="openshift-lws-operator/lws-controller-manager-68cc75d5fc-895fk"
Jul 28 10:17:47.088212 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:47.088175    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-68cc75d5fc-895fk"
Jul 28 10:17:47.240139 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:47.240114    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-68cc75d5fc-895fk"]
Jul 28 10:17:47.242849 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:17:47.242818    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f952314_6d4c_485f_8af8_2693950462e2.slice/crio-a300741c31f73c3457de011bcfe82a265e23f86a75d435aff47f478a137e2c13 WatchSource:0}: Error finding container a300741c31f73c3457de011bcfe82a265e23f86a75d435aff47f478a137e2c13: Status 404 returned error can't find the container with id a300741c31f73c3457de011bcfe82a265e23f86a75d435aff47f478a137e2c13
Jul 28 10:17:47.951273 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:47.951235    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-68cc75d5fc-895fk" event={"ID":"4f952314-6d4c-485f-8af8-2693950462e2","Type":"ContainerStarted","Data":"a300741c31f73c3457de011bcfe82a265e23f86a75d435aff47f478a137e2c13"}
Jul 28 10:17:48.958447 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:48.958412    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-68cc75d5fc-895fk" event={"ID":"4f952314-6d4c-485f-8af8-2693950462e2","Type":"ContainerStarted","Data":"de9fe051a5ba0a8afdf9e3c18d72ab0c6665904a6bada0dafc84fcaea9974dd4"}
Jul 28 10:17:48.958840 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:48.958480    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-68cc75d5fc-895fk"
Jul 28 10:17:48.977079 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:48.976971    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-68cc75d5fc-895fk" podStartSLOduration=1.506531752 podStartE2EDuration="2.976954767s" totalImagesPullingTime="1.470423015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:17:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:17:48.976136894 +0000 UTC m=+612.417678232" watchObservedRunningTime="2026-07-28 10:17:48.976954767 +0000 UTC m=+612.418496105"
Jul 28 10:17:59.966246 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:17:59.966171    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-68cc75d5fc-895fk"
Jul 28 10:18:21.572023 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:18:21.571991    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-srd6m"]
Jul 28 10:18:21.575455 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:18:21.575437    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-srd6m"
Jul 28 10:18:21.577537 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:18:21.577508    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 28 10:18:21.577714 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:18:21.577529    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 28 10:18:21.577823 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:18:21.577620    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 28 10:18:21.578226 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:18:21.578212    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 28 10:18:21.578287 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:18:21.578214    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-hvd92\""
Jul 28 10:18:21.585533 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:18:21.585507    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-srd6m"]
Jul 28 10:18:21.653394 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:18:21.653323    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/90ab7b32-9e16-40dc-a7ca-a97e62d2265e-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-srd6m\" (UID: \"90ab7b32-9e16-40dc-a7ca-a97e62d2265e\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-srd6m"
Jul 28 10:18:21.653394 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:18:21.653393    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6jwv\" (UniqueName: \"kubernetes.io/projected/90ab7b32-9e16-40dc-a7ca-a97e62d2265e-kube-api-access-s6jwv\") pod \"kuadrant-console-plugin-598f86468b-srd6m\" (UID: \"90ab7b32-9e16-40dc-a7ca-a97e62d2265e\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-srd6m"
Jul 28 10:18:21.653606 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:18:21.653466    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/90ab7b32-9e16-40dc-a7ca-a97e62d2265e-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-srd6m\" (UID: \"90ab7b32-9e16-40dc-a7ca-a97e62d2265e\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-srd6m"
Jul 28 10:18:21.754629 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:18:21.754595    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/90ab7b32-9e16-40dc-a7ca-a97e62d2265e-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-srd6m\" (UID: \"90ab7b32-9e16-40dc-a7ca-a97e62d2265e\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-srd6m"
Jul 28 10:18:21.754629 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:18:21.754635    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6jwv\" (UniqueName: \"kubernetes.io/projected/90ab7b32-9e16-40dc-a7ca-a97e62d2265e-kube-api-access-s6jwv\") pod \"kuadrant-console-plugin-598f86468b-srd6m\" (UID: \"90ab7b32-9e16-40dc-a7ca-a97e62d2265e\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-srd6m"
Jul 28 10:18:21.754864 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:18:21.754758    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/90ab7b32-9e16-40dc-a7ca-a97e62d2265e-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-srd6m\" (UID: \"90ab7b32-9e16-40dc-a7ca-a97e62d2265e\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-srd6m"
Jul 28 10:18:21.754923 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:18:21.754909    2563 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 28 10:18:21.754999 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:18:21.754986    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90ab7b32-9e16-40dc-a7ca-a97e62d2265e-plugin-serving-cert podName:90ab7b32-9e16-40dc-a7ca-a97e62d2265e nodeName:}" failed. No retries permitted until 2026-07-28 10:18:22.254963546 +0000 UTC m=+645.696504877 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/90ab7b32-9e16-40dc-a7ca-a97e62d2265e-plugin-serving-cert") pod "kuadrant-console-plugin-598f86468b-srd6m" (UID: "90ab7b32-9e16-40dc-a7ca-a97e62d2265e") : secret "plugin-serving-cert" not found
Jul 28 10:18:21.755237 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:18:21.755217    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/90ab7b32-9e16-40dc-a7ca-a97e62d2265e-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-srd6m\" (UID: \"90ab7b32-9e16-40dc-a7ca-a97e62d2265e\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-srd6m"
Jul 28 10:18:21.764278 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:18:21.764246    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6jwv\" (UniqueName: \"kubernetes.io/projected/90ab7b32-9e16-40dc-a7ca-a97e62d2265e-kube-api-access-s6jwv\") pod \"kuadrant-console-plugin-598f86468b-srd6m\" (UID: \"90ab7b32-9e16-40dc-a7ca-a97e62d2265e\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-srd6m"
Jul 28 10:18:22.258977 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:18:22.258932    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/90ab7b32-9e16-40dc-a7ca-a97e62d2265e-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-srd6m\" (UID: \"90ab7b32-9e16-40dc-a7ca-a97e62d2265e\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-srd6m"
Jul 28 10:18:22.261597 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:18:22.261573    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/90ab7b32-9e16-40dc-a7ca-a97e62d2265e-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-srd6m\" (UID: \"90ab7b32-9e16-40dc-a7ca-a97e62d2265e\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-srd6m"
Jul 28 10:18:22.486674 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:18:22.486632    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-srd6m"
Jul 28 10:18:22.636007 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:18:22.635979    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-srd6m"]
Jul 28 10:18:22.638870 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:18:22.638836    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod90ab7b32_9e16_40dc_a7ca_a97e62d2265e.slice/crio-5045c6b00392272d43f014d1f673db328ee28581d816f936bcb2895def07c6c6 WatchSource:0}: Error finding container 5045c6b00392272d43f014d1f673db328ee28581d816f936bcb2895def07c6c6: Status 404 returned error can't find the container with id 5045c6b00392272d43f014d1f673db328ee28581d816f936bcb2895def07c6c6
Jul 28 10:18:23.133949 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:18:23.133908    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-srd6m" event={"ID":"90ab7b32-9e16-40dc-a7ca-a97e62d2265e","Type":"ContainerStarted","Data":"5045c6b00392272d43f014d1f673db328ee28581d816f936bcb2895def07c6c6"}
Jul 28 10:18:28.162933 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:18:28.162896    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-srd6m" event={"ID":"90ab7b32-9e16-40dc-a7ca-a97e62d2265e","Type":"ContainerStarted","Data":"86be23b28012a54fe1e8cdc8eba96da9613656eba8ea08e7785d1efc1c5ebd9a"}
Jul 28 10:18:28.187908 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:18:28.187849    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-srd6m" podStartSLOduration=2.685884744 podStartE2EDuration="7.187830797s" totalImagesPullingTime="4.501946053s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:18:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:18:28.185733455 +0000 UTC m=+651.627274792" watchObservedRunningTime="2026-07-28 10:18:28.187830797 +0000 UTC m=+651.629372134"
Jul 28 10:19:05.691577 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:05.691533    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-2zlsl"]
Jul 28 10:19:05.694062 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:05.694041    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-2zlsl"
Jul 28 10:19:05.696344 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:05.696317    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 28 10:19:05.702563 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:05.702537    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-2zlsl"]
Jul 28 10:19:05.735437 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:05.735401    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-2zlsl"]
Jul 28 10:19:05.763580 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:05.763537    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4jg9\" (UniqueName: \"kubernetes.io/projected/3493d220-0745-4284-91fd-bdad0ba84e8a-kube-api-access-s4jg9\") pod \"limitador-limitador-85d9db4d5d-2zlsl\" (UID: \"3493d220-0745-4284-91fd-bdad0ba84e8a\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-2zlsl"
Jul 28 10:19:05.763757 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:05.763596    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3493d220-0745-4284-91fd-bdad0ba84e8a-config-file\") pod \"limitador-limitador-85d9db4d5d-2zlsl\" (UID: \"3493d220-0745-4284-91fd-bdad0ba84e8a\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-2zlsl"
Jul 28 10:19:05.864697 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:05.864660    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4jg9\" (UniqueName: \"kubernetes.io/projected/3493d220-0745-4284-91fd-bdad0ba84e8a-kube-api-access-s4jg9\") pod \"limitador-limitador-85d9db4d5d-2zlsl\" (UID: \"3493d220-0745-4284-91fd-bdad0ba84e8a\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-2zlsl"
Jul 28 10:19:05.864878 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:05.864716    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3493d220-0745-4284-91fd-bdad0ba84e8a-config-file\") pod \"limitador-limitador-85d9db4d5d-2zlsl\" (UID: \"3493d220-0745-4284-91fd-bdad0ba84e8a\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-2zlsl"
Jul 28 10:19:05.865397 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:05.865349    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3493d220-0745-4284-91fd-bdad0ba84e8a-config-file\") pod \"limitador-limitador-85d9db4d5d-2zlsl\" (UID: \"3493d220-0745-4284-91fd-bdad0ba84e8a\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-2zlsl"
Jul 28 10:19:05.873157 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:05.873127    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4jg9\" (UniqueName: \"kubernetes.io/projected/3493d220-0745-4284-91fd-bdad0ba84e8a-kube-api-access-s4jg9\") pod \"limitador-limitador-85d9db4d5d-2zlsl\" (UID: \"3493d220-0745-4284-91fd-bdad0ba84e8a\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-2zlsl"
Jul 28 10:19:06.006421 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:06.006304    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-2zlsl"
Jul 28 10:19:06.105780 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:06.105746    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-b54df94b4-wfn8t"]
Jul 28 10:19:06.108326 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:06.108302    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-wfn8t"
Jul 28 10:19:06.110530 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:06.110508    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-xc8zx\""
Jul 28 10:19:06.120111 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:06.120084    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-wfn8t"]
Jul 28 10:19:06.154414 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:06.154382    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-2zlsl"]
Jul 28 10:19:06.157270 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:19:06.157239    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3493d220_0745_4284_91fd_bdad0ba84e8a.slice/crio-5aec53cd350a6e217e71b188b573465e2b963c25757fc8c6685ccd4cffdfb7be WatchSource:0}: Error finding container 5aec53cd350a6e217e71b188b573465e2b963c25757fc8c6685ccd4cffdfb7be: Status 404 returned error can't find the container with id 5aec53cd350a6e217e71b188b573465e2b963c25757fc8c6685ccd4cffdfb7be
Jul 28 10:19:06.166436 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:06.166410    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfmnd\" (UniqueName: \"kubernetes.io/projected/9cffbaf5-9b9d-4836-b2a6-0de34dfeac2e-kube-api-access-lfmnd\") pod \"authorino-b54df94b4-wfn8t\" (UID: \"9cffbaf5-9b9d-4836-b2a6-0de34dfeac2e\") " pod="kuadrant-system/authorino-b54df94b4-wfn8t"
Jul 28 10:19:06.267029 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:06.266937    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfmnd\" (UniqueName: \"kubernetes.io/projected/9cffbaf5-9b9d-4836-b2a6-0de34dfeac2e-kube-api-access-lfmnd\") pod \"authorino-b54df94b4-wfn8t\" (UID: \"9cffbaf5-9b9d-4836-b2a6-0de34dfeac2e\") " pod="kuadrant-system/authorino-b54df94b4-wfn8t"
Jul 28 10:19:06.275995 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:06.275962    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfmnd\" (UniqueName: \"kubernetes.io/projected/9cffbaf5-9b9d-4836-b2a6-0de34dfeac2e-kube-api-access-lfmnd\") pod \"authorino-b54df94b4-wfn8t\" (UID: \"9cffbaf5-9b9d-4836-b2a6-0de34dfeac2e\") " pod="kuadrant-system/authorino-b54df94b4-wfn8t"
Jul 28 10:19:06.351267 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:06.351227    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-2zlsl" event={"ID":"3493d220-0745-4284-91fd-bdad0ba84e8a","Type":"ContainerStarted","Data":"5aec53cd350a6e217e71b188b573465e2b963c25757fc8c6685ccd4cffdfb7be"}
Jul 28 10:19:06.421200 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:06.421160    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-wfn8t"
Jul 28 10:19:06.565617 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:06.565527    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-b54df94b4-wfn8t"]
Jul 28 10:19:06.568419 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:19:06.568387    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9cffbaf5_9b9d_4836_b2a6_0de34dfeac2e.slice/crio-f756c3472c975d95f78458d7b2b836d0a1d38e3c374b79605c81ebb0b3f16e48 WatchSource:0}: Error finding container f756c3472c975d95f78458d7b2b836d0a1d38e3c374b79605c81ebb0b3f16e48: Status 404 returned error can't find the container with id f756c3472c975d95f78458d7b2b836d0a1d38e3c374b79605c81ebb0b3f16e48
Jul 28 10:19:07.357643 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:07.357613    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-wfn8t" event={"ID":"9cffbaf5-9b9d-4836-b2a6-0de34dfeac2e","Type":"ContainerStarted","Data":"f756c3472c975d95f78458d7b2b836d0a1d38e3c374b79605c81ebb0b3f16e48"}
Jul 28 10:19:09.439749 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:09.439715    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-wfn8t"]
Jul 28 10:19:10.375540 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:10.375502    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-wfn8t" event={"ID":"9cffbaf5-9b9d-4836-b2a6-0de34dfeac2e","Type":"ContainerStarted","Data":"ef8e084f6ddda5fb5d7efd8514e56c5f0dfd983d8ad69a7f3ce364953a637ad6"}
Jul 28 10:19:10.375748 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:10.375518    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-b54df94b4-wfn8t" podUID="9cffbaf5-9b9d-4836-b2a6-0de34dfeac2e" containerName="authorino" containerID="cri-o://ef8e084f6ddda5fb5d7efd8514e56c5f0dfd983d8ad69a7f3ce364953a637ad6" gracePeriod=30
Jul 28 10:19:10.377032 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:10.376991    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-2zlsl" event={"ID":"3493d220-0745-4284-91fd-bdad0ba84e8a","Type":"ContainerStarted","Data":"fd859da0330a74af9c53802f9a9fd081f83bfa2b8ecb3cb2f0a7967ec11b3121"}
Jul 28 10:19:10.377136 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:10.377093    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-2zlsl"
Jul 28 10:19:10.390566 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:10.390521    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-b54df94b4-wfn8t" podStartSLOduration=1.337747975 podStartE2EDuration="4.390505237s" totalImagesPullingTime="3.052757262s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:19:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:19:10.389912299 +0000 UTC m=+693.831453639" watchObservedRunningTime="2026-07-28 10:19:10.390505237 +0000 UTC m=+693.832046573"
Jul 28 10:19:10.405554 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:10.405501    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-85d9db4d5d-2zlsl" podStartSLOduration=1.9410920919999999 podStartE2EDuration="5.405484461s" totalImagesPullingTime="3.464392369s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:19:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:19:10.40489491 +0000 UTC m=+693.846436247" watchObservedRunningTime="2026-07-28 10:19:10.405484461 +0000 UTC m=+693.847025794"
Jul 28 10:19:10.630366 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:10.630287    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-wfn8t"
Jul 28 10:19:10.708241 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:10.708208    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lfmnd\" (UniqueName: \"kubernetes.io/projected/9cffbaf5-9b9d-4836-b2a6-0de34dfeac2e-kube-api-access-lfmnd\") pod \"9cffbaf5-9b9d-4836-b2a6-0de34dfeac2e\" (UID: \"9cffbaf5-9b9d-4836-b2a6-0de34dfeac2e\") "
Jul 28 10:19:10.710751 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:10.710713    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cffbaf5-9b9d-4836-b2a6-0de34dfeac2e-kube-api-access-lfmnd" (OuterVolumeSpecName: "kube-api-access-lfmnd") pod "9cffbaf5-9b9d-4836-b2a6-0de34dfeac2e" (UID: "9cffbaf5-9b9d-4836-b2a6-0de34dfeac2e"). InnerVolumeSpecName "kube-api-access-lfmnd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 10:19:10.809762 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:10.809721    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfmnd\" (UniqueName: \"kubernetes.io/projected/9cffbaf5-9b9d-4836-b2a6-0de34dfeac2e-kube-api-access-lfmnd\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 10:19:11.382506 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:11.382470    2563 generic.go:356] "Generic (PLEG): container finished" podID="9cffbaf5-9b9d-4836-b2a6-0de34dfeac2e" containerID="ef8e084f6ddda5fb5d7efd8514e56c5f0dfd983d8ad69a7f3ce364953a637ad6" exitCode=0
Jul 28 10:19:11.382657 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:11.382521    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-b54df94b4-wfn8t"
Jul 28 10:19:11.382657 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:11.382564    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-wfn8t" event={"ID":"9cffbaf5-9b9d-4836-b2a6-0de34dfeac2e","Type":"ContainerDied","Data":"ef8e084f6ddda5fb5d7efd8514e56c5f0dfd983d8ad69a7f3ce364953a637ad6"}
Jul 28 10:19:11.382657 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:11.382607    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-b54df94b4-wfn8t" event={"ID":"9cffbaf5-9b9d-4836-b2a6-0de34dfeac2e","Type":"ContainerDied","Data":"f756c3472c975d95f78458d7b2b836d0a1d38e3c374b79605c81ebb0b3f16e48"}
Jul 28 10:19:11.382657 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:11.382629    2563 scope.go:117] "RemoveContainer" containerID="ef8e084f6ddda5fb5d7efd8514e56c5f0dfd983d8ad69a7f3ce364953a637ad6"
Jul 28 10:19:11.392157 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:11.392135    2563 scope.go:117] "RemoveContainer" containerID="ef8e084f6ddda5fb5d7efd8514e56c5f0dfd983d8ad69a7f3ce364953a637ad6"
Jul 28 10:19:11.392506 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:19:11.392487    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef8e084f6ddda5fb5d7efd8514e56c5f0dfd983d8ad69a7f3ce364953a637ad6\": container with ID starting with ef8e084f6ddda5fb5d7efd8514e56c5f0dfd983d8ad69a7f3ce364953a637ad6 not found: ID does not exist" containerID="ef8e084f6ddda5fb5d7efd8514e56c5f0dfd983d8ad69a7f3ce364953a637ad6"
Jul 28 10:19:11.392583 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:11.392516    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef8e084f6ddda5fb5d7efd8514e56c5f0dfd983d8ad69a7f3ce364953a637ad6"} err="failed to get container status \"ef8e084f6ddda5fb5d7efd8514e56c5f0dfd983d8ad69a7f3ce364953a637ad6\": rpc error: code = NotFound desc = could not find container \"ef8e084f6ddda5fb5d7efd8514e56c5f0dfd983d8ad69a7f3ce364953a637ad6\": container with ID starting with ef8e084f6ddda5fb5d7efd8514e56c5f0dfd983d8ad69a7f3ce364953a637ad6 not found: ID does not exist"
Jul 28 10:19:11.398620 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:11.398589    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-b54df94b4-wfn8t"]
Jul 28 10:19:11.403473 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:11.403444    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-b54df94b4-wfn8t"]
Jul 28 10:19:13.132112 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:13.132072    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9cffbaf5-9b9d-4836-b2a6-0de34dfeac2e" path="/var/lib/kubelet/pods/9cffbaf5-9b9d-4836-b2a6-0de34dfeac2e/volumes"
Jul 28 10:19:21.383314 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:21.383284    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-2zlsl"
Jul 28 10:19:38.521597 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:38.521546    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["observability/jaeger-6bbd7d4c9d-x5v27"]
Jul 28 10:19:38.522060 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:38.521931    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cffbaf5-9b9d-4836-b2a6-0de34dfeac2e" containerName="authorino"
Jul 28 10:19:38.522060 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:38.521943    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cffbaf5-9b9d-4836-b2a6-0de34dfeac2e" containerName="authorino"
Jul 28 10:19:38.522060 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:38.522031    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cffbaf5-9b9d-4836-b2a6-0de34dfeac2e" containerName="authorino"
Jul 28 10:19:38.526560 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:38.526536    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="observability/jaeger-6bbd7d4c9d-x5v27"
Jul 28 10:19:38.528747 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:38.528722    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"observability\"/\"kube-root-ca.crt\""
Jul 28 10:19:38.529275 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:38.529257    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"observability\"/\"jaeger-dockercfg-z44dc\""
Jul 28 10:19:38.529275 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:38.529271    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"observability\"/\"openshift-service-ca.crt\""
Jul 28 10:19:38.535843 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:38.535819    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["observability/jaeger-6bbd7d4c9d-x5v27"]
Jul 28 10:19:38.671746 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:38.671706    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77h7p\" (UniqueName: \"kubernetes.io/projected/ff6dfcf3-3c1b-44ae-b83b-00135a7318fb-kube-api-access-77h7p\") pod \"jaeger-6bbd7d4c9d-x5v27\" (UID: \"ff6dfcf3-3c1b-44ae-b83b-00135a7318fb\") " pod="observability/jaeger-6bbd7d4c9d-x5v27"
Jul 28 10:19:38.773208 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:38.773113    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77h7p\" (UniqueName: \"kubernetes.io/projected/ff6dfcf3-3c1b-44ae-b83b-00135a7318fb-kube-api-access-77h7p\") pod \"jaeger-6bbd7d4c9d-x5v27\" (UID: \"ff6dfcf3-3c1b-44ae-b83b-00135a7318fb\") " pod="observability/jaeger-6bbd7d4c9d-x5v27"
Jul 28 10:19:38.782220 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:38.782185    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77h7p\" (UniqueName: \"kubernetes.io/projected/ff6dfcf3-3c1b-44ae-b83b-00135a7318fb-kube-api-access-77h7p\") pod \"jaeger-6bbd7d4c9d-x5v27\" (UID: \"ff6dfcf3-3c1b-44ae-b83b-00135a7318fb\") " pod="observability/jaeger-6bbd7d4c9d-x5v27"
Jul 28 10:19:38.837037 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:38.836982    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="observability/jaeger-6bbd7d4c9d-x5v27"
Jul 28 10:19:38.984093 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:38.984069    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["observability/jaeger-6bbd7d4c9d-x5v27"]
Jul 28 10:19:38.986237 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:19:38.986206    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff6dfcf3_3c1b_44ae_b83b_00135a7318fb.slice/crio-920c59ac61833a832b63311116a20ccd28aa4f548063faa1cd5d4ac6ca228376 WatchSource:0}: Error finding container 920c59ac61833a832b63311116a20ccd28aa4f548063faa1cd5d4ac6ca228376: Status 404 returned error can't find the container with id 920c59ac61833a832b63311116a20ccd28aa4f548063faa1cd5d4ac6ca228376
Jul 28 10:19:39.525716 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:39.525678    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="observability/jaeger-6bbd7d4c9d-x5v27" event={"ID":"ff6dfcf3-3c1b-44ae-b83b-00135a7318fb","Type":"ContainerStarted","Data":"920c59ac61833a832b63311116a20ccd28aa4f548063faa1cd5d4ac6ca228376"}
Jul 28 10:19:41.537466 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:41.537426    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="observability/jaeger-6bbd7d4c9d-x5v27" event={"ID":"ff6dfcf3-3c1b-44ae-b83b-00135a7318fb","Type":"ContainerStarted","Data":"fba3bf1a7fd1a6af9ed84fa16b104c16f3490cdadbda5b67b9de79e296058be1"}
Jul 28 10:19:41.537919 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:41.537485    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="observability/jaeger-6bbd7d4c9d-x5v27"
Jul 28 10:19:41.539939 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:41.539915    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="observability/jaeger-6bbd7d4c9d-x5v27"
Jul 28 10:19:41.553913 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:19:41.553867    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="observability/jaeger-6bbd7d4c9d-x5v27" podStartSLOduration=1.7539382940000001 podStartE2EDuration="3.553851481s" totalImagesPullingTime="1.799913187s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:19:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:19:41.552581075 +0000 UTC m=+724.994122413" watchObservedRunningTime="2026-07-28 10:19:41.553851481 +0000 UTC m=+724.995392818"
Jul 28 10:22:37.048065 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:22:37.048032    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 10:22:37.048590 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:22:37.048517    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 10:27:37.086902 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:27:37.086870    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 10:27:37.087472 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:27:37.087299    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 10:32:20.213895 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:20.213854    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6"]
Jul 28 10:32:20.218229 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:20.218198    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6"
Jul 28 10:32:20.222716 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:20.222687    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-15d6126c\"/\"scheduler-configmap-ref-test-kserve-self-signed-certs\""
Jul 28 10:32:20.223427 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:20.223404    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-15d6126c\"/\"openshift-service-ca.crt\""
Jul 28 10:32:20.223536 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:20.223404    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-15d6126c\"/\"kube-root-ca.crt\""
Jul 28 10:32:20.223536 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:20.223445    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-15d6126c\"/\"default-dockercfg-fxjnj\""
Jul 28 10:32:20.231458 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:20.231430    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6"]
Jul 28 10:32:20.386584 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:20.386540    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/991c53e7-1985-4559-95be-9a0caf170aad-model-cache\") pod \"scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6\" (UID: \"991c53e7-1985-4559-95be-9a0caf170aad\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6"
Jul 28 10:32:20.386584 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:20.386580    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/991c53e7-1985-4559-95be-9a0caf170aad-dshm\") pod \"scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6\" (UID: \"991c53e7-1985-4559-95be-9a0caf170aad\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6"
Jul 28 10:32:20.386804 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:20.386631    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/991c53e7-1985-4559-95be-9a0caf170aad-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6\" (UID: \"991c53e7-1985-4559-95be-9a0caf170aad\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6"
Jul 28 10:32:20.386804 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:20.386654    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpcqx\" (UniqueName: \"kubernetes.io/projected/991c53e7-1985-4559-95be-9a0caf170aad-kube-api-access-gpcqx\") pod \"scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6\" (UID: \"991c53e7-1985-4559-95be-9a0caf170aad\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6"
Jul 28 10:32:20.386804 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:20.386680    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/991c53e7-1985-4559-95be-9a0caf170aad-home\") pod \"scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6\" (UID: \"991c53e7-1985-4559-95be-9a0caf170aad\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6"
Jul 28 10:32:20.386804 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:20.386761    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/991c53e7-1985-4559-95be-9a0caf170aad-tmp-dir\") pod \"scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6\" (UID: \"991c53e7-1985-4559-95be-9a0caf170aad\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6"
Jul 28 10:32:20.487847 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:20.487751    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/991c53e7-1985-4559-95be-9a0caf170aad-home\") pod \"scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6\" (UID: \"991c53e7-1985-4559-95be-9a0caf170aad\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6"
Jul 28 10:32:20.487847 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:20.487804    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/991c53e7-1985-4559-95be-9a0caf170aad-tmp-dir\") pod \"scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6\" (UID: \"991c53e7-1985-4559-95be-9a0caf170aad\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6"
Jul 28 10:32:20.487847 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:20.487838    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/991c53e7-1985-4559-95be-9a0caf170aad-model-cache\") pod \"scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6\" (UID: \"991c53e7-1985-4559-95be-9a0caf170aad\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6"
Jul 28 10:32:20.488104 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:20.487856    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/991c53e7-1985-4559-95be-9a0caf170aad-dshm\") pod \"scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6\" (UID: \"991c53e7-1985-4559-95be-9a0caf170aad\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6"
Jul 28 10:32:20.488104 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:20.487890    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/991c53e7-1985-4559-95be-9a0caf170aad-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6\" (UID: \"991c53e7-1985-4559-95be-9a0caf170aad\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6"
Jul 28 10:32:20.488104 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:20.487913    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpcqx\" (UniqueName: \"kubernetes.io/projected/991c53e7-1985-4559-95be-9a0caf170aad-kube-api-access-gpcqx\") pod \"scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6\" (UID: \"991c53e7-1985-4559-95be-9a0caf170aad\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6"
Jul 28 10:32:20.488253 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:20.488180    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/991c53e7-1985-4559-95be-9a0caf170aad-home\") pod \"scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6\" (UID: \"991c53e7-1985-4559-95be-9a0caf170aad\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6"
Jul 28 10:32:20.488253 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:20.488229    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/991c53e7-1985-4559-95be-9a0caf170aad-tmp-dir\") pod \"scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6\" (UID: \"991c53e7-1985-4559-95be-9a0caf170aad\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6"
Jul 28 10:32:20.488253 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:20.488247    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/991c53e7-1985-4559-95be-9a0caf170aad-model-cache\") pod \"scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6\" (UID: \"991c53e7-1985-4559-95be-9a0caf170aad\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6"
Jul 28 10:32:20.490474 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:20.490451    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/991c53e7-1985-4559-95be-9a0caf170aad-dshm\") pod \"scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6\" (UID: \"991c53e7-1985-4559-95be-9a0caf170aad\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6"
Jul 28 10:32:20.490605 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:20.490587    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/991c53e7-1985-4559-95be-9a0caf170aad-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6\" (UID: \"991c53e7-1985-4559-95be-9a0caf170aad\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6"
Jul 28 10:32:20.499121 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:20.499076    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpcqx\" (UniqueName: \"kubernetes.io/projected/991c53e7-1985-4559-95be-9a0caf170aad-kube-api-access-gpcqx\") pod \"scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6\" (UID: \"991c53e7-1985-4559-95be-9a0caf170aad\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6"
Jul 28 10:32:20.530954 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:20.530893    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6"
Jul 28 10:32:20.683173 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:20.683133    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6"]
Jul 28 10:32:20.687159 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:32:20.687123    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod991c53e7_1985_4559_95be_9a0caf170aad.slice/crio-caf24185f7c4f300764151a9510bc5d721861c9f7c7f458f2a6b2c4a8812147e WatchSource:0}: Error finding container caf24185f7c4f300764151a9510bc5d721861c9f7c7f458f2a6b2c4a8812147e: Status 404 returned error can't find the container with id caf24185f7c4f300764151a9510bc5d721861c9f7c7f458f2a6b2c4a8812147e
Jul 28 10:32:20.689190 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:20.689170    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 10:32:21.379568 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:21.379527    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6" event={"ID":"991c53e7-1985-4559-95be-9a0caf170aad","Type":"ContainerStarted","Data":"caf24185f7c4f300764151a9510bc5d721861c9f7c7f458f2a6b2c4a8812147e"}
Jul 28 10:32:23.392021 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:23.391981    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6" event={"ID":"991c53e7-1985-4559-95be-9a0caf170aad","Type":"ContainerStarted","Data":"4cc98bf10e679436dd1a4d9c79f60ef7b9472a15ebcdbf2455952e5fecc3d996"}
Jul 28 10:32:23.409782 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:23.409720    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6" podStartSLOduration=1.664757368 podStartE2EDuration="3.409696413s" totalImagesPullingTime="1.744939045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:32:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:32:23.408550085 +0000 UTC m=+1486.850091422" watchObservedRunningTime="2026-07-28 10:32:23.409696413 +0000 UTC m=+1486.851237753"
Jul 28 10:32:23.531258 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:23.531162    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6"
Jul 28 10:32:30.531780 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:30.531738    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6"
Jul 28 10:32:30.544741 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:30.544714    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6"
Jul 28 10:32:31.443892 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:31.443857    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6"
Jul 28 10:32:37.126008 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:37.125973    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 10:32:37.127308 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:37.127284    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 10:32:56.685133 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:56.685089    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6"]
Jul 28 10:32:56.695848 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:32:56.695805    2563 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 28 10:32:56.695848 ip-10-0-130-235 kubenswrapper[2563]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 28 10:32:56.695848 ip-10-0-130-235 kubenswrapper[2563]:         , stdout: , stderr: , exit code -1
Jul 28 10:32:56.695848 ip-10-0-130-235 kubenswrapper[2563]:  > containerID="4cc98bf10e679436dd1a4d9c79f60ef7b9472a15ebcdbf2455952e5fecc3d996" cmd=["/bin/sleep","15"]
Jul 28 10:32:56.696445 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:56.695869    2563 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6" message="" err=<
Jul 28 10:32:56.696445 ip-10-0-130-235 kubenswrapper[2563]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 28 10:32:56.696445 ip-10-0-130-235 kubenswrapper[2563]:         , stdout: , stderr: , exit code -1
Jul 28 10:32:56.696445 ip-10-0-130-235 kubenswrapper[2563]:  >
Jul 28 10:32:56.696445 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:32:56.695881    2563 kuberuntime_container.go:755] "PreStop hook failed" err=<
Jul 28 10:32:56.696445 ip-10-0-130-235 kubenswrapper[2563]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 28 10:32:56.696445 ip-10-0-130-235 kubenswrapper[2563]:         , stdout: , stderr: , exit code -1
Jul 28 10:32:56.696445 ip-10-0-130-235 kubenswrapper[2563]:  > pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6" podUID="991c53e7-1985-4559-95be-9a0caf170aad" containerName="main" containerID="cri-o://4cc98bf10e679436dd1a4d9c79f60ef7b9472a15ebcdbf2455952e5fecc3d996"
Jul 28 10:32:56.696445 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:56.695931    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6" podUID="991c53e7-1985-4559-95be-9a0caf170aad" containerName="main" containerID="cri-o://4cc98bf10e679436dd1a4d9c79f60ef7b9472a15ebcdbf2455952e5fecc3d996" gracePeriod=60
Jul 28 10:32:57.054458 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:57.054425    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6"
Jul 28 10:32:57.125149 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:57.125118    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/991c53e7-1985-4559-95be-9a0caf170aad-model-cache\") pod \"991c53e7-1985-4559-95be-9a0caf170aad\" (UID: \"991c53e7-1985-4559-95be-9a0caf170aad\") "
Jul 28 10:32:57.125352 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:57.125189    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/991c53e7-1985-4559-95be-9a0caf170aad-dshm\") pod \"991c53e7-1985-4559-95be-9a0caf170aad\" (UID: \"991c53e7-1985-4559-95be-9a0caf170aad\") "
Jul 28 10:32:57.125352 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:57.125243    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/991c53e7-1985-4559-95be-9a0caf170aad-home\") pod \"991c53e7-1985-4559-95be-9a0caf170aad\" (UID: \"991c53e7-1985-4559-95be-9a0caf170aad\") "
Jul 28 10:32:57.125352 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:57.125275    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gpcqx\" (UniqueName: \"kubernetes.io/projected/991c53e7-1985-4559-95be-9a0caf170aad-kube-api-access-gpcqx\") pod \"991c53e7-1985-4559-95be-9a0caf170aad\" (UID: \"991c53e7-1985-4559-95be-9a0caf170aad\") "
Jul 28 10:32:57.125352 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:57.125315    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/991c53e7-1985-4559-95be-9a0caf170aad-tls-certs\") pod \"991c53e7-1985-4559-95be-9a0caf170aad\" (UID: \"991c53e7-1985-4559-95be-9a0caf170aad\") "
Jul 28 10:32:57.125352 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:57.125340    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/991c53e7-1985-4559-95be-9a0caf170aad-tmp-dir\") pod \"991c53e7-1985-4559-95be-9a0caf170aad\" (UID: \"991c53e7-1985-4559-95be-9a0caf170aad\") "
Jul 28 10:32:57.125643 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:57.125505    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/991c53e7-1985-4559-95be-9a0caf170aad-model-cache" (OuterVolumeSpecName: "model-cache") pod "991c53e7-1985-4559-95be-9a0caf170aad" (UID: "991c53e7-1985-4559-95be-9a0caf170aad"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 10:32:57.125643 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:57.125539    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/991c53e7-1985-4559-95be-9a0caf170aad-home" (OuterVolumeSpecName: "home") pod "991c53e7-1985-4559-95be-9a0caf170aad" (UID: "991c53e7-1985-4559-95be-9a0caf170aad"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 10:32:57.125749 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:57.125723    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/991c53e7-1985-4559-95be-9a0caf170aad-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "991c53e7-1985-4559-95be-9a0caf170aad" (UID: "991c53e7-1985-4559-95be-9a0caf170aad"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 10:32:57.125749 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:57.125734    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/991c53e7-1985-4559-95be-9a0caf170aad-model-cache\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 10:32:57.125828 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:57.125754    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/991c53e7-1985-4559-95be-9a0caf170aad-home\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 10:32:57.127992 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:57.127956    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/991c53e7-1985-4559-95be-9a0caf170aad-kube-api-access-gpcqx" (OuterVolumeSpecName: "kube-api-access-gpcqx") pod "991c53e7-1985-4559-95be-9a0caf170aad" (UID: "991c53e7-1985-4559-95be-9a0caf170aad"). InnerVolumeSpecName "kube-api-access-gpcqx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 10:32:57.128127 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:57.128049    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/991c53e7-1985-4559-95be-9a0caf170aad-dshm" (OuterVolumeSpecName: "dshm") pod "991c53e7-1985-4559-95be-9a0caf170aad" (UID: "991c53e7-1985-4559-95be-9a0caf170aad"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 10:32:57.128194 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:57.128112    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/991c53e7-1985-4559-95be-9a0caf170aad-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "991c53e7-1985-4559-95be-9a0caf170aad" (UID: "991c53e7-1985-4559-95be-9a0caf170aad"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 10:32:57.227229 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:57.227137    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gpcqx\" (UniqueName: \"kubernetes.io/projected/991c53e7-1985-4559-95be-9a0caf170aad-kube-api-access-gpcqx\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 10:32:57.227229 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:57.227164    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/991c53e7-1985-4559-95be-9a0caf170aad-tls-certs\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 10:32:57.227229 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:57.227173    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/991c53e7-1985-4559-95be-9a0caf170aad-tmp-dir\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 10:32:57.227229 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:57.227182    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/991c53e7-1985-4559-95be-9a0caf170aad-dshm\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 10:32:57.566341 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:57.566241    2563 generic.go:356] "Generic (PLEG): container finished" podID="991c53e7-1985-4559-95be-9a0caf170aad" containerID="4cc98bf10e679436dd1a4d9c79f60ef7b9472a15ebcdbf2455952e5fecc3d996" exitCode=0
Jul 28 10:32:57.566341 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:57.566289    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6" event={"ID":"991c53e7-1985-4559-95be-9a0caf170aad","Type":"ContainerDied","Data":"4cc98bf10e679436dd1a4d9c79f60ef7b9472a15ebcdbf2455952e5fecc3d996"}
Jul 28 10:32:57.566341 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:57.566326    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6"
Jul 28 10:32:57.566341 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:57.566345    2563 scope.go:117] "RemoveContainer" containerID="4cc98bf10e679436dd1a4d9c79f60ef7b9472a15ebcdbf2455952e5fecc3d996"
Jul 28 10:32:57.566690 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:57.566334    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6" event={"ID":"991c53e7-1985-4559-95be-9a0caf170aad","Type":"ContainerDied","Data":"caf24185f7c4f300764151a9510bc5d721861c9f7c7f458f2a6b2c4a8812147e"}
Jul 28 10:32:57.576381 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:57.576344    2563 scope.go:117] "RemoveContainer" containerID="4cc98bf10e679436dd1a4d9c79f60ef7b9472a15ebcdbf2455952e5fecc3d996"
Jul 28 10:32:57.576729 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:32:57.576706    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4cc98bf10e679436dd1a4d9c79f60ef7b9472a15ebcdbf2455952e5fecc3d996\": container with ID starting with 4cc98bf10e679436dd1a4d9c79f60ef7b9472a15ebcdbf2455952e5fecc3d996 not found: ID does not exist" containerID="4cc98bf10e679436dd1a4d9c79f60ef7b9472a15ebcdbf2455952e5fecc3d996"
Jul 28 10:32:57.576783 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:57.576752    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4cc98bf10e679436dd1a4d9c79f60ef7b9472a15ebcdbf2455952e5fecc3d996"} err="failed to get container status \"4cc98bf10e679436dd1a4d9c79f60ef7b9472a15ebcdbf2455952e5fecc3d996\": rpc error: code = NotFound desc = could not find container \"4cc98bf10e679436dd1a4d9c79f60ef7b9472a15ebcdbf2455952e5fecc3d996\": container with ID starting with 4cc98bf10e679436dd1a4d9c79f60ef7b9472a15ebcdbf2455952e5fecc3d996 not found: ID does not exist"
Jul 28 10:32:57.582301 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:57.582242    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6"]
Jul 28 10:32:57.584153 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:57.584127    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-5bff678dc7-mvgs6"]
Jul 28 10:32:59.131497 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:32:59.131461    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="991c53e7-1985-4559-95be-9a0caf170aad" path="/var/lib/kubelet/pods/991c53e7-1985-4559-95be-9a0caf170aad/volumes"
Jul 28 10:33:28.887110 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:33:28.887069    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8"]
Jul 28 10:33:28.887620 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:33:28.887509    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="991c53e7-1985-4559-95be-9a0caf170aad" containerName="main"
Jul 28 10:33:28.887620 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:33:28.887522    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="991c53e7-1985-4559-95be-9a0caf170aad" containerName="main"
Jul 28 10:33:28.887620 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:33:28.887611    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="991c53e7-1985-4559-95be-9a0caf170aad" containerName="main"
Jul 28 10:33:28.890892 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:33:28.890874    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8"
Jul 28 10:33:28.893241 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:33:28.893215    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"kube-root-ca.crt\""
Jul 28 10:33:28.893857 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:33:28.893839    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"default-dockercfg-4sjg9\""
Jul 28 10:33:28.893857 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:33:28.893847    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"openshift-service-ca.crt\""
Jul 28 10:33:28.893993 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:33:28.893872    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"scheduler-ha-replicas-test-kserve-self-signed-certs\""
Jul 28 10:33:28.901945 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:33:28.901918    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8"]
Jul 28 10:33:28.918209 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:33:28.918172    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db80e201-7b7e-431d-ae8d-cce1ae1db37c-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8\" (UID: \"db80e201-7b7e-431d-ae8d-cce1ae1db37c\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8"
Jul 28 10:33:28.918409 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:33:28.918226    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db80e201-7b7e-431d-ae8d-cce1ae1db37c-dshm\") pod \"scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8\" (UID: \"db80e201-7b7e-431d-ae8d-cce1ae1db37c\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8"
Jul 28 10:33:28.918409 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:33:28.918321    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db80e201-7b7e-431d-ae8d-cce1ae1db37c-home\") pod \"scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8\" (UID: \"db80e201-7b7e-431d-ae8d-cce1ae1db37c\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8"
Jul 28 10:33:28.918409 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:33:28.918386    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db80e201-7b7e-431d-ae8d-cce1ae1db37c-tmp-dir\") pod \"scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8\" (UID: \"db80e201-7b7e-431d-ae8d-cce1ae1db37c\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8"
Jul 28 10:33:28.918552 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:33:28.918412    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjjfj\" (UniqueName: \"kubernetes.io/projected/db80e201-7b7e-431d-ae8d-cce1ae1db37c-kube-api-access-jjjfj\") pod \"scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8\" (UID: \"db80e201-7b7e-431d-ae8d-cce1ae1db37c\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8"
Jul 28 10:33:28.918552 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:33:28.918472    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db80e201-7b7e-431d-ae8d-cce1ae1db37c-model-cache\") pod \"scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8\" (UID: \"db80e201-7b7e-431d-ae8d-cce1ae1db37c\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8"
Jul 28 10:33:29.019305 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:33:29.019263    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db80e201-7b7e-431d-ae8d-cce1ae1db37c-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8\" (UID: \"db80e201-7b7e-431d-ae8d-cce1ae1db37c\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8"
Jul 28 10:33:29.019305 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:33:29.019309    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db80e201-7b7e-431d-ae8d-cce1ae1db37c-dshm\") pod \"scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8\" (UID: \"db80e201-7b7e-431d-ae8d-cce1ae1db37c\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8"
Jul 28 10:33:29.019591 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:33:29.019326    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db80e201-7b7e-431d-ae8d-cce1ae1db37c-home\") pod \"scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8\" (UID: \"db80e201-7b7e-431d-ae8d-cce1ae1db37c\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8"
Jul 28 10:33:29.019591 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:33:29.019341    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db80e201-7b7e-431d-ae8d-cce1ae1db37c-tmp-dir\") pod \"scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8\" (UID: \"db80e201-7b7e-431d-ae8d-cce1ae1db37c\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8"
Jul 28 10:33:29.019591 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:33:29.019387    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjjfj\" (UniqueName: \"kubernetes.io/projected/db80e201-7b7e-431d-ae8d-cce1ae1db37c-kube-api-access-jjjfj\") pod \"scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8\" (UID: \"db80e201-7b7e-431d-ae8d-cce1ae1db37c\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8"
Jul 28 10:33:29.019591 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:33:29.019406    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db80e201-7b7e-431d-ae8d-cce1ae1db37c-model-cache\") pod \"scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8\" (UID: \"db80e201-7b7e-431d-ae8d-cce1ae1db37c\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8"
Jul 28 10:33:29.019897 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:33:29.019868    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db80e201-7b7e-431d-ae8d-cce1ae1db37c-home\") pod \"scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8\" (UID: \"db80e201-7b7e-431d-ae8d-cce1ae1db37c\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8"
Jul 28 10:33:29.020052 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:33:29.019936    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db80e201-7b7e-431d-ae8d-cce1ae1db37c-tmp-dir\") pod \"scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8\" (UID: \"db80e201-7b7e-431d-ae8d-cce1ae1db37c\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8"
Jul 28 10:33:29.020052 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:33:29.019979    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db80e201-7b7e-431d-ae8d-cce1ae1db37c-model-cache\") pod \"scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8\" (UID: \"db80e201-7b7e-431d-ae8d-cce1ae1db37c\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8"
Jul 28 10:33:29.021934 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:33:29.021910    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db80e201-7b7e-431d-ae8d-cce1ae1db37c-dshm\") pod \"scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8\" (UID: \"db80e201-7b7e-431d-ae8d-cce1ae1db37c\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8"
Jul 28 10:33:29.022178 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:33:29.022158    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db80e201-7b7e-431d-ae8d-cce1ae1db37c-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8\" (UID: \"db80e201-7b7e-431d-ae8d-cce1ae1db37c\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8"
Jul 28 10:33:29.028079 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:33:29.028055    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjjfj\" (UniqueName: \"kubernetes.io/projected/db80e201-7b7e-431d-ae8d-cce1ae1db37c-kube-api-access-jjjfj\") pod \"scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8\" (UID: \"db80e201-7b7e-431d-ae8d-cce1ae1db37c\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8"
Jul 28 10:33:29.203071 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:33:29.202972    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8"
Jul 28 10:33:29.356623 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:33:29.356590    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8"]
Jul 28 10:33:29.359172 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:33:29.359140    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb80e201_7b7e_431d_ae8d_cce1ae1db37c.slice/crio-c98e209f9f160b5e9ddd91da7ebf2c5d8e4db62d32e941d289da40198fb278bb WatchSource:0}: Error finding container c98e209f9f160b5e9ddd91da7ebf2c5d8e4db62d32e941d289da40198fb278bb: Status 404 returned error can't find the container with id c98e209f9f160b5e9ddd91da7ebf2c5d8e4db62d32e941d289da40198fb278bb
Jul 28 10:33:29.730022 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:33:29.729985    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8" event={"ID":"db80e201-7b7e-431d-ae8d-cce1ae1db37c","Type":"ContainerStarted","Data":"3a5ebeb77a69b1fe1ae091f2a816bac6406f04fb939cc7f6449af594927b4488"}
Jul 28 10:33:29.730217 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:33:29.730049    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8" event={"ID":"db80e201-7b7e-431d-ae8d-cce1ae1db37c","Type":"ContainerStarted","Data":"c98e209f9f160b5e9ddd91da7ebf2c5d8e4db62d32e941d289da40198fb278bb"}
Jul 28 10:33:29.760831 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:33:29.760775    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8" podStartSLOduration=1.760756151 podStartE2EDuration="1.760756151s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:33:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:33:29.759534654 +0000 UTC m=+1553.201075991" watchObservedRunningTime="2026-07-28 10:33:29.760756151 +0000 UTC m=+1553.202297487"
Jul 28 10:33:30.203625 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:33:30.203587    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8"
Jul 28 10:33:39.203912 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:33:39.203866    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8"
Jul 28 10:33:39.216890 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:33:39.216858    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8"
Jul 28 10:33:39.792312 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:33:39.792278    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8"
Jul 28 10:34:02.735394 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:02.735335    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8"]
Jul 28 10:34:02.747185 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:34:02.747140    2563 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 28 10:34:02.747185 ip-10-0-130-235 kubenswrapper[2563]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 28 10:34:02.747185 ip-10-0-130-235 kubenswrapper[2563]:         , stdout: , stderr: , exit code -1
Jul 28 10:34:02.747185 ip-10-0-130-235 kubenswrapper[2563]:  > containerID="3a5ebeb77a69b1fe1ae091f2a816bac6406f04fb939cc7f6449af594927b4488" cmd=["/bin/sleep","15"]
Jul 28 10:34:02.747530 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:02.747212    2563 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8" message="" err=<
Jul 28 10:34:02.747530 ip-10-0-130-235 kubenswrapper[2563]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 28 10:34:02.747530 ip-10-0-130-235 kubenswrapper[2563]:         , stdout: , stderr: , exit code -1
Jul 28 10:34:02.747530 ip-10-0-130-235 kubenswrapper[2563]:  >
Jul 28 10:34:02.747530 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:34:02.747226    2563 kuberuntime_container.go:755] "PreStop hook failed" err=<
Jul 28 10:34:02.747530 ip-10-0-130-235 kubenswrapper[2563]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 28 10:34:02.747530 ip-10-0-130-235 kubenswrapper[2563]:         , stdout: , stderr: , exit code -1
Jul 28 10:34:02.747530 ip-10-0-130-235 kubenswrapper[2563]:  > pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8" podUID="db80e201-7b7e-431d-ae8d-cce1ae1db37c" containerName="main" containerID="cri-o://3a5ebeb77a69b1fe1ae091f2a816bac6406f04fb939cc7f6449af594927b4488"
Jul 28 10:34:02.747530 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:02.747285    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8" podUID="db80e201-7b7e-431d-ae8d-cce1ae1db37c" containerName="main" containerID="cri-o://3a5ebeb77a69b1fe1ae091f2a816bac6406f04fb939cc7f6449af594927b4488" gracePeriod=60
Jul 28 10:34:02.782072 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:02.782011    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8" podUID="db80e201-7b7e-431d-ae8d-cce1ae1db37c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.36:8000/health\": dial tcp 10.132.0.36:8000: connect: connection refused"
Jul 28 10:34:03.112256 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:03.112227    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8"
Jul 28 10:34:03.229606 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:03.229573    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db80e201-7b7e-431d-ae8d-cce1ae1db37c-home\") pod \"db80e201-7b7e-431d-ae8d-cce1ae1db37c\" (UID: \"db80e201-7b7e-431d-ae8d-cce1ae1db37c\") "
Jul 28 10:34:03.229884 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:03.229646    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db80e201-7b7e-431d-ae8d-cce1ae1db37c-dshm\") pod \"db80e201-7b7e-431d-ae8d-cce1ae1db37c\" (UID: \"db80e201-7b7e-431d-ae8d-cce1ae1db37c\") "
Jul 28 10:34:03.229884 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:03.229692    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db80e201-7b7e-431d-ae8d-cce1ae1db37c-tls-certs\") pod \"db80e201-7b7e-431d-ae8d-cce1ae1db37c\" (UID: \"db80e201-7b7e-431d-ae8d-cce1ae1db37c\") "
Jul 28 10:34:03.229884 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:03.229734    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db80e201-7b7e-431d-ae8d-cce1ae1db37c-tmp-dir\") pod \"db80e201-7b7e-431d-ae8d-cce1ae1db37c\" (UID: \"db80e201-7b7e-431d-ae8d-cce1ae1db37c\") "
Jul 28 10:34:03.229884 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:03.229770    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jjjfj\" (UniqueName: \"kubernetes.io/projected/db80e201-7b7e-431d-ae8d-cce1ae1db37c-kube-api-access-jjjfj\") pod \"db80e201-7b7e-431d-ae8d-cce1ae1db37c\" (UID: \"db80e201-7b7e-431d-ae8d-cce1ae1db37c\") "
Jul 28 10:34:03.229884 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:03.229848    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db80e201-7b7e-431d-ae8d-cce1ae1db37c-model-cache\") pod \"db80e201-7b7e-431d-ae8d-cce1ae1db37c\" (UID: \"db80e201-7b7e-431d-ae8d-cce1ae1db37c\") "
Jul 28 10:34:03.230167 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:03.229898    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db80e201-7b7e-431d-ae8d-cce1ae1db37c-home" (OuterVolumeSpecName: "home") pod "db80e201-7b7e-431d-ae8d-cce1ae1db37c" (UID: "db80e201-7b7e-431d-ae8d-cce1ae1db37c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 10:34:03.230167 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:03.230109    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db80e201-7b7e-431d-ae8d-cce1ae1db37c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "db80e201-7b7e-431d-ae8d-cce1ae1db37c" (UID: "db80e201-7b7e-431d-ae8d-cce1ae1db37c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 10:34:03.230167 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:03.230147    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db80e201-7b7e-431d-ae8d-cce1ae1db37c-model-cache" (OuterVolumeSpecName: "model-cache") pod "db80e201-7b7e-431d-ae8d-cce1ae1db37c" (UID: "db80e201-7b7e-431d-ae8d-cce1ae1db37c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 10:34:03.230263 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:03.230177    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db80e201-7b7e-431d-ae8d-cce1ae1db37c-home\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 10:34:03.232120 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:03.232086    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db80e201-7b7e-431d-ae8d-cce1ae1db37c-dshm" (OuterVolumeSpecName: "dshm") pod "db80e201-7b7e-431d-ae8d-cce1ae1db37c" (UID: "db80e201-7b7e-431d-ae8d-cce1ae1db37c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 10:34:03.232247 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:03.232132    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db80e201-7b7e-431d-ae8d-cce1ae1db37c-kube-api-access-jjjfj" (OuterVolumeSpecName: "kube-api-access-jjjfj") pod "db80e201-7b7e-431d-ae8d-cce1ae1db37c" (UID: "db80e201-7b7e-431d-ae8d-cce1ae1db37c"). InnerVolumeSpecName "kube-api-access-jjjfj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 10:34:03.232247 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:03.232130    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db80e201-7b7e-431d-ae8d-cce1ae1db37c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "db80e201-7b7e-431d-ae8d-cce1ae1db37c" (UID: "db80e201-7b7e-431d-ae8d-cce1ae1db37c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 10:34:03.331430 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:03.331391    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db80e201-7b7e-431d-ae8d-cce1ae1db37c-dshm\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 10:34:03.331430 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:03.331420    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db80e201-7b7e-431d-ae8d-cce1ae1db37c-tls-certs\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 10:34:03.331430 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:03.331434    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db80e201-7b7e-431d-ae8d-cce1ae1db37c-tmp-dir\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 10:34:03.331684 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:03.331446    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjjfj\" (UniqueName: \"kubernetes.io/projected/db80e201-7b7e-431d-ae8d-cce1ae1db37c-kube-api-access-jjjfj\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 10:34:03.331684 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:03.331459    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db80e201-7b7e-431d-ae8d-cce1ae1db37c-model-cache\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 10:34:03.910072 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:03.910035    2563 generic.go:356] "Generic (PLEG): container finished" podID="db80e201-7b7e-431d-ae8d-cce1ae1db37c" containerID="3a5ebeb77a69b1fe1ae091f2a816bac6406f04fb939cc7f6449af594927b4488" exitCode=0
Jul 28 10:34:03.910523 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:03.910091    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8"
Jul 28 10:34:03.910523 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:03.910136    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8" event={"ID":"db80e201-7b7e-431d-ae8d-cce1ae1db37c","Type":"ContainerDied","Data":"3a5ebeb77a69b1fe1ae091f2a816bac6406f04fb939cc7f6449af594927b4488"}
Jul 28 10:34:03.910523 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:03.910200    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8" event={"ID":"db80e201-7b7e-431d-ae8d-cce1ae1db37c","Type":"ContainerDied","Data":"c98e209f9f160b5e9ddd91da7ebf2c5d8e4db62d32e941d289da40198fb278bb"}
Jul 28 10:34:03.910523 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:03.910225    2563 scope.go:117] "RemoveContainer" containerID="3a5ebeb77a69b1fe1ae091f2a816bac6406f04fb939cc7f6449af594927b4488"
Jul 28 10:34:03.920123 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:03.920105    2563 scope.go:117] "RemoveContainer" containerID="3a5ebeb77a69b1fe1ae091f2a816bac6406f04fb939cc7f6449af594927b4488"
Jul 28 10:34:03.920455 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:34:03.920435    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a5ebeb77a69b1fe1ae091f2a816bac6406f04fb939cc7f6449af594927b4488\": container with ID starting with 3a5ebeb77a69b1fe1ae091f2a816bac6406f04fb939cc7f6449af594927b4488 not found: ID does not exist" containerID="3a5ebeb77a69b1fe1ae091f2a816bac6406f04fb939cc7f6449af594927b4488"
Jul 28 10:34:03.920536 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:03.920464    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a5ebeb77a69b1fe1ae091f2a816bac6406f04fb939cc7f6449af594927b4488"} err="failed to get container status \"3a5ebeb77a69b1fe1ae091f2a816bac6406f04fb939cc7f6449af594927b4488\": rpc error: code = NotFound desc = could not find container \"3a5ebeb77a69b1fe1ae091f2a816bac6406f04fb939cc7f6449af594927b4488\": container with ID starting with 3a5ebeb77a69b1fe1ae091f2a816bac6406f04fb939cc7f6449af594927b4488 not found: ID does not exist"
Jul 28 10:34:03.932754 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:03.932719    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8"]
Jul 28 10:34:03.941351 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:03.941317    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-75747c94b5-mzrt8"]
Jul 28 10:34:05.131290 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:05.131255    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db80e201-7b7e-431d-ae8d-cce1ae1db37c" path="/var/lib/kubelet/pods/db80e201-7b7e-431d-ae8d-cce1ae1db37c/volumes"
Jul 28 10:34:38.185993 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:38.185950    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm"]
Jul 28 10:34:38.186453 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:38.186347    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db80e201-7b7e-431d-ae8d-cce1ae1db37c" containerName="main"
Jul 28 10:34:38.186453 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:38.186373    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="db80e201-7b7e-431d-ae8d-cce1ae1db37c" containerName="main"
Jul 28 10:34:38.186453 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:38.186447    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="db80e201-7b7e-431d-ae8d-cce1ae1db37c" containerName="main"
Jul 28 10:34:38.190681 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:38.190661    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm"
Jul 28 10:34:38.193726 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:38.193695    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4\"/\"kube-root-ca.crt\""
Jul 28 10:34:38.193896 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:38.193696    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4\"/\"openshift-service-ca.crt\""
Jul 28 10:34:38.193896 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:38.193695    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4\"/\"autoscale-hpa-deploy-kserve-self-signed-certs\""
Jul 28 10:34:38.193896 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:38.193703    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4\"/\"default-dockercfg-9wxqq\""
Jul 28 10:34:38.201677 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:38.201630    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm"]
Jul 28 10:34:38.353030 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:38.352988    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6803cbb3-65e2-4d59-881e-c7a88233fe17-model-cache\") pod \"autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm\" (UID: \"6803cbb3-65e2-4d59-881e-c7a88233fe17\") " pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm"
Jul 28 10:34:38.353237 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:38.353051    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6803cbb3-65e2-4d59-881e-c7a88233fe17-dshm\") pod \"autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm\" (UID: \"6803cbb3-65e2-4d59-881e-c7a88233fe17\") " pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm"
Jul 28 10:34:38.353237 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:38.353128    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6803cbb3-65e2-4d59-881e-c7a88233fe17-tmp-dir\") pod \"autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm\" (UID: \"6803cbb3-65e2-4d59-881e-c7a88233fe17\") " pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm"
Jul 28 10:34:38.353237 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:38.353148    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6803cbb3-65e2-4d59-881e-c7a88233fe17-tls-certs\") pod \"autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm\" (UID: \"6803cbb3-65e2-4d59-881e-c7a88233fe17\") " pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm"
Jul 28 10:34:38.353237 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:38.353163    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ht9ph\" (UniqueName: \"kubernetes.io/projected/6803cbb3-65e2-4d59-881e-c7a88233fe17-kube-api-access-ht9ph\") pod \"autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm\" (UID: \"6803cbb3-65e2-4d59-881e-c7a88233fe17\") " pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm"
Jul 28 10:34:38.353476 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:38.353296    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6803cbb3-65e2-4d59-881e-c7a88233fe17-home\") pod \"autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm\" (UID: \"6803cbb3-65e2-4d59-881e-c7a88233fe17\") " pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm"
Jul 28 10:34:38.454598 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:38.454501    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6803cbb3-65e2-4d59-881e-c7a88233fe17-dshm\") pod \"autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm\" (UID: \"6803cbb3-65e2-4d59-881e-c7a88233fe17\") " pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm"
Jul 28 10:34:38.454598 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:38.454551    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6803cbb3-65e2-4d59-881e-c7a88233fe17-tmp-dir\") pod \"autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm\" (UID: \"6803cbb3-65e2-4d59-881e-c7a88233fe17\") " pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm"
Jul 28 10:34:38.454598 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:38.454570    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6803cbb3-65e2-4d59-881e-c7a88233fe17-tls-certs\") pod \"autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm\" (UID: \"6803cbb3-65e2-4d59-881e-c7a88233fe17\") " pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm"
Jul 28 10:34:38.454868 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:38.454684    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ht9ph\" (UniqueName: \"kubernetes.io/projected/6803cbb3-65e2-4d59-881e-c7a88233fe17-kube-api-access-ht9ph\") pod \"autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm\" (UID: \"6803cbb3-65e2-4d59-881e-c7a88233fe17\") " pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm"
Jul 28 10:34:38.454868 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:38.454744    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6803cbb3-65e2-4d59-881e-c7a88233fe17-home\") pod \"autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm\" (UID: \"6803cbb3-65e2-4d59-881e-c7a88233fe17\") " pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm"
Jul 28 10:34:38.454868 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:38.454796    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6803cbb3-65e2-4d59-881e-c7a88233fe17-model-cache\") pod \"autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm\" (UID: \"6803cbb3-65e2-4d59-881e-c7a88233fe17\") " pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm"
Jul 28 10:34:38.455010 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:38.454960    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6803cbb3-65e2-4d59-881e-c7a88233fe17-tmp-dir\") pod \"autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm\" (UID: \"6803cbb3-65e2-4d59-881e-c7a88233fe17\") " pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm"
Jul 28 10:34:38.455065 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:38.455043    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6803cbb3-65e2-4d59-881e-c7a88233fe17-home\") pod \"autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm\" (UID: \"6803cbb3-65e2-4d59-881e-c7a88233fe17\") " pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm"
Jul 28 10:34:38.455110 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:38.455094    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6803cbb3-65e2-4d59-881e-c7a88233fe17-model-cache\") pod \"autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm\" (UID: \"6803cbb3-65e2-4d59-881e-c7a88233fe17\") " pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm"
Jul 28 10:34:38.457085 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:38.457058    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6803cbb3-65e2-4d59-881e-c7a88233fe17-dshm\") pod \"autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm\" (UID: \"6803cbb3-65e2-4d59-881e-c7a88233fe17\") " pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm"
Jul 28 10:34:38.457339 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:38.457321    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6803cbb3-65e2-4d59-881e-c7a88233fe17-tls-certs\") pod \"autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm\" (UID: \"6803cbb3-65e2-4d59-881e-c7a88233fe17\") " pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm"
Jul 28 10:34:38.462627 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:38.462595    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ht9ph\" (UniqueName: \"kubernetes.io/projected/6803cbb3-65e2-4d59-881e-c7a88233fe17-kube-api-access-ht9ph\") pod \"autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm\" (UID: \"6803cbb3-65e2-4d59-881e-c7a88233fe17\") " pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm"
Jul 28 10:34:38.503594 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:38.503555    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm"
Jul 28 10:34:38.655134 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:38.655094    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm"]
Jul 28 10:34:38.660144 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:34:38.660098    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6803cbb3_65e2_4d59_881e_c7a88233fe17.slice/crio-f28cef1cac0f974ecc3233c8790a36db3f2b935d65e6b6d5bcd59af263ed63ec WatchSource:0}: Error finding container f28cef1cac0f974ecc3233c8790a36db3f2b935d65e6b6d5bcd59af263ed63ec: Status 404 returned error can't find the container with id f28cef1cac0f974ecc3233c8790a36db3f2b935d65e6b6d5bcd59af263ed63ec
Jul 28 10:34:39.093751 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:39.093707    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm" event={"ID":"6803cbb3-65e2-4d59-881e-c7a88233fe17","Type":"ContainerStarted","Data":"d2ea5dc1654a7dd97b15096921db23bb25564e7c79744b78bf5fd886cb39cd2e"}
Jul 28 10:34:39.093926 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:39.093796    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm" event={"ID":"6803cbb3-65e2-4d59-881e-c7a88233fe17","Type":"ContainerStarted","Data":"f28cef1cac0f974ecc3233c8790a36db3f2b935d65e6b6d5bcd59af263ed63ec"}
Jul 28 10:34:39.112079 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:39.112021    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm" podStartSLOduration=1.112004826 podStartE2EDuration="1.112004826s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:34:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:34:39.110433389 +0000 UTC m=+1622.551974728" watchObservedRunningTime="2026-07-28 10:34:39.112004826 +0000 UTC m=+1622.553546153"
Jul 28 10:34:39.504225 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:39.504120    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm"
Jul 28 10:34:48.504384 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:48.504304    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm"
Jul 28 10:34:48.517056 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:48.517023    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm"
Jul 28 10:34:49.164615 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:34:49.164577    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm"
Jul 28 10:37:37.162432 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:37:37.162402    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 10:37:37.167247 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:37:37.167222    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 10:42:37.195905 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:42:37.195876    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 10:42:37.202619 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:42:37.202596    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 10:47:37.227884 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:47:37.227850    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 10:47:37.236867 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:47:37.236836    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 10:49:26.832100 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:49:26.831999    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm"]
Jul 28 10:49:26.844192 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:49:26.844151    2563 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 28 10:49:26.844192 ip-10-0-130-235 kubenswrapper[2563]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 28 10:49:26.844192 ip-10-0-130-235 kubenswrapper[2563]:         , stdout: , stderr: , exit code -1
Jul 28 10:49:26.844192 ip-10-0-130-235 kubenswrapper[2563]:  > containerID="d2ea5dc1654a7dd97b15096921db23bb25564e7c79744b78bf5fd886cb39cd2e" cmd=["/bin/sleep","15"]
Jul 28 10:49:26.844551 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:49:26.844212    2563 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm" message="" err=<
Jul 28 10:49:26.844551 ip-10-0-130-235 kubenswrapper[2563]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 28 10:49:26.844551 ip-10-0-130-235 kubenswrapper[2563]:         , stdout: , stderr: , exit code -1
Jul 28 10:49:26.844551 ip-10-0-130-235 kubenswrapper[2563]:  >
Jul 28 10:49:26.844551 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:49:26.844225    2563 kuberuntime_container.go:755] "PreStop hook failed" err=<
Jul 28 10:49:26.844551 ip-10-0-130-235 kubenswrapper[2563]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 28 10:49:26.844551 ip-10-0-130-235 kubenswrapper[2563]:         , stdout: , stderr: , exit code -1
Jul 28 10:49:26.844551 ip-10-0-130-235 kubenswrapper[2563]:  > pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm" podUID="6803cbb3-65e2-4d59-881e-c7a88233fe17" containerName="main" containerID="cri-o://d2ea5dc1654a7dd97b15096921db23bb25564e7c79744b78bf5fd886cb39cd2e"
Jul 28 10:49:26.844551 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:49:26.844279    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm" podUID="6803cbb3-65e2-4d59-881e-c7a88233fe17" containerName="main" containerID="cri-o://d2ea5dc1654a7dd97b15096921db23bb25564e7c79744b78bf5fd886cb39cd2e" gracePeriod=60
Jul 28 10:49:27.209592 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:49:27.209564    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm"
Jul 28 10:49:27.344630 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:49:27.344583    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6803cbb3-65e2-4d59-881e-c7a88233fe17-tmp-dir\") pod \"6803cbb3-65e2-4d59-881e-c7a88233fe17\" (UID: \"6803cbb3-65e2-4d59-881e-c7a88233fe17\") "
Jul 28 10:49:27.344630 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:49:27.344641    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6803cbb3-65e2-4d59-881e-c7a88233fe17-model-cache\") pod \"6803cbb3-65e2-4d59-881e-c7a88233fe17\" (UID: \"6803cbb3-65e2-4d59-881e-c7a88233fe17\") "
Jul 28 10:49:27.344862 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:49:27.344721    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6803cbb3-65e2-4d59-881e-c7a88233fe17-tls-certs\") pod \"6803cbb3-65e2-4d59-881e-c7a88233fe17\" (UID: \"6803cbb3-65e2-4d59-881e-c7a88233fe17\") "
Jul 28 10:49:27.344862 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:49:27.344755    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6803cbb3-65e2-4d59-881e-c7a88233fe17-dshm\") pod \"6803cbb3-65e2-4d59-881e-c7a88233fe17\" (UID: \"6803cbb3-65e2-4d59-881e-c7a88233fe17\") "
Jul 28 10:49:27.344862 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:49:27.344779    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ht9ph\" (UniqueName: \"kubernetes.io/projected/6803cbb3-65e2-4d59-881e-c7a88233fe17-kube-api-access-ht9ph\") pod \"6803cbb3-65e2-4d59-881e-c7a88233fe17\" (UID: \"6803cbb3-65e2-4d59-881e-c7a88233fe17\") "
Jul 28 10:49:27.344862 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:49:27.344805    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6803cbb3-65e2-4d59-881e-c7a88233fe17-home\") pod \"6803cbb3-65e2-4d59-881e-c7a88233fe17\" (UID: \"6803cbb3-65e2-4d59-881e-c7a88233fe17\") "
Jul 28 10:49:27.345132 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:49:27.344888    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6803cbb3-65e2-4d59-881e-c7a88233fe17-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6803cbb3-65e2-4d59-881e-c7a88233fe17" (UID: "6803cbb3-65e2-4d59-881e-c7a88233fe17"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 10:49:27.345132 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:49:27.344948    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6803cbb3-65e2-4d59-881e-c7a88233fe17-model-cache" (OuterVolumeSpecName: "model-cache") pod "6803cbb3-65e2-4d59-881e-c7a88233fe17" (UID: "6803cbb3-65e2-4d59-881e-c7a88233fe17"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 10:49:27.345132 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:49:27.345109    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6803cbb3-65e2-4d59-881e-c7a88233fe17-tmp-dir\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 10:49:27.345132 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:49:27.345126    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6803cbb3-65e2-4d59-881e-c7a88233fe17-model-cache\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 10:49:27.345293 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:49:27.345158    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6803cbb3-65e2-4d59-881e-c7a88233fe17-home" (OuterVolumeSpecName: "home") pod "6803cbb3-65e2-4d59-881e-c7a88233fe17" (UID: "6803cbb3-65e2-4d59-881e-c7a88233fe17"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 10:49:27.347220 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:49:27.347188    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6803cbb3-65e2-4d59-881e-c7a88233fe17-kube-api-access-ht9ph" (OuterVolumeSpecName: "kube-api-access-ht9ph") pod "6803cbb3-65e2-4d59-881e-c7a88233fe17" (UID: "6803cbb3-65e2-4d59-881e-c7a88233fe17"). InnerVolumeSpecName "kube-api-access-ht9ph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 10:49:27.347220 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:49:27.347198    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6803cbb3-65e2-4d59-881e-c7a88233fe17-dshm" (OuterVolumeSpecName: "dshm") pod "6803cbb3-65e2-4d59-881e-c7a88233fe17" (UID: "6803cbb3-65e2-4d59-881e-c7a88233fe17"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 10:49:27.347403 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:49:27.347217    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6803cbb3-65e2-4d59-881e-c7a88233fe17-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6803cbb3-65e2-4d59-881e-c7a88233fe17" (UID: "6803cbb3-65e2-4d59-881e-c7a88233fe17"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 10:49:27.446465 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:49:27.446352    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6803cbb3-65e2-4d59-881e-c7a88233fe17-tls-certs\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 10:49:27.446465 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:49:27.446409    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6803cbb3-65e2-4d59-881e-c7a88233fe17-dshm\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 10:49:27.446465 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:49:27.446418    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ht9ph\" (UniqueName: \"kubernetes.io/projected/6803cbb3-65e2-4d59-881e-c7a88233fe17-kube-api-access-ht9ph\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 10:49:27.446465 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:49:27.446428    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6803cbb3-65e2-4d59-881e-c7a88233fe17-home\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 10:49:27.694441 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:49:27.694405    2563 generic.go:356] "Generic (PLEG): container finished" podID="6803cbb3-65e2-4d59-881e-c7a88233fe17" containerID="d2ea5dc1654a7dd97b15096921db23bb25564e7c79744b78bf5fd886cb39cd2e" exitCode=0
Jul 28 10:49:27.694618 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:49:27.694478    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm"
Jul 28 10:49:27.694618 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:49:27.694490    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm" event={"ID":"6803cbb3-65e2-4d59-881e-c7a88233fe17","Type":"ContainerDied","Data":"d2ea5dc1654a7dd97b15096921db23bb25564e7c79744b78bf5fd886cb39cd2e"}
Jul 28 10:49:27.694618 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:49:27.694583    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm" event={"ID":"6803cbb3-65e2-4d59-881e-c7a88233fe17","Type":"ContainerDied","Data":"f28cef1cac0f974ecc3233c8790a36db3f2b935d65e6b6d5bcd59af263ed63ec"}
Jul 28 10:49:27.694618 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:49:27.694615    2563 scope.go:117] "RemoveContainer" containerID="d2ea5dc1654a7dd97b15096921db23bb25564e7c79744b78bf5fd886cb39cd2e"
Jul 28 10:49:27.704739 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:49:27.704720    2563 scope.go:117] "RemoveContainer" containerID="d2ea5dc1654a7dd97b15096921db23bb25564e7c79744b78bf5fd886cb39cd2e"
Jul 28 10:49:27.705049 ip-10-0-130-235 kubenswrapper[2563]: E0728 10:49:27.705030    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2ea5dc1654a7dd97b15096921db23bb25564e7c79744b78bf5fd886cb39cd2e\": container with ID starting with d2ea5dc1654a7dd97b15096921db23bb25564e7c79744b78bf5fd886cb39cd2e not found: ID does not exist" containerID="d2ea5dc1654a7dd97b15096921db23bb25564e7c79744b78bf5fd886cb39cd2e"
Jul 28 10:49:27.705094 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:49:27.705059    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2ea5dc1654a7dd97b15096921db23bb25564e7c79744b78bf5fd886cb39cd2e"} err="failed to get container status \"d2ea5dc1654a7dd97b15096921db23bb25564e7c79744b78bf5fd886cb39cd2e\": rpc error: code = NotFound desc = could not find container \"d2ea5dc1654a7dd97b15096921db23bb25564e7c79744b78bf5fd886cb39cd2e\": container with ID starting with d2ea5dc1654a7dd97b15096921db23bb25564e7c79744b78bf5fd886cb39cd2e not found: ID does not exist"
Jul 28 10:49:27.716336 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:49:27.716302    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm"]
Jul 28 10:49:27.719680 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:49:27.719646    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm"]
Jul 28 10:49:28.154523 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:49:28.154474    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-autoscaling-hpa-deployment-9ba6f3f4/autoscale-hpa-deploy-kserve-66746bd8b9-8gpjm" podUID="6803cbb3-65e2-4d59-881e-c7a88233fe17" containerName="main" probeResult="failure" output="Get \"https://10.132.0.37:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Jul 28 10:49:29.130524 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:49:29.130482    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6803cbb3-65e2-4d59-881e-c7a88233fe17" path="/var/lib/kubelet/pods/6803cbb3-65e2-4d59-881e-c7a88233fe17/volumes"
Jul 28 10:50:08.543895 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:50:08.543857    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh"]
Jul 28 10:50:08.544454 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:50:08.544226    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6803cbb3-65e2-4d59-881e-c7a88233fe17" containerName="main"
Jul 28 10:50:08.544454 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:50:08.544237    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6803cbb3-65e2-4d59-881e-c7a88233fe17" containerName="main"
Jul 28 10:50:08.544454 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:50:08.544314    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6803cbb3-65e2-4d59-881e-c7a88233fe17" containerName="main"
Jul 28 10:50:08.547331 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:50:08.547314    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh"
Jul 28 10:50:08.555314 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:50:08.555279    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-autoscaling-keda-deployment-b2150d0b\"/\"autoscale-keda-deploy-kserve-self-signed-certs\""
Jul 28 10:50:08.555590 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:50:08.555303    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-autoscaling-keda-deployment-b2150d0b\"/\"kube-root-ca.crt\""
Jul 28 10:50:08.555726 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:50:08.555403    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-autoscaling-keda-deployment-b2150d0b\"/\"default-dockercfg-54svx\""
Jul 28 10:50:08.555850 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:50:08.555438    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-autoscaling-keda-deployment-b2150d0b\"/\"openshift-service-ca.crt\""
Jul 28 10:50:08.558253 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:50:08.558229    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh"]
Jul 28 10:50:08.630014 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:50:08.629976    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29c4f9e2-4b50-4435-9bdf-89c9bd613966-model-cache\") pod \"autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh\" (UID: \"29c4f9e2-4b50-4435-9bdf-89c9bd613966\") " pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh"
Jul 28 10:50:08.630232 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:50:08.630029    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29c4f9e2-4b50-4435-9bdf-89c9bd613966-tls-certs\") pod \"autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh\" (UID: \"29c4f9e2-4b50-4435-9bdf-89c9bd613966\") " pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh"
Jul 28 10:50:08.630232 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:50:08.630155    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29c4f9e2-4b50-4435-9bdf-89c9bd613966-home\") pod \"autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh\" (UID: \"29c4f9e2-4b50-4435-9bdf-89c9bd613966\") " pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh"
Jul 28 10:50:08.630232 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:50:08.630195    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtz9g\" (UniqueName: \"kubernetes.io/projected/29c4f9e2-4b50-4435-9bdf-89c9bd613966-kube-api-access-jtz9g\") pod \"autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh\" (UID: \"29c4f9e2-4b50-4435-9bdf-89c9bd613966\") " pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh"
Jul 28 10:50:08.630232 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:50:08.630222    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29c4f9e2-4b50-4435-9bdf-89c9bd613966-dshm\") pod \"autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh\" (UID: \"29c4f9e2-4b50-4435-9bdf-89c9bd613966\") " pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh"
Jul 28 10:50:08.630463 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:50:08.630323    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29c4f9e2-4b50-4435-9bdf-89c9bd613966-tmp-dir\") pod \"autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh\" (UID: \"29c4f9e2-4b50-4435-9bdf-89c9bd613966\") " pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh"
Jul 28 10:50:08.730864 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:50:08.730819    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29c4f9e2-4b50-4435-9bdf-89c9bd613966-tls-certs\") pod \"autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh\" (UID: \"29c4f9e2-4b50-4435-9bdf-89c9bd613966\") " pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh"
Jul 28 10:50:08.731067 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:50:08.730887    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29c4f9e2-4b50-4435-9bdf-89c9bd613966-home\") pod \"autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh\" (UID: \"29c4f9e2-4b50-4435-9bdf-89c9bd613966\") " pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh"
Jul 28 10:50:08.731067 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:50:08.730911    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtz9g\" (UniqueName: \"kubernetes.io/projected/29c4f9e2-4b50-4435-9bdf-89c9bd613966-kube-api-access-jtz9g\") pod \"autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh\" (UID: \"29c4f9e2-4b50-4435-9bdf-89c9bd613966\") " pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh"
Jul 28 10:50:08.731067 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:50:08.730935    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29c4f9e2-4b50-4435-9bdf-89c9bd613966-dshm\") pod \"autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh\" (UID: \"29c4f9e2-4b50-4435-9bdf-89c9bd613966\") " pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh"
Jul 28 10:50:08.731067 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:50:08.730989    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29c4f9e2-4b50-4435-9bdf-89c9bd613966-tmp-dir\") pod \"autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh\" (UID: \"29c4f9e2-4b50-4435-9bdf-89c9bd613966\") " pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh"
Jul 28 10:50:08.731067 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:50:08.731039    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29c4f9e2-4b50-4435-9bdf-89c9bd613966-model-cache\") pod \"autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh\" (UID: \"29c4f9e2-4b50-4435-9bdf-89c9bd613966\") " pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh"
Jul 28 10:50:08.731413 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:50:08.731388    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29c4f9e2-4b50-4435-9bdf-89c9bd613966-home\") pod \"autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh\" (UID: \"29c4f9e2-4b50-4435-9bdf-89c9bd613966\") " pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh"
Jul 28 10:50:08.731522 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:50:08.731489    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29c4f9e2-4b50-4435-9bdf-89c9bd613966-tmp-dir\") pod \"autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh\" (UID: \"29c4f9e2-4b50-4435-9bdf-89c9bd613966\") " pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh"
Jul 28 10:50:08.731585 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:50:08.731570    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29c4f9e2-4b50-4435-9bdf-89c9bd613966-model-cache\") pod \"autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh\" (UID: \"29c4f9e2-4b50-4435-9bdf-89c9bd613966\") " pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh"
Jul 28 10:50:08.733454 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:50:08.733432    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29c4f9e2-4b50-4435-9bdf-89c9bd613966-dshm\") pod \"autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh\" (UID: \"29c4f9e2-4b50-4435-9bdf-89c9bd613966\") " pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh"
Jul 28 10:50:08.733590 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:50:08.733572    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29c4f9e2-4b50-4435-9bdf-89c9bd613966-tls-certs\") pod \"autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh\" (UID: \"29c4f9e2-4b50-4435-9bdf-89c9bd613966\") " pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh"
Jul 28 10:50:08.740435 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:50:08.740408    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtz9g\" (UniqueName: \"kubernetes.io/projected/29c4f9e2-4b50-4435-9bdf-89c9bd613966-kube-api-access-jtz9g\") pod \"autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh\" (UID: \"29c4f9e2-4b50-4435-9bdf-89c9bd613966\") " pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh"
Jul 28 10:50:08.865614 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:50:08.865576    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh"
Jul 28 10:50:09.018038 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:50:09.018002    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh"]
Jul 28 10:50:09.021269 ip-10-0-130-235 kubenswrapper[2563]: W0728 10:50:09.021243    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29c4f9e2_4b50_4435_9bdf_89c9bd613966.slice/crio-41fb83420f4c5606d75c1e70a6f1ca43dd73ec7f97bb738e62607c91bfd53c18 WatchSource:0}: Error finding container 41fb83420f4c5606d75c1e70a6f1ca43dd73ec7f97bb738e62607c91bfd53c18: Status 404 returned error can't find the container with id 41fb83420f4c5606d75c1e70a6f1ca43dd73ec7f97bb738e62607c91bfd53c18
Jul 28 10:50:09.907755 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:50:09.907716    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh" event={"ID":"29c4f9e2-4b50-4435-9bdf-89c9bd613966","Type":"ContainerStarted","Data":"220f9a5587f60d1522f5b2874e424c99c7e24a53571f51ebcfc5d3e4ce659074"}
Jul 28 10:50:09.908234 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:50:09.907766    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh" event={"ID":"29c4f9e2-4b50-4435-9bdf-89c9bd613966","Type":"ContainerStarted","Data":"41fb83420f4c5606d75c1e70a6f1ca43dd73ec7f97bb738e62607c91bfd53c18"}
Jul 28 10:50:09.926400 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:50:09.926314    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh" podStartSLOduration=1.926292057 podStartE2EDuration="1.926292057s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 10:50:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 10:50:09.924495491 +0000 UTC m=+2553.366036829" watchObservedRunningTime="2026-07-28 10:50:09.926292057 +0000 UTC m=+2553.367833396"
Jul 28 10:50:10.866070 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:50:10.866031    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh"
Jul 28 10:50:18.866017 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:50:18.865969    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh"
Jul 28 10:50:18.878858 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:50:18.878827    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh"
Jul 28 10:50:18.964500 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:50:18.964460    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh"
Jul 28 10:52:37.259729 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:52:37.259699    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 10:52:37.269210 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:52:37.269186    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 10:57:37.291496 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:57:37.291467    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 10:57:37.303125 ip-10-0-130-235 kubenswrapper[2563]: I0728 10:57:37.303095    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 11:02:37.326050 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:02:37.326018    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 11:02:37.338508 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:02:37.338482    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 11:05:06.015297 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:06.015253    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh"]
Jul 28 11:05:06.027705 ip-10-0-130-235 kubenswrapper[2563]: E0728 11:05:06.027663    2563 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 28 11:05:06.027705 ip-10-0-130-235 kubenswrapper[2563]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 28 11:05:06.027705 ip-10-0-130-235 kubenswrapper[2563]:         , stdout: , stderr: , exit code -1
Jul 28 11:05:06.027705 ip-10-0-130-235 kubenswrapper[2563]:  > containerID="220f9a5587f60d1522f5b2874e424c99c7e24a53571f51ebcfc5d3e4ce659074" cmd=["/bin/sleep","15"]
Jul 28 11:05:06.027984 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:06.027724    2563 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh" message="" err=<
Jul 28 11:05:06.027984 ip-10-0-130-235 kubenswrapper[2563]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 28 11:05:06.027984 ip-10-0-130-235 kubenswrapper[2563]:         , stdout: , stderr: , exit code -1
Jul 28 11:05:06.027984 ip-10-0-130-235 kubenswrapper[2563]:  >
Jul 28 11:05:06.027984 ip-10-0-130-235 kubenswrapper[2563]: E0728 11:05:06.027738    2563 kuberuntime_container.go:755] "PreStop hook failed" err=<
Jul 28 11:05:06.027984 ip-10-0-130-235 kubenswrapper[2563]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 28 11:05:06.027984 ip-10-0-130-235 kubenswrapper[2563]:         , stdout: , stderr: , exit code -1
Jul 28 11:05:06.027984 ip-10-0-130-235 kubenswrapper[2563]:  > pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh" podUID="29c4f9e2-4b50-4435-9bdf-89c9bd613966" containerName="main" containerID="cri-o://220f9a5587f60d1522f5b2874e424c99c7e24a53571f51ebcfc5d3e4ce659074"
Jul 28 11:05:06.027984 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:06.027790    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh" podUID="29c4f9e2-4b50-4435-9bdf-89c9bd613966" containerName="main" containerID="cri-o://220f9a5587f60d1522f5b2874e424c99c7e24a53571f51ebcfc5d3e4ce659074" gracePeriod=60
Jul 28 11:05:06.398096 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:06.398071    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh"
Jul 28 11:05:06.433048 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:06.433014    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29c4f9e2-4b50-4435-9bdf-89c9bd613966-tls-certs\") pod \"29c4f9e2-4b50-4435-9bdf-89c9bd613966\" (UID: \"29c4f9e2-4b50-4435-9bdf-89c9bd613966\") "
Jul 28 11:05:06.433248 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:06.433056    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29c4f9e2-4b50-4435-9bdf-89c9bd613966-model-cache\") pod \"29c4f9e2-4b50-4435-9bdf-89c9bd613966\" (UID: \"29c4f9e2-4b50-4435-9bdf-89c9bd613966\") "
Jul 28 11:05:06.433248 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:06.433122    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jtz9g\" (UniqueName: \"kubernetes.io/projected/29c4f9e2-4b50-4435-9bdf-89c9bd613966-kube-api-access-jtz9g\") pod \"29c4f9e2-4b50-4435-9bdf-89c9bd613966\" (UID: \"29c4f9e2-4b50-4435-9bdf-89c9bd613966\") "
Jul 28 11:05:06.433248 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:06.433210    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29c4f9e2-4b50-4435-9bdf-89c9bd613966-tmp-dir\") pod \"29c4f9e2-4b50-4435-9bdf-89c9bd613966\" (UID: \"29c4f9e2-4b50-4435-9bdf-89c9bd613966\") "
Jul 28 11:05:06.433447 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:06.433329    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29c4f9e2-4b50-4435-9bdf-89c9bd613966-home\") pod \"29c4f9e2-4b50-4435-9bdf-89c9bd613966\" (UID: \"29c4f9e2-4b50-4435-9bdf-89c9bd613966\") "
Jul 28 11:05:06.433447 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:06.433377    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29c4f9e2-4b50-4435-9bdf-89c9bd613966-dshm\") pod \"29c4f9e2-4b50-4435-9bdf-89c9bd613966\" (UID: \"29c4f9e2-4b50-4435-9bdf-89c9bd613966\") "
Jul 28 11:05:06.433574 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:06.433447    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29c4f9e2-4b50-4435-9bdf-89c9bd613966-model-cache" (OuterVolumeSpecName: "model-cache") pod "29c4f9e2-4b50-4435-9bdf-89c9bd613966" (UID: "29c4f9e2-4b50-4435-9bdf-89c9bd613966"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:05:06.433670 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:06.433644    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29c4f9e2-4b50-4435-9bdf-89c9bd613966-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "29c4f9e2-4b50-4435-9bdf-89c9bd613966" (UID: "29c4f9e2-4b50-4435-9bdf-89c9bd613966"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:05:06.433737 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:06.433694    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29c4f9e2-4b50-4435-9bdf-89c9bd613966-model-cache\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 11:05:06.433855 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:06.433830    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29c4f9e2-4b50-4435-9bdf-89c9bd613966-home" (OuterVolumeSpecName: "home") pod "29c4f9e2-4b50-4435-9bdf-89c9bd613966" (UID: "29c4f9e2-4b50-4435-9bdf-89c9bd613966"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:05:06.435700 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:06.435665    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29c4f9e2-4b50-4435-9bdf-89c9bd613966-dshm" (OuterVolumeSpecName: "dshm") pod "29c4f9e2-4b50-4435-9bdf-89c9bd613966" (UID: "29c4f9e2-4b50-4435-9bdf-89c9bd613966"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:05:06.435700 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:06.435687    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29c4f9e2-4b50-4435-9bdf-89c9bd613966-kube-api-access-jtz9g" (OuterVolumeSpecName: "kube-api-access-jtz9g") pod "29c4f9e2-4b50-4435-9bdf-89c9bd613966" (UID: "29c4f9e2-4b50-4435-9bdf-89c9bd613966"). InnerVolumeSpecName "kube-api-access-jtz9g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 11:05:06.435865 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:06.435718    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29c4f9e2-4b50-4435-9bdf-89c9bd613966-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "29c4f9e2-4b50-4435-9bdf-89c9bd613966" (UID: "29c4f9e2-4b50-4435-9bdf-89c9bd613966"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 11:05:06.487706 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:06.487669    2563 generic.go:356] "Generic (PLEG): container finished" podID="29c4f9e2-4b50-4435-9bdf-89c9bd613966" containerID="220f9a5587f60d1522f5b2874e424c99c7e24a53571f51ebcfc5d3e4ce659074" exitCode=0
Jul 28 11:05:06.487917 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:06.487722    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh"
Jul 28 11:05:06.487917 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:06.487758    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh" event={"ID":"29c4f9e2-4b50-4435-9bdf-89c9bd613966","Type":"ContainerDied","Data":"220f9a5587f60d1522f5b2874e424c99c7e24a53571f51ebcfc5d3e4ce659074"}
Jul 28 11:05:06.487917 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:06.487818    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh" event={"ID":"29c4f9e2-4b50-4435-9bdf-89c9bd613966","Type":"ContainerDied","Data":"41fb83420f4c5606d75c1e70a6f1ca43dd73ec7f97bb738e62607c91bfd53c18"}
Jul 28 11:05:06.487917 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:06.487847    2563 scope.go:117] "RemoveContainer" containerID="220f9a5587f60d1522f5b2874e424c99c7e24a53571f51ebcfc5d3e4ce659074"
Jul 28 11:05:06.497817 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:06.497796    2563 scope.go:117] "RemoveContainer" containerID="220f9a5587f60d1522f5b2874e424c99c7e24a53571f51ebcfc5d3e4ce659074"
Jul 28 11:05:06.498171 ip-10-0-130-235 kubenswrapper[2563]: E0728 11:05:06.498153    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"220f9a5587f60d1522f5b2874e424c99c7e24a53571f51ebcfc5d3e4ce659074\": container with ID starting with 220f9a5587f60d1522f5b2874e424c99c7e24a53571f51ebcfc5d3e4ce659074 not found: ID does not exist" containerID="220f9a5587f60d1522f5b2874e424c99c7e24a53571f51ebcfc5d3e4ce659074"
Jul 28 11:05:06.498254 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:06.498181    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"220f9a5587f60d1522f5b2874e424c99c7e24a53571f51ebcfc5d3e4ce659074"} err="failed to get container status \"220f9a5587f60d1522f5b2874e424c99c7e24a53571f51ebcfc5d3e4ce659074\": rpc error: code = NotFound desc = could not find container \"220f9a5587f60d1522f5b2874e424c99c7e24a53571f51ebcfc5d3e4ce659074\": container with ID starting with 220f9a5587f60d1522f5b2874e424c99c7e24a53571f51ebcfc5d3e4ce659074 not found: ID does not exist"
Jul 28 11:05:06.514342 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:06.514307    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh"]
Jul 28 11:05:06.518638 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:06.518610    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-autoscaling-keda-deployment-b2150d0b/autoscale-keda-deploy-kserve-77fc6d7cf9-22kqh"]
Jul 28 11:05:06.535014 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:06.534980    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jtz9g\" (UniqueName: \"kubernetes.io/projected/29c4f9e2-4b50-4435-9bdf-89c9bd613966-kube-api-access-jtz9g\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 11:05:06.535014 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:06.535012    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29c4f9e2-4b50-4435-9bdf-89c9bd613966-tmp-dir\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 11:05:06.535230 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:06.535030    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29c4f9e2-4b50-4435-9bdf-89c9bd613966-home\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 11:05:06.535230 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:06.535041    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29c4f9e2-4b50-4435-9bdf-89c9bd613966-dshm\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 11:05:06.535230 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:06.535051    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29c4f9e2-4b50-4435-9bdf-89c9bd613966-tls-certs\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 11:05:07.131232 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:07.131197    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29c4f9e2-4b50-4435-9bdf-89c9bd613966" path="/var/lib/kubelet/pods/29c4f9e2-4b50-4435-9bdf-89c9bd613966/volumes"
Jul 28 11:05:46.986653 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:46.986608    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"]
Jul 28 11:05:46.987181 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:46.987078    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29c4f9e2-4b50-4435-9bdf-89c9bd613966" containerName="main"
Jul 28 11:05:46.987181 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:46.987097    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="29c4f9e2-4b50-4435-9bdf-89c9bd613966" containerName="main"
Jul 28 11:05:46.987181 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:46.987170    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="29c4f9e2-4b50-4435-9bdf-89c9bd613966" containerName="main"
Jul 28 11:05:46.991505 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:46.991485    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 11:05:46.994491 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:46.994466    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2\"/\"default-dockercfg-m7rhc\""
Jul 28 11:05:46.994638 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:46.994519    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2\"/\"openshift-service-ca.crt\""
Jul 28 11:05:46.994638 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:46.994573    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2\"/\"autoscale-hpa-lws-kserve-self-signed-certs\""
Jul 28 11:05:46.994638 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:46.994581    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2\"/\"autoscale-hpa-lws-kserve-mn-dockercfg-79sbd\""
Jul 28 11:05:46.994638 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:46.994586    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2\"/\"kube-root-ca.crt\""
Jul 28 11:05:47.002390 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:47.002332    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"]
Jul 28 11:05:47.005690 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:47.005659    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41620e77-941a-499d-9e80-67ae7c81232b-home\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"41620e77-941a-499d-9e80-67ae7c81232b\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 11:05:47.005836 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:47.005711    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41620e77-941a-499d-9e80-67ae7c81232b-tmp-dir\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"41620e77-941a-499d-9e80-67ae7c81232b\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 11:05:47.005836 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:47.005748    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41620e77-941a-499d-9e80-67ae7c81232b-dshm\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"41620e77-941a-499d-9e80-67ae7c81232b\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 11:05:47.005836 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:47.005793    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41620e77-941a-499d-9e80-67ae7c81232b-model-cache\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"41620e77-941a-499d-9e80-67ae7c81232b\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 11:05:47.005836 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:47.005819    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z77tz\" (UniqueName: \"kubernetes.io/projected/41620e77-941a-499d-9e80-67ae7c81232b-kube-api-access-z77tz\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"41620e77-941a-499d-9e80-67ae7c81232b\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 11:05:47.005995 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:47.005867    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41620e77-941a-499d-9e80-67ae7c81232b-tls-certs\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"41620e77-941a-499d-9e80-67ae7c81232b\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 11:05:47.106853 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:47.106797    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41620e77-941a-499d-9e80-67ae7c81232b-home\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"41620e77-941a-499d-9e80-67ae7c81232b\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 11:05:47.107045 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:47.106860    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41620e77-941a-499d-9e80-67ae7c81232b-tmp-dir\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"41620e77-941a-499d-9e80-67ae7c81232b\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 11:05:47.107045 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:47.106885    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41620e77-941a-499d-9e80-67ae7c81232b-dshm\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"41620e77-941a-499d-9e80-67ae7c81232b\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 11:05:47.107045 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:47.106903    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41620e77-941a-499d-9e80-67ae7c81232b-model-cache\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"41620e77-941a-499d-9e80-67ae7c81232b\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 11:05:47.107045 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:47.106927    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z77tz\" (UniqueName: \"kubernetes.io/projected/41620e77-941a-499d-9e80-67ae7c81232b-kube-api-access-z77tz\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"41620e77-941a-499d-9e80-67ae7c81232b\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 11:05:47.107045 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:47.106987    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41620e77-941a-499d-9e80-67ae7c81232b-tls-certs\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"41620e77-941a-499d-9e80-67ae7c81232b\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 11:05:47.107276 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:47.107176    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41620e77-941a-499d-9e80-67ae7c81232b-home\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"41620e77-941a-499d-9e80-67ae7c81232b\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 11:05:47.107334 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:47.107281    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41620e77-941a-499d-9e80-67ae7c81232b-model-cache\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"41620e77-941a-499d-9e80-67ae7c81232b\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 11:05:47.107436 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:47.107412    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41620e77-941a-499d-9e80-67ae7c81232b-tmp-dir\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"41620e77-941a-499d-9e80-67ae7c81232b\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 11:05:47.109392 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:47.109351    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41620e77-941a-499d-9e80-67ae7c81232b-dshm\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"41620e77-941a-499d-9e80-67ae7c81232b\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 11:05:47.109509 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:47.109483    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41620e77-941a-499d-9e80-67ae7c81232b-tls-certs\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"41620e77-941a-499d-9e80-67ae7c81232b\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 11:05:47.116130 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:47.116089    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z77tz\" (UniqueName: \"kubernetes.io/projected/41620e77-941a-499d-9e80-67ae7c81232b-kube-api-access-z77tz\") pod \"autoscale-hpa-lws-kserve-mn-0\" (UID: \"41620e77-941a-499d-9e80-67ae7c81232b\") " pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 11:05:47.304061 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:47.303970    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 11:05:47.463457 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:47.463416    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"]
Jul 28 11:05:47.466959 ip-10-0-130-235 kubenswrapper[2563]: W0728 11:05:47.466919    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41620e77_941a_499d_9e80_67ae7c81232b.slice/crio-4f54041e687c85662d4f47dc333963d6380e388a76fd81f0ce32eb82233b6942 WatchSource:0}: Error finding container 4f54041e687c85662d4f47dc333963d6380e388a76fd81f0ce32eb82233b6942: Status 404 returned error can't find the container with id 4f54041e687c85662d4f47dc333963d6380e388a76fd81f0ce32eb82233b6942
Jul 28 11:05:47.699830 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:47.699793    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0" event={"ID":"41620e77-941a-499d-9e80-67ae7c81232b","Type":"ContainerStarted","Data":"a4fd36dc7eb5286d337b9954ee3ad239e822b832dd799791136475f29e8ce185"}
Jul 28 11:05:47.700045 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:47.699850    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0" event={"ID":"41620e77-941a-499d-9e80-67ae7c81232b","Type":"ContainerStarted","Data":"4f54041e687c85662d4f47dc333963d6380e388a76fd81f0ce32eb82233b6942"}
Jul 28 11:05:47.721513 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:47.721436    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0" podStartSLOduration=1.721414271 podStartE2EDuration="1.721414271s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:05:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:05:47.720768364 +0000 UTC m=+3491.162309703" watchObservedRunningTime="2026-07-28 11:05:47.721414271 +0000 UTC m=+3491.162955609"
Jul 28 11:05:48.304481 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:48.304441    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 11:05:57.304797 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:57.304713    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 11:05:57.317913 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:57.317876    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 11:05:57.765438 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:05:57.765409    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 11:07:37.359228 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:07:37.359114    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 11:07:37.372183 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:07:37.372156    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 11:10:48.351499 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:10:48.351463    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0"]
Jul 28 11:10:48.355241 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:10:48.355218    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0"
Jul 28 11:10:48.357916 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:10:48.357885    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-7c3633d1\"/\"default-dockercfg-lfhh2\""
Jul 28 11:10:48.357916 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:10:48.357912    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-7c3633d1\"/\"openshift-service-ca.crt\""
Jul 28 11:10:48.358094 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:10:48.357924    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-7c3633d1\"/\"kube-root-ca.crt\""
Jul 28 11:10:48.358480 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:10:48.358459    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-7c3633d1\"/\"llmisvc-model-pvc-router-manage-2577e7-b09f84b1-dockercfg-zzdrr\""
Jul 28 11:10:48.358598 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:10:48.358522    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-7c3633d1\"/\"llmisv1dc4269d1ada5f2d28562215d180c57f-kserve-self-signed-certs\""
Jul 28 11:10:48.369881 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:10:48.369846    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0"]
Jul 28 11:10:48.484540 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:10:48.484494    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8cc1f64c-273d-47e1-add7-cfef2f51d3b0-tmp-dir\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0\" (UID: \"8cc1f64c-273d-47e1-add7-cfef2f51d3b0\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0"
Jul 28 11:10:48.484733 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:10:48.484550    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8cc1f64c-273d-47e1-add7-cfef2f51d3b0-tls-certs\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0\" (UID: \"8cc1f64c-273d-47e1-add7-cfef2f51d3b0\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0"
Jul 28 11:10:48.484733 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:10:48.484645    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8cc1f64c-273d-47e1-add7-cfef2f51d3b0-model-cache\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0\" (UID: \"8cc1f64c-273d-47e1-add7-cfef2f51d3b0\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0"
Jul 28 11:10:48.484733 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:10:48.484698    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8cc1f64c-273d-47e1-add7-cfef2f51d3b0-dshm\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0\" (UID: \"8cc1f64c-273d-47e1-add7-cfef2f51d3b0\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0"
Jul 28 11:10:48.484869 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:10:48.484743    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8cc1f64c-273d-47e1-add7-cfef2f51d3b0-home\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0\" (UID: \"8cc1f64c-273d-47e1-add7-cfef2f51d3b0\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0"
Jul 28 11:10:48.484869 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:10:48.484774    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whstl\" (UniqueName: \"kubernetes.io/projected/8cc1f64c-273d-47e1-add7-cfef2f51d3b0-kube-api-access-whstl\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0\" (UID: \"8cc1f64c-273d-47e1-add7-cfef2f51d3b0\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0"
Jul 28 11:10:48.586140 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:10:48.586094    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8cc1f64c-273d-47e1-add7-cfef2f51d3b0-model-cache\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0\" (UID: \"8cc1f64c-273d-47e1-add7-cfef2f51d3b0\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0"
Jul 28 11:10:48.586313 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:10:48.586155    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8cc1f64c-273d-47e1-add7-cfef2f51d3b0-dshm\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0\" (UID: \"8cc1f64c-273d-47e1-add7-cfef2f51d3b0\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0"
Jul 28 11:10:48.586313 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:10:48.586186    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8cc1f64c-273d-47e1-add7-cfef2f51d3b0-home\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0\" (UID: \"8cc1f64c-273d-47e1-add7-cfef2f51d3b0\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0"
Jul 28 11:10:48.586413 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:10:48.586345    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whstl\" (UniqueName: \"kubernetes.io/projected/8cc1f64c-273d-47e1-add7-cfef2f51d3b0-kube-api-access-whstl\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0\" (UID: \"8cc1f64c-273d-47e1-add7-cfef2f51d3b0\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0"
Jul 28 11:10:48.586497 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:10:48.586477    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8cc1f64c-273d-47e1-add7-cfef2f51d3b0-tmp-dir\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0\" (UID: \"8cc1f64c-273d-47e1-add7-cfef2f51d3b0\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0"
Jul 28 11:10:48.586645 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:10:48.586533    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8cc1f64c-273d-47e1-add7-cfef2f51d3b0-tls-certs\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0\" (UID: \"8cc1f64c-273d-47e1-add7-cfef2f51d3b0\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0"
Jul 28 11:10:48.586645 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:10:48.586588    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8cc1f64c-273d-47e1-add7-cfef2f51d3b0-home\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0\" (UID: \"8cc1f64c-273d-47e1-add7-cfef2f51d3b0\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0"
Jul 28 11:10:48.586645 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:10:48.586590    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8cc1f64c-273d-47e1-add7-cfef2f51d3b0-model-cache\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0\" (UID: \"8cc1f64c-273d-47e1-add7-cfef2f51d3b0\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0"
Jul 28 11:10:48.586829 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:10:48.586717    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8cc1f64c-273d-47e1-add7-cfef2f51d3b0-tmp-dir\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0\" (UID: \"8cc1f64c-273d-47e1-add7-cfef2f51d3b0\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0"
Jul 28 11:10:48.588529 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:10:48.588502    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8cc1f64c-273d-47e1-add7-cfef2f51d3b0-dshm\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0\" (UID: \"8cc1f64c-273d-47e1-add7-cfef2f51d3b0\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0"
Jul 28 11:10:48.589061 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:10:48.589044    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8cc1f64c-273d-47e1-add7-cfef2f51d3b0-tls-certs\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0\" (UID: \"8cc1f64c-273d-47e1-add7-cfef2f51d3b0\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0"
Jul 28 11:10:48.595283 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:10:48.595258    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whstl\" (UniqueName: \"kubernetes.io/projected/8cc1f64c-273d-47e1-add7-cfef2f51d3b0-kube-api-access-whstl\") pod \"llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0\" (UID: \"8cc1f64c-273d-47e1-add7-cfef2f51d3b0\") " pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0"
Jul 28 11:12:37.390994 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:12:37.390871    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 11:12:37.406032 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:12:37.406004    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 11:12:51.366369 ip-10-0-130-235 kubenswrapper[2563]: E0728 11:12:51.366319    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kserve-pvc-source], unattached volumes=[kserve-pvc-source], failed to process volumes=[]: context deadline exceeded" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0" podUID="8cc1f64c-273d-47e1-add7-cfef2f51d3b0"
Jul 28 11:12:51.916728 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:12:51.916696    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0"
Jul 28 11:14:54.917524 ip-10-0-130-235 kubenswrapper[2563]: E0728 11:14:54.917477    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kserve-pvc-source], unattached volumes=[kserve-pvc-source], failed to process volumes=[]: context deadline exceeded" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0" podUID="8cc1f64c-273d-47e1-add7-cfef2f51d3b0"
Jul 28 11:14:55.548135 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:14:55.548103    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0"
Jul 28 11:16:58.549150 ip-10-0-130-235 kubenswrapper[2563]: E0728 11:16:58.549102    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kserve-pvc-source], unattached volumes=[kserve-pvc-source], failed to process volumes=[]: context deadline exceeded" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0" podUID="8cc1f64c-273d-47e1-add7-cfef2f51d3b0"
Jul 28 11:16:59.190014 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:16:59.189983    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0"
Jul 28 11:17:37.423532 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:17:37.423413    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 11:17:37.440919 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:17:37.440891    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 11:19:02.191005 ip-10-0-130-235 kubenswrapper[2563]: E0728 11:19:02.190954    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kserve-pvc-source], unattached volumes=[kserve-pvc-source], failed to process volumes=[]: context deadline exceeded" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0" podUID="8cc1f64c-273d-47e1-add7-cfef2f51d3b0"
Jul 28 11:19:02.829400 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:19:02.829348    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0"
Jul 28 11:20:42.035450 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:20:42.035410    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"]
Jul 28 11:20:42.045041 ip-10-0-130-235 kubenswrapper[2563]: E0728 11:20:42.045009    2563 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 28 11:20:42.045041 ip-10-0-130-235 kubenswrapper[2563]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 28 11:20:42.045041 ip-10-0-130-235 kubenswrapper[2563]:         , stdout: , stderr: , exit code -1
Jul 28 11:20:42.045041 ip-10-0-130-235 kubenswrapper[2563]:  > containerID="a4fd36dc7eb5286d337b9954ee3ad239e822b832dd799791136475f29e8ce185" cmd=["/bin/sleep","15"]
Jul 28 11:20:42.045344 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:20:42.045057    2563 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0" message="" err=<
Jul 28 11:20:42.045344 ip-10-0-130-235 kubenswrapper[2563]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 28 11:20:42.045344 ip-10-0-130-235 kubenswrapper[2563]:         , stdout: , stderr: , exit code -1
Jul 28 11:20:42.045344 ip-10-0-130-235 kubenswrapper[2563]:  >
Jul 28 11:20:42.045344 ip-10-0-130-235 kubenswrapper[2563]: E0728 11:20:42.045072    2563 kuberuntime_container.go:755] "PreStop hook failed" err=<
Jul 28 11:20:42.045344 ip-10-0-130-235 kubenswrapper[2563]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 28 11:20:42.045344 ip-10-0-130-235 kubenswrapper[2563]:         , stdout: , stderr: , exit code -1
Jul 28 11:20:42.045344 ip-10-0-130-235 kubenswrapper[2563]:  > pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0" podUID="41620e77-941a-499d-9e80-67ae7c81232b" containerName="main" containerID="cri-o://a4fd36dc7eb5286d337b9954ee3ad239e822b832dd799791136475f29e8ce185"
Jul 28 11:20:42.045344 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:20:42.045109    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0" podUID="41620e77-941a-499d-9e80-67ae7c81232b" containerName="main" containerID="cri-o://a4fd36dc7eb5286d337b9954ee3ad239e822b832dd799791136475f29e8ce185" gracePeriod=60
Jul 28 11:20:42.352049 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:20:42.352018    2563 generic.go:356] "Generic (PLEG): container finished" podID="41620e77-941a-499d-9e80-67ae7c81232b" containerID="a4fd36dc7eb5286d337b9954ee3ad239e822b832dd799791136475f29e8ce185" exitCode=0
Jul 28 11:20:42.352223 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:20:42.352095    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0" event={"ID":"41620e77-941a-499d-9e80-67ae7c81232b","Type":"ContainerDied","Data":"a4fd36dc7eb5286d337b9954ee3ad239e822b832dd799791136475f29e8ce185"}
Jul 28 11:20:42.399507 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:20:42.399482    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 11:20:42.525152 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:20:42.525120    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z77tz\" (UniqueName: \"kubernetes.io/projected/41620e77-941a-499d-9e80-67ae7c81232b-kube-api-access-z77tz\") pod \"41620e77-941a-499d-9e80-67ae7c81232b\" (UID: \"41620e77-941a-499d-9e80-67ae7c81232b\") "
Jul 28 11:20:42.525392 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:20:42.525208    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41620e77-941a-499d-9e80-67ae7c81232b-home\") pod \"41620e77-941a-499d-9e80-67ae7c81232b\" (UID: \"41620e77-941a-499d-9e80-67ae7c81232b\") "
Jul 28 11:20:42.525392 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:20:42.525275    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41620e77-941a-499d-9e80-67ae7c81232b-tls-certs\") pod \"41620e77-941a-499d-9e80-67ae7c81232b\" (UID: \"41620e77-941a-499d-9e80-67ae7c81232b\") "
Jul 28 11:20:42.525392 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:20:42.525307    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41620e77-941a-499d-9e80-67ae7c81232b-dshm\") pod \"41620e77-941a-499d-9e80-67ae7c81232b\" (UID: \"41620e77-941a-499d-9e80-67ae7c81232b\") "
Jul 28 11:20:42.525392 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:20:42.525345    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41620e77-941a-499d-9e80-67ae7c81232b-tmp-dir\") pod \"41620e77-941a-499d-9e80-67ae7c81232b\" (UID: \"41620e77-941a-499d-9e80-67ae7c81232b\") "
Jul 28 11:20:42.525613 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:20:42.525397    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41620e77-941a-499d-9e80-67ae7c81232b-model-cache\") pod \"41620e77-941a-499d-9e80-67ae7c81232b\" (UID: \"41620e77-941a-499d-9e80-67ae7c81232b\") "
Jul 28 11:20:42.525613 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:20:42.525465    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41620e77-941a-499d-9e80-67ae7c81232b-home" (OuterVolumeSpecName: "home") pod "41620e77-941a-499d-9e80-67ae7c81232b" (UID: "41620e77-941a-499d-9e80-67ae7c81232b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:20:42.525712 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:20:42.525688    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41620e77-941a-499d-9e80-67ae7c81232b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "41620e77-941a-499d-9e80-67ae7c81232b" (UID: "41620e77-941a-499d-9e80-67ae7c81232b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:20:42.525712 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:20:42.525697    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41620e77-941a-499d-9e80-67ae7c81232b-model-cache" (OuterVolumeSpecName: "model-cache") pod "41620e77-941a-499d-9e80-67ae7c81232b" (UID: "41620e77-941a-499d-9e80-67ae7c81232b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:20:42.525822 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:20:42.525804    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/41620e77-941a-499d-9e80-67ae7c81232b-home\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 11:20:42.527853 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:20:42.527825    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41620e77-941a-499d-9e80-67ae7c81232b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "41620e77-941a-499d-9e80-67ae7c81232b" (UID: "41620e77-941a-499d-9e80-67ae7c81232b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 11:20:42.527989 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:20:42.527960    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41620e77-941a-499d-9e80-67ae7c81232b-dshm" (OuterVolumeSpecName: "dshm") pod "41620e77-941a-499d-9e80-67ae7c81232b" (UID: "41620e77-941a-499d-9e80-67ae7c81232b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:20:42.527989 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:20:42.527985    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41620e77-941a-499d-9e80-67ae7c81232b-kube-api-access-z77tz" (OuterVolumeSpecName: "kube-api-access-z77tz") pod "41620e77-941a-499d-9e80-67ae7c81232b" (UID: "41620e77-941a-499d-9e80-67ae7c81232b"). InnerVolumeSpecName "kube-api-access-z77tz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 11:20:42.626269 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:20:42.626232    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/41620e77-941a-499d-9e80-67ae7c81232b-tls-certs\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 11:20:42.626269 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:20:42.626262    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/41620e77-941a-499d-9e80-67ae7c81232b-dshm\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 11:20:42.626269 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:20:42.626272    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41620e77-941a-499d-9e80-67ae7c81232b-tmp-dir\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 11:20:42.626269 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:20:42.626279    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/41620e77-941a-499d-9e80-67ae7c81232b-model-cache\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 11:20:42.626572 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:20:42.626288    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z77tz\" (UniqueName: \"kubernetes.io/projected/41620e77-941a-499d-9e80-67ae7c81232b-kube-api-access-z77tz\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 11:20:43.358478 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:20:43.358448    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"
Jul 28 11:20:43.359027 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:20:43.358481    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0" event={"ID":"41620e77-941a-499d-9e80-67ae7c81232b","Type":"ContainerDied","Data":"4f54041e687c85662d4f47dc333963d6380e388a76fd81f0ce32eb82233b6942"}
Jul 28 11:20:43.359027 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:20:43.358538    2563 scope.go:117] "RemoveContainer" containerID="a4fd36dc7eb5286d337b9954ee3ad239e822b832dd799791136475f29e8ce185"
Jul 28 11:20:43.375374 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:20:43.375323    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"]
Jul 28 11:20:43.378849 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:20:43.378816    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-autoscaling-hpa-lws-d4cbcfd2/autoscale-hpa-lws-kserve-mn-0"]
Jul 28 11:20:45.137541 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:20:45.137502    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41620e77-941a-499d-9e80-67ae7c81232b" path="/var/lib/kubelet/pods/41620e77-941a-499d-9e80-67ae7c81232b/volumes"
Jul 28 11:21:05.830122 ip-10-0-130-235 kubenswrapper[2563]: E0728 11:21:05.830010    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kserve-pvc-source], unattached volumes=[kserve-pvc-source], failed to process volumes=[]: context deadline exceeded" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0" podUID="8cc1f64c-273d-47e1-add7-cfef2f51d3b0"
Jul 28 11:21:06.484439 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:21:06.484404    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0"
Jul 28 11:21:43.501822 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:21:43.501780    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"]
Jul 28 11:21:43.502298 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:21:43.502201    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41620e77-941a-499d-9e80-67ae7c81232b" containerName="main"
Jul 28 11:21:43.502298 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:21:43.502214    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="41620e77-941a-499d-9e80-67ae7c81232b" containerName="main"
Jul 28 11:21:43.502298 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:21:43.502286    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="41620e77-941a-499d-9e80-67ae7c81232b" containerName="main"
Jul 28 11:21:43.507126 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:21:43.507096    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 28 11:21:43.510285 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:21:43.510253    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-autoscaling-keda-lws-e541a132\"/\"openshift-service-ca.crt\""
Jul 28 11:21:43.510285 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:21:43.510279    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-autoscaling-keda-lws-e541a132\"/\"default-dockercfg-xzt9g\""
Jul 28 11:21:43.510532 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:21:43.510258    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-autoscaling-keda-lws-e541a132\"/\"kube-root-ca.crt\""
Jul 28 11:21:43.510532 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:21:43.510261    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-autoscaling-keda-lws-e541a132\"/\"autoscale-keda-lws-kserve-mn-dockercfg-qgccn\""
Jul 28 11:21:43.510532 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:21:43.510328    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-autoscaling-keda-lws-e541a132\"/\"autoscale-keda-lws-kserve-self-signed-certs\""
Jul 28 11:21:43.518217 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:21:43.518182    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"]
Jul 28 11:21:43.590435 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:21:43.590390    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1757716-58a8-4f25-91d0-cd53e879cdba-model-cache\") pod \"autoscale-keda-lws-kserve-mn-0\" (UID: \"d1757716-58a8-4f25-91d0-cd53e879cdba\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 28 11:21:43.590619 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:21:43.590450    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1757716-58a8-4f25-91d0-cd53e879cdba-tls-certs\") pod \"autoscale-keda-lws-kserve-mn-0\" (UID: \"d1757716-58a8-4f25-91d0-cd53e879cdba\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 28 11:21:43.590619 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:21:43.590544    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1757716-58a8-4f25-91d0-cd53e879cdba-tmp-dir\") pod \"autoscale-keda-lws-kserve-mn-0\" (UID: \"d1757716-58a8-4f25-91d0-cd53e879cdba\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 28 11:21:43.590619 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:21:43.590592    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1757716-58a8-4f25-91d0-cd53e879cdba-home\") pod \"autoscale-keda-lws-kserve-mn-0\" (UID: \"d1757716-58a8-4f25-91d0-cd53e879cdba\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 28 11:21:43.590745 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:21:43.590712    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1757716-58a8-4f25-91d0-cd53e879cdba-dshm\") pod \"autoscale-keda-lws-kserve-mn-0\" (UID: \"d1757716-58a8-4f25-91d0-cd53e879cdba\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 28 11:21:43.590781 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:21:43.590742    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lslmz\" (UniqueName: \"kubernetes.io/projected/d1757716-58a8-4f25-91d0-cd53e879cdba-kube-api-access-lslmz\") pod \"autoscale-keda-lws-kserve-mn-0\" (UID: \"d1757716-58a8-4f25-91d0-cd53e879cdba\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 28 11:21:43.691166 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:21:43.691132    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1757716-58a8-4f25-91d0-cd53e879cdba-model-cache\") pod \"autoscale-keda-lws-kserve-mn-0\" (UID: \"d1757716-58a8-4f25-91d0-cd53e879cdba\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 28 11:21:43.691166 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:21:43.691173    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1757716-58a8-4f25-91d0-cd53e879cdba-tls-certs\") pod \"autoscale-keda-lws-kserve-mn-0\" (UID: \"d1757716-58a8-4f25-91d0-cd53e879cdba\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 28 11:21:43.691426 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:21:43.691212    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1757716-58a8-4f25-91d0-cd53e879cdba-tmp-dir\") pod \"autoscale-keda-lws-kserve-mn-0\" (UID: \"d1757716-58a8-4f25-91d0-cd53e879cdba\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 28 11:21:43.691426 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:21:43.691240    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1757716-58a8-4f25-91d0-cd53e879cdba-home\") pod \"autoscale-keda-lws-kserve-mn-0\" (UID: \"d1757716-58a8-4f25-91d0-cd53e879cdba\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 28 11:21:43.691426 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:21:43.691289    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1757716-58a8-4f25-91d0-cd53e879cdba-dshm\") pod \"autoscale-keda-lws-kserve-mn-0\" (UID: \"d1757716-58a8-4f25-91d0-cd53e879cdba\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 28 11:21:43.691426 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:21:43.691305    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lslmz\" (UniqueName: \"kubernetes.io/projected/d1757716-58a8-4f25-91d0-cd53e879cdba-kube-api-access-lslmz\") pod \"autoscale-keda-lws-kserve-mn-0\" (UID: \"d1757716-58a8-4f25-91d0-cd53e879cdba\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 28 11:21:43.691642 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:21:43.691617    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1757716-58a8-4f25-91d0-cd53e879cdba-tmp-dir\") pod \"autoscale-keda-lws-kserve-mn-0\" (UID: \"d1757716-58a8-4f25-91d0-cd53e879cdba\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 28 11:21:43.691692 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:21:43.691634    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1757716-58a8-4f25-91d0-cd53e879cdba-model-cache\") pod \"autoscale-keda-lws-kserve-mn-0\" (UID: \"d1757716-58a8-4f25-91d0-cd53e879cdba\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 28 11:21:43.691692 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:21:43.691673    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1757716-58a8-4f25-91d0-cd53e879cdba-home\") pod \"autoscale-keda-lws-kserve-mn-0\" (UID: \"d1757716-58a8-4f25-91d0-cd53e879cdba\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 28 11:21:43.693747 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:21:43.693723    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1757716-58a8-4f25-91d0-cd53e879cdba-dshm\") pod \"autoscale-keda-lws-kserve-mn-0\" (UID: \"d1757716-58a8-4f25-91d0-cd53e879cdba\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 28 11:21:43.694100 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:21:43.694068    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1757716-58a8-4f25-91d0-cd53e879cdba-tls-certs\") pod \"autoscale-keda-lws-kserve-mn-0\" (UID: \"d1757716-58a8-4f25-91d0-cd53e879cdba\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 28 11:21:43.699298 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:21:43.699271    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lslmz\" (UniqueName: \"kubernetes.io/projected/d1757716-58a8-4f25-91d0-cd53e879cdba-kube-api-access-lslmz\") pod \"autoscale-keda-lws-kserve-mn-0\" (UID: \"d1757716-58a8-4f25-91d0-cd53e879cdba\") " pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 28 11:21:43.820264 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:21:43.820229    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 28 11:21:43.968124 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:21:43.968091    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"]
Jul 28 11:21:43.970777 ip-10-0-130-235 kubenswrapper[2563]: W0728 11:21:43.970747    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1757716_58a8_4f25_91d0_cd53e879cdba.slice/crio-0717f6f7bc301238af03600be8d5794cca9a480b464128ffddb722f72252e25a WatchSource:0}: Error finding container 0717f6f7bc301238af03600be8d5794cca9a480b464128ffddb722f72252e25a: Status 404 returned error can't find the container with id 0717f6f7bc301238af03600be8d5794cca9a480b464128ffddb722f72252e25a
Jul 28 11:21:44.675786 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:21:44.675745    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0" event={"ID":"d1757716-58a8-4f25-91d0-cd53e879cdba","Type":"ContainerStarted","Data":"b247da1e0f4987fd26018791eebf22e772e5dbc359990e0f40a030627fd94425"}
Jul 28 11:21:44.676293 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:21:44.675799    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0" event={"ID":"d1757716-58a8-4f25-91d0-cd53e879cdba","Type":"ContainerStarted","Data":"0717f6f7bc301238af03600be8d5794cca9a480b464128ffddb722f72252e25a"}
Jul 28 11:21:44.697805 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:21:44.697727    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0" podStartSLOduration=1.697699512 podStartE2EDuration="1.697699512s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:21:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:21:44.695429073 +0000 UTC m=+4448.136970423" watchObservedRunningTime="2026-07-28 11:21:44.697699512 +0000 UTC m=+4448.139240850"
Jul 28 11:21:44.821278 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:21:44.821234    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 28 11:21:53.821074 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:21:53.821039    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 28 11:21:53.836574 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:21:53.836545    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 28 11:21:54.737281 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:21:54.737244    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 28 11:22:37.464967 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:22:37.464842    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 11:22:37.480776 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:22:37.480746    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 11:23:09.485151 ip-10-0-130-235 kubenswrapper[2563]: E0728 11:23:09.485098    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kserve-pvc-source], unattached volumes=[kserve-pvc-source], failed to process volumes=[]: context deadline exceeded" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0" podUID="8cc1f64c-273d-47e1-add7-cfef2f51d3b0"
Jul 28 11:23:10.107032 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:23:10.106997    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0"
Jul 28 11:25:13.108022 ip-10-0-130-235 kubenswrapper[2563]: E0728 11:25:13.107968    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kserve-pvc-source], unattached volumes=[kserve-pvc-source], failed to process volumes=[]: context deadline exceeded" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0" podUID="8cc1f64c-273d-47e1-add7-cfef2f51d3b0"
Jul 28 11:25:13.744621 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:25:13.744587    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0"
Jul 28 11:27:16.746070 ip-10-0-130-235 kubenswrapper[2563]: E0728 11:27:16.745978    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kserve-pvc-source], unattached volumes=[kserve-pvc-source], failed to process volumes=[]: context deadline exceeded" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0" podUID="8cc1f64c-273d-47e1-add7-cfef2f51d3b0"
Jul 28 11:27:17.380290 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:27:17.380258    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0"
Jul 28 11:27:37.496117 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:27:37.496006    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 11:27:37.513274 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:27:37.513246    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 11:29:20.381505 ip-10-0-130-235 kubenswrapper[2563]: E0728 11:29:20.381454    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kserve-pvc-source], unattached volumes=[kserve-pvc-source], failed to process volumes=[]: context deadline exceeded" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0" podUID="8cc1f64c-273d-47e1-add7-cfef2f51d3b0"
Jul 28 11:29:21.006020 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:29:21.005986    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0"
Jul 28 11:31:24.007115 ip-10-0-130-235 kubenswrapper[2563]: E0728 11:31:24.007066    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kserve-pvc-source], unattached volumes=[kserve-pvc-source], failed to process volumes=[]: context deadline exceeded" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0" podUID="8cc1f64c-273d-47e1-add7-cfef2f51d3b0"
Jul 28 11:31:24.658870 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:31:24.658837    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0"
Jul 28 11:32:37.529252 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:32:37.529141    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 11:32:37.550343 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:32:37.550314    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 11:33:27.660238 ip-10-0-130-235 kubenswrapper[2563]: E0728 11:33:27.660191    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kserve-pvc-source], unattached volumes=[kserve-pvc-source], failed to process volumes=[]: context deadline exceeded" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0" podUID="8cc1f64c-273d-47e1-add7-cfef2f51d3b0"
Jul 28 11:33:28.307492 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:33:28.307461    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0"
Jul 28 11:35:31.308070 ip-10-0-130-235 kubenswrapper[2563]: E0728 11:35:31.308021    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kserve-pvc-source], unattached volumes=[kserve-pvc-source], failed to process volumes=[]: context deadline exceeded" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0" podUID="8cc1f64c-273d-47e1-add7-cfef2f51d3b0"
Jul 28 11:35:31.940212 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:35:31.940177    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0"
Jul 28 11:36:36.826978 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:36:36.826937    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"]
Jul 28 11:36:36.837335 ip-10-0-130-235 kubenswrapper[2563]: E0728 11:36:36.837282    2563 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 28 11:36:36.837335 ip-10-0-130-235 kubenswrapper[2563]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 28 11:36:36.837335 ip-10-0-130-235 kubenswrapper[2563]:         , stdout: , stderr: , exit code -1
Jul 28 11:36:36.837335 ip-10-0-130-235 kubenswrapper[2563]:  > containerID="b247da1e0f4987fd26018791eebf22e772e5dbc359990e0f40a030627fd94425" cmd=["/bin/sleep","15"]
Jul 28 11:36:36.837721 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:36:36.837349    2563 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0" message="" err=<
Jul 28 11:36:36.837721 ip-10-0-130-235 kubenswrapper[2563]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 28 11:36:36.837721 ip-10-0-130-235 kubenswrapper[2563]:         , stdout: , stderr: , exit code -1
Jul 28 11:36:36.837721 ip-10-0-130-235 kubenswrapper[2563]:  >
Jul 28 11:36:36.837721 ip-10-0-130-235 kubenswrapper[2563]: E0728 11:36:36.837379    2563 kuberuntime_container.go:755] "PreStop hook failed" err=<
Jul 28 11:36:36.837721 ip-10-0-130-235 kubenswrapper[2563]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 28 11:36:36.837721 ip-10-0-130-235 kubenswrapper[2563]:         , stdout: , stderr: , exit code -1
Jul 28 11:36:36.837721 ip-10-0-130-235 kubenswrapper[2563]:  > pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0" podUID="d1757716-58a8-4f25-91d0-cd53e879cdba" containerName="main" containerID="cri-o://b247da1e0f4987fd26018791eebf22e772e5dbc359990e0f40a030627fd94425"
Jul 28 11:36:36.837721 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:36:36.837448    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0" podUID="d1757716-58a8-4f25-91d0-cd53e879cdba" containerName="main" containerID="cri-o://b247da1e0f4987fd26018791eebf22e772e5dbc359990e0f40a030627fd94425" gracePeriod=60
Jul 28 11:36:37.201281 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:36:37.201246    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 28 11:36:37.263589 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:36:37.263544    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1757716-58a8-4f25-91d0-cd53e879cdba-model-cache\") pod \"d1757716-58a8-4f25-91d0-cd53e879cdba\" (UID: \"d1757716-58a8-4f25-91d0-cd53e879cdba\") "
Jul 28 11:36:37.263589 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:36:37.263592    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1757716-58a8-4f25-91d0-cd53e879cdba-tls-certs\") pod \"d1757716-58a8-4f25-91d0-cd53e879cdba\" (UID: \"d1757716-58a8-4f25-91d0-cd53e879cdba\") "
Jul 28 11:36:37.263867 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:36:37.263622    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1757716-58a8-4f25-91d0-cd53e879cdba-dshm\") pod \"d1757716-58a8-4f25-91d0-cd53e879cdba\" (UID: \"d1757716-58a8-4f25-91d0-cd53e879cdba\") "
Jul 28 11:36:37.263867 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:36:37.263841    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1757716-58a8-4f25-91d0-cd53e879cdba-model-cache" (OuterVolumeSpecName: "model-cache") pod "d1757716-58a8-4f25-91d0-cd53e879cdba" (UID: "d1757716-58a8-4f25-91d0-cd53e879cdba"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:36:37.263867 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:36:37.263860    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1757716-58a8-4f25-91d0-cd53e879cdba-home\") pod \"d1757716-58a8-4f25-91d0-cd53e879cdba\" (UID: \"d1757716-58a8-4f25-91d0-cd53e879cdba\") "
Jul 28 11:36:37.264040 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:36:37.263893    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lslmz\" (UniqueName: \"kubernetes.io/projected/d1757716-58a8-4f25-91d0-cd53e879cdba-kube-api-access-lslmz\") pod \"d1757716-58a8-4f25-91d0-cd53e879cdba\" (UID: \"d1757716-58a8-4f25-91d0-cd53e879cdba\") "
Jul 28 11:36:37.264040 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:36:37.263928    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1757716-58a8-4f25-91d0-cd53e879cdba-tmp-dir\") pod \"d1757716-58a8-4f25-91d0-cd53e879cdba\" (UID: \"d1757716-58a8-4f25-91d0-cd53e879cdba\") "
Jul 28 11:36:37.264149 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:36:37.264116    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1757716-58a8-4f25-91d0-cd53e879cdba-home" (OuterVolumeSpecName: "home") pod "d1757716-58a8-4f25-91d0-cd53e879cdba" (UID: "d1757716-58a8-4f25-91d0-cd53e879cdba"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:36:37.264202 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:36:37.264189    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1757716-58a8-4f25-91d0-cd53e879cdba-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d1757716-58a8-4f25-91d0-cd53e879cdba" (UID: "d1757716-58a8-4f25-91d0-cd53e879cdba"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:36:37.264398 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:36:37.264321    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1757716-58a8-4f25-91d0-cd53e879cdba-tmp-dir\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 11:36:37.264398 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:36:37.264347    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d1757716-58a8-4f25-91d0-cd53e879cdba-model-cache\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 11:36:37.264398 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:36:37.264377    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d1757716-58a8-4f25-91d0-cd53e879cdba-home\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 11:36:37.266220 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:36:37.266195    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1757716-58a8-4f25-91d0-cd53e879cdba-dshm" (OuterVolumeSpecName: "dshm") pod "d1757716-58a8-4f25-91d0-cd53e879cdba" (UID: "d1757716-58a8-4f25-91d0-cd53e879cdba"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:36:37.266220 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:36:37.266204    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1757716-58a8-4f25-91d0-cd53e879cdba-kube-api-access-lslmz" (OuterVolumeSpecName: "kube-api-access-lslmz") pod "d1757716-58a8-4f25-91d0-cd53e879cdba" (UID: "d1757716-58a8-4f25-91d0-cd53e879cdba"). InnerVolumeSpecName "kube-api-access-lslmz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 11:36:37.266408 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:36:37.266195    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d1757716-58a8-4f25-91d0-cd53e879cdba-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d1757716-58a8-4f25-91d0-cd53e879cdba" (UID: "d1757716-58a8-4f25-91d0-cd53e879cdba"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 11:36:37.273753 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:36:37.273718    2563 generic.go:356] "Generic (PLEG): container finished" podID="d1757716-58a8-4f25-91d0-cd53e879cdba" containerID="b247da1e0f4987fd26018791eebf22e772e5dbc359990e0f40a030627fd94425" exitCode=0
Jul 28 11:36:37.273907 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:36:37.273765    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"
Jul 28 11:36:37.273907 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:36:37.273800    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0" event={"ID":"d1757716-58a8-4f25-91d0-cd53e879cdba","Type":"ContainerDied","Data":"b247da1e0f4987fd26018791eebf22e772e5dbc359990e0f40a030627fd94425"}
Jul 28 11:36:37.273907 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:36:37.273873    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0" event={"ID":"d1757716-58a8-4f25-91d0-cd53e879cdba","Type":"ContainerDied","Data":"0717f6f7bc301238af03600be8d5794cca9a480b464128ffddb722f72252e25a"}
Jul 28 11:36:37.273907 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:36:37.273905    2563 scope.go:117] "RemoveContainer" containerID="b247da1e0f4987fd26018791eebf22e772e5dbc359990e0f40a030627fd94425"
Jul 28 11:36:37.290490 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:36:37.290471    2563 scope.go:117] "RemoveContainer" containerID="b247da1e0f4987fd26018791eebf22e772e5dbc359990e0f40a030627fd94425"
Jul 28 11:36:37.290810 ip-10-0-130-235 kubenswrapper[2563]: E0728 11:36:37.290790    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b247da1e0f4987fd26018791eebf22e772e5dbc359990e0f40a030627fd94425\": container with ID starting with b247da1e0f4987fd26018791eebf22e772e5dbc359990e0f40a030627fd94425 not found: ID does not exist" containerID="b247da1e0f4987fd26018791eebf22e772e5dbc359990e0f40a030627fd94425"
Jul 28 11:36:37.290852 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:36:37.290821    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b247da1e0f4987fd26018791eebf22e772e5dbc359990e0f40a030627fd94425"} err="failed to get container status \"b247da1e0f4987fd26018791eebf22e772e5dbc359990e0f40a030627fd94425\": rpc error: code = NotFound desc = could not find container \"b247da1e0f4987fd26018791eebf22e772e5dbc359990e0f40a030627fd94425\": container with ID starting with b247da1e0f4987fd26018791eebf22e772e5dbc359990e0f40a030627fd94425 not found: ID does not exist"
Jul 28 11:36:37.301865 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:36:37.301725    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"]
Jul 28 11:36:37.305506 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:36:37.305475    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-autoscaling-keda-lws-e541a132/autoscale-keda-lws-kserve-mn-0"]
Jul 28 11:36:37.365861 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:36:37.365763    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lslmz\" (UniqueName: \"kubernetes.io/projected/d1757716-58a8-4f25-91d0-cd53e879cdba-kube-api-access-lslmz\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 11:36:37.365861 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:36:37.365795    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1757716-58a8-4f25-91d0-cd53e879cdba-tls-certs\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 11:36:37.365861 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:36:37.365816    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d1757716-58a8-4f25-91d0-cd53e879cdba-dshm\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 11:36:39.131654 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:36:39.131615    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1757716-58a8-4f25-91d0-cd53e879cdba" path="/var/lib/kubelet/pods/d1757716-58a8-4f25-91d0-cd53e879cdba/volumes"
Jul 28 11:37:28.288990 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:37:28.288892    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h"]
Jul 28 11:37:28.289574 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:37:28.289522    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1757716-58a8-4f25-91d0-cd53e879cdba" containerName="main"
Jul 28 11:37:28.289574 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:37:28.289542    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1757716-58a8-4f25-91d0-cd53e879cdba" containerName="main"
Jul 28 11:37:28.289735 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:37:28.289721    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1757716-58a8-4f25-91d0-cd53e879cdba" containerName="main"
Jul 28 11:37:28.292794 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:37:28.292770    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h"
Jul 28 11:37:28.295679 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:37:28.295651    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40\"/\"openshift-service-ca.crt\""
Jul 28 11:37:28.295818 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:37:28.295659    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40\"/\"default-dockercfg-n45mf\""
Jul 28 11:37:28.295818 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:37:28.295661    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40\"/\"autoscale-cleanup-hpa-kserve-self-signed-certs\""
Jul 28 11:37:28.295818 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:37:28.295759    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40\"/\"kube-root-ca.crt\""
Jul 28 11:37:28.305835 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:37:28.305806    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h"]
Jul 28 11:37:28.443133 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:37:28.443093    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0c1c6bd-043e-4553-9388-f0e0fcf102ac-model-cache\") pod \"autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h\" (UID: \"f0c1c6bd-043e-4553-9388-f0e0fcf102ac\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h"
Jul 28 11:37:28.443133 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:37:28.443131    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0c1c6bd-043e-4553-9388-f0e0fcf102ac-tls-certs\") pod \"autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h\" (UID: \"f0c1c6bd-043e-4553-9388-f0e0fcf102ac\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h"
Jul 28 11:37:28.443385 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:37:28.443152    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0c1c6bd-043e-4553-9388-f0e0fcf102ac-home\") pod \"autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h\" (UID: \"f0c1c6bd-043e-4553-9388-f0e0fcf102ac\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h"
Jul 28 11:37:28.443385 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:37:28.443222    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kj7wm\" (UniqueName: \"kubernetes.io/projected/f0c1c6bd-043e-4553-9388-f0e0fcf102ac-kube-api-access-kj7wm\") pod \"autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h\" (UID: \"f0c1c6bd-043e-4553-9388-f0e0fcf102ac\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h"
Jul 28 11:37:28.443385 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:37:28.443268    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0c1c6bd-043e-4553-9388-f0e0fcf102ac-dshm\") pod \"autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h\" (UID: \"f0c1c6bd-043e-4553-9388-f0e0fcf102ac\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h"
Jul 28 11:37:28.443385 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:37:28.443285    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0c1c6bd-043e-4553-9388-f0e0fcf102ac-tmp-dir\") pod \"autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h\" (UID: \"f0c1c6bd-043e-4553-9388-f0e0fcf102ac\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h"
Jul 28 11:37:28.544649 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:37:28.544551    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0c1c6bd-043e-4553-9388-f0e0fcf102ac-model-cache\") pod \"autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h\" (UID: \"f0c1c6bd-043e-4553-9388-f0e0fcf102ac\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h"
Jul 28 11:37:28.544649 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:37:28.544589    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0c1c6bd-043e-4553-9388-f0e0fcf102ac-tls-certs\") pod \"autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h\" (UID: \"f0c1c6bd-043e-4553-9388-f0e0fcf102ac\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h"
Jul 28 11:37:28.544649 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:37:28.544614    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0c1c6bd-043e-4553-9388-f0e0fcf102ac-home\") pod \"autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h\" (UID: \"f0c1c6bd-043e-4553-9388-f0e0fcf102ac\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h"
Jul 28 11:37:28.544649 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:37:28.544644    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kj7wm\" (UniqueName: \"kubernetes.io/projected/f0c1c6bd-043e-4553-9388-f0e0fcf102ac-kube-api-access-kj7wm\") pod \"autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h\" (UID: \"f0c1c6bd-043e-4553-9388-f0e0fcf102ac\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h"
Jul 28 11:37:28.544997 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:37:28.544672    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0c1c6bd-043e-4553-9388-f0e0fcf102ac-dshm\") pod \"autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h\" (UID: \"f0c1c6bd-043e-4553-9388-f0e0fcf102ac\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h"
Jul 28 11:37:28.544997 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:37:28.544689    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0c1c6bd-043e-4553-9388-f0e0fcf102ac-tmp-dir\") pod \"autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h\" (UID: \"f0c1c6bd-043e-4553-9388-f0e0fcf102ac\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h"
Jul 28 11:37:28.545097 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:37:28.545019    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0c1c6bd-043e-4553-9388-f0e0fcf102ac-model-cache\") pod \"autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h\" (UID: \"f0c1c6bd-043e-4553-9388-f0e0fcf102ac\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h"
Jul 28 11:37:28.545097 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:37:28.545033    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0c1c6bd-043e-4553-9388-f0e0fcf102ac-home\") pod \"autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h\" (UID: \"f0c1c6bd-043e-4553-9388-f0e0fcf102ac\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h"
Jul 28 11:37:28.545214 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:37:28.545131    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0c1c6bd-043e-4553-9388-f0e0fcf102ac-tmp-dir\") pod \"autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h\" (UID: \"f0c1c6bd-043e-4553-9388-f0e0fcf102ac\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h"
Jul 28 11:37:28.547150 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:37:28.547127    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0c1c6bd-043e-4553-9388-f0e0fcf102ac-dshm\") pod \"autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h\" (UID: \"f0c1c6bd-043e-4553-9388-f0e0fcf102ac\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h"
Jul 28 11:37:28.547312 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:37:28.547294    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0c1c6bd-043e-4553-9388-f0e0fcf102ac-tls-certs\") pod \"autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h\" (UID: \"f0c1c6bd-043e-4553-9388-f0e0fcf102ac\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h"
Jul 28 11:37:28.553342 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:37:28.553318    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kj7wm\" (UniqueName: \"kubernetes.io/projected/f0c1c6bd-043e-4553-9388-f0e0fcf102ac-kube-api-access-kj7wm\") pod \"autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h\" (UID: \"f0c1c6bd-043e-4553-9388-f0e0fcf102ac\") " pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h"
Jul 28 11:37:28.603462 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:37:28.603425    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h"
Jul 28 11:37:28.762067 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:37:28.762034    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h"]
Jul 28 11:37:28.764608 ip-10-0-130-235 kubenswrapper[2563]: W0728 11:37:28.764583    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0c1c6bd_043e_4553_9388_f0e0fcf102ac.slice/crio-a4367ac03dc4f9bfe88a6b9aba35e2fc64d6066703e3be67e8fbdd34209b3139 WatchSource:0}: Error finding container a4367ac03dc4f9bfe88a6b9aba35e2fc64d6066703e3be67e8fbdd34209b3139: Status 404 returned error can't find the container with id a4367ac03dc4f9bfe88a6b9aba35e2fc64d6066703e3be67e8fbdd34209b3139
Jul 28 11:37:29.532153 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:37:29.532119    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h" event={"ID":"f0c1c6bd-043e-4553-9388-f0e0fcf102ac","Type":"ContainerStarted","Data":"d1573d63ccd475f2966f896589b0420ffc86b1f7a0e6785a28076d0538d0b796"}
Jul 28 11:37:29.532637 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:37:29.532170    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h" event={"ID":"f0c1c6bd-043e-4553-9388-f0e0fcf102ac","Type":"ContainerStarted","Data":"a4367ac03dc4f9bfe88a6b9aba35e2fc64d6066703e3be67e8fbdd34209b3139"}
Jul 28 11:37:29.551803 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:37:29.551741    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h" podStartSLOduration=1.55172534 podStartE2EDuration="1.55172534s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:37:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:37:29.549559592 +0000 UTC m=+5392.991100929" watchObservedRunningTime="2026-07-28 11:37:29.55172534 +0000 UTC m=+5392.993266677"
Jul 28 11:37:29.604208 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:37:29.604155    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h"
Jul 28 11:37:34.941132 ip-10-0-130-235 kubenswrapper[2563]: E0728 11:37:34.941089    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kserve-pvc-source], unattached volumes=[kserve-pvc-source], failed to process volumes=[]: context deadline exceeded" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0" podUID="8cc1f64c-273d-47e1-add7-cfef2f51d3b0"
Jul 28 11:37:35.562559 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:37:35.562529    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0"
Jul 28 11:37:37.560381 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:37:37.560263    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 11:37:37.584153 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:37:37.584128    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 11:37:38.603839 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:37:38.603808    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h"
Jul 28 11:37:38.616613 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:37:38.616587    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h"
Jul 28 11:37:39.593506 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:37:39.593478    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h"
Jul 28 11:39:38.563493 ip-10-0-130-235 kubenswrapper[2563]: E0728 11:39:38.563433    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kserve-pvc-source], unattached volumes=[kserve-pvc-source], failed to process volumes=[]: context deadline exceeded" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0" podUID="8cc1f64c-273d-47e1-add7-cfef2f51d3b0"
Jul 28 11:39:39.185455 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:39:39.185408    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0"
Jul 28 11:41:42.186647 ip-10-0-130-235 kubenswrapper[2563]: E0728 11:41:42.186594    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kserve-pvc-source], unattached volumes=[kserve-pvc-source], failed to process volumes=[]: context deadline exceeded" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0" podUID="8cc1f64c-273d-47e1-add7-cfef2f51d3b0"
Jul 28 11:41:42.816422 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:41:42.816388    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0"
Jul 28 11:42:37.593762 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:42:37.593731    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 11:42:37.624125 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:42:37.624095    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 11:43:45.817344 ip-10-0-130-235 kubenswrapper[2563]: E0728 11:43:45.817297    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kserve-pvc-source], unattached volumes=[kserve-pvc-source], failed to process volumes=[]: context deadline exceeded" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0" podUID="8cc1f64c-273d-47e1-add7-cfef2f51d3b0"
Jul 28 11:43:46.444184 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:43:46.444153    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0"
Jul 28 11:45:49.444890 ip-10-0-130-235 kubenswrapper[2563]: E0728 11:45:49.444839    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kserve-pvc-source], unattached volumes=[kserve-pvc-source], failed to process volumes=[]: context deadline exceeded" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0" podUID="8cc1f64c-273d-47e1-add7-cfef2f51d3b0"
Jul 28 11:45:50.071019 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:45:50.070983    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0"
Jul 28 11:47:37.634226 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:47:37.634111    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 11:47:37.656499 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:47:37.656471    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 11:47:53.071644 ip-10-0-130-235 kubenswrapper[2563]: E0728 11:47:53.071594    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kserve-pvc-source], unattached volumes=[kserve-pvc-source], failed to process volumes=[]: context deadline exceeded" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0" podUID="8cc1f64c-273d-47e1-add7-cfef2f51d3b0"
Jul 28 11:47:53.701990 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:47:53.701955    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0"
Jul 28 11:48:50.388332 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:48:50.388293    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-kj4vk"]
Jul 28 11:48:50.391986 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:48:50.391964    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-kj4vk"
Jul 28 11:48:50.394932 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:48:50.394908    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"rolling-upgrade-test-kserve-self-signed-certs\""
Jul 28 11:48:50.394932 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:48:50.394919    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"openshift-service-ca.crt\""
Jul 28 11:48:50.395132 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:48:50.394907    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"default-dockercfg-2tsmf\""
Jul 28 11:48:50.395280 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:48:50.395265    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"kube-root-ca.crt\""
Jul 28 11:48:50.403611 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:48:50.403586    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-kj4vk"]
Jul 28 11:48:50.510602 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:48:50.510559    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ebe4723-a9ed-42e8-a69e-3aa14ed707bc-dshm\") pod \"rolling-upgrade-test-kserve-74b79d7f77-kj4vk\" (UID: \"0ebe4723-a9ed-42e8-a69e-3aa14ed707bc\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-kj4vk"
Jul 28 11:48:50.510803 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:48:50.510693    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ebe4723-a9ed-42e8-a69e-3aa14ed707bc-model-cache\") pod \"rolling-upgrade-test-kserve-74b79d7f77-kj4vk\" (UID: \"0ebe4723-a9ed-42e8-a69e-3aa14ed707bc\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-kj4vk"
Jul 28 11:48:50.510803 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:48:50.510722    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ebe4723-a9ed-42e8-a69e-3aa14ed707bc-tls-certs\") pod \"rolling-upgrade-test-kserve-74b79d7f77-kj4vk\" (UID: \"0ebe4723-a9ed-42e8-a69e-3aa14ed707bc\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-kj4vk"
Jul 28 11:48:50.510803 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:48:50.510772    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqk2q\" (UniqueName: \"kubernetes.io/projected/0ebe4723-a9ed-42e8-a69e-3aa14ed707bc-kube-api-access-fqk2q\") pod \"rolling-upgrade-test-kserve-74b79d7f77-kj4vk\" (UID: \"0ebe4723-a9ed-42e8-a69e-3aa14ed707bc\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-kj4vk"
Jul 28 11:48:50.510803 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:48:50.510794    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ebe4723-a9ed-42e8-a69e-3aa14ed707bc-home\") pod \"rolling-upgrade-test-kserve-74b79d7f77-kj4vk\" (UID: \"0ebe4723-a9ed-42e8-a69e-3aa14ed707bc\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-kj4vk"
Jul 28 11:48:50.510981 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:48:50.510866    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ebe4723-a9ed-42e8-a69e-3aa14ed707bc-tmp-dir\") pod \"rolling-upgrade-test-kserve-74b79d7f77-kj4vk\" (UID: \"0ebe4723-a9ed-42e8-a69e-3aa14ed707bc\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-kj4vk"
Jul 28 11:48:50.611899 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:48:50.611865    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ebe4723-a9ed-42e8-a69e-3aa14ed707bc-dshm\") pod \"rolling-upgrade-test-kserve-74b79d7f77-kj4vk\" (UID: \"0ebe4723-a9ed-42e8-a69e-3aa14ed707bc\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-kj4vk"
Jul 28 11:48:50.612091 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:48:50.611946    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ebe4723-a9ed-42e8-a69e-3aa14ed707bc-model-cache\") pod \"rolling-upgrade-test-kserve-74b79d7f77-kj4vk\" (UID: \"0ebe4723-a9ed-42e8-a69e-3aa14ed707bc\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-kj4vk"
Jul 28 11:48:50.612091 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:48:50.611967    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ebe4723-a9ed-42e8-a69e-3aa14ed707bc-tls-certs\") pod \"rolling-upgrade-test-kserve-74b79d7f77-kj4vk\" (UID: \"0ebe4723-a9ed-42e8-a69e-3aa14ed707bc\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-kj4vk"
Jul 28 11:48:50.612091 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:48:50.612006    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqk2q\" (UniqueName: \"kubernetes.io/projected/0ebe4723-a9ed-42e8-a69e-3aa14ed707bc-kube-api-access-fqk2q\") pod \"rolling-upgrade-test-kserve-74b79d7f77-kj4vk\" (UID: \"0ebe4723-a9ed-42e8-a69e-3aa14ed707bc\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-kj4vk"
Jul 28 11:48:50.612091 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:48:50.612023    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ebe4723-a9ed-42e8-a69e-3aa14ed707bc-home\") pod \"rolling-upgrade-test-kserve-74b79d7f77-kj4vk\" (UID: \"0ebe4723-a9ed-42e8-a69e-3aa14ed707bc\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-kj4vk"
Jul 28 11:48:50.612091 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:48:50.612060    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ebe4723-a9ed-42e8-a69e-3aa14ed707bc-tmp-dir\") pod \"rolling-upgrade-test-kserve-74b79d7f77-kj4vk\" (UID: \"0ebe4723-a9ed-42e8-a69e-3aa14ed707bc\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-kj4vk"
Jul 28 11:48:50.612439 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:48:50.612416    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ebe4723-a9ed-42e8-a69e-3aa14ed707bc-model-cache\") pod \"rolling-upgrade-test-kserve-74b79d7f77-kj4vk\" (UID: \"0ebe4723-a9ed-42e8-a69e-3aa14ed707bc\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-kj4vk"
Jul 28 11:48:50.612531 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:48:50.612490    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ebe4723-a9ed-42e8-a69e-3aa14ed707bc-home\") pod \"rolling-upgrade-test-kserve-74b79d7f77-kj4vk\" (UID: \"0ebe4723-a9ed-42e8-a69e-3aa14ed707bc\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-kj4vk"
Jul 28 11:48:50.612590 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:48:50.612567    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ebe4723-a9ed-42e8-a69e-3aa14ed707bc-tmp-dir\") pod \"rolling-upgrade-test-kserve-74b79d7f77-kj4vk\" (UID: \"0ebe4723-a9ed-42e8-a69e-3aa14ed707bc\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-kj4vk"
Jul 28 11:48:50.614422 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:48:50.614395    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ebe4723-a9ed-42e8-a69e-3aa14ed707bc-dshm\") pod \"rolling-upgrade-test-kserve-74b79d7f77-kj4vk\" (UID: \"0ebe4723-a9ed-42e8-a69e-3aa14ed707bc\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-kj4vk"
Jul 28 11:48:50.614808 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:48:50.614787    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ebe4723-a9ed-42e8-a69e-3aa14ed707bc-tls-certs\") pod \"rolling-upgrade-test-kserve-74b79d7f77-kj4vk\" (UID: \"0ebe4723-a9ed-42e8-a69e-3aa14ed707bc\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-kj4vk"
Jul 28 11:48:50.621049 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:48:50.621022    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqk2q\" (UniqueName: \"kubernetes.io/projected/0ebe4723-a9ed-42e8-a69e-3aa14ed707bc-kube-api-access-fqk2q\") pod \"rolling-upgrade-test-kserve-74b79d7f77-kj4vk\" (UID: \"0ebe4723-a9ed-42e8-a69e-3aa14ed707bc\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-kj4vk"
Jul 28 11:48:50.703215 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:48:50.703114    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-kj4vk"
Jul 28 11:48:50.848404 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:48:50.848377    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-kj4vk"]
Jul 28 11:48:50.851386 ip-10-0-130-235 kubenswrapper[2563]: W0728 11:48:50.851334    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ebe4723_a9ed_42e8_a69e_3aa14ed707bc.slice/crio-158b900c32b1ff6cd9ec929e7bcdbc3ed52658868de233e4ec58de45184f4305 WatchSource:0}: Error finding container 158b900c32b1ff6cd9ec929e7bcdbc3ed52658868de233e4ec58de45184f4305: Status 404 returned error can't find the container with id 158b900c32b1ff6cd9ec929e7bcdbc3ed52658868de233e4ec58de45184f4305
Jul 28 11:48:50.994074 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:48:50.993966    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-kj4vk" event={"ID":"0ebe4723-a9ed-42e8-a69e-3aa14ed707bc","Type":"ContainerStarted","Data":"6a6fa2883a693112402fd0126cd598b298cc94c1b21f9014c5ef92c129173b2d"}
Jul 28 11:48:50.994074 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:48:50.994034    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-kj4vk" event={"ID":"0ebe4723-a9ed-42e8-a69e-3aa14ed707bc","Type":"ContainerStarted","Data":"158b900c32b1ff6cd9ec929e7bcdbc3ed52658868de233e4ec58de45184f4305"}
Jul 28 11:48:51.014273 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:48:51.014190    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-kj4vk" podStartSLOduration=1.014172943 podStartE2EDuration="1.014172943s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:48:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:48:51.012720468 +0000 UTC m=+6074.454261805" watchObservedRunningTime="2026-07-28 11:48:51.014172943 +0000 UTC m=+6074.455714279"
Jul 28 11:48:51.703723 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:48:51.703688    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-kj4vk"
Jul 28 11:49:00.703693 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:49:00.703662    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-kj4vk"
Jul 28 11:49:00.716299 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:49:00.716270    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-kj4vk"
Jul 28 11:49:01.060963 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:49:01.060884    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-kj4vk"
Jul 28 11:49:56.703579 ip-10-0-130-235 kubenswrapper[2563]: E0728 11:49:56.703525    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kserve-pvc-source], unattached volumes=[kserve-pvc-source], failed to process volumes=[]: context deadline exceeded" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0" podUID="8cc1f64c-273d-47e1-add7-cfef2f51d3b0"
Jul 28 11:49:57.342723 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:49:57.342687    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0"
Jul 28 11:50:14.615660 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:50:14.615620    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-kj4vk"]
Jul 28 11:50:14.626320 ip-10-0-130-235 kubenswrapper[2563]: E0728 11:50:14.626280    2563 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 28 11:50:14.626320 ip-10-0-130-235 kubenswrapper[2563]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 28 11:50:14.626320 ip-10-0-130-235 kubenswrapper[2563]:         , stdout: , stderr: , exit code -1
Jul 28 11:50:14.626320 ip-10-0-130-235 kubenswrapper[2563]:  > containerID="6a6fa2883a693112402fd0126cd598b298cc94c1b21f9014c5ef92c129173b2d" cmd=["/bin/sleep","15"]
Jul 28 11:50:14.626666 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:50:14.626342    2563 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-kj4vk" message="" err=<
Jul 28 11:50:14.626666 ip-10-0-130-235 kubenswrapper[2563]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 28 11:50:14.626666 ip-10-0-130-235 kubenswrapper[2563]:         , stdout: , stderr: , exit code -1
Jul 28 11:50:14.626666 ip-10-0-130-235 kubenswrapper[2563]:  >
Jul 28 11:50:14.626666 ip-10-0-130-235 kubenswrapper[2563]: E0728 11:50:14.626383    2563 kuberuntime_container.go:755] "PreStop hook failed" err=<
Jul 28 11:50:14.626666 ip-10-0-130-235 kubenswrapper[2563]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 28 11:50:14.626666 ip-10-0-130-235 kubenswrapper[2563]:         , stdout: , stderr: , exit code -1
Jul 28 11:50:14.626666 ip-10-0-130-235 kubenswrapper[2563]:  > pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-kj4vk" podUID="0ebe4723-a9ed-42e8-a69e-3aa14ed707bc" containerName="main" containerID="cri-o://6a6fa2883a693112402fd0126cd598b298cc94c1b21f9014c5ef92c129173b2d"
Jul 28 11:50:14.626666 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:50:14.626433    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-kj4vk" podUID="0ebe4723-a9ed-42e8-a69e-3aa14ed707bc" containerName="main" containerID="cri-o://6a6fa2883a693112402fd0126cd598b298cc94c1b21f9014c5ef92c129173b2d" gracePeriod=60
Jul 28 11:50:14.990912 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:50:14.990885    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-kj4vk"
Jul 28 11:50:15.095714 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:50:15.095675    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ebe4723-a9ed-42e8-a69e-3aa14ed707bc-model-cache\") pod \"0ebe4723-a9ed-42e8-a69e-3aa14ed707bc\" (UID: \"0ebe4723-a9ed-42e8-a69e-3aa14ed707bc\") "
Jul 28 11:50:15.095714 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:50:15.095718    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ebe4723-a9ed-42e8-a69e-3aa14ed707bc-tls-certs\") pod \"0ebe4723-a9ed-42e8-a69e-3aa14ed707bc\" (UID: \"0ebe4723-a9ed-42e8-a69e-3aa14ed707bc\") "
Jul 28 11:50:15.095979 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:50:15.095748    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqk2q\" (UniqueName: \"kubernetes.io/projected/0ebe4723-a9ed-42e8-a69e-3aa14ed707bc-kube-api-access-fqk2q\") pod \"0ebe4723-a9ed-42e8-a69e-3aa14ed707bc\" (UID: \"0ebe4723-a9ed-42e8-a69e-3aa14ed707bc\") "
Jul 28 11:50:15.095979 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:50:15.095773    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ebe4723-a9ed-42e8-a69e-3aa14ed707bc-home\") pod \"0ebe4723-a9ed-42e8-a69e-3aa14ed707bc\" (UID: \"0ebe4723-a9ed-42e8-a69e-3aa14ed707bc\") "
Jul 28 11:50:15.095979 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:50:15.095826    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ebe4723-a9ed-42e8-a69e-3aa14ed707bc-tmp-dir\") pod \"0ebe4723-a9ed-42e8-a69e-3aa14ed707bc\" (UID: \"0ebe4723-a9ed-42e8-a69e-3aa14ed707bc\") "
Jul 28 11:50:15.095979 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:50:15.095858    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ebe4723-a9ed-42e8-a69e-3aa14ed707bc-dshm\") pod \"0ebe4723-a9ed-42e8-a69e-3aa14ed707bc\" (UID: \"0ebe4723-a9ed-42e8-a69e-3aa14ed707bc\") "
Jul 28 11:50:15.096181 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:50:15.096013    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ebe4723-a9ed-42e8-a69e-3aa14ed707bc-model-cache" (OuterVolumeSpecName: "model-cache") pod "0ebe4723-a9ed-42e8-a69e-3aa14ed707bc" (UID: "0ebe4723-a9ed-42e8-a69e-3aa14ed707bc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:50:15.096181 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:50:15.096073    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ebe4723-a9ed-42e8-a69e-3aa14ed707bc-home" (OuterVolumeSpecName: "home") pod "0ebe4723-a9ed-42e8-a69e-3aa14ed707bc" (UID: "0ebe4723-a9ed-42e8-a69e-3aa14ed707bc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:50:15.096181 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:50:15.096145    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ebe4723-a9ed-42e8-a69e-3aa14ed707bc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0ebe4723-a9ed-42e8-a69e-3aa14ed707bc" (UID: "0ebe4723-a9ed-42e8-a69e-3aa14ed707bc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:50:15.096342 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:50:15.096251    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ebe4723-a9ed-42e8-a69e-3aa14ed707bc-model-cache\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 11:50:15.096342 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:50:15.096263    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ebe4723-a9ed-42e8-a69e-3aa14ed707bc-home\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 11:50:15.096342 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:50:15.096274    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ebe4723-a9ed-42e8-a69e-3aa14ed707bc-tmp-dir\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 11:50:15.098313 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:50:15.098291    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ebe4723-a9ed-42e8-a69e-3aa14ed707bc-dshm" (OuterVolumeSpecName: "dshm") pod "0ebe4723-a9ed-42e8-a69e-3aa14ed707bc" (UID: "0ebe4723-a9ed-42e8-a69e-3aa14ed707bc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:50:15.098523 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:50:15.098505    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ebe4723-a9ed-42e8-a69e-3aa14ed707bc-kube-api-access-fqk2q" (OuterVolumeSpecName: "kube-api-access-fqk2q") pod "0ebe4723-a9ed-42e8-a69e-3aa14ed707bc" (UID: "0ebe4723-a9ed-42e8-a69e-3aa14ed707bc"). InnerVolumeSpecName "kube-api-access-fqk2q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 11:50:15.098597 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:50:15.098580    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ebe4723-a9ed-42e8-a69e-3aa14ed707bc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0ebe4723-a9ed-42e8-a69e-3aa14ed707bc" (UID: "0ebe4723-a9ed-42e8-a69e-3aa14ed707bc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 11:50:15.197488 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:50:15.197400    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ebe4723-a9ed-42e8-a69e-3aa14ed707bc-tls-certs\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 11:50:15.197488 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:50:15.197428    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqk2q\" (UniqueName: \"kubernetes.io/projected/0ebe4723-a9ed-42e8-a69e-3aa14ed707bc-kube-api-access-fqk2q\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 11:50:15.197488 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:50:15.197436    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ebe4723-a9ed-42e8-a69e-3aa14ed707bc-dshm\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 11:50:15.433739 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:50:15.433705    2563 generic.go:356] "Generic (PLEG): container finished" podID="0ebe4723-a9ed-42e8-a69e-3aa14ed707bc" containerID="6a6fa2883a693112402fd0126cd598b298cc94c1b21f9014c5ef92c129173b2d" exitCode=0
Jul 28 11:50:15.433939 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:50:15.433751    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-kj4vk" event={"ID":"0ebe4723-a9ed-42e8-a69e-3aa14ed707bc","Type":"ContainerDied","Data":"6a6fa2883a693112402fd0126cd598b298cc94c1b21f9014c5ef92c129173b2d"}
Jul 28 11:50:15.433939 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:50:15.433757    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-kj4vk"
Jul 28 11:50:15.433939 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:50:15.433794    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-kj4vk" event={"ID":"0ebe4723-a9ed-42e8-a69e-3aa14ed707bc","Type":"ContainerDied","Data":"158b900c32b1ff6cd9ec929e7bcdbc3ed52658868de233e4ec58de45184f4305"}
Jul 28 11:50:15.433939 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:50:15.433817    2563 scope.go:117] "RemoveContainer" containerID="6a6fa2883a693112402fd0126cd598b298cc94c1b21f9014c5ef92c129173b2d"
Jul 28 11:50:15.443241 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:50:15.443217    2563 scope.go:117] "RemoveContainer" containerID="6a6fa2883a693112402fd0126cd598b298cc94c1b21f9014c5ef92c129173b2d"
Jul 28 11:50:15.443659 ip-10-0-130-235 kubenswrapper[2563]: E0728 11:50:15.443633    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a6fa2883a693112402fd0126cd598b298cc94c1b21f9014c5ef92c129173b2d\": container with ID starting with 6a6fa2883a693112402fd0126cd598b298cc94c1b21f9014c5ef92c129173b2d not found: ID does not exist" containerID="6a6fa2883a693112402fd0126cd598b298cc94c1b21f9014c5ef92c129173b2d"
Jul 28 11:50:15.443759 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:50:15.443667    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a6fa2883a693112402fd0126cd598b298cc94c1b21f9014c5ef92c129173b2d"} err="failed to get container status \"6a6fa2883a693112402fd0126cd598b298cc94c1b21f9014c5ef92c129173b2d\": rpc error: code = NotFound desc = could not find container \"6a6fa2883a693112402fd0126cd598b298cc94c1b21f9014c5ef92c129173b2d\": container with ID starting with 6a6fa2883a693112402fd0126cd598b298cc94c1b21f9014c5ef92c129173b2d not found: ID does not exist"
Jul 28 11:50:15.450197 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:50:15.450122    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-kj4vk"]
Jul 28 11:50:15.456827 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:50:15.456797    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-74b79d7f77-kj4vk"]
Jul 28 11:50:17.131945 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:50:17.131914    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ebe4723-a9ed-42e8-a69e-3aa14ed707bc" path="/var/lib/kubelet/pods/0ebe4723-a9ed-42e8-a69e-3aa14ed707bc/volumes"
Jul 28 11:51:38.894606 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:51:38.894569    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/llmisvc-controller-manager-9547d8dfc-d5x59"]
Jul 28 11:51:38.895213 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:51:38.894843    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve/llmisvc-controller-manager-9547d8dfc-d5x59" podUID="14da0de4-5f97-4f12-9577-1778aab5fd4b" containerName="manager" containerID="cri-o://ab5900ed3464d936741e6fbf3ec946e511ca0d62e47d8156f073548e2d07f7f7" gracePeriod=30
Jul 28 11:51:39.863418 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:51:39.863388    2563 generic.go:356] "Generic (PLEG): container finished" podID="14da0de4-5f97-4f12-9577-1778aab5fd4b" containerID="ab5900ed3464d936741e6fbf3ec946e511ca0d62e47d8156f073548e2d07f7f7" exitCode=0
Jul 28 11:51:39.863593 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:51:39.863463    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-9547d8dfc-d5x59" event={"ID":"14da0de4-5f97-4f12-9577-1778aab5fd4b","Type":"ContainerDied","Data":"ab5900ed3464d936741e6fbf3ec946e511ca0d62e47d8156f073548e2d07f7f7"}
Jul 28 11:51:40.448866 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:51:40.448838    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-9547d8dfc-d5x59"
Jul 28 11:51:40.507065 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:51:40.507021    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w5288\" (UniqueName: \"kubernetes.io/projected/14da0de4-5f97-4f12-9577-1778aab5fd4b-kube-api-access-w5288\") pod \"14da0de4-5f97-4f12-9577-1778aab5fd4b\" (UID: \"14da0de4-5f97-4f12-9577-1778aab5fd4b\") "
Jul 28 11:51:40.507277 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:51:40.507137    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/14da0de4-5f97-4f12-9577-1778aab5fd4b-cert\") pod \"14da0de4-5f97-4f12-9577-1778aab5fd4b\" (UID: \"14da0de4-5f97-4f12-9577-1778aab5fd4b\") "
Jul 28 11:51:40.509627 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:51:40.509591    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14da0de4-5f97-4f12-9577-1778aab5fd4b-cert" (OuterVolumeSpecName: "cert") pod "14da0de4-5f97-4f12-9577-1778aab5fd4b" (UID: "14da0de4-5f97-4f12-9577-1778aab5fd4b"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 11:51:40.509627 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:51:40.509605    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14da0de4-5f97-4f12-9577-1778aab5fd4b-kube-api-access-w5288" (OuterVolumeSpecName: "kube-api-access-w5288") pod "14da0de4-5f97-4f12-9577-1778aab5fd4b" (UID: "14da0de4-5f97-4f12-9577-1778aab5fd4b"). InnerVolumeSpecName "kube-api-access-w5288". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 11:51:40.608326 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:51:40.608283    2563 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/14da0de4-5f97-4f12-9577-1778aab5fd4b-cert\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 11:51:40.608326 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:51:40.608316    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w5288\" (UniqueName: \"kubernetes.io/projected/14da0de4-5f97-4f12-9577-1778aab5fd4b-kube-api-access-w5288\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 11:51:40.869305 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:51:40.869219    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-9547d8dfc-d5x59"
Jul 28 11:51:40.869305 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:51:40.869226    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-9547d8dfc-d5x59" event={"ID":"14da0de4-5f97-4f12-9577-1778aab5fd4b","Type":"ContainerDied","Data":"ff98ddcbbf74ad51e0b7ae19056f2f6f9ca83cc5807f313be1bf996d6d43edc3"}
Jul 28 11:51:40.869305 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:51:40.869283    2563 scope.go:117] "RemoveContainer" containerID="ab5900ed3464d936741e6fbf3ec946e511ca0d62e47d8156f073548e2d07f7f7"
Jul 28 11:51:40.890290 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:51:40.890259    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/llmisvc-controller-manager-9547d8dfc-d5x59"]
Jul 28 11:51:40.893275 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:51:40.893246    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/llmisvc-controller-manager-9547d8dfc-d5x59"]
Jul 28 11:51:41.131374 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:51:41.131273    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14da0de4-5f97-4f12-9577-1778aab5fd4b" path="/var/lib/kubelet/pods/14da0de4-5f97-4f12-9577-1778aab5fd4b/volumes"
Jul 28 11:52:00.344034 ip-10-0-130-235 kubenswrapper[2563]: E0728 11:52:00.343983    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kserve-pvc-source], unattached volumes=[kserve-pvc-source], failed to process volumes=[]: context deadline exceeded" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0" podUID="8cc1f64c-273d-47e1-add7-cfef2f51d3b0"
Jul 28 11:52:00.968401 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:00.968300    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0"
Jul 28 11:52:03.529694 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:03.529653    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h"]
Jul 28 11:52:03.539861 ip-10-0-130-235 kubenswrapper[2563]: E0728 11:52:03.539816    2563 log.go:32] "ExecSync cmd from runtime service failed" err=<
Jul 28 11:52:03.539861 ip-10-0-130-235 kubenswrapper[2563]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 28 11:52:03.539861 ip-10-0-130-235 kubenswrapper[2563]:         , stdout: , stderr: , exit code -1
Jul 28 11:52:03.539861 ip-10-0-130-235 kubenswrapper[2563]:  > containerID="d1573d63ccd475f2966f896589b0420ffc86b1f7a0e6785a28076d0538d0b796" cmd=["/bin/sleep","15"]
Jul 28 11:52:03.540158 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:03.539882    2563 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h" message="" err=<
Jul 28 11:52:03.540158 ip-10-0-130-235 kubenswrapper[2563]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 28 11:52:03.540158 ip-10-0-130-235 kubenswrapper[2563]:         , stdout: , stderr: , exit code -1
Jul 28 11:52:03.540158 ip-10-0-130-235 kubenswrapper[2563]:  >
Jul 28 11:52:03.540158 ip-10-0-130-235 kubenswrapper[2563]: E0728 11:52:03.539895    2563 kuberuntime_container.go:755] "PreStop hook failed" err=<
Jul 28 11:52:03.540158 ip-10-0-130-235 kubenswrapper[2563]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Jul 28 11:52:03.540158 ip-10-0-130-235 kubenswrapper[2563]:         , stdout: , stderr: , exit code -1
Jul 28 11:52:03.540158 ip-10-0-130-235 kubenswrapper[2563]:  > pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h" podUID="f0c1c6bd-043e-4553-9388-f0e0fcf102ac" containerName="main" containerID="cri-o://d1573d63ccd475f2966f896589b0420ffc86b1f7a0e6785a28076d0538d0b796"
Jul 28 11:52:03.540158 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:03.539948    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h" podUID="f0c1c6bd-043e-4553-9388-f0e0fcf102ac" containerName="main" containerID="cri-o://d1573d63ccd475f2966f896589b0420ffc86b1f7a0e6785a28076d0538d0b796" gracePeriod=60
Jul 28 11:52:03.583153 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:03.583105    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h" podUID="f0c1c6bd-043e-4553-9388-f0e0fcf102ac" containerName="main" probeResult="failure" output="Get \"https://10.132.0.42:8000/health\": dial tcp 10.132.0.42:8000: connect: connection refused"
Jul 28 11:52:03.898618 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:03.898588    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h"
Jul 28 11:52:03.987418 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:03.987344    2563 generic.go:356] "Generic (PLEG): container finished" podID="f0c1c6bd-043e-4553-9388-f0e0fcf102ac" containerID="d1573d63ccd475f2966f896589b0420ffc86b1f7a0e6785a28076d0538d0b796" exitCode=0
Jul 28 11:52:03.987596 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:03.987426    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h"
Jul 28 11:52:03.987596 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:03.987434    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h" event={"ID":"f0c1c6bd-043e-4553-9388-f0e0fcf102ac","Type":"ContainerDied","Data":"d1573d63ccd475f2966f896589b0420ffc86b1f7a0e6785a28076d0538d0b796"}
Jul 28 11:52:03.987596 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:03.987491    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h" event={"ID":"f0c1c6bd-043e-4553-9388-f0e0fcf102ac","Type":"ContainerDied","Data":"a4367ac03dc4f9bfe88a6b9aba35e2fc64d6066703e3be67e8fbdd34209b3139"}
Jul 28 11:52:03.987596 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:03.987516    2563 scope.go:117] "RemoveContainer" containerID="d1573d63ccd475f2966f896589b0420ffc86b1f7a0e6785a28076d0538d0b796"
Jul 28 11:52:03.997106 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:03.997083    2563 scope.go:117] "RemoveContainer" containerID="d1573d63ccd475f2966f896589b0420ffc86b1f7a0e6785a28076d0538d0b796"
Jul 28 11:52:03.997536 ip-10-0-130-235 kubenswrapper[2563]: E0728 11:52:03.997511    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1573d63ccd475f2966f896589b0420ffc86b1f7a0e6785a28076d0538d0b796\": container with ID starting with d1573d63ccd475f2966f896589b0420ffc86b1f7a0e6785a28076d0538d0b796 not found: ID does not exist" containerID="d1573d63ccd475f2966f896589b0420ffc86b1f7a0e6785a28076d0538d0b796"
Jul 28 11:52:03.997597 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:03.997549    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1573d63ccd475f2966f896589b0420ffc86b1f7a0e6785a28076d0538d0b796"} err="failed to get container status \"d1573d63ccd475f2966f896589b0420ffc86b1f7a0e6785a28076d0538d0b796\": rpc error: code = NotFound desc = could not find container \"d1573d63ccd475f2966f896589b0420ffc86b1f7a0e6785a28076d0538d0b796\": container with ID starting with d1573d63ccd475f2966f896589b0420ffc86b1f7a0e6785a28076d0538d0b796 not found: ID does not exist"
Jul 28 11:52:04.025881 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:04.025847    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0c1c6bd-043e-4553-9388-f0e0fcf102ac-model-cache\") pod \"f0c1c6bd-043e-4553-9388-f0e0fcf102ac\" (UID: \"f0c1c6bd-043e-4553-9388-f0e0fcf102ac\") "
Jul 28 11:52:04.026006 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:04.025928    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0c1c6bd-043e-4553-9388-f0e0fcf102ac-tls-certs\") pod \"f0c1c6bd-043e-4553-9388-f0e0fcf102ac\" (UID: \"f0c1c6bd-043e-4553-9388-f0e0fcf102ac\") "
Jul 28 11:52:04.026065 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:04.026015    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0c1c6bd-043e-4553-9388-f0e0fcf102ac-tmp-dir\") pod \"f0c1c6bd-043e-4553-9388-f0e0fcf102ac\" (UID: \"f0c1c6bd-043e-4553-9388-f0e0fcf102ac\") "
Jul 28 11:52:04.026065 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:04.026050    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kj7wm\" (UniqueName: \"kubernetes.io/projected/f0c1c6bd-043e-4553-9388-f0e0fcf102ac-kube-api-access-kj7wm\") pod \"f0c1c6bd-043e-4553-9388-f0e0fcf102ac\" (UID: \"f0c1c6bd-043e-4553-9388-f0e0fcf102ac\") "
Jul 28 11:52:04.026163 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:04.026106    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0c1c6bd-043e-4553-9388-f0e0fcf102ac-home\") pod \"f0c1c6bd-043e-4553-9388-f0e0fcf102ac\" (UID: \"f0c1c6bd-043e-4553-9388-f0e0fcf102ac\") "
Jul 28 11:52:04.026163 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:04.026147    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0c1c6bd-043e-4553-9388-f0e0fcf102ac-dshm\") pod \"f0c1c6bd-043e-4553-9388-f0e0fcf102ac\" (UID: \"f0c1c6bd-043e-4553-9388-f0e0fcf102ac\") "
Jul 28 11:52:04.026265 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:04.026186    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0c1c6bd-043e-4553-9388-f0e0fcf102ac-model-cache" (OuterVolumeSpecName: "model-cache") pod "f0c1c6bd-043e-4553-9388-f0e0fcf102ac" (UID: "f0c1c6bd-043e-4553-9388-f0e0fcf102ac"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:52:04.026425 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:04.026399    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0c1c6bd-043e-4553-9388-f0e0fcf102ac-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f0c1c6bd-043e-4553-9388-f0e0fcf102ac" (UID: "f0c1c6bd-043e-4553-9388-f0e0fcf102ac"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:52:04.026490 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:04.026415    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0c1c6bd-043e-4553-9388-f0e0fcf102ac-home" (OuterVolumeSpecName: "home") pod "f0c1c6bd-043e-4553-9388-f0e0fcf102ac" (UID: "f0c1c6bd-043e-4553-9388-f0e0fcf102ac"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:52:04.026490 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:04.026466    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0c1c6bd-043e-4553-9388-f0e0fcf102ac-model-cache\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 11:52:04.028513 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:04.028484    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0c1c6bd-043e-4553-9388-f0e0fcf102ac-dshm" (OuterVolumeSpecName: "dshm") pod "f0c1c6bd-043e-4553-9388-f0e0fcf102ac" (UID: "f0c1c6bd-043e-4553-9388-f0e0fcf102ac"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:52:04.028623 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:04.028545    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0c1c6bd-043e-4553-9388-f0e0fcf102ac-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f0c1c6bd-043e-4553-9388-f0e0fcf102ac" (UID: "f0c1c6bd-043e-4553-9388-f0e0fcf102ac"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 11:52:04.028623 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:04.028554    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0c1c6bd-043e-4553-9388-f0e0fcf102ac-kube-api-access-kj7wm" (OuterVolumeSpecName: "kube-api-access-kj7wm") pod "f0c1c6bd-043e-4553-9388-f0e0fcf102ac" (UID: "f0c1c6bd-043e-4553-9388-f0e0fcf102ac"). InnerVolumeSpecName "kube-api-access-kj7wm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 11:52:04.127138 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:04.127104    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0c1c6bd-043e-4553-9388-f0e0fcf102ac-tmp-dir\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 11:52:04.127138 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:04.127136    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kj7wm\" (UniqueName: \"kubernetes.io/projected/f0c1c6bd-043e-4553-9388-f0e0fcf102ac-kube-api-access-kj7wm\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 11:52:04.127138 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:04.127145    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0c1c6bd-043e-4553-9388-f0e0fcf102ac-home\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 11:52:04.127431 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:04.127153    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0c1c6bd-043e-4553-9388-f0e0fcf102ac-dshm\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 11:52:04.127431 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:04.127161    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0c1c6bd-043e-4553-9388-f0e0fcf102ac-tls-certs\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 11:52:04.311566 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:04.311526    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h"]
Jul 28 11:52:04.314784 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:04.314753    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-autoscaling-cleanup-hpa-a41f0e40/autoscale-cleanup-hpa-kserve-6cfdf55fc9-24z8h"]
Jul 28 11:52:05.132089 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:05.132048    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0c1c6bd-043e-4553-9388-f0e0fcf102ac" path="/var/lib/kubelet/pods/f0c1c6bd-043e-4553-9388-f0e0fcf102ac/volumes"
Jul 28 11:52:24.812542 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:24.812501    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-47lr5/must-gather-786cj"]
Jul 28 11:52:24.813080 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:24.812896    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ebe4723-a9ed-42e8-a69e-3aa14ed707bc" containerName="main"
Jul 28 11:52:24.813080 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:24.812907    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ebe4723-a9ed-42e8-a69e-3aa14ed707bc" containerName="main"
Jul 28 11:52:24.813080 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:24.812927    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0c1c6bd-043e-4553-9388-f0e0fcf102ac" containerName="main"
Jul 28 11:52:24.813080 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:24.812933    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0c1c6bd-043e-4553-9388-f0e0fcf102ac" containerName="main"
Jul 28 11:52:24.813080 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:24.812946    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14da0de4-5f97-4f12-9577-1778aab5fd4b" containerName="manager"
Jul 28 11:52:24.813080 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:24.812953    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="14da0de4-5f97-4f12-9577-1778aab5fd4b" containerName="manager"
Jul 28 11:52:24.813080 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:24.813016    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="14da0de4-5f97-4f12-9577-1778aab5fd4b" containerName="manager"
Jul 28 11:52:24.813080 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:24.813026    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0c1c6bd-043e-4553-9388-f0e0fcf102ac" containerName="main"
Jul 28 11:52:24.813080 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:24.813034    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ebe4723-a9ed-42e8-a69e-3aa14ed707bc" containerName="main"
Jul 28 11:52:24.816278 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:24.816257    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-47lr5/must-gather-786cj"
Jul 28 11:52:24.818898 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:24.818877    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-47lr5\"/\"kube-root-ca.crt\""
Jul 28 11:52:24.819028 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:24.818972    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-47lr5\"/\"default-dockercfg-db4kp\""
Jul 28 11:52:24.819559 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:24.819542    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-47lr5\"/\"openshift-service-ca.crt\""
Jul 28 11:52:24.824638 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:24.824617    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-47lr5/must-gather-786cj"]
Jul 28 11:52:24.926582 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:24.926531    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/600dc39e-6213-468b-b63b-9483a3dc44d5-must-gather-output\") pod \"must-gather-786cj\" (UID: \"600dc39e-6213-468b-b63b-9483a3dc44d5\") " pod="openshift-must-gather-47lr5/must-gather-786cj"
Jul 28 11:52:24.926799 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:24.926593    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkr2m\" (UniqueName: \"kubernetes.io/projected/600dc39e-6213-468b-b63b-9483a3dc44d5-kube-api-access-bkr2m\") pod \"must-gather-786cj\" (UID: \"600dc39e-6213-468b-b63b-9483a3dc44d5\") " pod="openshift-must-gather-47lr5/must-gather-786cj"
Jul 28 11:52:25.027704 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:25.027666    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/600dc39e-6213-468b-b63b-9483a3dc44d5-must-gather-output\") pod \"must-gather-786cj\" (UID: \"600dc39e-6213-468b-b63b-9483a3dc44d5\") " pod="openshift-must-gather-47lr5/must-gather-786cj"
Jul 28 11:52:25.027704 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:25.027713    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkr2m\" (UniqueName: \"kubernetes.io/projected/600dc39e-6213-468b-b63b-9483a3dc44d5-kube-api-access-bkr2m\") pod \"must-gather-786cj\" (UID: \"600dc39e-6213-468b-b63b-9483a3dc44d5\") " pod="openshift-must-gather-47lr5/must-gather-786cj"
Jul 28 11:52:25.028030 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:25.028010    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/600dc39e-6213-468b-b63b-9483a3dc44d5-must-gather-output\") pod \"must-gather-786cj\" (UID: \"600dc39e-6213-468b-b63b-9483a3dc44d5\") " pod="openshift-must-gather-47lr5/must-gather-786cj"
Jul 28 11:52:25.041199 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:25.041176    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkr2m\" (UniqueName: \"kubernetes.io/projected/600dc39e-6213-468b-b63b-9483a3dc44d5-kube-api-access-bkr2m\") pod \"must-gather-786cj\" (UID: \"600dc39e-6213-468b-b63b-9483a3dc44d5\") " pod="openshift-must-gather-47lr5/must-gather-786cj"
Jul 28 11:52:25.127078 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:25.127046    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-47lr5/must-gather-786cj"
Jul 28 11:52:25.273889 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:25.273863    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-47lr5/must-gather-786cj"]
Jul 28 11:52:25.276098 ip-10-0-130-235 kubenswrapper[2563]: W0728 11:52:25.276072    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod600dc39e_6213_468b_b63b_9483a3dc44d5.slice/crio-e3d0a24fc9470a9cb5b416e30d1340b98721c9794ba28dd401455e702c9b3b70 WatchSource:0}: Error finding container e3d0a24fc9470a9cb5b416e30d1340b98721c9794ba28dd401455e702c9b3b70: Status 404 returned error can't find the container with id e3d0a24fc9470a9cb5b416e30d1340b98721c9794ba28dd401455e702c9b3b70
Jul 28 11:52:25.277866 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:25.277849    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 11:52:26.108562 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:26.108471    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-47lr5/must-gather-786cj" event={"ID":"600dc39e-6213-468b-b63b-9483a3dc44d5","Type":"ContainerStarted","Data":"e3d0a24fc9470a9cb5b416e30d1340b98721c9794ba28dd401455e702c9b3b70"}
Jul 28 11:52:32.147596 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:32.147558    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-47lr5/must-gather-786cj" event={"ID":"600dc39e-6213-468b-b63b-9483a3dc44d5","Type":"ContainerStarted","Data":"b3d740be61e057ef8ee090b4bc8bc0990def2357900ee2292b048d4842a2b5d5"}
Jul 28 11:52:32.148007 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:32.147604    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-47lr5/must-gather-786cj" event={"ID":"600dc39e-6213-468b-b63b-9483a3dc44d5","Type":"ContainerStarted","Data":"a637a11c44c35afa5f84c685292db387ba3be2a6d4eecd8fa9c0ddef0af5a53c"}
Jul 28 11:52:32.165012 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:32.164958    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-47lr5/must-gather-786cj" podStartSLOduration=1.686127875 podStartE2EDuration="8.164941727s" totalImagesPullingTime="6.478813852s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:52:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:52:32.164384897 +0000 UTC m=+6295.605926233" watchObservedRunningTime="2026-07-28 11:52:32.164941727 +0000 UTC m=+6295.606483064"
Jul 28 11:52:37.674558 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:37.674458    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 11:52:37.691983 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:37.691800    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 11:52:49.112108 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:49.112077    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-65b54959b5-rhznz_70a2f95e-67d1-4353-b4d9-d59a5c23f970/router/0.log"
Jul 28 11:52:50.059565 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:50.059538    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-65b54959b5-rhznz_70a2f95e-67d1-4353-b4d9-d59a5c23f970/router/0.log"
Jul 28 11:52:51.042329 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:51.042298    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-65b54959b5-rhznz_70a2f95e-67d1-4353-b4d9-d59a5c23f970/router/0.log"
Jul 28 11:52:52.082980 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:52.082945    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-65b54959b5-rhznz_70a2f95e-67d1-4353-b4d9-d59a5c23f970/router/0.log"
Jul 28 11:52:53.146175 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:53.146146    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-srd6m_90ab7b32-9e16-40dc-a7ca-a97e62d2265e/kuadrant-console-plugin/0.log"
Jul 28 11:52:53.313527 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:53.313497    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-2zlsl_3493d220-0745-4284-91fd-bdad0ba84e8a/limitador/0.log"
Jul 28 11:52:54.574137 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:54.574105    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-srd6m_90ab7b32-9e16-40dc-a7ca-a97e62d2265e/kuadrant-console-plugin/0.log"
Jul 28 11:52:54.745083 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:54.745057    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-2zlsl_3493d220-0745-4284-91fd-bdad0ba84e8a/limitador/0.log"
Jul 28 11:52:55.907339 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:55.907311    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-srd6m_90ab7b32-9e16-40dc-a7ca-a97e62d2265e/kuadrant-console-plugin/0.log"
Jul 28 11:52:56.087400 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:56.087378    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-2zlsl_3493d220-0745-4284-91fd-bdad0ba84e8a/limitador/0.log"
Jul 28 11:52:57.186516 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:57.186489    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-srd6m_90ab7b32-9e16-40dc-a7ca-a97e62d2265e/kuadrant-console-plugin/0.log"
Jul 28 11:52:57.358144 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:57.358113    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-2zlsl_3493d220-0745-4284-91fd-bdad0ba84e8a/limitador/0.log"
Jul 28 11:52:58.455168 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:58.455137    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-srd6m_90ab7b32-9e16-40dc-a7ca-a97e62d2265e/kuadrant-console-plugin/0.log"
Jul 28 11:52:58.631594 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:58.631564    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-2zlsl_3493d220-0745-4284-91fd-bdad0ba84e8a/limitador/0.log"
Jul 28 11:52:59.731828 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:59.731797    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-srd6m_90ab7b32-9e16-40dc-a7ca-a97e62d2265e/kuadrant-console-plugin/0.log"
Jul 28 11:52:59.898732 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:52:59.898700    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-2zlsl_3493d220-0745-4284-91fd-bdad0ba84e8a/limitador/0.log"
Jul 28 11:53:00.980471 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:00.980393    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-srd6m_90ab7b32-9e16-40dc-a7ca-a97e62d2265e/kuadrant-console-plugin/0.log"
Jul 28 11:53:01.149009 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:01.148976    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-2zlsl_3493d220-0745-4284-91fd-bdad0ba84e8a/limitador/0.log"
Jul 28 11:53:02.233500 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:02.233472    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-srd6m_90ab7b32-9e16-40dc-a7ca-a97e62d2265e/kuadrant-console-plugin/0.log"
Jul 28 11:53:02.407581 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:02.407546    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-2zlsl_3493d220-0745-4284-91fd-bdad0ba84e8a/limitador/0.log"
Jul 28 11:53:03.504900 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:03.504872    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-srd6m_90ab7b32-9e16-40dc-a7ca-a97e62d2265e/kuadrant-console-plugin/0.log"
Jul 28 11:53:03.684621 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:03.684596    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-2zlsl_3493d220-0745-4284-91fd-bdad0ba84e8a/limitador/0.log"
Jul 28 11:53:04.799823 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:04.799795    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-srd6m_90ab7b32-9e16-40dc-a7ca-a97e62d2265e/kuadrant-console-plugin/0.log"
Jul 28 11:53:04.974039 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:04.974009    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-2zlsl_3493d220-0745-4284-91fd-bdad0ba84e8a/limitador/0.log"
Jul 28 11:53:06.084670 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:06.084644    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-srd6m_90ab7b32-9e16-40dc-a7ca-a97e62d2265e/kuadrant-console-plugin/0.log"
Jul 28 11:53:06.249377 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:06.249323    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-2zlsl_3493d220-0745-4284-91fd-bdad0ba84e8a/limitador/0.log"
Jul 28 11:53:07.330641 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:07.330612    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-srd6m_90ab7b32-9e16-40dc-a7ca-a97e62d2265e/kuadrant-console-plugin/0.log"
Jul 28 11:53:07.501608 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:07.501581    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-2zlsl_3493d220-0745-4284-91fd-bdad0ba84e8a/limitador/0.log"
Jul 28 11:53:08.615008 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:08.614977    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-srd6m_90ab7b32-9e16-40dc-a7ca-a97e62d2265e/kuadrant-console-plugin/0.log"
Jul 28 11:53:08.789847 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:08.789812    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-2zlsl_3493d220-0745-4284-91fd-bdad0ba84e8a/limitador/0.log"
Jul 28 11:53:09.882662 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:09.882635    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-srd6m_90ab7b32-9e16-40dc-a7ca-a97e62d2265e/kuadrant-console-plugin/0.log"
Jul 28 11:53:10.048866 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:10.048841    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-2zlsl_3493d220-0745-4284-91fd-bdad0ba84e8a/limitador/0.log"
Jul 28 11:53:11.124838 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:11.124809    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-srd6m_90ab7b32-9e16-40dc-a7ca-a97e62d2265e/kuadrant-console-plugin/0.log"
Jul 28 11:53:11.296065 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:11.296037    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-2zlsl_3493d220-0745-4284-91fd-bdad0ba84e8a/limitador/0.log"
Jul 28 11:53:12.393969 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:12.393834    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-srd6m_90ab7b32-9e16-40dc-a7ca-a97e62d2265e/kuadrant-console-plugin/0.log"
Jul 28 11:53:12.561804 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:12.561773    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-2zlsl_3493d220-0745-4284-91fd-bdad0ba84e8a/limitador/0.log"
Jul 28 11:53:13.640554 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:13.640526    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-srd6m_90ab7b32-9e16-40dc-a7ca-a97e62d2265e/kuadrant-console-plugin/0.log"
Jul 28 11:53:13.813747 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:13.813721    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-2zlsl_3493d220-0745-4284-91fd-bdad0ba84e8a/limitador/0.log"
Jul 28 11:53:14.962995 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:14.962965    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-srd6m_90ab7b32-9e16-40dc-a7ca-a97e62d2265e/kuadrant-console-plugin/0.log"
Jul 28 11:53:15.130409 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:15.130352    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-2zlsl_3493d220-0745-4284-91fd-bdad0ba84e8a/limitador/0.log"
Jul 28 11:53:16.221141 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:16.221110    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-srd6m_90ab7b32-9e16-40dc-a7ca-a97e62d2265e/kuadrant-console-plugin/0.log"
Jul 28 11:53:16.391019 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:16.390993    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-2zlsl_3493d220-0745-4284-91fd-bdad0ba84e8a/limitador/0.log"
Jul 28 11:53:17.488593 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:17.488567    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-srd6m_90ab7b32-9e16-40dc-a7ca-a97e62d2265e/kuadrant-console-plugin/0.log"
Jul 28 11:53:17.655427 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:17.655400    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-2zlsl_3493d220-0745-4284-91fd-bdad0ba84e8a/limitador/0.log"
Jul 28 11:53:18.777380 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:18.777326    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-srd6m_90ab7b32-9e16-40dc-a7ca-a97e62d2265e/kuadrant-console-plugin/0.log"
Jul 28 11:53:18.947974 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:18.947942    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-2zlsl_3493d220-0745-4284-91fd-bdad0ba84e8a/limitador/0.log"
Jul 28 11:53:20.048178 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:20.048148    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-srd6m_90ab7b32-9e16-40dc-a7ca-a97e62d2265e/kuadrant-console-plugin/0.log"
Jul 28 11:53:20.215326 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:20.215285    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-2zlsl_3493d220-0745-4284-91fd-bdad0ba84e8a/limitador/0.log"
Jul 28 11:53:21.318329 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:21.318304    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-srd6m_90ab7b32-9e16-40dc-a7ca-a97e62d2265e/kuadrant-console-plugin/0.log"
Jul 28 11:53:21.491462 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:21.491434    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-2zlsl_3493d220-0745-4284-91fd-bdad0ba84e8a/limitador/0.log"
Jul 28 11:53:22.591852 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:22.591821    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-srd6m_90ab7b32-9e16-40dc-a7ca-a97e62d2265e/kuadrant-console-plugin/0.log"
Jul 28 11:53:22.759705 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:22.759675    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-2zlsl_3493d220-0745-4284-91fd-bdad0ba84e8a/limitador/0.log"
Jul 28 11:53:23.882110 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:23.882084    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-srd6m_90ab7b32-9e16-40dc-a7ca-a97e62d2265e/kuadrant-console-plugin/0.log"
Jul 28 11:53:24.069142 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:24.069113    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-2zlsl_3493d220-0745-4284-91fd-bdad0ba84e8a/limitador/0.log"
Jul 28 11:53:25.457866 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:25.457830    2563 generic.go:356] "Generic (PLEG): container finished" podID="600dc39e-6213-468b-b63b-9483a3dc44d5" containerID="a637a11c44c35afa5f84c685292db387ba3be2a6d4eecd8fa9c0ddef0af5a53c" exitCode=0
Jul 28 11:53:25.458303 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:25.457903    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-47lr5/must-gather-786cj" event={"ID":"600dc39e-6213-468b-b63b-9483a3dc44d5","Type":"ContainerDied","Data":"a637a11c44c35afa5f84c685292db387ba3be2a6d4eecd8fa9c0ddef0af5a53c"}
Jul 28 11:53:25.458303 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:25.458298    2563 scope.go:117] "RemoveContainer" containerID="a637a11c44c35afa5f84c685292db387ba3be2a6d4eecd8fa9c0ddef0af5a53c"
Jul 28 11:53:25.655143 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:25.655098    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-47lr5_must-gather-786cj_600dc39e-6213-468b-b63b-9483a3dc44d5/gather/0.log"
Jul 28 11:53:26.272315 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:26.272228    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wxwvv/must-gather-hwxvc"]
Jul 28 11:53:26.276958 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:26.276939    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wxwvv/must-gather-hwxvc"
Jul 28 11:53:26.279115 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:26.279100    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-wxwvv\"/\"default-dockercfg-crk46\""
Jul 28 11:53:26.279369 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:26.279334    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wxwvv\"/\"kube-root-ca.crt\""
Jul 28 11:53:26.279440 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:26.279380    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wxwvv\"/\"openshift-service-ca.crt\""
Jul 28 11:53:26.283809 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:26.283785    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wxwvv/must-gather-hwxvc"]
Jul 28 11:53:26.338047 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:26.338014    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6hkh\" (UniqueName: \"kubernetes.io/projected/9e8784cb-a8b3-4db7-a13d-2f6f5344e378-kube-api-access-w6hkh\") pod \"must-gather-hwxvc\" (UID: \"9e8784cb-a8b3-4db7-a13d-2f6f5344e378\") " pod="openshift-must-gather-wxwvv/must-gather-hwxvc"
Jul 28 11:53:26.338213 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:26.338099    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9e8784cb-a8b3-4db7-a13d-2f6f5344e378-must-gather-output\") pod \"must-gather-hwxvc\" (UID: \"9e8784cb-a8b3-4db7-a13d-2f6f5344e378\") " pod="openshift-must-gather-wxwvv/must-gather-hwxvc"
Jul 28 11:53:26.439204 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:26.439171    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9e8784cb-a8b3-4db7-a13d-2f6f5344e378-must-gather-output\") pod \"must-gather-hwxvc\" (UID: \"9e8784cb-a8b3-4db7-a13d-2f6f5344e378\") " pod="openshift-must-gather-wxwvv/must-gather-hwxvc"
Jul 28 11:53:26.439409 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:26.439271    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6hkh\" (UniqueName: \"kubernetes.io/projected/9e8784cb-a8b3-4db7-a13d-2f6f5344e378-kube-api-access-w6hkh\") pod \"must-gather-hwxvc\" (UID: \"9e8784cb-a8b3-4db7-a13d-2f6f5344e378\") " pod="openshift-must-gather-wxwvv/must-gather-hwxvc"
Jul 28 11:53:26.439559 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:26.439539    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9e8784cb-a8b3-4db7-a13d-2f6f5344e378-must-gather-output\") pod \"must-gather-hwxvc\" (UID: \"9e8784cb-a8b3-4db7-a13d-2f6f5344e378\") " pod="openshift-must-gather-wxwvv/must-gather-hwxvc"
Jul 28 11:53:26.447418 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:26.447388    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6hkh\" (UniqueName: \"kubernetes.io/projected/9e8784cb-a8b3-4db7-a13d-2f6f5344e378-kube-api-access-w6hkh\") pod \"must-gather-hwxvc\" (UID: \"9e8784cb-a8b3-4db7-a13d-2f6f5344e378\") " pod="openshift-must-gather-wxwvv/must-gather-hwxvc"
Jul 28 11:53:26.587790 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:26.587759    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wxwvv/must-gather-hwxvc"
Jul 28 11:53:26.730735 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:26.730703    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wxwvv/must-gather-hwxvc"]
Jul 28 11:53:26.733742 ip-10-0-130-235 kubenswrapper[2563]: W0728 11:53:26.733719    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9e8784cb_a8b3_4db7_a13d_2f6f5344e378.slice/crio-4d9606c7d9c7034aea1854b2a3d6e278056b04937ece4522048f3bec418cd5e7 WatchSource:0}: Error finding container 4d9606c7d9c7034aea1854b2a3d6e278056b04937ece4522048f3bec418cd5e7: Status 404 returned error can't find the container with id 4d9606c7d9c7034aea1854b2a3d6e278056b04937ece4522048f3bec418cd5e7
Jul 28 11:53:27.469669 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:27.469624    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wxwvv/must-gather-hwxvc" event={"ID":"9e8784cb-a8b3-4db7-a13d-2f6f5344e378","Type":"ContainerStarted","Data":"4d9606c7d9c7034aea1854b2a3d6e278056b04937ece4522048f3bec418cd5e7"}
Jul 28 11:53:28.479676 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:28.479641    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wxwvv/must-gather-hwxvc" event={"ID":"9e8784cb-a8b3-4db7-a13d-2f6f5344e378","Type":"ContainerStarted","Data":"fa803bb9fc956482b9c0c74d79e0ed0de158e90c2e3a41ad4dc0cf6179a98638"}
Jul 28 11:53:28.480073 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:28.479685    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wxwvv/must-gather-hwxvc" event={"ID":"9e8784cb-a8b3-4db7-a13d-2f6f5344e378","Type":"ContainerStarted","Data":"bcedc579f9fa54ade2e0ac510d5d16aa10056d2d0fc244679d600a64b53390aa"}
Jul 28 11:53:28.496744 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:28.496675    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-wxwvv/must-gather-hwxvc" podStartSLOduration=1.5366185940000001 podStartE2EDuration="2.496652743s" totalImagesPullingTime="960.034149ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 11:53:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:53:28.494816817 +0000 UTC m=+6351.936358162" watchObservedRunningTime="2026-07-28 11:53:28.496652743 +0000 UTC m=+6351.938194082"
Jul 28 11:53:29.400016 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:29.399957    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-nz956_5f9bc3b4-d2b8-40de-8c04-846085757396/global-pull-secret-syncer/0.log"
Jul 28 11:53:29.500716 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:29.500687    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8bh2v_c4838d1b-fa7d-4770-a990-66c67c14c8b9/konnectivity-agent/0.log"
Jul 28 11:53:29.601998 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:29.601969    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-235.ec2.internal_b79e08bd50776ee222be14a701a056dc/haproxy/0.log"
Jul 28 11:53:31.107753 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:31.107716    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-47lr5/must-gather-786cj"]
Jul 28 11:53:31.108619 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:31.108559    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-47lr5/must-gather-786cj" podUID="600dc39e-6213-468b-b63b-9483a3dc44d5" containerName="copy" containerID="cri-o://b3d740be61e057ef8ee090b4bc8bc0990def2357900ee2292b048d4842a2b5d5" gracePeriod=2
Jul 28 11:53:31.111000 ip-10-0-130-235 kubenswrapper[2563]: E0728 11:53:31.110892    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-786cj\" is forbidden: User \"system:node:ip-10-0-130-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-47lr5\": no relationship found between node 'ip-10-0-130-235.ec2.internal' and this object" podUID="600dc39e-6213-468b-b63b-9483a3dc44d5" pod="openshift-must-gather-47lr5/must-gather-786cj"
Jul 28 11:53:31.113700 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:31.113655    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-47lr5/must-gather-786cj"]
Jul 28 11:53:31.505388 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:31.504872    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-47lr5_must-gather-786cj_600dc39e-6213-468b-b63b-9483a3dc44d5/copy/0.log"
Jul 28 11:53:31.511147 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:31.510296    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-47lr5/must-gather-786cj"
Jul 28 11:53:31.512777 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:31.511882    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-47lr5_must-gather-786cj_600dc39e-6213-468b-b63b-9483a3dc44d5/copy/0.log"
Jul 28 11:53:31.512777 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:31.512293    2563 generic.go:356] "Generic (PLEG): container finished" podID="600dc39e-6213-468b-b63b-9483a3dc44d5" containerID="b3d740be61e057ef8ee090b4bc8bc0990def2357900ee2292b048d4842a2b5d5" exitCode=143
Jul 28 11:53:31.512777 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:31.512488    2563 scope.go:117] "RemoveContainer" containerID="b3d740be61e057ef8ee090b4bc8bc0990def2357900ee2292b048d4842a2b5d5"
Jul 28 11:53:31.534549 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:31.534093    2563 scope.go:117] "RemoveContainer" containerID="a637a11c44c35afa5f84c685292db387ba3be2a6d4eecd8fa9c0ddef0af5a53c"
Jul 28 11:53:31.569571 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:31.569512    2563 scope.go:117] "RemoveContainer" containerID="b3d740be61e057ef8ee090b4bc8bc0990def2357900ee2292b048d4842a2b5d5"
Jul 28 11:53:31.570376 ip-10-0-130-235 kubenswrapper[2563]: E0728 11:53:31.570050    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3d740be61e057ef8ee090b4bc8bc0990def2357900ee2292b048d4842a2b5d5\": container with ID starting with b3d740be61e057ef8ee090b4bc8bc0990def2357900ee2292b048d4842a2b5d5 not found: ID does not exist" containerID="b3d740be61e057ef8ee090b4bc8bc0990def2357900ee2292b048d4842a2b5d5"
Jul 28 11:53:31.570376 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:31.570114    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3d740be61e057ef8ee090b4bc8bc0990def2357900ee2292b048d4842a2b5d5"} err="failed to get container status \"b3d740be61e057ef8ee090b4bc8bc0990def2357900ee2292b048d4842a2b5d5\": rpc error: code = NotFound desc = could not find container \"b3d740be61e057ef8ee090b4bc8bc0990def2357900ee2292b048d4842a2b5d5\": container with ID starting with b3d740be61e057ef8ee090b4bc8bc0990def2357900ee2292b048d4842a2b5d5 not found: ID does not exist"
Jul 28 11:53:31.570376 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:31.570140    2563 scope.go:117] "RemoveContainer" containerID="a637a11c44c35afa5f84c685292db387ba3be2a6d4eecd8fa9c0ddef0af5a53c"
Jul 28 11:53:31.570627 ip-10-0-130-235 kubenswrapper[2563]: E0728 11:53:31.570526    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a637a11c44c35afa5f84c685292db387ba3be2a6d4eecd8fa9c0ddef0af5a53c\": container with ID starting with a637a11c44c35afa5f84c685292db387ba3be2a6d4eecd8fa9c0ddef0af5a53c not found: ID does not exist" containerID="a637a11c44c35afa5f84c685292db387ba3be2a6d4eecd8fa9c0ddef0af5a53c"
Jul 28 11:53:31.570627 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:31.570558    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a637a11c44c35afa5f84c685292db387ba3be2a6d4eecd8fa9c0ddef0af5a53c"} err="failed to get container status \"a637a11c44c35afa5f84c685292db387ba3be2a6d4eecd8fa9c0ddef0af5a53c\": rpc error: code = NotFound desc = could not find container \"a637a11c44c35afa5f84c685292db387ba3be2a6d4eecd8fa9c0ddef0af5a53c\": container with ID starting with a637a11c44c35afa5f84c685292db387ba3be2a6d4eecd8fa9c0ddef0af5a53c not found: ID does not exist"
Jul 28 11:53:31.592860 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:31.592822    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bkr2m\" (UniqueName: \"kubernetes.io/projected/600dc39e-6213-468b-b63b-9483a3dc44d5-kube-api-access-bkr2m\") pod \"600dc39e-6213-468b-b63b-9483a3dc44d5\" (UID: \"600dc39e-6213-468b-b63b-9483a3dc44d5\") "
Jul 28 11:53:31.593083 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:31.592915    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/600dc39e-6213-468b-b63b-9483a3dc44d5-must-gather-output\") pod \"600dc39e-6213-468b-b63b-9483a3dc44d5\" (UID: \"600dc39e-6213-468b-b63b-9483a3dc44d5\") "
Jul 28 11:53:31.602040 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:31.600527    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/600dc39e-6213-468b-b63b-9483a3dc44d5-kube-api-access-bkr2m" (OuterVolumeSpecName: "kube-api-access-bkr2m") pod "600dc39e-6213-468b-b63b-9483a3dc44d5" (UID: "600dc39e-6213-468b-b63b-9483a3dc44d5"). InnerVolumeSpecName "kube-api-access-bkr2m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 11:53:31.606378 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:31.602910    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/600dc39e-6213-468b-b63b-9483a3dc44d5-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "600dc39e-6213-468b-b63b-9483a3dc44d5" (UID: "600dc39e-6213-468b-b63b-9483a3dc44d5"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 11:53:31.694419 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:31.694379    2563 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/600dc39e-6213-468b-b63b-9483a3dc44d5-must-gather-output\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 11:53:31.694419 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:31.694419    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bkr2m\" (UniqueName: \"kubernetes.io/projected/600dc39e-6213-468b-b63b-9483a3dc44d5-kube-api-access-bkr2m\") on node \"ip-10-0-130-235.ec2.internal\" DevicePath \"\""
Jul 28 11:53:32.521809 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:32.521771    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-47lr5/must-gather-786cj"
Jul 28 11:53:33.135040 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:33.134999    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="600dc39e-6213-468b-b63b-9483a3dc44d5" path="/var/lib/kubelet/pods/600dc39e-6213-468b-b63b-9483a3dc44d5/volumes"
Jul 28 11:53:34.515707 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:34.515671    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-srd6m_90ab7b32-9e16-40dc-a7ca-a97e62d2265e/kuadrant-console-plugin/0.log"
Jul 28 11:53:34.714659 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:34.714625    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-2zlsl_3493d220-0745-4284-91fd-bdad0ba84e8a/limitador/0.log"
Jul 28 11:53:35.717890 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:35.717853    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e/alertmanager/0.log"
Jul 28 11:53:35.740638 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:35.740612    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e/config-reloader/0.log"
Jul 28 11:53:35.769502 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:35.769460    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e/kube-rbac-proxy-web/0.log"
Jul 28 11:53:35.794579 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:35.794552    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e/kube-rbac-proxy/0.log"
Jul 28 11:53:35.818065 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:35.818019    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e/kube-rbac-proxy-metric/0.log"
Jul 28 11:53:35.841648 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:35.841603    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e/prom-label-proxy/0.log"
Jul 28 11:53:35.865468 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:35.865429    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_bb5b0c00-bbb6-4377-acd1-2f4a3ed8714e/init-config-reloader/0.log"
Jul 28 11:53:35.919930 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:35.919888    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-x6bhh_3014f727-2c79-465d-ba86-bbea763014e5/cluster-monitoring-operator/0.log"
Jul 28 11:53:36.058202 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:36.058127    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-blv5n_fae66faf-70d8-4771-a93b-81c86a4dbb89/monitoring-plugin/0.log"
Jul 28 11:53:36.164744 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:36.164710    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gxdc6_995d66bd-d908-4743-90f3-e5db8d8f5fcb/node-exporter/0.log"
Jul 28 11:53:36.199566 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:36.199535    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gxdc6_995d66bd-d908-4743-90f3-e5db8d8f5fcb/kube-rbac-proxy/0.log"
Jul 28 11:53:36.224140 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:36.224108    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-gxdc6_995d66bd-d908-4743-90f3-e5db8d8f5fcb/init-textfile/0.log"
Jul 28 11:53:36.332486 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:36.332458    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-g2lz6_a33c1bbf-f7c5-4302-93f0-3bf51a4b1f7f/kube-rbac-proxy-main/0.log"
Jul 28 11:53:36.357392 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:36.357347    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-g2lz6_a33c1bbf-f7c5-4302-93f0-3bf51a4b1f7f/kube-rbac-proxy-self/0.log"
Jul 28 11:53:36.383381 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:36.383339    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-g2lz6_a33c1bbf-f7c5-4302-93f0-3bf51a4b1f7f/openshift-state-metrics/0.log"
Jul 28 11:53:36.681283 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:36.681208    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-9pwjg_82569a5b-837e-48eb-8f0e-efd6e2538b19/prometheus-operator-admission-webhook/0.log"
Jul 28 11:53:36.728189 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:36.728161    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5fcc6cf86-n5c9g_8a0366a2-b544-4511-b5b7-ff44f5c85b80/telemeter-client/0.log"
Jul 28 11:53:36.773318 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:36.773290    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5fcc6cf86-n5c9g_8a0366a2-b544-4511-b5b7-ff44f5c85b80/reload/0.log"
Jul 28 11:53:36.820159 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:36.820128    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5fcc6cf86-n5c9g_8a0366a2-b544-4511-b5b7-ff44f5c85b80/kube-rbac-proxy/0.log"
Jul 28 11:53:36.870432 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:36.870404    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d57b56d8-ws28t_5ab933ad-8909-4f95-8f91-bf3ddeeb6876/thanos-query/0.log"
Jul 28 11:53:36.914338 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:36.914311    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d57b56d8-ws28t_5ab933ad-8909-4f95-8f91-bf3ddeeb6876/kube-rbac-proxy-web/0.log"
Jul 28 11:53:36.956785 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:36.956713    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d57b56d8-ws28t_5ab933ad-8909-4f95-8f91-bf3ddeeb6876/kube-rbac-proxy/0.log"
Jul 28 11:53:36.997181 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:36.997157    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d57b56d8-ws28t_5ab933ad-8909-4f95-8f91-bf3ddeeb6876/prom-label-proxy/0.log"
Jul 28 11:53:37.035201 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:37.035170    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d57b56d8-ws28t_5ab933ad-8909-4f95-8f91-bf3ddeeb6876/kube-rbac-proxy-rules/0.log"
Jul 28 11:53:37.075443 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:37.075410    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d57b56d8-ws28t_5ab933ad-8909-4f95-8f91-bf3ddeeb6876/kube-rbac-proxy-metrics/0.log"
Jul 28 11:53:38.444236 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:38.444206    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/1.log"
Jul 28 11:53:38.450883 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:38.450852    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-g7s47_bebcadc4-87cd-4b77-997e-bf1daa3bdfb5/console-operator/2.log"
Jul 28 11:53:39.428106 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:39.428068    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-nljwq_7f4d862e-1588-468a-bfce-d45f2a036b02/volume-data-source-validator/0.log"
Jul 28 11:53:40.465824 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:40.465792    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-thhvn_826225a0-88f8-4113-9f7d-05a75796d40d/dns/0.log"
Jul 28 11:53:40.489550 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:40.489525    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-thhvn_826225a0-88f8-4113-9f7d-05a75796d40d/kube-rbac-proxy/0.log"
Jul 28 11:53:40.613351 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:40.613315    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hpntr_0f2a806a-05a5-4fd1-be4f-be8b64a5013b/dns-node-resolver/0.log"
Jul 28 11:53:41.208216 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:41.208183    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-d48vp_4b1075b5-7b36-420f-aa9f-90022e20d840/node-ca/0.log"
Jul 28 11:53:42.535128 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:42.535104    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-65b54959b5-rhznz_70a2f95e-67d1-4353-b4d9-d59a5c23f970/router/0.log"
Jul 28 11:53:43.244325 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:43.244282    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-287lq_e6813c6f-ee42-4551-af42-d4fd8af7cf98/serve-healthcheck-canary/0.log"
Jul 28 11:53:44.144541 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:44.144506    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-r6ptx_f6411b5d-e39d-4922-a1fc-67c4952bbb01/insights-operator/0.log"
Jul 28 11:53:44.145706 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:44.145679    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-r6ptx_f6411b5d-e39d-4922-a1fc-67c4952bbb01/insights-operator/1.log"
Jul 28 11:53:44.358966 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:44.358920    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vj25w_562ec3c6-ef0d-41f0-9279-498b868331e8/kube-rbac-proxy/0.log"
Jul 28 11:53:44.384198 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:44.384163    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vj25w_562ec3c6-ef0d-41f0-9279-498b868331e8/exporter/0.log"
Jul 28 11:53:44.411143 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:44.411116    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vj25w_562ec3c6-ef0d-41f0-9279-498b868331e8/extractor/0.log"
Jul 28 11:53:45.299728 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:45.299641    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wxwvv/perf-node-gather-daemonset-x42bk"]
Jul 28 11:53:45.300332 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:45.300225    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="600dc39e-6213-468b-b63b-9483a3dc44d5" containerName="copy"
Jul 28 11:53:45.300332 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:45.300241    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="600dc39e-6213-468b-b63b-9483a3dc44d5" containerName="copy"
Jul 28 11:53:45.300332 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:45.300267    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="600dc39e-6213-468b-b63b-9483a3dc44d5" containerName="gather"
Jul 28 11:53:45.300332 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:45.300276    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="600dc39e-6213-468b-b63b-9483a3dc44d5" containerName="gather"
Jul 28 11:53:45.300612 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:45.300407    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="600dc39e-6213-468b-b63b-9483a3dc44d5" containerName="copy"
Jul 28 11:53:45.300612 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:45.300428    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="600dc39e-6213-468b-b63b-9483a3dc44d5" containerName="gather"
Jul 28 11:53:45.303935 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:45.303906    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wxwvv/perf-node-gather-daemonset-x42bk"
Jul 28 11:53:45.314792 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:45.314763    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wxwvv/perf-node-gather-daemonset-x42bk"]
Jul 28 11:53:45.442059 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:45.442021    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/484f231f-c36b-4f20-a69b-db8369753d75-lib-modules\") pod \"perf-node-gather-daemonset-x42bk\" (UID: \"484f231f-c36b-4f20-a69b-db8369753d75\") " pod="openshift-must-gather-wxwvv/perf-node-gather-daemonset-x42bk"
Jul 28 11:53:45.442059 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:45.442066    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/484f231f-c36b-4f20-a69b-db8369753d75-proc\") pod \"perf-node-gather-daemonset-x42bk\" (UID: \"484f231f-c36b-4f20-a69b-db8369753d75\") " pod="openshift-must-gather-wxwvv/perf-node-gather-daemonset-x42bk"
Jul 28 11:53:45.442302 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:45.442105    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/484f231f-c36b-4f20-a69b-db8369753d75-sys\") pod \"perf-node-gather-daemonset-x42bk\" (UID: \"484f231f-c36b-4f20-a69b-db8369753d75\") " pod="openshift-must-gather-wxwvv/perf-node-gather-daemonset-x42bk"
Jul 28 11:53:45.442302 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:45.442145    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sxfs\" (UniqueName: \"kubernetes.io/projected/484f231f-c36b-4f20-a69b-db8369753d75-kube-api-access-4sxfs\") pod \"perf-node-gather-daemonset-x42bk\" (UID: \"484f231f-c36b-4f20-a69b-db8369753d75\") " pod="openshift-must-gather-wxwvv/perf-node-gather-daemonset-x42bk"
Jul 28 11:53:45.442302 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:45.442183    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/484f231f-c36b-4f20-a69b-db8369753d75-podres\") pod \"perf-node-gather-daemonset-x42bk\" (UID: \"484f231f-c36b-4f20-a69b-db8369753d75\") " pod="openshift-must-gather-wxwvv/perf-node-gather-daemonset-x42bk"
Jul 28 11:53:45.543317 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:45.543271    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/484f231f-c36b-4f20-a69b-db8369753d75-proc\") pod \"perf-node-gather-daemonset-x42bk\" (UID: \"484f231f-c36b-4f20-a69b-db8369753d75\") " pod="openshift-must-gather-wxwvv/perf-node-gather-daemonset-x42bk"
Jul 28 11:53:45.543546 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:45.543336    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/484f231f-c36b-4f20-a69b-db8369753d75-sys\") pod \"perf-node-gather-daemonset-x42bk\" (UID: \"484f231f-c36b-4f20-a69b-db8369753d75\") " pod="openshift-must-gather-wxwvv/perf-node-gather-daemonset-x42bk"
Jul 28 11:53:45.543546 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:45.543397    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sxfs\" (UniqueName: \"kubernetes.io/projected/484f231f-c36b-4f20-a69b-db8369753d75-kube-api-access-4sxfs\") pod \"perf-node-gather-daemonset-x42bk\" (UID: \"484f231f-c36b-4f20-a69b-db8369753d75\") " pod="openshift-must-gather-wxwvv/perf-node-gather-daemonset-x42bk"
Jul 28 11:53:45.543546 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:45.543426    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/484f231f-c36b-4f20-a69b-db8369753d75-proc\") pod \"perf-node-gather-daemonset-x42bk\" (UID: \"484f231f-c36b-4f20-a69b-db8369753d75\") " pod="openshift-must-gather-wxwvv/perf-node-gather-daemonset-x42bk"
Jul 28 11:53:45.543546 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:45.543465    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/484f231f-c36b-4f20-a69b-db8369753d75-podres\") pod \"perf-node-gather-daemonset-x42bk\" (UID: \"484f231f-c36b-4f20-a69b-db8369753d75\") " pod="openshift-must-gather-wxwvv/perf-node-gather-daemonset-x42bk"
Jul 28 11:53:45.543546 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:45.543497    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/484f231f-c36b-4f20-a69b-db8369753d75-sys\") pod \"perf-node-gather-daemonset-x42bk\" (UID: \"484f231f-c36b-4f20-a69b-db8369753d75\") " pod="openshift-must-gather-wxwvv/perf-node-gather-daemonset-x42bk"
Jul 28 11:53:45.543546 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:45.543536    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/484f231f-c36b-4f20-a69b-db8369753d75-lib-modules\") pod \"perf-node-gather-daemonset-x42bk\" (UID: \"484f231f-c36b-4f20-a69b-db8369753d75\") " pod="openshift-must-gather-wxwvv/perf-node-gather-daemonset-x42bk"
Jul 28 11:53:45.543817 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:45.543623    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/484f231f-c36b-4f20-a69b-db8369753d75-podres\") pod \"perf-node-gather-daemonset-x42bk\" (UID: \"484f231f-c36b-4f20-a69b-db8369753d75\") " pod="openshift-must-gather-wxwvv/perf-node-gather-daemonset-x42bk"
Jul 28 11:53:45.543817 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:45.543668    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/484f231f-c36b-4f20-a69b-db8369753d75-lib-modules\") pod \"perf-node-gather-daemonset-x42bk\" (UID: \"484f231f-c36b-4f20-a69b-db8369753d75\") " pod="openshift-must-gather-wxwvv/perf-node-gather-daemonset-x42bk"
Jul 28 11:53:45.551736 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:45.551660    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sxfs\" (UniqueName: \"kubernetes.io/projected/484f231f-c36b-4f20-a69b-db8369753d75-kube-api-access-4sxfs\") pod \"perf-node-gather-daemonset-x42bk\" (UID: \"484f231f-c36b-4f20-a69b-db8369753d75\") " pod="openshift-must-gather-wxwvv/perf-node-gather-daemonset-x42bk"
Jul 28 11:53:45.617542 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:45.617494    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wxwvv/perf-node-gather-daemonset-x42bk"
Jul 28 11:53:45.784614 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:45.784556    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wxwvv/perf-node-gather-daemonset-x42bk"]
Jul 28 11:53:45.787222 ip-10-0-130-235 kubenswrapper[2563]: W0728 11:53:45.787184    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod484f231f_c36b_4f20_a69b_db8369753d75.slice/crio-8a7a4c7d344dd1b3a55e62a793d70a0bd8e11c9c45f0e38d669be321e70f39be WatchSource:0}: Error finding container 8a7a4c7d344dd1b3a55e62a793d70a0bd8e11c9c45f0e38d669be321e70f39be: Status 404 returned error can't find the container with id 8a7a4c7d344dd1b3a55e62a793d70a0bd8e11c9c45f0e38d669be321e70f39be
Jul 28 11:53:46.614134 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:46.614088    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wxwvv/perf-node-gather-daemonset-x42bk" event={"ID":"484f231f-c36b-4f20-a69b-db8369753d75","Type":"ContainerStarted","Data":"adfb5cc862e02f464d20bae7dc2c7038f0bddffe4a88b6133fc7b6517b783520"}
Jul 28 11:53:46.614632 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:46.614159    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wxwvv/perf-node-gather-daemonset-x42bk" event={"ID":"484f231f-c36b-4f20-a69b-db8369753d75","Type":"ContainerStarted","Data":"8a7a4c7d344dd1b3a55e62a793d70a0bd8e11c9c45f0e38d669be321e70f39be"}
Jul 28 11:53:46.614997 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:46.614972    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-wxwvv/perf-node-gather-daemonset-x42bk"
Jul 28 11:53:46.630398 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:46.630327    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-wxwvv/perf-node-gather-daemonset-x42bk" podStartSLOduration=1.630307918 podStartE2EDuration="1.630307918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 11:53:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 11:53:46.629052457 +0000 UTC m=+6370.070593794" watchObservedRunningTime="2026-07-28 11:53:46.630307918 +0000 UTC m=+6370.071849256"
Jul 28 11:53:47.450642 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:47.450613    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-68cc75d5fc-895fk_4f952314-6d4c-485f-8af8-2693950462e2/manager/0.log"
Jul 28 11:53:47.477288 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:47.477262    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-mzlfv_f2d727c1-a530-483a-b99b-b3ab2e3b71da/openshift-lws-operator/0.log"
Jul 28 11:53:53.641786 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:53.640844    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-wxwvv/perf-node-gather-daemonset-x42bk"
Jul 28 11:53:54.200741 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:54.200700    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-czhcl_f8b50686-28d7-4749-9430-54bd0f88f2f2/kube-storage-version-migrator-operator/1.log"
Jul 28 11:53:54.201634 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:54.201602    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-czhcl_f8b50686-28d7-4749-9430-54bd0f88f2f2/kube-storage-version-migrator-operator/0.log"
Jul 28 11:53:55.553171 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:55.553083    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-29wrb_5f8ff51d-90c2-4146-b040-a71013fed650/kube-multus/0.log"
Jul 28 11:53:55.586289 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:55.586263    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bmkbw_9e2fbc65-eba0-483e-ad62-d8f17b9bd452/kube-multus-additional-cni-plugins/0.log"
Jul 28 11:53:55.612612 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:55.612569    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bmkbw_9e2fbc65-eba0-483e-ad62-d8f17b9bd452/egress-router-binary-copy/0.log"
Jul 28 11:53:55.642033 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:55.642000    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bmkbw_9e2fbc65-eba0-483e-ad62-d8f17b9bd452/cni-plugins/0.log"
Jul 28 11:53:55.672136 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:55.672105    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bmkbw_9e2fbc65-eba0-483e-ad62-d8f17b9bd452/bond-cni-plugin/0.log"
Jul 28 11:53:55.695166 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:55.695127    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bmkbw_9e2fbc65-eba0-483e-ad62-d8f17b9bd452/routeoverride-cni/0.log"
Jul 28 11:53:55.721478 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:55.721445    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bmkbw_9e2fbc65-eba0-483e-ad62-d8f17b9bd452/whereabouts-cni-bincopy/0.log"
Jul 28 11:53:55.744876 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:55.744845    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bmkbw_9e2fbc65-eba0-483e-ad62-d8f17b9bd452/whereabouts-cni/0.log"
Jul 28 11:53:56.253004 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:56.252876    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-299qd_bfe1e00b-5d1b-4127-b270-abe1d44d3dd0/network-metrics-daemon/0.log"
Jul 28 11:53:56.277137 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:56.277071    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-299qd_bfe1e00b-5d1b-4127-b270-abe1d44d3dd0/kube-rbac-proxy/0.log"
Jul 28 11:53:56.814343 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:56.814227    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-65b54959b5-rhznz_70a2f95e-67d1-4353-b4d9-d59a5c23f970/router/0.log"
Jul 28 11:53:57.489881 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:57.489848    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4sg48_53d373c1-808b-4f32-ab54-d47bbeb3a151/ovn-controller/0.log"
Jul 28 11:53:57.509479 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:57.509439    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4sg48_53d373c1-808b-4f32-ab54-d47bbeb3a151/ovn-acl-logging/0.log"
Jul 28 11:53:57.535871 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:57.535837    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4sg48_53d373c1-808b-4f32-ab54-d47bbeb3a151/kube-rbac-proxy-node/0.log"
Jul 28 11:53:57.559620 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:57.559586    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4sg48_53d373c1-808b-4f32-ab54-d47bbeb3a151/kube-rbac-proxy-ovn-metrics/0.log"
Jul 28 11:53:57.580224 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:57.580186    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4sg48_53d373c1-808b-4f32-ab54-d47bbeb3a151/northd/0.log"
Jul 28 11:53:57.605051 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:57.605017    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4sg48_53d373c1-808b-4f32-ab54-d47bbeb3a151/nbdb/0.log"
Jul 28 11:53:57.628289 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:57.628253    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4sg48_53d373c1-808b-4f32-ab54-d47bbeb3a151/sbdb/0.log"
Jul 28 11:53:57.694746 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:57.694709    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4sg48_53d373c1-808b-4f32-ab54-d47bbeb3a151/ovnkube-controller/0.log"
Jul 28 11:53:58.137867 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:58.137840    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-65b54959b5-rhznz_70a2f95e-67d1-4353-b4d9-d59a5c23f970/router/0.log"
Jul 28 11:53:59.718793 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:59.718757    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-7d88cb55d8-vf6dm_b486a7fe-4047-4522-b588-de2eb9aa52aa/check-endpoints/0.log"
Jul 28 11:53:59.776061 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:53:59.776030    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-cmrjd_598884b3-5fd8-42ae-87ee-4f24d6c4bdb3/network-check-target-container/0.log"
Jul 28 11:54:00.934293 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:54:00.934258    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-bzknt_af6b006d-6148-4b30-a1e9-bd2e14118b06/iptables-alerter/0.log"
Jul 28 11:54:01.763099 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:54:01.763065    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-fmf4x_493a6ea0-265b-4330-8e11-b4911d3a3cb3/tuned/0.log"
Jul 28 11:54:03.968974 ip-10-0-130-235 kubenswrapper[2563]: E0728 11:54:03.968911    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kserve-pvc-source], unattached volumes=[kserve-pvc-source], failed to process volumes=[]: context deadline exceeded" pod="e2e-test-llm-inference-service-7c3633d1/llmisvc-model-pvc-router-manage-2577e794-kserve-mn-0" podUID="8cc1f64c-273d-47e1-add7-cfef2f51d3b0"
Jul 28 11:54:03.974326 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:54:03.974298    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-6748489656-vjpcz_94b8b045-295f-4194-a022-913c35c0529f/cluster-samples-operator/0.log"
Jul 28 11:54:03.992427 ip-10-0-130-235 kubenswrapper[2563]: I0728 11:54:03.992399    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-6748489656-vjpcz_94b8b045-295f-4194-a022-913c35c0529f/cluster-samples-operator-watch/0.log"